Block Collections

A block collection is a repository that groups related blocks together. All blocks in a collection share the same programming language and build system. Collections are the unit of distribution in Spade: you install, version, and publish collections, not individual blocks.

What collections are for🔗

Collections serve several purposes:

• Organization — Group related blocks by domain (e.g., raster processing, machine learning, data ingestion)
• Shared code — Blocks in the same collection can share utility functions, data types, and dependencies
• Consistent toolchain — All blocks in a collection use the same language and dependency management
• Versioning — The entire collection is versioned together, so all its blocks stay in sync

For example, you might have a raster collection containing blocks for reprojection, resampling, and mosaic operations, all written in Python and sharing a common set of GDAL bindings.

Directory structure🔗

A collection has this general layout:

my-collection/
  blocks/
    reproject.yaml      # Block manifest
    resample.yaml       # Block manifest
    mosaic.yaml         # Block manifest
  src/                  # Language-specific source code
    ...
  <project-file>        # Language-specific project file (see below)

The blocks/ directory contains one YAML manifest per block. Each manifest declares the block's interface (inputs, outputs, parameters) as described in Blocks. The rest of the repository contains the source code and configuration for the chosen language.

Language detection🔗

Spade determines the collection's language by looking for a project file in the repository root:

When you run spade init --language <lang>, Spade scaffolds the appropriate project file and directory structure. If you create a collection manually, just make sure the correct project file is present at the root.

Language-specific layouts🔗

Python collections use pyproject.toml and place source code under src/<package_name>/:

my-collection/
  pyproject.toml
  blocks/
    summarize.yaml
  src/
    my_collection/
      __init__.py
      summarize.py

R collections are the default when no recognized project file is found. Source code lives under R/, and dependencies (CRAN packages plus spadelib) are declared in a DESCRIPTION file:

my-collection/
  DESCRIPTION
  blocks/
    interpolate.yaml
  R/
    interpolate.R

Go collections use go.mod:

my-collection/
  go.mod
  go.sum
  blocks/
    convert.yaml
  cmd/
    convert/
      main.go

Rust collections use Cargo.toml:

my-collection/
  Cargo.toml
  blocks/
    detect.yaml
  src/
    bin/
      detect.rs

TypeScript collections use package.json:

my-collection/
  package.json
  tsconfig.json
  blocks/
    transform.yaml
  src/
    transform.ts

Collection name🔗

Every block has a unique ID in the form <collection>.<block>. The block name comes from the manifest filename (without the .yaml extension). The collection name comes from the language's own root manifest:

R is the odd one out: Spade never reads DESCRIPTION's Package field for naming purposes, it just uses whatever directory the collection lives in. This is why the built-in sae and stats collections resolve to sae/stats block IDs even though their DESCRIPTION files declare Package: sae.collection and Package: stats.collection — that field only names the R package for R's own tooling, it has no effect on the Spade collection name.

There is currently no supported way to override the resolved name for any language — whatever the table above produces is exactly what appears in block IDs, in ~/.spade/blocks/<collection>/, and in the registry. If you need a different collection name than your manifest's name field (or, for R, your directory name) produces, rename that directly rather than looking for a separate override setting.

This naming convention ensures block IDs are globally unique and immediately tell you which collection a block belongs to. When you reference a block in a pipeline, you use this full <collection>.<block> ID.

Versioning🔗

Collections use semantic versioning (e.g., 1.0.0, 0.3.2). The version is specified in each block's manifest file under the version field. All blocks in a collection should share the same version number, since they are built and installed together.

When you install a new version of a collection, it is stored alongside previous versions. This means multiple versions of the same collection can coexist, and pipelines can pin to a specific version if needed.

Installing collections🔗

The spade install command builds a collection from source — from a Git repository or a local directory. There is no separate mode for fetching a prebuilt artifact today; every install compiles or resolves dependencies locally.

# From a Git repository (shallow-cloned into a temp directory, then built)
spade install https://github.com/example/gdal-blocks.git

# From a local directory (built in place)
spade install .
spade install ./my-collection

For either source, Spade:

1. Clones the repo (or uses the local directory in place)
2. Detects the language from the project file and reads the collection's name/version
3. Builds the collection using the appropriate toolchain (cargo build --release for Rust, go build for Go, uv sync for Python, bun install + bun build --compile for TypeScript, pak/Rscript for R)
4. Installs the built artifacts and manifests to ~/.spade/blocks/<collection>/<version>/ and registers each block

See spade install for the full breakdown, including how each language's build step works.

The installed layout looks like:

~/.spade/blocks/
  raster/
    0.2.1/
      blocks/
        reproject.yaml
        resample.yaml
        mosaic.yaml
      <built artifacts>
    0.3.0/
      ...
  ml/
    1.0.0/
      blocks/
        classify.yaml
      <built artifacts>

Sharing collections🔗

There's no central registry to fetch collections from today — to share one, point spade install at wherever it lives, a Git repository or a local path, as above.

spade upload (see CLI reference) packages the current collection's blocks/, language manifest, and source into a <name>-<version>.tar.gz archive, intended for the cloud screening and build pipeline. The server-side upload endpoint isn't wired up yet, so today this only produces a local archive — it prints "Upload endpoint is not yet configured" and stops there.

Planned: a cloud registry that screens, builds, signs, and versions collections centrally — a version would move through states like submitted → screening → screened → building → available (and later deprecated / yanked / recalled), and spade install <collection>@<version> would fetch a signed, prebuilt artifact without needing a local toolchain. None of this is implemented yet; see spade upload for what actually exists today.

Creating a new collection🔗

To create a new collection:

mkdir my-blocks && cd my-blocks
spade init --language python

This scaffolds the project structure for the chosen language. Then add blocks with:

spade add my-block-name

This creates the manifest file at blocks/my-block-name.yaml and a starter source file in the appropriate location for the language.

Validate your collection at any time:

spade check

This verifies that all manifests are well-formed, entrypoints exist, and there are no conflicting block IDs.