Your First Pipeline

A pipeline is a series of processing steps connected together. Each step is a block — a self-contained unit of computation. In this guide, you will create a simple pipeline that fetches a raster and reprojects it to a new coordinate system.

What you'll build🔗

This pipeline uses two blocks, both from Spade's built-in collections:

1. data.read — Fetches a single object (here, a raster) from any supported backend
2. gdal.warp — Reprojects, resamples, or warps a raster using gdalwarp

Data flows from the first block's output into the second block's input.

Prerequisites🔗

Make sure you have:

• The Spade CLI installed (Installation guide)
• The data and gdal block collections installed from the Spade repository checkout:

spade install file:///path/to/blocks/data
spade install file:///path/to/blocks/gdal

(See the Block Catalog for every built-in collection and its blocks.)

Write the pipeline YAML🔗

Create a file called reproject-pipeline.yaml:

name: reproject-example
version: "1.0"
description: Fetch a raster and reproject it

blocks:
  - id: "@source"
    name: data.read
    inputs: []
    args:
      uri: "s3://example-bucket/scenes/2025-06-01.tif"
      format: "GeoTIFF"

  - id: "@reproject"
    name: gdal.warp
    inputs:
      - "@source"
    args:
      target_crs: "EPSG:4326"
      resolution: 0
      resampling: "bilinear"
      output_format: "GTiff"

Block IDs use @source and @reproject — short codes — rather than long UUID strings. Short codes are the recommended form for hand-authored pipelines: they are readable, diff-friendly, and easy to type correctly. The CLI resolves them to stable UUIDs automatically on the first spade check or spade run and stores the bindings in a sibling reproject-pipeline.lock.yaml file.

Let's walk through each field:

• name — A human-readable name for the pipeline
• version — The pipeline version (must be a quoted string)
• blocks — The list of processing steps

For each block:

• id — A short code (@<identifier>) uniquely identifying this block invocation within the pipeline. Short codes must start with a letter or underscore and contain only letters, digits, and underscores after the @.
• name — Which block to run (format: collection.block)
• inputs — Which earlier blocks provide data to this one. An empty list [] means this block has no dependencies (it is a source block that runs first).
• args — Parameters passed to the block at runtime

The second block lists "@source" in its inputs. This tells Spade that the second block depends on the first block's output. Spade automatically matches the file-typed output of data.read to the file-typed input expected by gdal.warp.

Validate the pipeline🔗

Before running, check that the pipeline is valid:

spade check reproject-pipeline.yaml

If everything is correct, you'll see:

Pipeline "reproject-example" is valid.

The first time you run spade check (or spade run), the CLI also creates a reproject-pipeline.lock.yaml file alongside your pipeline:

# reproject-pipeline.lock.yaml
pipeline: reproject-example
version: "1.0"
bindings:
  "@source":    019cf4bc-1111-7000-0000-000000000001
  "@reproject": 019cf4bc-2222-7000-0000-000000000002

This file stores the UUID assigned to each short code so that reruns use the same IDs — enabling Spade's result cache to work correctly.

If there's an issue — for example, a missing block or an invalid reference — spade check will describe the problem precisely.

Run the pipeline🔗

Execute the pipeline locally:

spade run reproject-pipeline.yaml

Spade will:

1. Resolve block dependencies
2. Execute data.read first (since it has no inputs)
3. Pass its output to gdal.warp
4. Execute gdal.warp
5. Report success

You should see output like:

Loaded pipeline: reproject-example (019cf4bc-0000-7000-0000-000000000000)
Executing pipeline with 2 block(s)...
  [1/2] data.read running...
  [1/2] data.read complete
  [2/2] gdal.warp running...
  [2/2] gdal.warp complete

Pipeline complete: 2 block(s) executed in 1.2s

There is also a --no-ui flag, but it currently has no effect on this output -- the line-by-line format above is the only progress display spade run implements today, regardless of whether you pass it.

Inspect the results🔗

Pipeline working directories are stored in ~/.spade/pipelines/. To keep the working directory after the pipeline finishes (it's normally cleaned up), use:

spade run --keep-work-dir reproject-pipeline.yaml

Inside the working directory, each block invocation has its own folder with inputs/, outputs/, and logs/ subdirectories.

Next steps🔗

Now that you've run a pipeline, learn how to create your own block:

• Your First Block (Python)
• Your First Block (R)
• Or go straight to the library documentation for your language.