Input References

When a block needs data from an upstream block, you list the upstream block's invocation ID in the inputs field. Spade supports two reference styles: bare references and explicit references. You can mix both styles in the same inputs list.

Bare references🔗

A bare reference is the simplest form. You provide only the invocation ID of the upstream block:

inputs:
  - "@upstream"

When Spade encounters a bare reference, it uses type matching to determine which output from the upstream block connects to which input on the current block. This works well when the connection is unambiguous -- for example, when the upstream block produces a single file output and the current block expects a single file input of a compatible type. If more than one input could take that output, Spade reports a real ambiguity error for bare references (see Walkthrough: ambiguous resolution below) -- this is the one case in the whole algorithm where ambiguity is actually detected and rejected.

Example: unambiguous bare reference🔗

Consider two real blocks:

• base.csv_to_parquet produces one output: result (type: file, format: Parquet)
• base.filter_rows expects one file input: table (type: file, format: Parquet) plus a scalar expression argument

The pipeline:

blocks:
  - id: "@raw"
    name: data.read
    inputs: []
    args:
      uri: "s3://example-bucket/raw.csv"
      format: "CSV"

  - id: "@source"
    name: base.csv_to_parquet
    inputs:
      - "@raw"
    args:
      delimiter: ","
      has_header: true

  - id: "@filtered"
    name: base.filter_rows
    inputs:
      - "@source"
    args:
      expression: "state = 'ME'"

Spade sees that base.csv_to_parquet produces a Parquet file output and base.filter_rows expects a Parquet file input. There is exactly one way to match them, so the bare reference is sufficient.

Explicit references🔗

An explicit reference is an object with up to three keys:

inputs:
  - block: "@classify"
    output: classified_raster

Explicit references are necessary when a bare reference would be ambiguous. This happens when the upstream block produces multiple outputs of the same type, or when there are multiple upstream blocks whose outputs could match the same input -- bare references genuinely reject both of these cases with an error.

as matters even more in a further, more specific case: when two or more explicit references in the same inputs list target outputs of the same type, and the current block has two or more unmatched inputs of that same type.

Example: explicit reference needed🔗

Consider base.join, a real block that merges two tables on shared key columns. Its manifest declares two file inputs of the same type:

• left (type: file, format: Parquet)
• right (type: file, format: Parquet)

Suppose two independent upstream blocks each produce a table:

blocks:
  - id: "@estimates"
    name: data.read
    inputs: []
    args:
      uri: "s3://example-bucket/area-estimates.parquet"
      format: "Parquet"

  - id: "@covariates"
    name: data.read
    inputs: []
    args:
      uri: "s3://example-bucket/county-covariates.parquet"
      format: "Parquet"

Both produce a single output named file, of type file. If you tried bare references here:

  - id: "@joined"
    name: base.join
    inputs:
      - "@estimates"
      - "@covariates"
    args:
      on: county_fips
      how: inner

spade check rejects this with a real ambiguity error -- as soon as Spade processes the first bare reference (@estimates), it finds that its file output is compatible with both still-unmatched inputs (left and right), and bare references do check for exactly this:

Pipeline validation failed with 1 error(s):
  - block <joined-id> (base.join): ambiguous type match: output <estimates-id>.file (type "file") matches multiple inputs: [left right]

Naming the output with block+output alone does not fix this -- both left and right are file, so whichever reference comes first in the list simply claims left and the other claims right, without any error:

  # THIS VALIDATES AND RUNS, BUT MAY SILENTLY SWAP left/right
  - id: "@joined"
    name: base.join
    inputs:
      - block: "@covariates"
        output: file
      - block: "@estimates"
        output: file
    args:
      on: county_fips
      how: inner

Here @covariates happens to land in left and @estimates in right, purely because of the order the two references are written in -- not because that's what either output or input name suggests. Swap the order of these two lines and the wiring silently swaps too, with no warning from spade check.

You need as to pin each one directly:

  - id: "@joined"
    name: base.join
    inputs:
      - block: "@estimates"
        output: file
        as: left
      - block: "@covariates"
        output: file
        as: right
    args:
      on: county_fips
      how: inner

Here, as wires the @estimates output directly to the left input and the @covariates output directly to the right input, regardless of the order the references are written in -- no ambiguity, silent or otherwise.

Mixed references🔗

You can combine bare and explicit references in the same inputs list. Spade resolves all explicit references first, then resolves the remaining bare references against whatever inputs are still unmatched. This ordering is exactly what makes the following pattern work:

  - id: "@joined"
    name: base.join
    inputs:
      # Explicit reference: pin @estimates to `left` directly.
      - block: "@estimates"
        output: file
        as: left

      # Bare reference: by the time Spade reaches this, `left` is
      # already matched, so `right` is the only remaining
      # type-compatible input -- no ambiguity, even without `as`.
      - "@covariates"
    args:
      on: county_fips
      how: inner

This is useful when one connection needs to be pinned explicitly (because its type is shared with another input) while another connection is left to resolve on its own once the explicit reference has narrowed the field.

Type-matching algorithm🔗

When Spade processes a block's inputs list, it runs the following algorithm to wire upstream outputs to the current block's declared inputs:

Step 1: Gather declared inputs. Read the current block's manifest to find all declared inputs and their types. Separate file-type inputs (which come from upstream blocks) from scalar inputs (which come from args).

Step 2: Resolve explicit references. For each explicit reference in the inputs list, in order:

1. Look up the upstream block invocation by its block ID, and the named output in that upstream block's manifest.
2. If the reference has an as key, check that the named input exists and that its type is compatible with the output's type; if so wire them directly and mark both as resolved. If the type is incompatible, report an error.
3. Otherwise (no as), scan the current block's declared inputs in alphabetical order by name and take the first one that is both unmatched so far and type-compatible with the output. Wire them together immediately.
4. If no input in the entire scan is type-compatible, report a "no matching input" error.
5. If more than one input would have matched, Spade does not notice. It has already committed to the first (alphabetically) compatible input by the time a second candidate would be considered. No ambiguity error is possible on this path.

Step 3: Resolve bare references. For each bare reference in the inputs list, and for each of that upstream block's not-yet-claimed outputs:

1. Collect every not-yet-matched input on the current block that is type-compatible with this output.
2. If there is exactly one such input, wire them together and mark both as resolved.
3. If there is more than one, report a real ambiguity error and suggest using explicit references.
4. If there are none, that output is simply left unconnected (it is not an error by itself -- see Step 4).

Step 4: Verify completeness. After resolving all references, check that every non-scalar (file/collection/json/etc.) input on the current block has been wired to an upstream output. If any such input remains unwired, report a missing-input error. Scalar inputs (string, number, boolean) are not checked here -- they come from args and are checked separately (see Pipeline Validation).

Walkthrough: unambiguous resolution🔗

Suppose Block A produces:

• result (type: file, format: Parquet)

And Block B expects:

• table (type: file, format: Parquet)
• expression (type: string, from args)

Block B's pipeline entry:

- id: "@filtered"
  name: base.filter_rows
  inputs:
    - "@source"
  args:
    expression: "state = 'ME'"

Resolution proceeds as:

1. Declared file-type inputs: table (Parquet). Scalar inputs: expression (from args).
2. No explicit references.
3. Bare reference to Block A (@source). Block A has one unclaimed output: result (Parquet). Exactly one unmatched input on Block B is type-compatible: table. Wire result to table.
4. All required inputs are satisfied.

Walkthrough: ambiguous resolution🔗

Suppose @estimates and @covariates each produce one output:

• file (type: file, format: Parquet)

And base.join expects two file-type inputs of the same type:

• left (type: file, format: Parquet)
• right (type: file, format: Parquet)

Pipeline entry using bare references:

- id: "@joined"
  name: base.join
  inputs:
    - "@estimates"
    - "@covariates"
  args:
    on: county_fips
    how: inner

Resolution proceeds as:

1. Declared file-type inputs: left (Parquet), right (Parquet).
2. No explicit references.
3. Bare reference to @estimates. Its one unclaimed output, file (Parquet), is compatible with two still-unmatched inputs: left and right.
4. Ambiguity error, reported immediately -- Spade never even reaches the second bare reference to @covariates.

Pipeline validation failed with 1 error(s):
  - block <joined-id> (base.join): ambiguous type match: output <estimates-id>.file (type "file") matches multiple inputs: [left right]

The fix is to use explicit references with as, as shown in Example: explicit reference needed above.

When to use each style🔗

As a general rule: start with bare references for simplicity. If spade check reports an ambiguity error, switch to explicit references and always add as for the affected connections -- block+output alone only tells Spade which output to use, not which input to put it in, and (as covered above) Spade will not warn you if it guesses wrong.