User Uploads

Pipelines often need your own data — a custom raster, a boundary vector, a table of plots — not just public datasets. Spade brings these in through uploads: you upload a file once, and pipelines reference it like any other data, share it without re-uploading, and consume it through an ordinary block.

The model🔗

An upload is a first-class, persistent asset, not a per-pipeline attachment:

• Bytes live in Spade's object storage.
• Metadata (name, size, type, owner, storage key) lives in a catalog.

A pipeline references an asset by a stable object key, never by a URL. The key is owner-namespaced, which is also the authorization anchor: a pipeline may read an asset only when the asset belongs to the pipeline's owner.

Uploads are pulled into a pipeline by a first-party upload block rather than a special pipeline primitive — so the "everything is a node" model stays uniform. See the cloud collection for the blocks themselves.

Uploading a file🔗

Web UI. Drag an Upload node onto the canvas. It lets you upload a new file or pick an existing asset from your catalog, and shows a type dropdown defaulted from the filename (raster / vector / table / file) that you can override. Connect the node's output to the block that should consume it. The node serializes to the matching cloud.upload_<type> block with the asset's id and object key — the variant is hidden from you.

CLI. spade data upload boundary.tif uploads the file and prints an asset id. See spade data.

Because assets are addressed by a stable key, uploading the same file once and referencing it from several pipelines reuses the same object — no re-transfer, no duplication.

Access at run time (pre-signed URLs)🔗

No standing storage credential ever lives inside a block. When a cloud run reaches an upload block, the worker mints a short-lived, single-object pre-signed GET URL for the asset and injects it into the block, which fetches the bytes and writes them to its output. The URL expires quickly and is never logged; a redelivered job mints a fresh one. This mirrors the local/cloud split used for secrets: a pipeline references data by a stable id, and where that id resolves depends on where the pipeline runs.

Running locally🔗

Upload blocks run in the cloud only — a local spade run has no object storage or URL minting. To run an equivalent pipeline on your machine, replace the upload block with data.read pointed at the file on disk:

# cloud:
- id: 019cf4bc-1111-7000-0000-000000000000
  name: cloud.upload_raster
  args:
    asset_id: 019cf4bc-aaaa-7000-0000-000000000000
    object_key: data/<owner>/<file>/boundary.tif

# local equivalent:
- id: 019cf4bc-1111-7000-0000-000000000000
  name: data.read
  args:
    uri: "file:///abs/path/boundary.tif"
    format: "GeoTIFF"

Both produce a single file output, so the rest of the pipeline is unchanged.

Out of scope🔗

Cross-user sharing beyond the owner-read model, retention/cleanup of unreferenced assets, and format conversion on ingest are not covered here — the upload block passes bytes through, and the declared type is your assertion about the file.