platform

The platform collection holds first-party, deliberately-trusted blocks — blocks the platform grants capabilities that would be unsafe to hand an arbitrary third-party plugin. Its first members are the upload blocks, which materialize a file you uploaded (through the web UI or spade data upload) as a normal pipeline input.

Trust is granted by provenance: these blocks live in the Spade repository and are authored by the Spade team. That is what earns them their one elevated capability — receiving a short-lived, single-object pre-signed URL for your uploaded asset, minted by the worker at dispatch. The capability is gated by a worker-held allow-list of block names, never by anything a manifest declares, so a third-party block cannot obtain it by imitation.

• Language: Go
• Network: yes in the cloud (the block fetches the asset over HTTPS); not needed locally once the asset is cached
• Runs: both in the cloud and locally — see Running locally below.

Upload blocks🔗

There is one variant per Spade type. All four share a single implementation and differ only in their declared output type; the web UI presents them as a single Upload node with a type dropdown, so you never choose a variant by hand.

The declared type is an assertion about the file you provided — the block passes the bytes through unchanged and never converts formats. The type is chosen at upload time and defaults from the file extension (overridable).

How it works🔗

An upload block has no block inputs. The pipeline stores a stable reference to the asset — its object_key (and asset_id for provenance) — never a URL:

- id: 019cf4bc-1111-7000-0000-000000000000
  name: platform.upload_raster
  inputs: []
  args:
    asset_id: 019cf4bc-aaaa-7000-0000-000000000000
    object_key: data/019cf4bc-owner-.../.../boundary.tif

- id: 019cf4bc-2222-7000-0000-000000000000
  name: raster.reproject
  inputs:
    - 019cf4bc-1111-7000-0000-000000000000   # consumes the upload's output
  args:
    target_crs: "EPSG:4326"

At dispatch the worker recognizes the allow-listed block, confirms the object_key belongs to the pipeline owner, mints a short-TTL pre-signed GET URL, and writes it into the block's url parameter. The block GETs the URL and writes the file to its output. The URL is a bearer capability that expires quickly and is never logged; a redelivered job simply mints a fresh one.

Downstream blocks consume the upload's output exactly like any other block output, and type matching resolves through the variant's static output declaration.

Folder uploads🔗

You can also upload a whole folder (spade data upload <folder>) and read it in a pipeline as either a flattened collection of files or a structure-preserving directory. The stored asset is the same either way — one object per file — and the choice is made by which block you reference, at pipeline-authoring time:

A collection is ideal for map/fan-out over independent files; a directory is for folders whose internal structure matters (shapefile sets, .gdb, sidecar files). The pipeline carries the folder's object_prefix (instead of a single file's object_key); at dispatch the worker LISTs the prefix, presigns each member, and injects an ordered urls list the block materializes.

- id: 019cf4bc-1111-7000-0000-000000000000
  name: platform.upload_raster_collection
  inputs: []
  args:
    asset_id: 019cf4bc-ffff-7000-0000-000000000000
    object_prefix: data/019cf4bc-owner-.../019cf4bc-ffff-.../

Reference VRTs🔗

The collection's one member that is not an upload block:

It receives a different capability through the same allow-list path: after its inputs are staged, the worker injects a references map from each staged file to its object key, and the block rewrites the VRT to name its sources by those keys. The runtime resolves them again when the VRT is staged into a consumer — signed URLs for a block with network access, the sources fetched beside it for any other. Locally, spade run derives the map from the staged files and resolves references to paths on disk, so the pipeline is the same in both places.

Running locally🔗

Upload blocks are not cloud-only. There is no object storage on your machine, so instead of a pre-signed URL spade run resolves the asset to a file on disk and hands the block a file:// URL — the block fetches both schemes through one code path, so nothing about the pipeline changes.

Resolution tries three things, most specific first:

1. --asset <asset-id>=<path> — point the block at a local copy explicitly. Repeatable, one per asset.
2. The local asset cache — a copy under ~/.spade/assets/<asset-id>/ from an earlier run or download.
3. A fetch from your catalog, which then fills the cache. If the catalog knows the asset's content hash, the download is verified against it and a mismatch is discarded rather than kept.

The first two need neither a network nor a login, so once an asset has been fetched even once, that pipeline runs offline. spade check reports which assets are cached and which a run would fetch, without downloading anything.

spade run pipeline.yaml
spade run pipeline.yaml --asset 019cf4bc-aaaa-7000-0000-000000000000=./boundary.tif

A folder block resolves the same way: the folder is materialized under ~/.spade/assets/<asset-id>/ and its members are injected in the same sorted-by-path order the worker produces from a sorted object listing, so a local run and a cloud run see the collection identically.

Coming from a local pipeline🔗

If you authored a pipeline with data.read pointed at a file on your machine, spade submit converts it: it matches each local file against your catalog by content hash, uploads what is missing, rewrites the block into the matching platform.upload_* block, and runs it in the cloud. Because upload blocks also run locally, the converted pipeline still works on your machine — there is no need to swap anything back.

Creating an asset🔗

Upload a file and get an asset you can reference:

• Web UI — drag an Upload node onto the canvas, upload a new file or pick an existing one, and connect it downstream. See User uploads.
• CLI — spade data upload boundary.tif. See spade data.