platform.normalize_vrt

Rewrites a GDAL Virtual Raster so that every local reference names its source by its location in Spade's object storage — spade://outputs/<invocation>/<output>/<file> — instead of by path. The result is a few kilobytes that read their sources lazily wherever the runtime stages them, which is what makes a large raster surface cheap to broadcast into a map.

  • Kind: standard
  • Network: no

Inputs🔗

NameTypeDescription
vrtfile (VRT)A VRT, typically the bundle gdal.build_vrt or gdal.reduce_vrt publishes. Every file it references must have arrived alongside it as part of this input.

Outputs🔗

NameTypeDescription
vrtfile (VRT)The same raster, referencing its sources in object storage. Nested VRTs are rewritten and published beside it.

How it works🔗

The block cannot work out the object keys on its own — it sees inputs/vrt/sources/0000_a.tif and nothing else — so the runtime injects a references map from each staged input file to its key, through the same allow-listed capability path as the upload blocks' url (see the platform collection). It is deliberately not a declared input. A reference to a file the runtime did not stage fails the block rather than shipping a broken VRT; references that are already portable (/vsis3/, /vsicurl/) pass through untouched.

References are keys, never URLs, so the output is byte-stable across runs and the result cache stays honest for it and for everything downstream. What a consumer can open is decided when the runtime stages the file into it:

ConsumerReceives
a block with network: true, in the cloudsigned URLs — it reads windows straight from storage, one tile at a time
any other block, in the cloudthe sources fetched beside the VRT (cached per worker) and referenced relatively
any block, locallyabsolute paths into the pipeline directory

So a normalized VRT is readable by every block; network access is a performance choice for the consumer, not a requirement.

Example🔗

Build a DEM surface once and broadcast it into a map over many boundaries:

- id: "@dem_tiles"
  name: platform.upload_raster_collection
  inputs: []
  args:
    asset_id: 019cf4bc-dddd-7000-8000-000000000001
    object_prefix: data/019cf4bc-owner-.../019cf4bc-dddd-.../

- id: "@dem"
  name: gdal.build_vrt
  inputs: ["@dem_tiles"]
  args: { resolution: highest, separate: false }

- id: "@surface"
  name: platform.normalize_vrt
  inputs: ["@dem"]
  args: {}

- id: "@clip"
  name: gdal.clip_raster_by_vector
  inputs:
    - "@each_boundary"                    # mapped
    - { block: "@surface", output: vrt, as: source }   # broadcast: a few KB per invocation
  args: { crop_to_cutline: true, all_touched: false }

See examples/cloud/06-dem-broadcast.yaml for the whole pipeline.

Limits🔗

  • A reference VRT fed into a kind: reduce block as one member of the gathered collection is fine — it has no sidecar directory to rename. The bundle it was made from is not (see gdal.build_vrt), which is one more reason to normalize before any fan-in.
  • The block only rewrites references to files that arrived as its input. A VRT over files from somewhere else — a hand-written one, say — fails with the path it could not map.