Types

The Rust library defines types as structs that implement a set of traits. Each type provides metadata for manifest generation, handles loading from the inputs/ directory, and knows how to write itself to outputs/.

Traits🔗

SpadeType🔗

Provides metadata for manifest generation and output naming:

pub trait SpadeType {
    fn type_name() -> &'static str;
    fn default_output_name() -> &'static str;
    fn manifest_entry() -> ManifestInfo;
}

FromInput🔗

Constructs a typed value from raw filesystem input data:

pub trait FromInput: Sized {
    fn from_single_file(path: String) -> Result<Self>;
    fn from_multiple_files(paths: Vec<String>) -> Result<Self>;
    fn from_directory(path: String) -> Result<Self>;
}

IntoOutput🔗

Writes a typed value to an output subdirectory:

pub trait IntoOutput {
    fn write_to(self: Box<Self>, output_dir: &Path) -> Result<()>;
    fn default_output_name(&self) -> &'static str;
}

Type reference🔗

Single-file types🔗

Each has a constructor: File::new(path), RasterFile::new(path), etc. The new method accepts any type implementing Into<String>.

All single-file types are generated by the define_file_type! macro, which implements SpadeType, FromInput, IntoOutput, Clone, Debug, and PartialEq.

Directory type🔗

Constructor: Directory::new(path). The Directory type is implemented manually (not via macro) because its IntoOutput behavior copies the directory recursively.

Collection types🔗

Each has a constructor: FileCollection::new(paths), etc. Collection types are generated by the define_collection_type! macro.

Scalar types (manifest-only)🔗

These SpadeType implementations are for use with the ManifestBuilder. They are not used as inputs or outputs at runtime:

Using args.input and args.param🔗

File inputs🔗

The Args::input method retrieves a typed file input:

let source: RasterFile = args.input("source")?;
// source.path contains the file path

let tiles: RasterFileCollection = args.input("tiles")?;
// tiles.paths contains a Vec<String> of file paths

let dir: Directory = args.input("data")?;
// dir.path contains the directory path

The type is inferred from the variable binding. The library scans inputs/<name>/ and calls the appropriate FromInput method.

Scalar parameters🔗

The Args::param method retrieves a scalar parameter from params.yaml. The type must implement serde::de::DeserializeOwned:

let resolution: f64 = args.param("resolution")?;
let method: String = args.param("method")?;
let normalize: bool = args.param("normalize")?;
let count: i64 = args.param("count")?;

Checking for optional inputs🔗

Use has_input and has_param to check before accessing optional arguments:

if args.has_input("mask") {
    let mask: VectorFile = args.input("mask")?;
    // ...
}

if args.has_param("buffer") {
    let buffer: f64 = args.param("buffer")?;
    // ...
}

Constructing output values🔗

When returning from your handler, construct the appropriate type:

// Single file
Ok(RasterFile::new("result.tif"))

// Directory
Ok(Directory::new("output_dir"))

// Collection
Ok(RasterFileCollection::new(vec![
    "tile_001.tif".into(),
    "tile_002.tif".into(),
]))

ManifestInfo🔗

Each type's manifest_entry() returns a ManifestInfo:

pub struct ManifestInfo {
    pub type_name: &'static str,     // "file", "directory", "collection", etc.
    pub format: Option<&'static str>, // Some("GeoTIFF"), Some("GeoJSON"), etc.
    pub item_type: Option<&'static str>, // Some("file") for collections
}

The macros🔗

define_file_type!🔗

Generates a single-file type with all trait implementations:

define_file_type!(RasterFile, "file", "raster", Some("GeoTIFF"));

Arguments: struct name, type name, default output name, format option.

define_collection_type!🔗

Generates a collection type with all trait implementations:

define_collection_type!(RasterFileCollection, "file", "rasters", Some("GeoTIFF"));

Arguments: struct name, item type name, default output name, format option.

Error types🔗

The SpadeError enum covers all error cases:

Importing🔗

All public types are re-exported from the crate root:

use spadelib::{
    File, RasterFile, VectorFile, TabularFile, JsonFile,
    Directory,
    FileCollection, RasterFileCollection, VectorFileCollection, TabularFileCollection,
    SpadeType, FromInput, IntoOutput,
    Args, Outputs,
    run, build, ManifestBuilder,
    SpadeError, Result,
};

Tables, columns and models🔗

Since spadelib 0.4 (ModelFile and the Envelope shape: 0.5).

A column selector is the string a user writes to name columns of a table input: comma-separated names, globs (*, ?, [...], matched against whole column names), backtick-quoted literals for names containing ,, * or a leading -, and - exclusions. B*,MB*,-B0 is every B and MB column except B0; if the first item is an exclusion, the selection starts from every eligible column. Selection is strict: a name that is not a column, a glob that matches nothing, or an exclusion that removes nothing is an error, and the message suggests near matches and lists the columns that do exist -- so a user who did not know the upstream schema learns it from the first failed run. Columns passed as reserved (the target, a weights column) are skipped by globs and refused by name, with the role in the message.

use spadelib::{select_columns, read_model};

let features = select_columns(&names, "B*,MB*,-B0", &[("ESTIMATE", "the target")], Some("data"))?;
let model = read_model(&model_file.path, "model")?;   // Model { envelope, payload }

read_table is behind the tables cargo feature (spadelib = { version = "0.5", features = ["tables"] }), since polars is a large dependency most blocks never need. There is no write_table; use polars' writers.

ModelFile (type: file, format: SpadeModel, default output name model) is the type of a fitted model; see the type reference above.

The selector grammar, the error messages and the model archive are specified in spec/models.md (§4 and §6); a shared fixture keeps the Python, R and Rust implementations identical.