Types

The Rust library defines types as structs that implement a set of traits. Each type provides metadata for manifest generation, handles loading from the inputs/ directory, and knows how to write itself to outputs/.

Traits🔗

SpadeType🔗

Provides metadata for manifest generation and output naming:

pub trait SpadeType {
    fn type_name() -> &'static str;
    fn default_output_name() -> &'static str;
    fn manifest_entry() -> ManifestInfo;
}

FromInput🔗

Constructs a typed value from raw filesystem input data:

pub trait FromInput: Sized {
    fn from_single_file(path: String) -> Result<Self>;
    fn from_multiple_files(paths: Vec<String>) -> Result<Self>;
    fn from_directory(path: String) -> Result<Self>;
}

IntoOutput🔗

Writes a typed value to an output subdirectory:

pub trait IntoOutput {
    fn write_to(self: Box<Self>, output_dir: &Path) -> Result<()>;
    fn default_output_name(&self) -> &'static str;
}

Type reference🔗

Single-file types🔗

Typetype_name()Default output nameFormatField
File"file"file--path: String
RasterFile"file"rasterGeoTIFFpath: String
VectorFile"file"vectorGeoJSONpath: String
TabularFile"file"tabularCSVpath: String
JsonFile"json"json--path: String

Each has a constructor: File::new(path), RasterFile::new(path), etc. The new method accepts any type implementing Into<String>.

All single-file types are generated by the define_file_type! macro, which implements SpadeType, FromInput, IntoOutput, Clone, Debug, and PartialEq.

Directory type🔗

Typetype_name()Default output nameField
Directory"directory"directorypath: String

Constructor: Directory::new(path). The Directory type is implemented manually (not via macro) because its IntoOutput behavior copies the directory recursively.

Collection types🔗

Typetype_name()Default output nameFormatField
FileCollection"collection"files--paths: Vec<String>
RasterFileCollection"collection"rastersGeoTIFFpaths: Vec<String>
VectorFileCollection"collection"vectorsGeoJSONpaths: Vec<String>
TabularFileCollection"collection"tablesCSVpaths: Vec<String>

Each has a constructor: FileCollection::new(paths), etc. Collection types are generated by the define_collection_type! macro.

Scalar types (manifest-only)🔗

These SpadeType implementations are for use with the ManifestBuilder. They are not used as inputs or outputs at runtime:

Rust typetype_name()Manifest type
String"string"string
f64"number"number
f32"number"number
i64"number"number
i32"number"number
bool"boolean"boolean

Using args.input and args.param🔗

File inputs🔗

The Args::input method retrieves a typed file input:

let source: RasterFile = args.input("source")?;
// source.path contains the file path

let tiles: RasterFileCollection = args.input("tiles")?;
// tiles.paths contains a Vec<String> of file paths

let dir: Directory = args.input("data")?;
// dir.path contains the directory path

The type is inferred from the variable binding. The library scans inputs/<name>/ and calls the appropriate FromInput method.

Scalar parameters🔗

The Args::param method retrieves a scalar parameter from params.yaml. The type must implement serde::de::DeserializeOwned:

let resolution: f64 = args.param("resolution")?;
let method: String = args.param("method")?;
let normalize: bool = args.param("normalize")?;
let count: i64 = args.param("count")?;

Checking for optional inputs🔗

Use has_input and has_param to check before accessing optional arguments:

if args.has_input("mask") {
    let mask: VectorFile = args.input("mask")?;
    // ...
}

if args.has_param("buffer") {
    let buffer: f64 = args.param("buffer")?;
    // ...
}

Constructing output values🔗

When returning from your handler, construct the appropriate type:

// Single file
Ok(RasterFile::new("result.tif"))

// Directory
Ok(Directory::new("output_dir"))

// Collection
Ok(RasterFileCollection::new(vec![
    "tile_001.tif".into(),
    "tile_002.tif".into(),
]))

ManifestInfo🔗

Each type's manifest_entry() returns a ManifestInfo:

pub struct ManifestInfo {
    pub type_name: &'static str,     // "file", "directory", "collection", etc.
    pub format: Option<&'static str>, // Some("GeoTIFF"), Some("GeoJSON"), etc.
    pub item_type: Option<&'static str>, // Some("file") for collections
}

The macros🔗

define_file_type!🔗

Generates a single-file type with all trait implementations:

define_file_type!(RasterFile, "file", "raster", Some("GeoTIFF"));

Arguments: struct name, type name, default output name, format option.

define_collection_type!🔗

Generates a collection type with all trait implementations:

define_collection_type!(RasterFileCollection, "file", "rasters", Some("GeoTIFF"));

Arguments: struct name, item type name, default output name, format option.

Error types🔗

The SpadeError enum covers all error cases:

VariantWhen it occurs
InputNotFound { name }args.input() called with a name not in inputs/
ParamNotFound { name }args.param() called with a name not in params.yaml
EmptyInputDir { name }An input subdirectory exists but contains no files
TypeMismatch { name, expected, found }Input data cannot be converted to the requested type
SecretNotFound { name }get_secret() called with a name not declared in the pipeline's secrets: map, or resolution failed
IoError(std::io::Error)Filesystem errors
YamlError(serde_yaml::Error)YAML parsing errors
JsonError(serde_json::Error)JSON parsing errors
HandlerError(Box<dyn Error>)Errors from user handler functions

Importing🔗

All public types are re-exported from the crate root:

use spade::{
    File, RasterFile, VectorFile, TabularFile, JsonFile,
    Directory,
    FileCollection, RasterFileCollection, VectorFileCollection, TabularFileCollection,
    SpadeType, FromInput, IntoOutput,
    Args, Outputs,
    run, build, ManifestBuilder,
    SpadeError, Result,
};