Types
The Rust library defines types as structs that implement a set of traits. Each type provides metadata for manifest generation, handles loading from the inputs/ directory, and knows how to write itself to outputs/.
Traits🔗
SpadeType🔗
Provides metadata for manifest generation and output naming:
pub trait SpadeType {
fn type_name() -> &'static str;
fn default_output_name() -> &'static str;
fn manifest_entry() -> ManifestInfo;
}FromInput🔗
Constructs a typed value from raw filesystem input data:
pub trait FromInput: Sized {
fn from_single_file(path: String) -> Result<Self>;
fn from_multiple_files(paths: Vec<String>) -> Result<Self>;
fn from_directory(path: String) -> Result<Self>;
}IntoOutput🔗
Writes a typed value to an output subdirectory:
pub trait IntoOutput {
fn write_to(self: Box<Self>, output_dir: &Path) -> Result<()>;
fn default_output_name(&self) -> &'static str;
}Type reference🔗
Single-file types🔗
| Type | type_name() | Default output name | Format | Field |
|---|---|---|---|---|
File | "file" | file | -- | path: String |
RasterFile | "file" | raster | GeoTIFF | path: String |
VectorFile | "file" | vector | GeoJSON | path: String |
TabularFile | "file" | tabular | CSV | path: String |
JsonFile | "json" | json | -- | path: String |
Each has a constructor: File::new(path), RasterFile::new(path), etc. The new method accepts any type implementing Into<String>.
All single-file types are generated by the define_file_type! macro, which implements SpadeType, FromInput, IntoOutput, Clone, Debug, and PartialEq.
Directory type🔗
| Type | type_name() | Default output name | Field |
|---|---|---|---|
Directory | "directory" | directory | path: String |
Constructor: Directory::new(path). The Directory type is implemented manually (not via macro) because its IntoOutput behavior copies the directory recursively.
Collection types🔗
| Type | type_name() | Default output name | Format | Field |
|---|---|---|---|---|
FileCollection | "collection" | files | -- | paths: Vec<String> |
RasterFileCollection | "collection" | rasters | GeoTIFF | paths: Vec<String> |
VectorFileCollection | "collection" | vectors | GeoJSON | paths: Vec<String> |
TabularFileCollection | "collection" | tables | CSV | paths: Vec<String> |
Each has a constructor: FileCollection::new(paths), etc. Collection types are generated by the define_collection_type! macro.
Scalar types (manifest-only)🔗
These SpadeType implementations are for use with the ManifestBuilder. They are not used as inputs or outputs at runtime:
| Rust type | type_name() | Manifest type |
|---|---|---|
String | "string" | string |
f64 | "number" | number |
f32 | "number" | number |
i64 | "number" | number |
i32 | "number" | number |
bool | "boolean" | boolean |
Using args.input and args.param🔗
File inputs🔗
The Args::input method retrieves a typed file input:
let source: RasterFile = args.input("source")?;
// source.path contains the file path
let tiles: RasterFileCollection = args.input("tiles")?;
// tiles.paths contains a Vec<String> of file paths
let dir: Directory = args.input("data")?;
// dir.path contains the directory pathThe type is inferred from the variable binding. The library scans inputs/<name>/ and calls the appropriate FromInput method.
Scalar parameters🔗
The Args::param method retrieves a scalar parameter from params.yaml. The type must implement serde::de::DeserializeOwned:
let resolution: f64 = args.param("resolution")?;
let method: String = args.param("method")?;
let normalize: bool = args.param("normalize")?;
let count: i64 = args.param("count")?;Checking for optional inputs🔗
Use has_input and has_param to check before accessing optional arguments:
if args.has_input("mask") {
let mask: VectorFile = args.input("mask")?;
// ...
}
if args.has_param("buffer") {
let buffer: f64 = args.param("buffer")?;
// ...
}Constructing output values🔗
When returning from your handler, construct the appropriate type:
// Single file
Ok(RasterFile::new("result.tif"))
// Directory
Ok(Directory::new("output_dir"))
// Collection
Ok(RasterFileCollection::new(vec![
"tile_001.tif".into(),
"tile_002.tif".into(),
]))ManifestInfo🔗
Each type's manifest_entry() returns a ManifestInfo:
pub struct ManifestInfo {
pub type_name: &'static str, // "file", "directory", "collection", etc.
pub format: Option<&'static str>, // Some("GeoTIFF"), Some("GeoJSON"), etc.
pub item_type: Option<&'static str>, // Some("file") for collections
}The macros🔗
define_file_type!🔗
Generates a single-file type with all trait implementations:
define_file_type!(RasterFile, "file", "raster", Some("GeoTIFF"));Arguments: struct name, type name, default output name, format option.
define_collection_type!🔗
Generates a collection type with all trait implementations:
define_collection_type!(RasterFileCollection, "file", "rasters", Some("GeoTIFF"));Arguments: struct name, item type name, default output name, format option.
Error types🔗
The SpadeError enum covers all error cases:
| Variant | When it occurs |
|---|---|
InputNotFound { name } | args.input() called with a name not in inputs/ |
ParamNotFound { name } | args.param() called with a name not in params.yaml |
EmptyInputDir { name } | An input subdirectory exists but contains no files |
TypeMismatch { name, expected, found } | Input data cannot be converted to the requested type |
SecretNotFound { name } | get_secret() called with a name not declared in the pipeline's secrets: map, or resolution failed |
IoError(std::io::Error) | Filesystem errors |
YamlError(serde_yaml::Error) | YAML parsing errors |
JsonError(serde_json::Error) | JSON parsing errors |
HandlerError(Box<dyn Error>) | Errors from user handler functions |
Importing🔗
All public types are re-exported from the crate root:
use spade::{
File, RasterFile, VectorFile, TabularFile, JsonFile,
Directory,
FileCollection, RasterFileCollection, VectorFileCollection, TabularFileCollection,
SpadeType, FromInput, IntoOutput,
Args, Outputs,
run, build, ManifestBuilder,
SpadeError, Result,
};