models.predict

Score a table with a fitted SpadeModel from stats, sae, ml or this collection. The table is read and the predictions written one batch at a time, so a table of any size scores in the memory of one batch (20 million rows in about 66 MB). Linear, glm and Fay-Herriot models score from the coefficients in the model file; random forests and gradient boosting score through their ONNX graph, which needs ONNX Runtime (ORT_DYLIB_PATH; the worker image provides it).

The table is checked against the model before any row is scored: missing columns (with near-match suggestions), text where a number is needed, and categories the model never saw in training all fail with the reason. A Fay-Herriot model gives its synthetic prediction here; sae.fay_herriot_mse gives EBLUP estimates and their MSE.

Kind: standard
Network: no

Inputs🔗

Name	Type	Description
`data`	file (Parquet or CSV)	The table to score, with every column the model was fitted on. The predictions are written in the same format.
`model`	file (SpadeModel)	The fitted model.
`id_columns`	string	Column selector for columns to carry into the predictions unchanged (e.g. `STATECD,COUNTYCD`), so each prediction can be joined back. Kept as text when reading CSV. Default empty.
`na_action`	string	A row with a missing value in a model column: `fail` (default, naming the row and column), `drop` it from the output, or `keep` it with a null prediction so the output lines up row for row with the input.
`interval`	number	Interval level, e.g. `0.95`, adding `se`, `lower` and `upper` columns; `0` (default) for none. Linear and glm models only.
`interval_kind`	string	`prediction` (default): the interval for a new observation (linear models). `confidence`: the interval for the mean response; a glm gives a confidence interval only, on the link scale mapped to the response scale.

Outputs🔗

Name	Type	Description
`predictions`	file (Parquet or CSV, as the input)	The id columns, then `prediction` on the response scale; with an interval, `se` (standard error of the fitted mean), `lower` and `upper`. A classifier writes `label` and one `prob_<class>` column per class instead.

Example🔗

- id: "@predict"
  name: models.predict
  inputs:
    - block: "@plots"
      output: file
      as: data
    - block: "@fit"
      output: model
      as: model
  args:
    id_columns: "plot_id"
    interval: 0.95