sae

The sae collection implements Fay-Herriot area-level small-area estimation: a model-based technique for producing more precise estimates for small geographic or demographic areas than their direct (design-based) survey estimates allow.

A direct estimate for a small area — e.g. from fiadb or data.census_acs — carries large sampling variance when few samples fall in that area. The Fay-Herriot model treats each area's direct estimate as its true value plus sampling error, and relates the true value to area-level auxiliary covariates through a linear regression with an area-level random effect. Fitting the model (Fisher scoring, with REML/ML/FH variance estimation) borrows strength across areas to shrink noisy direct estimates toward the regression line, producing an EBLUP (Empirical Best Linear Unbiased Predictor) that is typically more precise than the direct estimate alone.

Three blocks build on the same model class and chain through a fitted model, a SpadeModel file (see models):

1. sae.all_subsets_fh — predictor selection. Given a pool of candidate covariates (a column selector such as B*,MB*), enumerates every subset up to a size limit, fits each by Fisher-scoring EBLUP, and selects the best model by AIC among those whose coefficients are all significant. Its model output is the selected model, fitted.
2. sae.fay_herriot — model fit. Fits a Fay-Herriot model on a given formula or set of features and writes the fitted model, the coefficient table and goodness-of-fit diagnostics.
3. sae.fay_herriot_mse — estimates. Takes a fitted model from either block above and returns per-area EBLUP estimates with their Prasad-Rao mean squared error and precision diagnostics (SE ratio, relative efficiency versus the direct estimate). It does not refit: the model carries everything the MSE needs. Areas without a direct estimate get the synthetic (regression-only) prediction.

In practice: wire all_subsets_fh's model output straight into fay_herriot_mse -- no predictors to copy by hand. models.predict also scores a Fay-Herriot model, giving the synthetic prediction.

All three take Parquet or CSV and share the model-fitting arguments of the stats model blocks: target, a features column selector, id_columns, categorical, na_action. A missing or misspelled column fails with near-match suggestions and the list of columns that exist.

A fourth block, sae.hello_world, is a diagnostic/smoke-test block that verifies the R runtime executes correctly in a given environment — it is not a modeling tool, but a real, registered block that spade check/spade list will surface.

All blocks in this collection declare network: false — small-area estimation runs entirely on data already local to the pipeline.

Blocks🔗

Installation🔗

spade install file:///path/to/blocks/sae