ml.select_best

Close a hyperparameter grid: rank the gathered models on a metric from their fit statistics and output the best one unchanged, with a ranking of every candidate and the map item values that produced it. Use it as the reduce after a fit block mapped over base.map_grid. With direction: auto, metrics containing accuracy or r_squared are maximised and all others (rmse, mae, log_loss, aic, ...) minimised; ties keep map order. An unknown metric fails, listing the metrics the models have.

  • Kind: reduce
  • Network: no

Inputs🔗

NameTypeDescription
modelscollection (SpadeModel)The models a mapped fit block produced, one per grid point.
metricstringMetric to rank on, from the models' fit metrics, e.g. cv_rmse, cv_r_squared, cv_accuracy, cv_log_loss.
directionstringauto (default) maximises accuracy and r_squared metrics and minimises the rest; or min, max.

Outputs🔗

NameTypeDescription
modelfile (SpadeModel)The winning model, byte for byte.
rankingfile (CSV)Every candidate, best first: rank, ordinal, key, the map item's values, and the metric.

Example🔗

- id: "@best"
  name: ml.select_best
  inputs:
    - block: "@fit"
      output: model
      as: models
  args:
    metric: "cv_rmse"