eval_toolkit.metric_specs#

First-party MetricSpec instances consumed by scorecard(). The namespace ships threshold-free specs only (Decision R). Each unparameterized metric is a module-level singleton; parameterized metrics (currently just the ECE family) are factory callables with LRU-cached identity.

brier

Brier-score singleton spec (mean squared error of probabilistic predictions).

ece

Factory for an ECE MetricSpec with configurable bins + strategy.

make_spec_name

Canonicalize a parameterized MetricSpec name.

pr_auc

PR-AUC singleton spec (area under the precision-recall curve).

roc_auc

ROC-AUC singleton spec (area under the receiver-operating-characteristic curve).