eval_toolkit.loaders#

DataFrameLoader

Wraps a single in-memory dataframe + a split column.

DatasetLoader

Yields one or more EvalSlice keyed by split name.

HFDatasetsLoader

Load a HuggingFace datasets repo as {split: EvalSlice}.

OodManifestLoader

Wrap ood_dataset_from_manifest() as a DatasetLoader.

ParquetGlobLoader

Load + concat parquet files matched by a glob; hash each for the manifest.

SingleSliceLoader

Wraps a single pre-built EvalSlice as {"all": slice}.

ood_dataset_from_manifest

Load multiple OOD eval slates declaratively from a YAML manifest.