datasets-and-metrics

# AIF360 datasets and metrics router Use this sub-skill when the task is about AIF360's legacy dataset objects and metric classes: constructing `StructuredDataset`, `BinaryLabelDataset`, `StandardDataset`, or `RegressionDataset` instances; loading legacy raw-dataset wrappers; defining privileged/unprivileged protected groups; and computing dataset, classification, regression, sample-distortion, or MDSS classification metrics. ## Read first - [Data formats](references/data-formats.md): dataset object conventions, in-memory pandas construction, built-in dataset wrapper caveats, and group/label encoding rules. - [API reference](references/api-reference.md): constructor signatures, metric class selection, method groups, and optional OT metric caveats. - [Workflows](references/workflows.md): runnable patterns for synthetic datasets, prediction metric reports, raw wrappers, sample distortion, and regression metrics. - [Troubleshooting](references/troubleshooting.md): import/install warnings, optional dependencies, raw-data failures, group/schema errors, and workflow-specific metric failures. ## Fast routing 1. **In-memory legacy dataset**: read [data formats](references/data-formats.md#in-memory-binarylabeldataset-from-pandas) and build a numeric, NA-free pandas `DataFrame`; pass `label_names`, `protected_attribute_names`, and explicit `favorable_label`/`unfavorable_label` when labels are not `1.0`/`0.0`. 2. **Built-in wrappers**: read [built-in dataset wrappers](references/data-formats.md#built-in-legacy-dataset-wrappers) before calling `AdultDataset`, `GermanDataset`, `CompasDataset`, `BankDataset`, `MEPSDataset19/20/21`, or `LawSchoolGPADataset`; legacy wrappers are not safe smoke tests because public raw files may be absent or network-bound. 3. **Metric choice**: use `BinaryLabelDatasetMetric` for one true dataset, `ClassificationMetric` for true-vs-predicted `BinaryLabelDataset` pairs, `SampleDistortionMetric` for original-vs-distorted `StructuredDataset` pairs, `RegressionDatasetMetric` for ranked regression datasets, and `MDSSClassificationMetric` only when a classification metric object must score a known group. 4. **Group definitions**: create `privileged_groups` and `unprivileged_groups` as lists of dictionaries keyed by `dataset.protected_attribute_names`; the dictionary values must match the encoded numeric protected attributes. 5. **Smoke check**: run the bundled no-data script with `python scripts/metric_report_smoke.py --pretty` from this sub-skill directory, or pass its path to a Python interpreter from another working directory. ## Route away when appropriate - If the task asks for bias mitigation `fit`, `transform`, `predict`, postprocessing thresholds, or algorithm selection after metric diagnosis, route to [mitigation-algorithms](../mitigation-algorithms/SKILL.md). - If the task asks for the preferred pandas/scikit-learn interface, `aif360.sklearn.datasets.fetch_*`, protected attributes in pandas indexes, sklearn scorers, or sklearn pipelines, route to [sklearn-interface](../sklearn-interface/SKILL.md). - If the task asks for subgroup search, FACTS, bias scanning beyond `MDSSClassificationMetric.score_groups`, or metric text/JSON explainers, route to [detectors-and-explainers](../detectors-and-explainers/SKILL.md). ## Minimal operating checklist - Confirm `aif360` imports and note that base dataset/metric workflows are CPU-only. - Keep raw benchmark data and network access out of smoke tests; use synthetic `BinaryLabelDataset` examples first. - Check `dataset.protected_attribute_names`, `dataset.privileged_protected_attributes`, and `dataset.unprivileged_protected_attributes` before constructing metric group dictionaries. - For `ClassificationMetric`, make the predicted dataset by deep-copying the true dataset and changing only `labels` and optionally `scores`; otherwise equality validation will fail. - Mark optional-dependency metrics such as optimal transport as optional/unverified unless the required extra has been installed and tested in the current task environment.