Skip to content
  • FDA AI-DSF lifecycle guidance: DRAFT (Jan 2025) — not for implementation
  • PCCP guidance: final (Dec 2024, updated Aug 2025)
  • FDA Statistical Guidance on diagnostic tests: final (2007)
  • Guidance map v1

Input schema and mapping

ProofPack reads one CSV — UTF-8, a header row, a design target of 100,000 rows. The column names below are the canonical roles the engine sees after mapping. Your original headers are recorded in mapping.json on your machine and never leave it: no error message, no report and no egress payload carries one. The source of truth is the engine’s schema/schema_v1.json; this page is its prose.

Column Type Required What it is and which gate watches it
y_true string or integer required Ground truth. Values must be a subset of the declared classes.positive, classes.negative and indeterminates.values, else H02.
score number or null required unless y_pred is present The continuous output. Orientation and type are declared in criteria.yaml, never inferred. With score.type: probability, a value outside [0, 1] is H03.
y_pred string, integer or null optional A predicted label. If present it must equal the declared rule applied to score at every operating point on every row, else H04.
row_id string, non-empty required for a paired compare Unique per row; the join key between version N and N−1. A duplicate is H05.
case_id string, non-empty required if clustering.unit: case_id The cluster unit (patient, study). More rows than cases puts the run on the clustered path.
indeterminate 0, 1 or null optional The alternative to listing indeterminate values in y_true / y_pred.
age number or null optional Banded by the bands you declare for it; no default bands exist — a numeric age column with none declared is H08.
age_band string or null optional Used as-is when age is absent.
sex, race, ethnicity, site, device, protocol, severity string or null optional The attribute columns. Levels are your strings; a missing value is tabulated as an “Unknown/missing” level, never dropped. site, device and protocol levels are pseudonymised in any egress payload.
attr_<name> string, number or null optional Your extra attributes (^attr_[a-z0-9_]{1,26}$). A numeric extra needs declared bands.
event_date ISO date string or null optional Coarsened at ingest to the declared period.granularity; the raw date never appears in any output. Present with no period declaration → H11.
period string or null optional An already-coarsened period label, for example 2026-Q1.
model_version string or null optional Long format: rows for two versions in one file.
dataset dev, test or null optional dev rows carry attributes only and feed the Table 1 similarity check.
rater_<k> any optional Echoed in Table 1 only (“rater columns present: k — not analysed in v1”); never used to construct y_true.
anything else Ignored and listed in the mapping report as unused.

Identifier names — canonical columns and attr_* extras — match ^[a-z][a-z0-9_]{0,31}$; that pattern is also the header whitelist for the one (deferred) egress channel that could ever carry a column name.

These tokens are read as missing, in every column:

"" (empty), NA, N/A, na, n/a, NaN, nan, null, NULL, None, unknown, Unknown, UNKNOWN, missing, Missing.

A row missing y_true, or missing score where a score column exists, is excluded from analysis and counted in the flow table (included / excluded-missing / indeterminate) that every pack opens with; dev rows are never analysed. A missing attribute value is not an exclusion: the row is tabulated under “Unknown/missing”.

  • One binary problem per run. A multi-finding device is one CSV (or one score/y_true pair) per finding, run separately.
  • Dates are identifiers for privacy purposes. Declare a period and let the engine coarsen them.
  • Long format (model_version) or --prior are the two ways to compare versions; see version comparison.
Terminal window
proofpack map --input test.csv --criteria criteria.yaml --out mapping.json

The mapper reads the header row and assigns each header a canonical role and a confidence:

Header Role Confidence
exactly a canonical name (y_true, score, site, …; case-insensitive) that column high
attr_* or rater_* itself high
a synonym from the table below the canonical column medium
anything else unmapped (unused) low

Synonyms, lower-cased: label, truth, gt, ground_truth, outcome, target, reference, yy_true · prob, probability, p1, pred_prob, confidence, output, riskscore · prediction, predicted, pred_label, classy_pred · gendersex · centre, center, hospital, institution, facilitysite · scanner, manufacturer, vendor, model_namedevice · patient_id, subject_id, study_id, mrn_hashcase_id.

Two headers mapping to the same role is H07.

mapping.json records the order-independent SHA-256 of the header set, the roles, the confidences, who decided (stub today, interactive or file) and a timestamp. The hash goes into the run manifest, so a pack can be tied to the exact header set it was rendered from without the headers themselves leaving your machine.

Engine 0.1.0.dev1 is a header-only mapper. The value-level summaries (unique counts, top values, min/max, missing %, suppressed before display), the type-consistency check and the interactive accept/edit step are later engine days. Until they land, a header the synonym table does not know is left at low confidence, and the way to map it is to rename the column.

--yes on map, run or compare declares the run non-interactive (CI mode). It is accepted only when both hold:

  1. a prior mapping.json exists (the --mapping path, or --out for map) whose header-set hash equals the current table’s; and
  2. every mapped role in that file is at high confidence.

Otherwise the run is H07 with a message naming which condition failed — no prior file, a hash that differs, or roles below high. Without --yes, a matching prior mapping is reused as it stands and a non-matching one is replaced by a fresh header-only mapping.

It never infers a declaration. The positive class, the score’s orientation and type, the thresholds, the reference-standard type, the indeterminate values and the clustering unit come from criteria.yaml only, and are echoed in the T8 manifest of every pack.