Skip to content
  • FDA AI-DSF lifecycle guidance: DRAFT (Jan 2025) — not for implementation
  • PCCP guidance: final (Dec 2024, updated Aug 2025)
  • FDA Statistical Guidance on diagnostic tests: final (2007)
  • Guidance map v1

Quickstart

Three commands, in this order. Every one of them runs inside your environment and none of them opens a network connection; --offline makes that a hard rule rather than a default.

  • A CSV of one row per prediction with, at minimum, a ground-truth column and either a score column or a predicted-label column. The schema page lists every column the engine reads.
  • A criteria.yaml you have authored. There is no default file to copy: the positive class, score orientation, thresholds, intended-use prevalence and subgroup attributes are yours to declare, and the engine halts without them. The criteria.yaml page walks every block.
  • A bundled synthetic sample table is not shipped with engine 0.1.0.dev1. The engine’s own test suite generates a seeded synthetic cohort, and the site’s browser demo (a later build day) will carry pre-computed samples; until then the commands below use your own table, called test.csv here.
Terminal window
proofpack doctor --offline

One row per check — interpreter, numpy, yaml, jsonschema, scipy (optional extra), the docx extra, the five schema files, guidance_map_v1.csv, write access, licence, network (skipped with --offline), reference platform, engine version — each [ok ] or [FAIL]. Exit 0 means every essential check passed.

Terminal window
proofpack map --input test.csv --criteria criteria.yaml --out mapping.json

Reads the header row and writes mapping.json: original header → canonical role → confidence. Original headers stay in that file on your machine and never appear in an error message. Review the roles before you run. In engine 0.1.0.dev1 the mapper is header-only (exact name or synonym); the value-level summaries and the interactive accept/edit step are later engine days, and the schema and mapping page says exactly what the current one does.

Passing --criteria here lets the mapper apply gate H11 (a date-like column with no period declaration) to the header list before H07 and before the mapping is written. map reads the whole file into memory first — a ragged row is S04 before H11 — and types nothing.

Terminal window
proofpack run --input test.csv --criteria criteria.yaml --mapping mapping.json --out ./pack --offline

Loads and validates the declarations first (the H08 and H09 checks on the file itself fire here, before the table is opened), then reads, maps and types the table and runs the remaining gates in the order the HALT gates page lists. On success it writes ingest_report.json and mapping.json into --out and prints ingest ok: <included> rows included of <rows>, any [Wxx] finding, and the report path; the full flow counts (excluded for missing y_true, excluded for missing score, indeterminate) are in the flow object of ingest_report.json, not on the terminal. In engine 0.1.0.dev1 that is where run stops: the statistics and the templates land on later engine build days, and the command’s own last line says so. On a HALT nothing is written to --out at all.

--yes makes the run non-interactive: it is accepted only when a prior mapping.json exists with the same header-set hash and every mapped role at high confidence, otherwise the run halts with H07 (see non-interactive runs).

Code Meaning
0 ok
2 warnings only (findings such as W06 or W10 are in the report; the run completed)
3 HALT — a gate fired; no document was written
4 licence
5 internal error

These are the engine’s constants (errors.py) and the CLI’s main() maps every exception to one of them; a traceback is never the answer.

--offline (zero sockets), --quiet and --json-log (one JSON object per message, on stdout for results and stderr for a HALT) are accepted both before and after the subcommand: proofpack --offline run … and proofpack run … --offline are the same command.

The three things that usually stop a first run, in the order they arrive:

  1. Docker on Windows. Take the pip path; it is the primary install. See Install → Windows.
  2. A mapping you did not expect. A header the synonym table does not know is left unmapped at low confidence. Rename the column to its canonical name, or wait for the interactive mapper. See proofpack map.
  3. A HALT. The CLI prints HALT Hxx: <message>, a detail: line when the gate carries one (a count, a statistic such as the AUROC for H01, a hash prefix for H07, a schema path for H08 or a criterion id and field for H09 — never a row of your table) and No document was written. Look the code up on the HALT gates page; every H-, S- and W-code has its own section.

If you are stuck on any of them, support answers computational-correctness questions within one business day. Send the proofpack map output — roles and confidences only — never the table.