How ProofPack runs inside your environment

ProofPack is a command-line engine you run where your test set already is — a CI job, a VM, a laptop. It reads one CSV and thecriteria.yaml you authored, refuses to compute if a gate fires, and otherwise writes an evidence pack to your disk. Nothing about the table is needed anywhere else, and the one call that can leave is listed field by field below.

What runs where

How ProofPack runs inside your environment: CSV and criteria.yaml in, DOCX, HTML and JSON out, on your diskProofPack runs inside your environment. Your CSV and criteria.yaml stay there; the engine maps, gates, computes and renders there; DOCX, HTML and JSON are written to your disk. Only aggregates ever leave, and only by the channels on the egress table below: at launch at most one outbound call, the telemetry POST, carrying the fields the engine’s egress schema lists — no column names, no row-level values, no metric values. --offline opens zero sockets.Your environmentCI job, VM or laptop — the engine runs heretest.csvone row per predictioncriteria.yamlauthored by youProofPack enginemapHALT gates H01–H12statistics with intervalsrendera HALT writes nothingno network neededyour diskDOCXHTMLJSONmapping.jsonmanifest (T8)--offline opens zero sockets. Without it, at most one call leaves:the telemetry POST — field list on the right, generated from the engine's egress schema.Global Phoenix Solutions Ltdnever receives your tabletelemetry (optional)schemalicence_idrun_idengine_versionplatformmanifest_sha256duration_shalt_coderow_count_buckettimestampskipped with --offlineNo column names. No row-level values.No metric values. No free text.
The engine, the table and the pack all stay inside the box on the left. The box on the right receives, at most, the telemetry fields it shows — and nothing at all with --offline.

The run, step by step

  1. Declare. You author criteria.yaml: the positive class, the score's orientation and type, the thresholds, the intended-use prevalence, the subgroups and — if you set any — the acceptance criteria with an author, a date and a justification. The engine infers none of these and ships no default for any of them. Every block and field →
  2. Map. proofpack map reads the header row and assigns each column a canonical role and a confidence. You review it. The original headers stay in mapping.jsonon your machine. Schema and mapping →
  3. Gate. Twelve HALT gates run before any statistic is computed: a declared orientation the data contradicts, a label outside the declared classes, a probability outside [0, 1], a predicted label the threshold does not reproduce, duplicate ids, a single-class set, a mapping that no longer matches, a missing declaration, a criterion that names something that does not exist, a date column with no period declared, an unpaired comparison. Eleven of them exit 3 and write nothing; the twelfth, the prevalence check, is a flag: the run continues, prints the finding and exits 2.The gates →
  4. Compute. Overall and subgroup performance with a confidence interval, the method and the sample size on every number; calibration; version comparison against your own margin. Where a number cannot be estimated the pack is specified to say so with a typed reason rather than switching method. Engine0.1.0.dev1 stops after step 3. run today computes the AUROC point estimate gate H01 tests (kept in memory, written to no file, on the terminal only in the HALT's detail when H01 halts) and the observed prevalence gate H10 compares with your declared value (written to ingest_report.json when the flag fires). The interval library exists understats/; gates.py imports one name from it, auroc_mann_whitney, and the stringsbootstrap, delong, wald_ci,logit_ci, auroc_number,subgroup and calibrat occur in neithercli.py nor gates.py (the site's tests/day5-repair2.test.mjs greps both files for those seven strings; changelog).
  5. Render. DOCX, HTML and JSON on your disk, in the shape of an FDA AI-enabled-device evidence pack and a PCCP performance-evaluation report. The FDA draft guidance on AI-enabled device software functions (January 2025), not for implementation, is the guidance the FDA templates are mapped to; every anchor names the document, its date and its draft or final status. The T8 manifest is specified to echo your criteria.yamlverbatim. No renderer and no manifest writer is in engine0.1.0.dev1; this step is a later engine build day.

Everything that may ever leave

This table's rows are generated at build time from the engine's own egress schema by the row loaderthe trust page also uses, where the schema is published verbatim with the engine commit it was vendored from. A build test parses this table and holds every row to the manifest, the schema and the trust page's rows.

ChannelWhenFields (exhaustive)Limits and suppressionExists today?
Demo — uploadnever—No endpoint exists that accepts uploads. To be verified by the visible network counter and by a CI network-capture test (Playwright HAR).Not built yet

The in-browser demo is a later lane-S day. Nothing on this site accepts an upload today because no upload route exists in the repository at all.

Demo — eventon run completion, unless opted out
  • source
  • rows_bucket
  • duration_ms
  • halted
No headers, no numbers, no file names. `source` is one of `sample` or `upload`.Not built yet

No analytics of any kind is wired up on this site yet — not even Cloudflare Web Analytics.

Runner — licence checknever (removed before launch)—There is no licence server. A licence file is an Ed25519-signed document that verifies offline against the public key published on the trust page (/trust#licence-key), so the runner never phones home to check it.Removed

D2 §5.1 listed a licence-check call. D1 §6 and §7 removed it: at launch the runner makes at most one outbound call, and it is not this one. The row is kept so the removal is on the record rather than silently absent.

Runner — telemetryrun end; skipped entirely with `--offline` or `egress.telemetry: false`
  • schema
  • licence_id
  • run_id
  • engine_version
  • platform
  • manifest_sha256
  • duration_s
  • halt_code
  • row_count_bucket
  • timestamp

Generated from $defs/telemetry in the engine's egress schema.

The manifest hash only — no column names, no counts below bucket level, no metric values, no row-level field, no free-text declaration field. `row_count_bucket` is one of `<1k`, `1k-10k`, `10k-100k`, `>100k`; `halt_code` is a typed code or null.Not built yet

The runner's telemetry module is a lane-E day that has not run yet. The field list beside this row is generated from the schema the engine will validate the payload against, not typed by hand.

Runner — narrative (optional, online, off by default)deferred past v1.0 — no code path exists—If it is ever built: column headers passing the whitelist regex, and a claims JSON of aggregates only, validated against the `proofpack-aggregates/1` branch of the egress schema, which the trust page publishes verbatim (/trust#egress). Cells with n<10, events<5 or non-events<5 suppressed before egress; sites and devices pseudonymised (Site A, Site B…) and mapped back locally; no free text, no dates, no row-level fields. The aggregate rules are already in the schema and already CI-tested, so the rule exists before the feature.Deferred

There is no LLM anywhere in the pack pipeline at launch. Nothing in ProofPack v1.0 sends anything to a language model.

Support — support@only when a person at your company emails us
  • whatever that person chooses to put in the email
We never ask for test-set data, model outputs or patient data in a support ticket. If a support answerer is ever built, it will be tested against a corpus of tickets containing pasted data to prove it makes no model call on them, before it is switched on — no such module or corpus exists today. If you send us data anyway it is in our mailbox, which is the one egress route on this table that your own people control rather than our code.Yes

Our company email is hosted on Google Workspace today, which is why Google is on the sub-processor list (/trust#sub-processors and /legal/sub-processors). The `support@` alias itself has not been created yet, so mail to us reaches the founder’s mailbox. The tested support answerer is a lane-A day that has not run yet.

Site — buying ProofPackwhen you start a checkout or ask for a quote
  • legal entity
  • buyer name and email
  • billing address
  • VAT/EIN
  • company registration number
  • purchase order number
  • plan and model count
  • the model name you type on the form
  • accounts-payable email, if you give one
  • which procurement documents you need (DPA countersignature, security questionnaire, W-8BEN-E, vendor form)
  • on a quote acceptance: the name and role of the person who accepts, their confirmation that they are authorised to bind the company, their IP address, their browser user-agent string, and the time of acceptance
Held by Stripe (payments, tax, invoicing) and in our own database; quote PDFs are stored in a private bucket and served through our own route. The IP address and user-agent are recorded only as the audit record for a quote acceptance, which the acceptance page itself states before you submit it. No test-set data, no model output and no patient data reaches any of it, because none of it is ever asked for.Yes

Built and tested against a Stripe mock. No live Stripe, database or email account exists yet, so nothing has actually been stored anywhere.

Any aggregate cell with n < 10, fewer than 5 events or fewer than 5 non-events is serialised as suppressed with null values. A customer may tighten those numbers and may not loosen them. Sites, devices and protocols are pseudonymised to “Site A”, “Device A”, “Level A” before egress, and the map back is written only to your own machine.

These are never serialised at all, in any channel:

  • original headers
  • row-level values
  • justification
  • description
  • source
  • raw dates

The full account — the security controls in place and the ones we do not hold, sub-processors, retention, the public licence-verification key — is on the trust page. The engine's reference is in the docs, starting withegress and --offline.

The browser demo

The live demo is scheduled for a later build day and is not linked until it exists. When it does, the same engine runs inside your browser tab under Pyodide: a file you upload is parsed and analysed there, there is no upload endpoint, and closing the tab discards the data. The trust page carries that statement verbatim.