Every grade on this page is provable.
FilingsCheck asks 8 models for figures reported in SEC filings and verifies every claim against the primary source as filed. Each grade is sealed, and a commitment to it is entered in the witness-cosigned public log. The latest edition, FilingsCheck v1.3, the frontier edition (v1.3), ran through the production API, and every grade in it re-checks offline against the published keys.
FilingsCheck v1.3
- Published
- 2026-10-07
- Sealed grades
- 2530
- Models tested
- 8
- Model spend, all runs
- $11.081638
The engine’s own scorecard
Adversarial probes measure Artizar’s own accuracy, not the models under test. Every run injects known-answer probes (true controls plus deliberate falsehoods) and grades the engine on catching them.
| Adversarial probes, all runs | Figure |
|---|---|
| probes graded as expected, across all runs | 2032 / 2032 |
| false greens: planted falsehoods graded confirmed | 0 |
| false alarms: true controls graded contradicted | 0 |
By run
- claude-haiku-4-5254 / 254 (0 false greens, 0 false alarms)
- claude-sonnet-5254 / 254 (0 false greens, 0 false alarms)
- gpt-5.4-mini254 / 254 (0 false greens, 0 false alarms)
- gpt-5.4254 / 254 (0 false greens, 0 false alarms)
- claude-sonnet-5-5254 / 254 (0 false greens, 0 false alarms)
- claude-opus-5-5254 / 254 (0 false greens, 0 false alarms)
- claude-fable-5-1254 / 254 (0 false greens, 0 false alarms)
- gpt-5.5254 / 254 (0 false greens, 0 false alarms)
A false green is a planted falsehood graded confirmed. It is the one failure this system must never produce.
Eight models, one standard of proof
FilingsCheck asks each model for exactly the fact the engine verifies; the model may abstain. Supported means the engine confirmed the model’s figure against the filing. The headline is supported over asserted, so abstentions leave the denominator.
Runs are under one of two conditions. A series run repeats the request every earlier edition sent; a frontier run uses the vendor’s top reasoning effort and its largest output budget (four runs here). Compare headlines within a condition, not across them.
- condition
- series
- supported
- 8
- contradicted
- 54
- abstained
- 20
- headline
- 12.7% (8/63)
- cost
- $0.049641
- condition
- series
- supported
- 43
- contradicted
- 17
- abstained
- 23
- headline
- 71.7% (43/60)
- cost
- $0.070916
- condition
- series
- supported
- 23
- contradicted
- 33
- abstained
- 27
- headline
- 41.1% (23/56)
- cost
- $0.020652
- condition
- series
- supported
- 38
- contradicted
- 7
- abstained
- 38
- headline
- 84.4% (38/45)
- cost
- $0.065675
- condition
- frontier, max effort
- supported
- 66
- contradicted
- 4
- abstained
- 13
- headline
- 94.3% (66/70)
- cost
- $1.943888
- condition
- frontier, max effort
- supported
- 69
- contradicted
- 3
- abstained
- 11
- headline
- 95.8% (69/72)
- cost
- $0.963456
- condition
- frontier, max effort
- supported
- 67
- contradicted
- 4
- abstained
- 12
- headline
- 94.4% (67/71)
- cost
- $2.52339
- condition
- frontier, xhigh effort
- supported
- 51
- contradicted
- 11
- abstained
- 21
- headline
- 82.3% (51/62)
- cost
- $5.44402
How to read FilingsCheck
Models are asked for the fact itself.
Each task names a company, a metric, and a period, and asks for exactly the figure the engine verifies. The model answers or abstains; the engine checks the answer against the SEC filings as filed and seals the grade.
Supported over asserted.
Abstentions are honest and leave the denominator. The headline measures what a model chose to assert, not what it declined to guess.
A low headline is the engine working.
When a model asserts a wrong figure, the engine contradicts it and the record shows the catch. The scorecard above (zero false greens across eight runs) is what makes the matrix worth reading.
How this edition is verified
Each run ships as a sealed bundle. One command per bundle checks, offline, every grade’s full chain: the Ed25519 seal signature, the commitment recompute, the audit-log entry hash, the witnessed inclusion proof, report-to-verdict consistency, and the prompt-version label against the claims. Any failure is a named error.
python -m artizar verify-bundle bundles/anthropic-claude-haiku-4-5
python -m artizar verify-bundle bundles/anthropic-claude-sonnet-5
python -m artizar verify-bundle bundles/openai-gpt-5.4-mini
python -m artizar verify-bundle bundles/openai-gpt-5.4
python -m artizar verify-bundle bundles/anthropic-claude-sonnet-5-5
python -m artizar verify-bundle bundles/anthropic-claude-opus-5-5
python -m artizar verify-bundle bundles/anthropic-claude-fable-5-1
python -m artizar verify-bundle bundles/openai-gpt-5.5- key_provenance
- deployed (operator-published; https://artizar.ai/.well-known/artizar-keys.json; S74 ceremony 2026-07)
The verification keys are public at the address above. The sealed bundles themselves are shared with design partners, who can run the commands above on their own machines. Request API access.
What, exactly, was measured
Two editions are comparable only when these labels match. Every label is read from the sealed artifacts, and an edition whose runs disagree on any of them refuses to build.
The live node stamps the same versions this edition was sealed under.
- taskset
- taskset-v12
- prompt
- v2 · template sha256 fb0029a26d6e91aea980639ce2e4b3b8cb2f2ed7c663d22bcc6f4875a9dfa6a7
- scoring_policy
- policy-v7
- period_binding
- binding-v4
- metric_lexicon
- lexicon-v7
- context_notes
- notes-v6
- verdict_schema
- verdict-v4
- harness
- filingscheck/v2
- model_price_table
- prices-2026-10-06
- bundle_format
- fcbundle/v1
Edition history
Editions are append-only: a new one never replaces an old one, and the sealed bundles behind each never change. The probe scorecard is Artizar’s own accuracy and reads across editions; the model headlines do not, because each edition uses a different taskset.
| Edition | Published | Sealed grades | Probes as expected | False greens | Metrics | Taskset | Model spend |
|---|---|---|---|---|---|---|---|
| v1.3 | 2026-10-07 | 2530 | 2032 / 2032 | 0 | 17 | taskset-v12 | $11.081638 |
| v1.2 | 2026-09-23 | 962 | 776 / 776 | 0 | 13 | taskset-v10 | $0.18945025 |
| v1.1 | 2026-08-31 | 806 | 632 / 632 | 0 | 12 | taskset-v8 | $0.1637795 (a lower bound; see the note below the table) |
| v1.0.1 | 2026-07-28 | 485 | 372 / 372 | 0 | 5 | taskset-v6 | $0.08118625 |
* A lower bound. That edition did not yet count the tokens of model answers the harness asked the model to repair, so its true spend was somewhat higher. Later editions count them.
Benchmark your model against the record.
Design partners can run their own model through FilingsCheck and keep the sealed grades, bundles included.