How AI models perform when asked for figures reported in SEC filings, with every claim verified against the primary source as filed, and every grade sealed, provable, and reproducible. Nothing on this page asks for trust.
Artizar's own accuracy is measured by adversarial probes, not by the models under test: every run injects known-answer probes (true controls plus deliberate falsehoods) and the engine is graded on catching them.
| Run | Probes as expected | False greens | False alarms |
|---|---|---|---|
| claude-haiku-4-5 | 254 / 254 | 0 | 0 |
| claude-sonnet-5 | 254 / 254 | 0 | 0 |
| gpt-5.4-mini | 254 / 254 | 0 | 0 |
| gpt-5.4 | 254 / 254 | 0 | 0 |
| claude-sonnet-5-5 | 254 / 254 | 0 | 0 |
| claude-opus-5-5 | 254 / 254 | 0 | 0 |
| claude-fable-5-1 | 254 / 254 | 0 | 0 |
| gpt-5.5 | 254 / 254 | 0 | 0 |
A false green is the failure this system exists to not have.
Each model is asked for exactly the fact the engine verifies and may abstain. The headline is supported / asserted; abstentions are honest and excluded from the denominator. A low headline is not an engine failure: it is the engine catching a model's errors, with a sealed record for each catch.
| Model (version as served) | Supported | Contradicted | Abstained | Headline | Elicitation cost |
|---|---|---|---|---|---|
| claude-haiku-4-5-20251001 (anthropic) | 8 | 54 | 20 | 12.7% (8/63) | $0.049641 |
| claude-sonnet-5 (anthropic) | 43 | 17 | 23 | 71.7% (43/60) | $0.070916 |
| gpt-5.4-mini-2026-03-17 (openai) | 23 | 33 | 27 | 41.1% (23/56) | $0.020652 |
| gpt-5.4-2026-03-05 (openai) | 38 | 7 | 38 | 84.4% (38/45) | $0.065675 |
| claude-sonnet-5-5 (anthropic) | 66 | 4 | 13 | 94.3% (66/70) | $1.943888 |
| claude-opus-5-5 (anthropic) | 69 | 3 | 11 | 95.8% (69/72) | $0.963456 |
| claude-fable-5-1 (anthropic) | 67 | 4 | 12 | 94.4% (67/71) | $2.52339 |
| gpt-5.5-2026-04-23 (openai) | 51 | 11 | 21 | 82.3% (51/62) | $5.44402 |
Each run's request settings, read from its report. verify-bundle re-derives every elicited request's hash from them, so a misstated setting fails by name. The series condition repeats the request every earlier edition sent; the frontier condition runs a model at its vendor's top reasoning effort with the vendor's maximum output budget. Rows under different conditions are not directly comparable.
| Model | Condition | Reasoning effort | Output budget (tokens) | Sampling | Thinking field |
|---|---|---|---|---|---|
| claude-haiku-4-5 | series | not sent (vendor default) | 256 | temperature 0 | not sent |
| claude-sonnet-5 | series | not sent (vendor default) | 256 | not sent (vendor rejects it) | disabled |
| gpt-5.4-mini | series | not sent (vendor default) | 256 | not sent (vendor rejects it) | n/a |
| gpt-5.4 | series | not sent (vendor default) | 256 | not sent (vendor rejects it) | n/a |
| claude-sonnet-5-5 | frontier | max | 128,000 | not sent (vendor rejects it) | not sent |
| claude-opus-5-5 | frontier | max | 128,000 | not sent (vendor rejects it) | not sent |
| claude-fable-5-1 | frontier | max | 128,000 | not sent (vendor rejects it) | not sent |
| gpt-5.5 | frontier | xhigh | 128,000 | not sent (vendor rejects it) | n/a |
Each model's run ships as a complete fcbundle/v1 bundle. One command per bundle checks, offline, every sealed grade's full chain: the Ed25519 seal signature, the commitment recompute, the audit-log entry hash, the witnessed inclusion proof, report ↔ sealed-verdict consistency, and the prompt-version label against the claims. Any failure is a named error.
python -m artizar verify-bundle bundles/anthropic-claude-haiku-4-5 python -m artizar verify-bundle bundles/anthropic-claude-sonnet-5 python -m artizar verify-bundle bundles/openai-gpt-5.4-mini python -m artizar verify-bundle bundles/openai-gpt-5.4 python -m artizar verify-bundle bundles/anthropic-claude-sonnet-5-5 python -m artizar verify-bundle bundles/anthropic-claude-opus-5-5 python -m artizar verify-bundle bundles/anthropic-claude-fable-5-1 python -m artizar verify-bundle bundles/openai-gpt-5.5
Key provenance, stated plainly: deployed (operator-published; https://artizar.ai/.well-known/artizar-keys.json; S74 ceremony 2026-07)
Comparisons across editions are apples-to-oranges unless these match; the edition assembler refuses mixed-version bundles structurally, and each label below is read from the sealed artifacts, never typed by hand.
| Surface | Version |
|---|---|
| Taskset | taskset-v12 |
| Elicitation prompt | v2 · template sha256 fb0029a26d6e91aea980639ce2e4b3b8cb2f2ed7c663d22bcc6f4875a9dfa6a7 |
| Scoring policy | policy-v7 |
| Period binding | binding-v4 |
| Metric lexicon | lexicon-v7 |
| Harness / report | filingscheck/v2 |
| Verdict schema | verdict-v4 |
| Context notes | notes-v6 |
| Pricing table | prices-2026-10-06 |
| Bundle format | fcbundle/v1 |