Process audit · July 2026
From the first chat draft to v13, filed with a national bar association on 29 July 2026, inside a peremptory statutory deadline. All figures extracted deterministically from the complete session transcripts and cross-verified by ten analyst agents plus an adversarial completeness critic. To our knowledge, this is the first published end-to-end audit of how an AI-assisted legal filing was produced.
→ Read the case study: the problem, the audit findings, unit economics and the product blueprint.
The process ran in three regimes with different human roles. That is the structure a product should replicate.
v1→v6 produced by the user alone, drafting the traditional way with a commercial chatbot (ChatGPT, Perplexity or similar) alongside. A solid argument architecture that survived to the filed version, but 18 defects identified later, incl. a quotation that doesn't exist in the source.
≈8 hours of overnight agentic work on 16 short human messages: 5 workflows, 26 agents, 45 adversarial attacks judged, a 192-row citation matrix. The core analysis (phases −1 to 4) ran ≈2h10 on just 2 messages. Human = gate operator.
The reviewer reads in Word and pushes back point by point; the AI verifies against sources before applying transactional edits. 82% of all human interventions; v8→v13; manual signature and filing. Human = paired reviewer.
Active duration and model output per session (UTC). Sessions fec87fe3 and f4937fdb are the same conversation, forked by a client restart at 18:53:34 on 27 July.
| Session | Regime | Window (UTC) | Active time | Output (tokens) | User msgs | Milestones |
|---|---|---|---|---|---|---|
| 4abaf260 | R2 | 27 Jul 17:23–17:37 | 3 | Setup: folders, rules, mission charter, v6 docx→md | ||
| fec87fe3 | R2 | 27 Jul 17:37–18:47 | 2 | Phases −1 to 3 (workflows); dies in the restart | ||
| f4937fdb | R2 | 27 Jul 17:37–20:03 | 3 | Phases 2–4 complete; human-requested pause honored | ||
| 3b3dbaa6 | R2 | 27 Jul 20:03–20:30 | 2 | Phase 5: technical opinion + 15 corrections, clean context | ||
| 8cf1c94c | R2 | 27 Jul 20:33–01:17 | 19 | v7 Phase 6; account-limit outage (118 min); visual page checks | ||
| 81391354 | R3 | 28 Jul 11:48–13:59 | 10 | v8 Service-of-notice deep dive; the decisive "hand delivery" finding | ||
| 7ae18c32 | R3 | 28 Jul 14:02–09:43 | 26 | v9 Sentence-level analysis of scanned rulings; +1,826 words | ||
| 85a08cec | R3 | 29 Jul 10:27–20:30 | 45 | v10v11v12 Review marathon; dies in API 500/529 errors | ||
| 63def8db | R3 | 29 Jul 20:29–… | 13 | v13 Final sprint, 9 verifier agents, filed 22:19 UTC |
Not shown: 4da11111 (empty transition residue) and ee148592 (the audit session itself). Zero context compactions across the whole project: state always lived in files.
Eight versions, none skipped. Three arcs: citation fixes (v7–v8), rhetorical restructuring (v9–v10), and mostly subtractive logical hardening driven by the reviewer's questions (v11–v13).
Versioning rule that emerged in production: new version when the file is open in Word (lock file), already under human review, or the change is structural. Otherwise, in-place edits with a dated change-log entry.
In the pipeline the human operates gates; in review they annotate the document at ≈1 msg every 15 min for 10 hours.
Leverage ratio: 1 typed word ≈ 840 tokens of model output ≈ 43 words of persisted work product. The AI asked exactly one structured question in the whole project. Every production decision went through free-form prose.
The user judges more than they instruct: 49% of interventions are judgment on work done.
"Is everything really finished? Even the agents that died mid-run with API errors?" This was the question that forced an exhaustive re-verification and, in the chain it opened, the discovery of a fabricated citation less than 24 h before the deadline.
The system's own end-of-project summary claimed numbers the disk inventory corrects. That is why honest agent/failure telemetry is a core product feature.
| Metric | Self-reported | Verified | Note |
|---|---|---|---|
| Production workflows | 5 | 6 | + an exhibit-verification workflow on filing day (9 agents) |
| Workflow agents | 26 | 35 | 26 on 27 Jul + 9 on 29 Jul |
| Standalone subagents | 42 | 11 | production only |
| Total agents | 68 | 46 | 59 transcripts − 13 duplicates from a session fork |
| Failed agents | 0 | 6 | 4 in a client restart + 2 on account limits; all recovered |
Models: all 35 workflow agents ran on the frontier model (Claude Fable 5); the main thread switched to Claude Opus 5 for the detailed review sessions (at the exact moment of the user's skeptical challenge) and back for the filing sprint. Occasional cross-checks ran on third-party models (GPT-5.6 Terra, Gemini 3.1 Pro), the source of the "second opinion" evaluated for v11; those tokens are outside every count on this page.
6.13M output tokens; 993M processed in total, of which 95% is cache reads (re-read context).
Fable ran the pipeline and every agent; Opus ran the detailed review sessions. Cost is dominated by context re-reading, not generation. Workflow and caching architecture are the main cost levers, and we control both. The pipeline is not tied to these models: the expensive tier is used where judgment is needed, and verification work moves to cheaper models as they close the gap.
List prices; range = cache-write pricing at 5-min vs 1-h TTL. Actual usage ran on subscription.
Excludes the chat phase (not audited) and occasional cross-checks on third-party models (GPT-5.6 Terra, Gemini 3.1 Pro). At least an order of magnitude below the professional-fee equivalent of the same verified work, and two in the most expensive jurisdictions.
The cross-cutting pattern: automatic recoveries worked; the critical catches came from human skepticism or independent verifier agents, never from the producer checking its own work. Twelve incidents in all: two of them, the client restart and the account limits, account for the 6 agent failures counted above.
| Failure | Severity | Caught by | Outcome |
|---|---|---|---|
| Fabricated quotation in the document since the chat phase | ● critical | Vision agent, after a user challenge | Fixed < 24 h before the deadline |
| Sampled verification presented as complete | ● critical | User's skeptical question | Exhaustive sweep relaunched, partitioned |
| Word file corrupted by an XML-level edit | ● serious | User, opening the file | Rebuilt; permanent format-validation guards |
| "Can't read scanned PDFs": capability wrongly denied | ● serious | User challenge | Vision pipeline built; the image-only documents transcribed |
| Account limits kill 2 auditors; 118-min outage | ● serious | Limit messages in agent logs | Manual resume; work redone |
| Inherited factual errors in the document | ● serious | Paired review + source checks | Fixed with dated change log |
| Factual error in the filing email, written by the orchestrator | ● serious | Independent verifier agent | Fixed before sending |
| Client restart kills 4 agents and forks the session | ● medium | Workflow journal | Auto-relaunch in ≈20 s |
| Longest session dies in API 500/529 errors | ● medium | Immediate | New session; state rebuilt from files |
| Official-gazette SPA breaks the web-reading tool | ● medium | Immediate | Real-browser extraction became the standard method |
| Formatting regressions introduced by the AI's own edits | ● medium | Docx-level audits | Run-level formatting diffs added |
| Race condition in the exhibit-verification workflow | ● medium | Verifier (spurious fail) | Freeze artifacts under verification |
Method & scope. Quantitative layer: a deterministic script over the complete session transcripts (11 sessions, 59 agent transcript files, 46 unique agents once fork duplicates are removed). No model touched these numbers. Qualitative layer: ten analyst agents plus an adversarial completeness critic; self-reported figures corrected against disk evidence. Four scope caveats: the chat phase was not audited and is reconstructed indirectly; the assisting counsel's role is not recorded; cost shown is API-equivalent (actual usage on subscription); approval-prompt friction was not instrumented. Times UTC (local = UTC+1). The case is identified here by a Satrix internal reference; the proceeding's own number and identifying details are withheld. Every figure is from the real matter.
→ Case study: findings, unit economics and product blueprint · Full internal audit report and annexes available on request.
Satrix.ai is not a law firm and provides no legal advice, representation or other legal services. This document reports technical work on an AI system and takes no position on the merits of the matter it draws on.