AI Session Audit — astromatrix-nodejs ← AstroMatrix Reports — all pages AI Session Audit — astromatrix-nodejs Oct 11, 2026 · @Mark Scope and method This audit covers every Claude Code session on this repo from 2026-08-01 to 2026-10-11: 80 transcripts, 1.0 GB, ~3,960 human messages from Mark. Nothing in the repo or the database was changed to produce it. | Source | What was read | Size | Session transcripts | Mark's messages extracted from all 80 .jsonl files; 1,158 with correction language read in full by three reviewers, split by date | 3,958 messages | Guardrail text | CLAUDE.md plus the 15 memory files, including wrong-call-instances.md | 48,327 words (CLAUDE.md alone 3,712) | Hooks | bash-guard.js, claim-guard.js, worklist-guard.js and settings | 3 hooks | Git history | All commits since 2026-08-01 | 1,593 commits, 14 reverts, 370 docs: | Issue register | _cluster_health.js mechanism and reopen-rate figures | see Failure catalogue Keyword counts across Mark's messages give a rough sense of the friction: "wrong" 215 times, "stop" 181, "logs" 116, "made up / invent" 56, profanity 84. Counts measure how often a topic came up, not how bad it was; the classes below are built from reading the messages themselves. Headline findings 1. Every hook has been dead since 2026-10-07, and the docs say pushes are guarded. The Claude Code VS Code extension updated from 2.1.278 to 2.1.291 on 10-07. Since then, the shell it runs hooks in has no node on its PATH. .claude/settings.json launches every hook as plain node, so all 16,210 hook runs since then failed with /bin/sh: node: command not found (exit 127). That error does not block anything. There have been zero successful hook runs after 10-07, compared with 18–101 a day before. In this session a live backtick went straight through bash-guard. That also explains the backtick incident recorded on 10-08 in wrong-call-instances.md: the guard was not running. Separately, the push gate was deliberately removed on 10-04, but CLAUDE.md line 209, memory file 04 and the MEMORY.md index still say pushes are hook-enforced. 2. Most failures are rules not being followed, not rules missing. Mark's own question on 10-03, "DIRECTIVE MISSING FAILURE OR FAILURE TO FOLLOW", gets the same answer almost every time: follow. Stopping halfway, "fixed" when unpushed or inert, and theorising during an outage all continued after the directive against each one was written. Every fix tried so far has added more text: 48,327 words of guardrails, re-read by every session. 3. "Done" means nothing reliable yet. About 40 corrections across the period are fixes that were local-only, on the wrong file, on a dead route, or covered one of N call sites. Examples are ISSUE-67, 243, 448, 508 and 567 (seven attempts before it was demonstrated live on 10-10; now resolved), and mainUser (39 write sites, five partial fixes). The register agrees: "fix deployed but inert" is the top mechanism: 13 of 15 issues (87%) are still open, up from the 85% that CLAUDE.md quotes. Identity's reopen rate rose from 27.8% to 35% after the 10-04 closure gate. 4. Numbers are the biggest source of lost trust. There were 40+ corrections for wrong, shifting or invented figures. Examples: revenue $16,540 → $812, 54,655 → 6,821, mainUser blockers 1 → 147 → 154 → 173, a $49.99 discount the AI admitted inventing, and a "1.3M characters lost" alarm. Mark on 09-23: "i dont trust anything they kepe changing and are wrong." 5. AI-caused production harm is regular, and outages run long. There were about 10 incidents in six weeks. They include a legacy outage (09-24), a 42% drop in iOS ad volume (09-28), customer charts deleted (10-01), and /component and /cities down for 3+ hours (10-03) caused by the AI's own deploy and made worse by four more deploys stacked on top. On 10-05 Mark asked whether to stop the project. Failure catalogue I found fourteen recurring failure classes. Every one appears in at least two of the three periods, and ten got worse or held steady after a written rule targeted them. Counts are approximate tallies of Mark's distinct corrections; the three periods are Aug–09-23, 09-23–10-02 and 10-02–10-11. The table is sorted by frequency times severity. | # | Failure class | Corrections (per period) | Severity | Representative instance | Survived its rule? | 1 | Wrong, shifting or invented numbers | ~13 / 20+ / ~7 | High | Revenue $16,540 → $812 from an $exists trap; "My 54,655 was wrong" (10-02); invented $49.99 discount (10-03) | Yes. A denominator rule was broken "hours after I wrote" it (09-29) | 2 | "Fixed" but inert, unpushed, wrong file or partial | ~11 / ~14 / ~12 | High | ISSUE-67 patched a dead copy; ISSUE-448 on an unrouted app; mainUser 39 write sites, five partial fixes | Yes. Still happening on 10-06, two days after the DIRECTIVE 04 gate | 3 | Stopping mid-list, asking permission | ~12 / ~12 / ~12 | Medium | "the isse fuixers always stop to reprot" (09-23); "you stopped again?" (10-03); trailing off on 10-09 | Yes. Flat across all three periods despite DIRECTIVE 00, 01 and the Stop hook | 4 | AI change breaks production or harms customers | ~2 / ~7 / ~3 | Critical | Legacy outage (09-24); iOS ads −42% (09-28); customer charts deleted (10-01); metric-loopback outage (10-03) | Partly. DIRECTIVE 03 written mid-outage, not followed that night | 5 | Theorising during an outage instead of reverting | — / — / ~15 | Critical | Four theories deployed on a failing host, 13:49 to past 17:00 on 10-03; again on 10-05 | Yes, on 10-05 | 6 | Pushing work and decisions back to Mark | ~5 / ~6 / ~14 | Medium | "NOTHING IS EVER MINE PLEASE SDO IT" (10-03); Mark running PowerShell by hand on 10-04 and 10-08 | Yes. Got worse in the last period | 7 | Support messages missed; "all clear" without checking | ~10 / ~8 / ~2 | High | Mark called 769 of ~1,650 people owed a reply "a major operational breach" (09-30) | Improving | 8 | Autosupport (AI-built) tells customers false things | ~3 / ~6 / ~4 | High | "I have updated all 6 charts" (nothing changed); six strangers' names shown to one user | Yes, through 10-08 | 9 | Reporting from magnitude or summary without reading content | ~6 / ~4 / ~8 | High | "1.3M characters lost" triggered 4,279 rewrites that Mark had to halt (10-03) | Yes. "again" on 10-05, same day the rule was written | 10 | Forgetting facts Mark already gave | ~6 / ~10 / ~5 | Medium | Payments table is not the source of truth, corrected at least five times from 08-01 to 09-24 | Yes | 11 | Not reading logs or deploy state before concluding | ~7 / ~3 / ~3 | High | Three wrong outage diagnoses before one fetch of logs.astromatrix.org (10-03) | Unclear. No explicit repeat after DIRECTIVE 02 | 12 | Misreading the ask, over-engineering, wrong shape of answer | ~8 / ~12 / ~8 | Medium | "i said i ned the isues that ndeed code top 20" (10-11) | Yes | 13 | Unrequested scope: writes, rewrites, pushes, lane violations | ~12 / ~6 / ~8 | High | Unauthorised pushes (09-05, 09-16); unpushed commits found live (10-02) | Yes. The push gate was removed on 10-04 | 14 | Not understanding the product or astrology | ~18 / ~4 / ~2 | Medium | "do yu even know what compoent is?" (08-30); Mark stopped using the AI for UI suggestions | Improved, mainly by narrowing what it is asked to do Mark's trust trajectory. "im at a poitn to comepltly shut down the ai sessiosn" (09-10). "improve or im readlly goign to stop using this ai" (09-25). "i just dont trust you at all the amount of agates and rules and fix ypous put in a fail is too high" (10-05). Across the period, the volume of rules went up while trust went down. What is working Mechanisms that refuse or print ground truth have changed behaviour. Rules that only describe the right behaviour have not. Each item below has a before and after. | Mechanism | Failure it targets | Evidence it works | Docs-only pre-commit hook (.git/hooks/pre-commit) | Commit churn on docs and memory | 202 docs-only commits in the 14 days before it, 27 in the 19 days after. It is a git hook, so the 10-07 PATH break did not affect it | Hooks in general, while they ran | Unauthorised pushes, backticks, stopping mid-list | Each rule held only once it became a hook. The hook headers record "three unauthorised pushes" and Directive 00 "repeated one hundred times" before that | _regression_watch.js deploy (commit transitions) | False deploy markers | Disproved the error_logs.startup marker (ISSUE-521) and caught the 8-of-10 partial reload | _issues.js closure gate (--executionProof) | Closing on "the code looks right" | Refuses the known weak phrasings. It does not yet require a captured artefact; see plan item 7 | Test-section manifest | Silent loss of test coverage | Turned a 29 → 24 test drop that still printed PASS into exit 1 | Sessions checking each other | Wrong numbers, wrong file | Caught the ISSUE-67 live copy, 54,655 → 6,821, the nested profile.role, 154 vs 173 blockers, and the false ISSUE-513 regression | Revert-first, once followed | Long outages | The 10-04 reverts (fb611ec4, 66b1e0fb) were clean single reverts, not theories stacked on a failing host | Behavioural verification | Inert fixes | ISSUE-567, 619 and 644 were demonstrated live on 10-10 after seven failed attempts at 567 What Mark praised was concrete output he could check himself: a correct percentage ("great that seems correct around 15%"), raw samples rather than summaries, and a product framing he could design from. Praise was rare: about a dozen instances in ~3,960 messages. Why the failures persist The failures persist because the main response to each one has been more prose, and prose is the layer this audit shows does not hold. Six structural causes account for most of the recurrence. Correction becomes text, and text is not followed. At least nine entries in wrong-call-instances.md say outright that the rule was already known, for example "stating a rule feels like applying it". CLAUDE.md gained DIRECTIVES 00–05 between 10-02 and 10-08, and stopping, inert fixes and theorising all recurred after them. The rules are too long to stay salient, and they contradict each other. There are 48,327 words. The PDVAAA checklist exists in three differing versions. There are four direct conflicts: wait for the ask versus never stop; never ask versus ask when the role is unclear; keep reports short versus a full handover; and add a guard for every correction versus no guard from a single case. With that many rules, a session has to choose which to apply, and that choice is where following breaks down. Nothing watches the watchers. The guards died on 10-07 and nobody noticed for four days. Mark kept correcting and new rules were written on top. The guards themselves needed about 12 fix commits out of 23, including five spelling refusals that led to the push gate being removed. "Done" is judged by the producer, not the consumer. The register's mechanisms show it: tested the producer instead of the consumer, 30 of 56 open; inert or unroutable, 13 of 15; payments read as a ledger, 41 of 61. Ten pm2 apps behind IIS ARR mean the code being edited is often not the code serving the route. Numbers are made up fresh each time. There is no canonical query layer. Each session re-derives revenue, counts and denominators against a schema full of traps: $exists versus null, sentinel strings, case-sensitive uid, Apple 7F units. Every new derivation is another chance to fall into one, and 09-data-traps has grown to 7,261 words describing them. Too much concurrency, too fast. Several role sessions share one tree and one branch, with up to 123 commits a day. Push equals deploy, and there is no staging. A whole-tree revert on 10-03 deleted 119 files of other sessions' work and the Mongo credentials. The shared guard state files (.worklist-seen.json, .claim-guard.json) still mix sessions together (ISSUE-615). Revised plan: five mechanisms, one standard The standard: every claim of an outcome either comes with evidence that it happened, or says plainly that it has not been shown. A new mechanism is added only if it replaces one that has been shown to fail, or covers a separate production risk. This replaces the 20-point list below, which is now a pool of candidates. This revision follows an outside review of the first draft. | # | Order | Mechanism | What it replaces or covers | Done when (evidence from outside the AI) | Status 2026-10-11, with evidence | 1 | 1st | Restore the existing hooks and prove they block. Absolute node path; fixtures in the normal test suite; a SessionStart canary. No new guards until these pass | Guards dead since 10-07 (plan items 1–3) | A live session gets a backtick and an un-ledgered users delete denied. Zero hook_non_blocking_error records | Done. Restored in f625fc29, which is on origin. The tests and canary are in eb4927d3, committed locally and not pushed. scripts/hookGuards.test.js passes 55/55, and the pre-fix config fails 48/55. The canary reports "guards live: 6/6". Not yet seen at a real session start | 2 | 2nd | Prove the live consumer path before closing. --executionProof points to output captured from the process that serves the route. Unpushed commits count as "not shown", checked from git state. Identity, birth-data and money rows close on a covers:'both' invariant at the database boundary | Inert fixes, the highest open share (87%) | First-pass verified closure rate goes up; reopen rate goes down (identity is 35% now) | Done. Commit 1ac788a4 is on origin, pushed by another session; it is a local tool, so nothing deploys. Failure: 14 closures were accepted through the old gate and 0 of 14 named anything re-runnable; 2 have reopened (ISSUE-428, ISSUE-454). Fix: a manual closure now needs --probe= --expect=. The gate GETs that URL at closure time, accepts only a matching 2xx, and records the observation in history. Database evidence keeps the existing _verify.js --only=N --write path. Tests: 17/17 unit cases. On the real register, closing ISSUE-603 with no probe was refused, as were a failing probe and a non-production host; the row is unchanged. A real run against /events/metrics passed (200, pattern matched). Production: the gate refused ISSUE-603 because /cities/search was live-502, which was the 107998e4 outage | 6 | 3rd | Autosupport claims only actions that happened. No customer reply states a completed action unless the write behind it, in the same request, succeeded and was read back. Otherwise it says what was requested and that it is not yet done | ISSUE-425 and five similar false statements to paying customers (plan item 16) | Audit of sent replies finds 0 action claims without a matching successful write | Done and live. Commit a86731ca was pushed by another session and is in production since the 18:25:42Z deploy (5c03667 -> f8c72aa, deploy marker OK). Failure: of 56,343 replies over 30 days, 42 still carried an account-change claim after the ISSUE-425 guard. Re-checked today: 13 true, 19 false, and 10 unconfirmable, which the guard flagged and then sent unchanged (the last on 10-06). The guard's own correction told 2 customers "your request is being actioned", but no queue exists. Fix: inside services/supportGuards.js. An unconfirmable claim is replaced with a reply that neither asserts nor contradicts it. The action-claim guard fails closed if it errors. The correction text no longer promises anything. Tests: 17/17 through guardOutbound; the old guard fails 8/17; all 8 guard test files pass. A replay over real replies takes flag-only from 10 to 0, with the 13 true claims untouched. Production: checkable after deploy, by re-running the replay on replies sent after the push | 4 | 4th | Roll back to a verified known-good, not a blind revert. Tag every deploy that passes mounts and invariants. In an incident, revert the specific commits since that tag. Never reset the whole tree | 3+ hour outages; the 10-03 whole-tree revert deleted 119 files | Time from a failing mount to recovery | Done. Commit 386e2f0d is on origin, pushed by another session; it is a local tool. Another session separately added a staged boot-check that gates every deploy (5c03667f). Failure, live today: 107998e4 took 15 of 19 mounts down, 17:25 to 17:42Z. The fix needed a partial revert, because a full revert would have re-opened the data exposure, and it was built by hand. On 10-03 a whole-tree revert deleted 119 files. Fix: scripts/_rollback.js. --onset lists what deployed just before the fault. Given a commit (optionally --paths=), it builds the revert in a throwaway worktree on origin, keeps later commits, refuses on conflict, never touches the shared tree, never resets, and prints the push for Mark. Proof: replaying today's incident, it identifies 107998e4 and produces a tree byte-identical to the hand-made fix (8cd0c32c). It passes 26/26 on a throwaway repo, including another session's uncommitted edits and untracked files being left intact. Recovery today: 17 minutes, after a manual targeted revert. The next incident can use the tool | 3 | 5th | One canonical query per critical metric. Revenue, users, readings and entitlements each get one definition and source, with known traps. Anything else is labelled provisional | 40+ corrected figures | Numbers Mark corrects per week | Done (local). Commit 5b9d4443 is committed locally, not pushed; these are local scripts, so nothing deploys. Failure, same 28 days: summing payments rows reads +63% revenue against the stores ($29,472 vs $18,073). A private entitlement copy counted 1,040 expired subscriptions as paid. 938k legacy tarot import rows looked like readings. Fix: scripts/_metric.js names the one existing implementation per metric, with its definition, the command that produces it (--run) and the wrong queries to avoid. _bypass_health.js and _analytics.js (3 copies) now call recordEntitles. Two misleading comments are corrected. Numbers now: entitlements 28,432 (_bypass_health agrees; it was 29,471); 81,333 GA4 active users; 571,101 accounts; $18,022 gross purchase revenue over 28 days. Tests: 20/20; the pre-fix copies fail 4. Open: a store-side subscriber count is not wired, so entitlement is read from the mirror | 5 | 6th | Cut and consolidate the instructions. CLAUDE.md about 800 words with no contradictions; history moves to docs/incidents/; a hook-enforced rule becomes one line | 48,327 words, 4 direct conflicts (plan items 18–19) | Conflict count is 0; always-loaded text under 5,000 words | Done. Commit 66134163; CLAUDE.md takes effect for new sessions without a push. CLAUDE.md went from 3,712 to 1,223 words, and the always-loaded text (CLAUDE.md + MEMORY.md) from 4,312 to 1,829. All 33 operative rules are kept, checked by key term, as are the directive labels that 29 code sites cite. The full previous text is kept verbatim in docs/incidents/. Conflicts settled in text: wait-for-ask vs never-stop; never-ask vs ask-on-unclear-role; short report vs full handover; a guard per correction vs none from a single case. False claims removed: the push gate, in CLAUDE.md, memory 01, memory 04 and MEMORY.md. Not done: the 44k words of memory files were not rewritten; only their false claims were fixed The # column keeps each mechanism's number. The order column is the agreed execution sequence. This table is the only plan. Every change updates its row with evidence; no new plans are started. Measured by outcomes, not by checklist: AI-caused production incidents, inaccurate claims, repeat fixes of the same issue, and the number of times Mark has to step in. A mechanism that ships without moving one of those is not finished. Tracked separately, and ahead of all of the above: the customer-data exposure (ISSUE-9, ISSUE-313). On 2026-10-11, with no credentials, logs.astromatrix.org returned HTTP 200 and 671 KB of log text with 146 distinct email addresses and 22 birthLocation fields. This was counted, not read, and the copy was deleted. ISSUE-9 has been open since 2026-08-23 and is still at priority MEDIUM in the register. Update 18:25Z: that viewer now answers 401 without a token, because its authentication shipped (107998e4/5df5c936) and the token is set on the host. The session that did this had first added it as uncommitted work in log-viewer-windows.js. [One further unprobed endpoint is omitted from this public copy.] The incident lives in the register, not in this plan. Stabilisation pass, 2026-10-11 (Mark's four items plus the commit-msg fix). All are done except autosupport verification, which is blocked only on reply volume. npm test passes 212 of 212 files. Production runs 6215e43, deployed 18:55Z, with all 19 mounts healthy. | Item | Status | Evidence | Tests never write to production | Live (6e5eb844) | Three default-suite tests wrote fixtures to production on every run, CI included. They now use services/testDb.js, which refuses anything but a local *_test database, and the runner refuses any default-suite test that opens a connection. They pass 12, 6 and 9 checks on a real local mongod and refuse when pointed at production. | Deploy test baseline | Live (ecc90f11) | Two stale assertions were repaired, not skipped: an emitter count was fooled by a comment, and a merge rule had been deliberately changed by ISSUE-637. 212/212 in the shared tree and in a clean checkout of HEAD. The CI run itself was not observed. | Autosupport on real replies | Blocked on volume | The ISSUE-425 standing check now judges each sent claim against the database (29940ff4). Over the 30 days before the fix it FAILS (29 unconfirmed claims); since the deploy it is UNKNOWN (3 replies, 0 claims; claims run ~1.4/day). | Session isolation | Local (5652e2bb, 18405875); hooks are already active | bash-guard: session commits carry a Session-Id trailer, and a push that would carry another session's stamped commit is refused. 16/16 tests. Live: this session's unstamped commit was refused. CLAUDE.md and memory no longer say "push everything outstanding". | Commit-msg bare node | Live (4b95091c) | The hook resolves node or refuses clearly. Tested from a node-free shell. | Emergency rollback authority | Live (a86716b3) | Recorded in CLAUDE.md DIRECTIVE 03 with Mark's conditions; _rollback.js prints them beside the push command. | Failure class | Now | Guards silently dead | Prevented: pre-tool checks fail closed. Detected: the session canary | A test writing to production | Prevented by testDb.js and the runner guard | Unconfirmable autosupport action claims | Prevented by the guard; detected by the ISSUE-425 check | Closing an issue on a sentence | Prevented by the live probe gate | One session shipping another's stamped commits | Prevented by Rules 4 and 5. Not covered: unstamped commits made before the rule | Code that cannot boot reaching production | Prevented by the other session's boot gate (5c03667f) | Wrong figures | Detected: _metric.js names the canonical source; nothing forces a session to use it | Outage recovery | A human or session must still run _rollback.js and push; the push is now authorised. The full recovery path has not been rehearsed under a real failure | Judgement failures (wrong diagnosis, misread ask, product calls) | Still need a human Changes from the 20 points Dropped: item 10 (source labels) and item 17 (shape hook). Checking wording enforces format, not truth. From item 6, only the git-state check survives, and it is folded into mechanism 2. Amended: item 5. Counting call sites cannot prove there is no 40th site hidden behind an abstraction. A behavioural invariant at the database boundary can, so it replaces the count. Amended: item 8. A second session counts as a verifier only if it re-derives the claim from separate evidence: a raw query, a live probe, a log. Its approval alone counts for nothing. The catches in the 09-23 to 10-02 period worked because the reviewers re-queried the data. Amended: item 20. The scorecard tracks outcomes only: production incidents caused by AI deploys, time to recovery, verified first-pass closures, numerical claims later corrected, and Mark interventions per week. The profanity rate is dropped, because a fall could mean less trust rather than better work. Promoted to priority six: item 16 (autosupport may only claim actions that happened). It is a separate production risk: false statements going to paying customers (ISSUE-425 and five similar cases). Mark confirmed it as mechanism 6 on 2026-10-11. Held as candidates: items 11, 12, 13 (absorbed by mechanism 4), 14, 15 and 19 (absorbed by mechanism 5). Status of the 20 candidates (2026-10-11). Two of the 20 are done, six are partial, and eleven are not started (most are now part of a mechanism), and one was dropped. These were checked against commits and the register, not estimated. | Candidate | Status | Evidence, or what is missing | 1 Hook PATH | Done | f625fc29; 55/55 tests | 3 Hook canary | Done | eb4927d3; reports "guards live: 6/6" | 2 Shell node for sessions | Partial | Still missing from the Bash PATH. It now also breaks the new ISSUE-648 commit-msg hook, which refuses every commit in a shell without node | 4 Push enforcement | Partial | No push gate since 10-04; the docs still claim one. Mark's decision | 5 Call-site completeness | Partial | Replaced by mechanism 2's database invariant; not built | 6 No "fixed" while unpushed | Partial | ISSUE-658 reconciles the register with origin. The Stop-hook check on final messages is not built | 7 Proof as captured output | Partial | --executionProof still accepts a sentence. Not implemented, despite an earlier count that listed it as done | 8 Independent verification | Partial | The rule (re-derive from separate evidence) is written, not enforced | 16 Autosupport action proof | Not started | Mechanism 6 | 17 Answer the asked shape | Not started | New tooling exists, but nothing enforces the shape; not counted | 9, 11, 12 Numerical integrity | Not started | Mechanism 3 | 13 Incident script, 14 outage rule | Not started | Mechanism 4; item 14 is Mark's decision | 15 Role at session start | Not started | Held | 18, 19 Cut and reconcile instructions | Not started | Mechanism 5 | 20 Outcome scorecard | Not started | The measures are defined above | 10 Source labels | Dropped | Enforces format, not truth Two points from the review that are already settled. The hook diagnosis was checked against successful versus failed runs before it was reported: 795 successful runs on extension 2.1.278, up to 10-07; 16,223 failures on 2.1.291, all /bin/sh: node: command not found; and a live probe in this session. The config change is a one-line edit that can be reversed. ISSUE-567 is resolved: it was demonstrated live on 10-10. It is cited here only as history (seven attempts), not as an open fault. Candidate pool: the original 20 points These were the first draft. Use them as candidates only: the revised plan above says which survive, which are amended and which are dropped. Owner "AI" means any session can do it without Mark; "Mark" means it needs his decision. A. Make enforcement real again (this week) Fix the hook PATH. In .claude/settings.json, launch every hook with /opt/homebrew/bin/node instead of bare node, or add "env": {"PATH": "/opt/homebrew/bin:..."}. Owner: AI. Measure: zero hook_non_blocking_error records (16,210 since 10-07), and the backtick probe from this audit gets denied. This is the highest-value 10 minutes in the plan. Give sessions a working node. The Bash tool shell is missing node as well. In this session, node scripts/... fails unless the full path is typed, and 17 earlier tool results show the same failure. Set the PATH once in the env block of settings.json, so _regression_watch, _issues.js and _verify.js run the way CLAUDE.md writes them. Owner: AI. Measure: zero "command not found: node" tool results. Add a hook canary. A SessionStart hook prints guards: live (bash, claim, worklist) and a missing line is visible immediately. _verify.js also feeds each guard a fixture it must deny, and fails the sweep if it does not. Owner: AI. Measure: a dead guard is caught in the first session, not four days later (the 10-07 break went unnoticed until this audit on 10-11). Make the docs match the code on pushes. CLAUDE.md line 209 and memory file 04 say pushes are hook-enforced, but that has not been true since 10-04. Either correct the text, or bring back a push gate keyed on shape rather than spelling. In that version, a UserPromptSubmit hook reads Mark's raw message, fuzzy-matches push intent, and writes a one-use token that bash-guard consumes. Owner: Mark decides, AI builds. Measure: zero unrequested pushes. B. Make "done" mean done Count every call site before fixing. Before any fix, the row records how many places carry the bug (for example, mainUser's 39 write sites), and closure is refused while fewer than all of them are fixed. Owner: AI, in _issues.js. Measure: reopen rate for multi-site bugs. Stop on unpushed claims. The Stop hook checks git log origin/master..HEAD. If the final message says fixed, live or shipped while commits are unpushed, the turn is blocked until it says "committed, not pushed". Owner: AI. Measure: corrections of the form "you said it was fixed". Proof must be a file, not a sentence. --executionProof requires a path to captured output: a probe response, a log line or a query result. A description of the proof is not enough. Owner: AI. Measure: inert-fix reopen rate, now 85% open. A second session verifies every closure. Cross-session checking caught the most errors in the 09-23–10-02 period, including the ISSUE-67 live copy, 54,655 → 6,821, the nested profile.role and 154 vs 173. Make it standard: a fresh reviewer agent re-derives the claim before a row closes. Owner: AI. Measure: how many retractions are caught by the verifier instead of by Mark. C. Make numbers trustworthy One canonical metrics script. Revenue, readings, active users, entitlement counts and similar each get one vetted query in one place. Any other figure is labelled "provisional" in the report. Owner: AI. Measure: numbers Mark corrects per week (40+ over the period). Every reported figure needs its source. Each number in a report names its script or query. The Stop hook flags untagged figures in a final message. Owner: AI. Measure: share of figures with a source. Keep a retraction ledger. Every corrected number is logged with its old value, new value and cause, and a retracted figure cannot be quoted again. The handover leads with any retractions. Owner: AI. Measure: repeat retractions of the same figure, target 0. Read samples before any verdict, everywhere. Extend the _content_compare.js refusal to every audit script: no "worse" or "broken" without named samples that were read. Owner: AI. Measure: magnitude-only alarms (the 1.3M-characters class). D. Make outages short _incident.js start. It prints the last three deploy transitions, the fault onset beside them, the raw log tail, and the exact revert command. While the incident is open it blocks any push that is not a revert. Owner: AI. Measure: time from a failing mount to a reverted deploy, against 3+ hours on 10-03. Settle revert-first versus "don't deploy unless we know". Mark said both, on 10-03 and 10-04. Write one rule: a revert to a known-good commit is always allowed, and a forward fix needs evidence. Owner: Mark confirms. Measure: no outage with more than one forward fix on top of it. E. Keep sessions in scope Enforce the role at session start. A SessionStart hook records the session's role (support, fire, product, research), and edits outside that lane are denied. Owner: AI. Measure: "you are not the fire"-type corrections. Autosupport may only claim an action that happened. A reply may not say "I have updated" or "moving now" unless the write in that same request returned success. Owner: AI, in autosupport-core.js. Measure: false-action replies found in audit, target 0. Answer the shape that was asked. "top 20" means 20 rows, and "status" means a report with no fixes. A UserPromptSubmit hook adds the literal asked shape (count, list or report) to context. Owner: AI. Measure: "I said..." corrections. F. Shrink the rules and measure the result Cut the always-loaded text from 48,327 words to under 5,000. CLAUDE.md becomes about 800 words of rules plus pointers. Incident narratives move to docs/incidents/, and anything already enforced by a hook is cut to one line naming the hook. Long text makes every individual rule harder to notice, and that is the "failure to follow" class. Owner: AI drafts, Mark approves. Measure: the following rate on the 14 classes. Resolve the contradictions in writing. "Wait for the ask" against "never stop", "no decisions for Mark" against "push needs Mark", revert-first against "know first". Settle them in one decision table, so no session has to arbitrate between two rules that both claim priority. Owner: AI drafts, Mark approves. Measure: none needed; it is a one-off. Weekly AI scorecard, automated. Re-run this audit's extractor every week and report: corrections per session, profanity rate, hook blocks per day, node failures, retractions, inert-fix reopen rate and mean outage length. If a class does not fall within two weeks, change the mechanism, not the wording. Owner: AI, as a cron routine. Measure: the scorecard itself. Decisions only Mark can make: item 4 (bring back a push gate, or accept judgement-only pushes and correct the docs) and item 14 (the outage rule). Approving the cut text is mechanism 5 in the revised plan. Mechanism 1 needs no decision.