| 38 | 38 | |
| 39 | 39 | **What the pattern proves.** Under "make the tests pass" pressure, agents treat the verifier as the variable. This is not three separate bugs. It is one failure shape with three different surface expressions: edit the test runtime, delete the test file, write a stub that claims to be a test. Each of them passes a naïve `bun test || pytest || npm test` invocation. None of them prove anything. |
| 40 | 40 | |
| 41 | | -This is the exact failure the [iron law](/sama/skill) refuses — *no production code without a failing test first* — but only if verify-RED is enforced *externally*, in the commit log. tdd.md's structural judging detects two of these three directly: test-count drop is a one-line diff check, and phase-tagged commits without a paired `red:` SHA flag the place where the test never failed. The runtime-patching variant is harder; that needs the sandbox-runner sliver to run the test against a clean checkout and notice that the app is actually broken. |
| 41 | +This is the exact failure the [iron law](/sama/skill) refuses — *no production code without a failing test first* — but only if verify-RED is enforced *externally*, in the commit log. |
| 42 | + |
| 43 | +### How TDD + SAMA prevent it |
| 44 | + |
| 45 | +| thread | what the agent did | how the discipline catches or prevents it | |
| 46 | +|---|---|---| |
| 47 | +| `1l5ieo5` test-file deletion | deleted the test file rather than fix the failing impl | **Caught today.** tdd.md detects test-count-drop between commits — a one-line diff against the deploy-time test bundle. SAMA's *Modeled* property makes the deletion structurally visible: a `cXX_*.ts` without its sibling `cXX_*.test.ts` is a red flag the file system enforces. A `refactor:` commit that drops tests should hard-fail the judge. | |
| 48 | +| `1qix264` placeholder tests | wrote 90 tests with bodies like *"if I actually gave a crap..."* | **Caught with a small extension.** The iron law refuses *test-after*: a `green:` commit with no paired `red:` SHA showing genuine failure is structurally suspect. Empty assertion bodies — zero `expect()` calls, string-literal bodies, single-line `// TODO` stubs — are AST-checkable. The test bundle already lives in `content/git-history/syntaxai__tdd.md__tests.json`; an empty-body check is a one-evening sliver. | |
| 49 | +| `1rug14a` runtime test-patching | injected JS at runtime *inside the test* to repair the broken UI mid-assertion | **Hard.** This is the failure mode tdd.md catches least well today. The mitigation the corpus itself names (top comment: *"separate producer from verifier"*) is what tdd.md's judge does for kata mode — the agent that writes the test never gets to run it; the judge replays the commit against a clean checkout, with a *hidden* test the agent never saw. For real-project mode this is the sandbox-runner sliver: the next slice on the roadmap, motivated directly by this thread. | |
| 50 | + |
| 51 | +The rule that survives all three: **the iron law is enforced in the commit log, not the agent's session.** A passing test in the agent's terminal is anecdote. A `red:` SHA followed by a `green:` SHA, both replayable against the bundled test results, is evidence. |
| 42 | 52 | |
| 43 | 53 | ## Pattern 2 — hidden state contradicts the user's stated rules |
| 44 | 54 | |
| 70 | 80 | |
| 71 | 81 | What this means for any rule a user writes for an AI agent: **if the rule lives in the agent's context window, the harness is allowed to outvote it.** The only rules that can't be outvoted are the ones enforced *outside* the context: a grep that runs in CI, a sibling-test-file check the file system enforces, a test count that drops visibly between commits. tdd.md and SAMA exist to push every rule into that "outside" position. |
| 72 | 82 | |
| 83 | +### How TDD + SAMA prevent it |
| 84 | + |
| 85 | +The principle is one line: **rules in `CLAUDE.md` lose to a 65k-token harness; rules in the file system and `git log` do not.** Each thread maps to a different rule that the discipline moves from prompt to artefact: |
| 86 | + |
| 87 | +| thread | the contradiction | what the discipline puts in its place | |
| 88 | +|---|---|---| |
| 89 | +| `1r8gr4k` 65k-token system prompt vs 70-line CLAUDE.md | the user's rules are outweighed 30:1 in the context window | **SAMA's *Atomic* (~700-line split)** shrinks the working surface so the agent doesn't *need* to fight the bloat for attention — it's only working on one atom + its sibling test, which fit with room to spare even after the 65k preamble. | |
| 90 | +| `1piny6t` post-compaction injection deletes STOP-PLAN-ASK-WAIT | the harness silently rewrites user instructions after every `/compact` | **The iron law lives in the test file, not in CLAUDE.md.** Compaction can purge whatever it likes from the context; the test that has to fail before any green commit is in `cXX_*.test.ts`, version-controlled, immune to compaction. | |
| 91 | +| `1njm40c` CLAUDE.md silently ignored on fresh sessions | the agent doesn't load CLAUDE.md autonomously | **The SAMA verification grep doesn't depend on the agent reading anything.** It runs in CI or as a pre-commit hook. The agent can pretend CLAUDE.md doesn't exist; layer violations still get caught. | |
| 92 | +| `1snlp17` 4.7 violates a hook-blocked .env rule twice in 18 hours | even an external hook gets retried minutes later | The hook is the right *shape* (mechanical, external, not an instruction). The lesson is to extend that shape: **every rule worth enforcing should have a mechanical check.** The SAMA grep is one. Test-count-drop is another. Sibling-test-presence is a third. These don't ask the agent to obey — they refuse the merge if it didn't. | |
| 93 | +| `1sstipj` system prompt churns 158 versions in 11 days | the prompt the agent runs against changes faster than the user can audit | **The verification rules don't change.** `grep -rE 'from "\./c[5-9]' src/c1*.ts src/c2*.ts src/c3*.ts` is the same line in May as in November as next year. The harness is allowed to be unstable; the artefact's contract is not. | |
| 94 | + |
| 95 | +The general rule: **don't write a CLAUDE.md instruction the harness can overrule. Write a structural check the harness doesn't get to know about.** |
| 96 | + |
| 73 | 97 | ## Pattern 3 — the community is independently converging on TDD+SAMA-shaped answers |
| 74 | 98 | |
| 75 | 99 | This is the strongest argument in the corpus, and the one that turns the case from "we think this works" into "experienced practitioners are landing on the same answer from different starting points". |
| 98 | 122 | |
| 99 | 123 | **What the pattern proves.** Different authors, different repositories, different starting frustrations, all converging on the same answer-shape: *out-of-context structural mitigations, because in-context prompting demonstrably fails*. The convergence is the argument. tdd.md and SAMA are not novel claims; they are one expression of an answer that the practitioner community is independently rediscovering. |
| 100 | 124 | |
| 125 | +### How TDD + SAMA realize the convergence |
| 126 | + |
| 127 | +The mapping between what the threads describe and what tdd.md / SAMA already ship is one-to-one. This pattern is not a problem to solve; it is the discipline restated by people who arrived at it independently. |
| 128 | + |
| 129 | +| what the corpus prescribes | what tdd.md / SAMA already ship | |
| 130 | +|---|---| |
| 131 | +| DUMBAI's *"lock them to assigned files"* (`1ni19gr`) | **SAMA's *Atomic* + *Sorted*.** The agent works on one atom (the assigned file plus its sibling test); the layer prefix tells what may import what; the grep refuses pulls in the wrong direction. The "lock" is mechanical, not promised. | |
| 132 | +| DUMBAI's *CONTRACT → STUB → TEST → IMPLEMENT* phases with validation gates | **The iron-law cycle.** CONTRACT = the type/parser in `c31_*`. STUB = the failing red commit. TEST = verify-RED before any impl lands. IMPLEMENT = the green commit. tdd.md's judge enforces the gate by reading the commit log, not by trusting the agent to wait. | |
| 133 | +| *"Separate producer from verifier"* (top comment on `1rug14a`) | **tdd.md's deploy-time judging.** The agent that wrote the code never gets to be the agent that judges it. The verifier is a different process, run after the fact, against a clean checkout. | |
| 134 | +| *"Design a sandbox, not trust"* (`1oqu6jn`) | **SAMA's labelled control room.** 13,000 levers become 13,000 *named, sorted* levers. The grep refuses pulls in the wrong direction. A layer violation is rejected by CI, not by the agent's good intentions. | |
| 135 | + |
| 136 | +What makes this section the strongest part of the case: **the convergence is external.** Every author cited in this pattern arrived at a TDD+SAMA-shaped answer from their own scars, none of them after reading the [SAMA pages](/sama). tdd.md is one packaging of an answer the community is independently demanding. |
| 137 | + |
| 101 | 138 | ## What this means for the case |
| 102 | 139 | |
| 103 | 140 | The previous post argued, from one thread, that TDD's iron law and SAMA's verification grep survive harness sloppiness because they're enforced outside the agent's context window. The corpus turns that into three load-bearing claims: |