1f9254a4e56a8721c125b9cb319eed5883d8ecc1 diff --git a/content/blog/deploy-that-lies-cascade.md b/content/blog/deploy-that-lies-cascade.md new file mode 100644 index 0000000000000000000000000000000000000000..fb935eaca61ad7d0c98285969665433963094e34 --- /dev/null +++ b/content/blog/deploy-that-lies-cascade.md @@ -0,0 +1,310 @@ +# When the deploy lies: three bugs hidden by one silent error suppressor + +The two prior posts in this thread were clean rounds: the verifier +named a violation, I produced the named artifact, the verifier flipped +green. Atomic-700 on `c21_app.ts` → split per domain → ✓. Modeled on +four `c32_*.ts` files → add the four siblings → ✓. Encouraging stories +about mechanical enforcement. + +This post is the messy round. It's the one that taught me that +mechanical enforcement only works if the pipeline that runs it is +itself running. + +## The visible bug + +`/reports/live` is the public live-data demo: real commit history for +this repo, rendered into a TDD-discipline scorecard, refreshed on every +deploy. On 2026-05-22 the header read: + +``` +tdd-discipline report · 2026-05-03 → 2026-05-10 +``` + +Twelve days of staleness on a page that calls itself "live." I'd +shipped seven commits across the previous rounds and none of them +appeared. + +## Why nobody noticed for 12 days + +The deploy script in git-mode invoked the snapshot generator over ssh: + +```bash +ssh "$SSH_HOST" "cd ~/$REMOTE_SRC_DIR && bun scripts/p620/snapshot-git-history.ts" 2>/dev/null \ + || echo " ⚠ snapshot-git-history skipped (script may live outside the rsync exclude — non-fatal)" +``` + +Two clauses are doing the damage: + +- `2>/dev/null` discards stderr — including the error message we'd want. +- `|| echo " ⚠ ... non-fatal"` turns a real failure into a printed + warning. Worse, the warning text *blames the wrong thing* + ("script may live outside the rsync exclude") so anyone who DID see + the warning would file it under "harmless artifact of rsync vs git + mode" and move on. + +The actual failure: there's no `bun` on the p620 host. Bun lives only +inside the tdd-md container image. The ssh tried to invoke a binary +that doesn't exist on PATH; the shell returned 127; the warning fired; +the deploy continued; the snapshot file's timestamp stayed at May 11. + +Twelve days. Every deploy. Both of the previous "clean rounds" deployed +through this same broken path and updated the *site* but not the +*live data*. The blog posts about going green were themselves served by +a deploy script that was lying about its own snapshot step. + +## Fix 1, and what it revealed + +The fix is structurally trivial: run the script *inside* the container +where bun lives, by mounting the working tree as a volume: + +```bash +ssh "$SSH_HOST" "podman run --rm \ + -v \$HOME/$REMOTE_SRC_DIR:/work:Z \ + --workdir /work \ + $IMAGE_TAG \ + bun scripts/p620/snapshot-git-history.ts" \ + || { echo '✗ snapshot-git-history failed'; exit 1; } +``` + +The `:Z` is the Fedora SELinux relabel — the script process inside +needs to be able to read/write the bind mount. The `|| +{ echo ✗; exit 1 }` replaces the swallow with a real failure mode. No +more silent skips. + +After this fix landed, `/reports/live` immediately caught up: + +``` +tdd-discipline report · 2026-05-03 → 2026-05-22 +``` + +So far so good. But the moment I looked at `/reports/live/tests`, the +sibling test-stability page, the timestamp said: + +``` +last run 2026-05-10 · 17 runs cumulative +``` + +Same staleness. Different cause. + +## The second silent failure + +Looking at the deploy script again, the **rsync** escape hatch runs +both snapshot scripts: + +```bash +( cd "$REPO_ROOT" && bun scripts/p620/snapshot-git-history.ts ) || ... +( cd "$REPO_ROOT" && bun scripts/p620/snapshot-tests.ts ) || ... +``` + +The **git-mode** happy path runs only the first one. When the deploy +flow switched from rsync to git as the default a while back, the +test-snapshot step got dropped on the floor and nobody noticed — +because the test-stability page was always 17 cumulative runs old, and +"old enough that nobody questioned the number" is one of the failure +modes that a verifier can't detect. + +Fix 2: add the second podman-run step, with one wrinkle. Unlike +`snapshot-git-history` (which is pure git + filesystem), `snapshot-tests` +calls `bun test`, which needs `node_modules` to resolve `marked` and +`node-html-parser`. The bind-mounted host directory has no +`node_modules` (the host has no Bun). But the image already ships +them at `/app/node_modules`. So: + +```bash +podman run --rm -v $HOME/src/tdd.md:/work:Z --workdir /work $IMAGE_TAG \ + sh -c 'ln -sfn /app/node_modules node_modules && bun scripts/p620/snapshot-tests.ts' +``` + +Symlink the container's `node_modules` into the work directory, then +let the script use it. The symlink persists on the host between +deploys but points at a path inside the container — harmless dead-link +outside the next podman-run, valid inside. + +## Two more bugs, surfaced by the snapshot actually running + +When the next deploy ran with both snapshots wired in, the live page +now read: + +``` +Total: 193 tests · 192 passing · 1 failing · 1 placeholder ⚠ +``` + +193 pass locally, every time I run them. 192 pass + 1 fail + 1 +placeholder on the container. Two bugs that had been hiding behind +"the test suite never actually ran in the deploy pipeline." + +### Bug A: a 1-in-16 flaky test + +The failing test was one I wrote in the prior round: + +```ts +test("verifySession rejects a cookie with a forged signature", async () => { + const cookie = await signSession("eve"); + const tampered = cookie.replace(/.$/, "0"); + const result = await verifySession(tampered); + expect(result).toBeNull(); +}); +``` + +`replace(/.$/, "0")` replaces the last character with "0". When the +HMAC signature's last hex digit *is already* "0" — which happens with +probability 1/16, since SHA-256 hex output is uniform — the +"tampered" string is identical to the original, the signature +verifies, the function returns `"eve"`, and the assertion fails. + +Local runs masked this because the random draws (the timestamp going +into the signed payload) happened to never produce a `0`-ending sig. +The first run that actually ran in CI hit the unlucky draw and +exposed it. + +Fix: read the last char, flip to a digit it definitely isn't: + +```ts +const lastChar = cookie.slice(-1); +const tampered = cookie.slice(0, -1) + (lastChar === "f" ? "0" : "f"); +expect(tampered).not.toBe(cookie); // loudly fail if a future regression collides +``` + +Five runs in a row, every one passes. Determinism restored. + +### Bug B: the verifier's own test, flagged by its own check + +The placeholder warning pointed at: + +``` +src/c32_sama_verify.test.ts > does nothing +``` + +`c32_sama_verify.ts` is the verifier itself. Its test file holds a +fixture: + +```ts +test("Atomic: placeholder test (zero expect calls) is flagged", () => { + const placeholderFixture = `test("does nothing", () => { /* TODO */ })`; + // ... feed it to the verifier, assert the verifier flags it +}); +``` + +The string `test("does nothing", () => { /* TODO */ })` is a *fixture* +— a literal example of what a placeholder test looks like, fed to the +verifier so we can assert the verifier catches it. It's not a real +test. + +The verifier itself handles this correctly. It uses a +`stripStringsAndComments` helper to mask out string literals before +running its `test()`-finder regex over the source. So when the +verifier scans `c32_sama_verify.test.ts`, it sees the fixture as +whitespace, doesn't pick it up, and reports zero placeholders in that +file. + +But `snapshot-tests.ts` — the deploy-time generator that feeds +`/reports/live/tests` — duplicated the regex *without* the +strip-strings step. So it grepped the raw source, found the fixture +inside the backtick string, treated it as a real `test()` call, walked +its (TODO-only) body, counted zero `expect()` calls, and flagged it. + +The deploy-time detector was flagging the very test that proves the +runtime detector works. + +Fix: export `stripStringsAndComments` from `c32_sama_verify.ts` and +use the same mask-index pattern in the snapshot script: + +```ts +import { stripStringsAndComments } from "../../src/c32_sama_verify.ts"; +// ... +const mask = stripStringsAndComments(content); +while ((m = re.exec(content)) !== null) { + // If the match position is whitespace in the mask, the original + // was inside a string or comment — skip. + if (mask[m.index] === " " || mask[m.index] === "\n") continue; + // ... rest of the body-walking logic +} +``` + +DRYing the helper across the two places that need the same string-aware +behaviour. Now the snapshot agrees with the verifier. + +## What the cascade was actually telling me + +The bug count for ronde 4 looks bad: a 12-day staleness, a flaky test, +a false-positive in the deploy-time detector. Three independent +problems. + +But the *order* is the part worth looking at. Each fix made the next +one visible: + +1. Deploy script ran the snapshot step → file's timestamp moved → + `/reports/live` started reporting current commits. +2. Deploy script ran the test snapshot → tests actually ran in the + deploy pipeline → the flaky test surfaced (because previously it + never ran in CI), and the false-positive surfaced (because + previously the snapshot was 12 days old and that particular + fixture had been added since then). +3. Each fix's success was the precondition for the next bug to be + visible. + +The cascade isn't proof the system is fragile. It's proof that the +system was *blind* — a layer of silent error suppression had hidden +every downstream failure, so they accumulated without being detected. +The fix was less "patch three things" than "remove the lie and watch +what falls out." + +This is the same shape as TDD's iron rule applied to *infrastructure* +rather than to source: you can't trust a pass you didn't run. The +deploy-pipeline checks `bun test` exits zero — but only if `bun test` +*ran*. If the call returns 127 (command not found) and the deploy +script swallows it, every later assertion is hollow. + +`/reports/live` showing all-green for 12 days was perfectly compatible +with the test suite being completely broken. The only way to know is +to delete the swallowing. + +## Why this is the empirical case for SAMA, not against it + +A naive reading is "the codebase had three bugs you didn't catch." +The fairer reading is: the codebase had *one* bug — silent error +suppression in a deploy script — and the other two were latent +consequences that the verifier *would have* caught the moment they +ran. Removing the silence took ~15 minutes. Once silence was gone, both +hidden bugs surfaced *on the very next deploy*, with line numbers and +file paths, in two cells of a public web page. + +That's the empirical pattern SAMA's pitch turns on, scaled to the +infrastructure layer: + +- **Verification has to be observable.** A check that runs into + `2>/dev/null` is indistinguishable from a check that passes. +- **The cost of removing silence is low.** A `||` swallow → `|| + { echo ✗; exit 1; }` is a one-line change. A `2>/dev/null` → + `2>&1` is one word. +- **Removing silence pays compounding returns.** Three bugs hidden by + one suppressor — each one would have been instantly diagnosable if + the surface had been honest. + +## What this still doesn't prove + +It doesn't prove that exposing every failure produces a useful signal. +Some failures *should* be tolerated (best-effort cleanup, optional +caches), and over-strict failure handling can break production for +trivial reasons. The judgement is *which* failures: in this case, +`snapshot-git-history` running was load-bearing for the public claim +that `/reports/live` reflects the current repo. Treating its failure +as "non-fatal" was a category error. + +The general principle the cascade demonstrates: in a system whose value +proposition is *the artefacts a reviewer can replay*, the pipeline +that produces those artefacts has the same audit requirements as the +source code does. Silent failures in the pipeline are violations of +the standard the same way silent failures in the source would be. + +--- + +**See for yourself:** + +- Live: (date window is now current) +- Live: ("193 passing · 0 placeholder") +- The PR that landed the three fixes: + +- Previous posts in this thread: + [the c21 Atomic-700 split](/blog/sama-empirical-c21-split) · + [greening the Modeled dogfood](/blog/sama-empirical-modeled-green) diff --git a/e2e/git-content-browse.spec.ts b/e2e/git-content-browse.spec.ts new file mode 100644 index 0000000000000000000000000000000000000000..6cd5fc28dc12e97f0525e61dbb0e19956de5bc00 --- /dev/null +++ b/e2e/git-content-browse.spec.ts @@ -0,0 +1,121 @@ +// E2E: every blog post in ALL_POSTS is reachable via /GIT/. +// +// Crawls the registry's slugs (lifted into a literal array here so +// the test file doesn't import server-side modules) and asserts: +// 1. /GIT/syntaxai/tdd.md/tree/main/content/blog lists each post +// 2. /GIT/syntaxai/tdd.md/blob/main/content/blog/.md renders +// the post (markdown rendered via marked into the chrome) +// 3. /GIT/syntaxai/tdd.md/raw/main/content/blog/.md serves +// the raw markdown +// Plus the tree home (/GIT/syntaxai/tdd.md/tree/main) shows the +// top-level directories (content/, src/, public/, scripts/, etc.). + +import { test, expect } from "@playwright/test"; +import * as fs from "fs"; +import * as path from "path"; + +// Mirror of c31_blog.ts ALL_POSTS slugs. If a post is added there, +// add the slug here too. Kept inline to avoid pulling server code +// into the test process. +const BLOG_SLUGS = [ + "sama-meets-git-cms", + "from-rules-to-checks", + "agentic-coding-corpus-three-patterns", + "claude-code-harness-postmortem", + "three-constraints-agentic-coding", + "tweag-handbook-tdd", + "aider-tdd", + "cursor-tdd", + "claude-code-tdd", +]; + +const SCREENSHOT_DIR = "test-results/git-content-browse"; + +test.beforeAll(() => { + fs.mkdirSync(SCREENSHOT_DIR, { recursive: true }); +}); + +test.describe("/GIT browses the local bare repo", () => { + test("repo root tree lists the top-level directories", async ({ page }) => { + const res = await page.goto("/GIT/syntaxai/tdd.md/tree/main"); + expect(res?.status()).toBe(200); + + // Top-level dirs we expect after the dev tree was pushed. + for (const dir of ["content", "src", "public", "scripts", "e2e"]) { + await expect( + page.locator(`a[href="/GIT/syntaxai/tdd.md/tree/main/${dir}"]`), + ).toBeVisible(); + } + // Top-level files + await expect( + page.locator('a[href="/GIT/syntaxai/tdd.md/blob/main/package.json"]'), + ).toBeVisible(); + + await page.screenshot({ + path: path.join(SCREENSHOT_DIR, "1-repo-root-tree.png"), + fullPage: true, + }); + }); + + test("content/blog tree lists every post in ALL_POSTS", async ({ page }) => { + const res = await page.goto("/GIT/syntaxai/tdd.md/tree/main/content/blog"); + expect(res?.status()).toBe(200); + for (const slug of BLOG_SLUGS) { + const link = page.locator( + `a[href="/GIT/syntaxai/tdd.md/blob/main/content/blog/${slug}.md"]`, + ); + await expect(link, `link to ${slug}.md must be present`).toBeVisible(); + } + + await page.screenshot({ + path: path.join(SCREENSHOT_DIR, "2-content-blog-tree.png"), + fullPage: true, + }); + }); + + for (const slug of BLOG_SLUGS) { + test(`blob view renders ${slug}.md as markdown via /GIT`, async ({ page }) => { + const res = await page.goto( + `/GIT/syntaxai/tdd.md/blob/main/content/blog/${slug}.md`, + ); + expect(res?.status()).toBe(200); + // The repo-blob-rendered container is what marked.parse output + // lands in. It must exist + be non-empty. + const rendered = page.locator(".repo-blob-rendered"); + await expect(rendered).toBeVisible(); + const text = (await rendered.textContent()) ?? ""; + expect(text.length).toBeGreaterThan(200); + // The breadcrumb must show the file path so users can climb. + await expect(page.locator(".commit-breadcrumb")).toContainText(`${slug}.md`); + }); + + test(`raw endpoint serves ${slug}.md as text/plain via /GIT`, async ({ request }) => { + const res = await request.get( + `/GIT/syntaxai/tdd.md/raw/main/content/blog/${slug}.md`, + ); + expect(res.status()).toBe(200); + expect(res.headers()["content-type"]).toMatch(/text\/plain/); + const body = await res.text(); + // Frontmatter or first heading — every blog post has one. + expect(body.length).toBeGreaterThan(200); + }); + } + + test("path traversal is rejected", async ({ request }) => { + for (const evil of [ + "/GIT/syntaxai/tdd.md/blob/main/../etc/passwd", + "/GIT/syntaxai/tdd.md/blob/main/content/../../etc/passwd", + "/GIT/syntaxai/tdd.md/tree/main//content", + ]) { + const res = await request.get(evil); + expect(res.status(), `${evil} must 404`).toBe(404); + } + }); + + test("non-allowed (owner, repo) 404s — only syntaxai/tdd.md is served", async ({ + request, + }) => { + const res = await request.get("/GIT/someone/random-repo/tree/main"); + expect(res.status()).toBe(404); + }); +}); diff --git a/feedback.md b/feedback.md index 3a279d036377b1d1ff0b0f4b310b4a0326243e81..a196671eacdc06db1a732443ffba734fd588f273 100644 --- a/feedback.md +++ b/feedback.md @@ -1,66 +1,114 @@ -Goede vraag. Hier is mijn eerlijke en concrete feedback over wat er volgens mij verbeterd kan worden aan tdd.md: -Wat al goed is +SAMA 2.0 — Voorstel voor de volgende evolutie - Strenge judge +Hier is een uitgewerkte, concrete versie van SAMA 2.0. Ik houd de kern intact (want die is sterk), maar maak het completer, toekomstbestendiger en nog beter geschikt voor agentic coding in 2026+. +Kern blijft hetzelfde (de vier letters) - Hidden tests + S — Sorted: Alphabetical + dependency direction (lager nummer importeert nooit hoger). - Git-based scoring + A — Architecture: Numerieke lagen met duidelijke contracten. - Publieke verdicts + M — Modeled: Types + sibling tests. -Wat ik zou verbeteren + A — Atomic: Bestanden klein houden (~700 regels) + split per domein. - Te puristisch / te streng - Op dit moment is het bijna “religieuze TDD”. Dit jaagt mensen weg. - Verbetering: Introduceer verschillende niveaus of modi: +Wat verandert / wordt toegevoegd in 2.0 - Strict Mode (huidige versie) +Ik voeg twee nieuwe letters toe → SAMA wordt SAMAX (of je houdt SAMA en maakt de extra’s optioneel). +Nieuwe letter: X — eXtensible & Vertical - Pragmatic Mode (zoals Kent Beck later bedoelde): toestaat spikes/exploratie, test-first is sterk aangemoedigd maar niet heilig. + Doel: Combineer de kracht van horizontale lagen (duidelijke dependency flow) met verticale slices (alles van één feature dicht bij elkaar). - Learning Mode: mildere straf voor beginners. + Regel: Optionele feature-prefix bovenop de laag: c32_user_auth.ts of feat_payment_c32_processor.ts. - Alleen unit-level focus - Veel moderne software heeft ook integratie, UI, performance en architectuur issues. - Verbetering: Voeg kata’s toe op verschillende lagen (niet alleen string calculator niveau), inclusief: + Voordeel voor agents: Een agent die aan “user authentication” werkt, ziet alle relevante bestanden gegroepeerd via zoekopdracht feat_user_*. - API-kata’s +Nieuwe letter: D — Documented (de vijfde discipline) - Database interactie + Regel: Elke module én elke feature-map heeft een README.md of .agent.md met: - UI/component testing + One-sentence responsibility - Geen onderscheid tussen exploratie en implementatie - In echte projecten doe je vaak eerst een spike. - Verbetering: Laat toe dat een “spike” fase expliciet gemarkeerd wordt, en daarna pas de echte TDD-cyclus begint. + Key types & contracts - Scoring is te binair - Momenteel voelt het soms als een spelletje “volg de regels perfect”. - Verbetering: Voeg kwaliteitsmetingen toe, zoals: + Acceptance criteria / invariants - Code simplicity / cyclomatic complexity + “Where to put new code” instructies - Hoe klein de stappen waren + Dit wordt de levende specificatie voor de agent. - Hoe goed de namen van tests en variabelen zijn +Uitgebreide Layer Mapping (Architecture 2.0) +Laag Naam Verantwoordelijkheid Voorbeelden Mag importeren van +c11 Entry / Composition Root App bootstrap, wiring main.ts, server.ts Alles +c13 Data / Persistence DB, repositories, queries c13_user_repo.ts c31 +c14 I/O Adapters HTTP, queues, external APIs c14_auth_controller.ts c21, c31, c32 +c21 Handlers / Presenters Request/response orchestration c21_login_handler.ts c32, c31 +c31 Models & Types Domain models, DTOs, Value Objects, Zod schemas c31_user.ts - (puur) +c32 Business Logic / Use Cases Pure functions, domain rules c32_user_auth.ts c31 +c40 Application Services Orchestratie van meerdere use cases (nieuw!) c40_user_service.ts c32, c31 +c51 UI / Presentation Components, pages, server components c51_user_profile.tsx c31, c32 +c60 Infrastructure / External Third-party clients, config, logging c60_stripe_client.ts - +c70 Cross-cutting Auth middleware, logging, monitoring, tracing c70_logger.ts alles (voorzichtig) - Of de code idiomatisch is voor de taal +Atomic-regel verfijnd: Max ~600-700 regels of max 1 feature per bestand (zelfs als het kleiner is). +Tooling & Automatisering (kritisch voor gold standard) - Te weinig feedback voor verbetering - Je krijgt een score, maar niet altijd waarom je slecht scoort op een begrijpelijke manier. - Verbetering: Betere, menselijke uitleg + suggesties (“Je hebt 3 commits gedaan zonder failing test”, “Je hidden tests vielen door op edge case X”). +SAMA 2.0 moet dit hebben: - Te weinig variatie in kata’s - Beginnend met string-calc is prima, maar er moet snel meer komen (bijv. een kleine web API, een game loop, een parser, etc.). + Officiële sama CLI - Community & educatie - Voeg een “Why” sectie toe die uitlegt wanneer strikte TDD zinvol is en wanneer niet. Op dit moment straalt het te veel “dit is de enige juiste manier” uit. + sama check → voert alle verificaties uit -Mijn ideale versie van tdd.md + sama new c32_user_auth → maakt bestand + template + .agent.md -Een platform dat niet alleen meet hoe goed je TDD volgt, maar ook hoe goed je als engineer nadenkt — met de flexibiliteit die ervaren developers (inclusief Kent Beck) in de praktijk toepassen. + sama split → helpt bij Atomic splits -Kort samengevat: -tdd.md is nu een strenge TDD-judge. -Ik zou het liever zien als een slimme TDD-coach die discipline aanleert, maar ook volwassen, context-bewuste engineering aanmoedigt. + sama verify-repo + + Pre-commit hooks + GitHub Action + + LSP / Editor integration (VS Code, Cursor, Windsurf) + + Claude / Cursor skill met automatische checks + +Extra verbeteringen + + Monorepo support: Package-prefixen (pkg_auth_c32_...) + + Graduele adoptie: SAMA Light (alleen Sorted + Atomic) → Full SAMAX + + Frontend + Backend harmonisatie (vooral Next.js App Router) + + Deprecation policy: Duidelijke manier om oude lagen/bestanden af te bouwen + + Performance & Security gates in Atomic-check (bijv. flag grote N+1 queries) + +Voorbeeld folder structuur (SAMA 2.0) + +src/ +├── c11_app.ts +├── features/ +│ └── user/ +│ ├── feat_user_c13_repo.ts +│ ├── feat_user_c31_types.ts +│ ├── feat_user_c32_auth.ts +│ ├── feat_user_c32_profile.ts +│ ├── feat_user_c40_service.ts +│ └── feat_user.agent.md +├── c60_stripe.ts +└── c70_auth_middleware.ts + +Samenvatting: Waarom dit de gold standard wordt + + Behoudt mechanische verifieerbaarheid (grep + CLI) + + Wordt hybride (horizontaal + verticaal) + + Voegt documentatie als first-class citizen toe + + Wordt echt toolbaar en adopteerbaar + + Schaalt van solo-agent project tot middelgroot team + +Dit is geen over-engineering — het lost precies de pijnpunten op die ontstaan als projecten groter worden dan 20-30k regels met agents. + +Wil je dat ik een volledige officiële spec uitwerk (zoals de originele tdd.md pagina), inclusief verificatie commando’s en templates? Of focus op één deel (bijv. de CLI spec of een concrete Next.js template)? \ No newline at end of file diff --git a/plan.md b/plan.md new file mode 100644 index 0000000000000000000000000000000000000000..ef5064aae89b01230c0d364cb0f80ecb5089e03b --- /dev/null +++ b/plan.md @@ -0,0 +1,321 @@ +# Plan — port podman/syntax CMS into tdd.md, SAMA-native + +**Doel.** Het CMS uit `~/Documents/podman` (sx-filter + sx-editor + sx-content + Ghost-compat theme) volledig overzetten naar tdd.md, in 100% SAMA-stijl, met de bestaande tdd.md content intact gemigreerd. + +**Niet-doel.** Podman, Caddy, of een tweede service-tier in tdd.md. Alles draait in één Bun-proces dat we al hebben (`c11_server.ts`). Caddy's rol (TLS + routing) doet onze deploy-laag op p620. + +--- + +## ⚠ Eerst beslissen — storage-canon + +Dit stuurt elke andere keuze. Twee opties; ik default naar **A** tenzij je flipt. + +### A. Git-canon (default — behoudt tdd.md identity) + +- Bron-van-waarheid blijft het bare repo `/app/repo` (huidige stack). +- **Elke save in de editor = een commit** via bestaande `c14_git.commitFile`. +- sxdoc-trees (typed blocks) leven als sidecar JSON naast de markdown: + `content/blog/foo.md` + `content/blog/foo.sxdoc.json`. +- SQLite (bestaande `c13_database`) krijgt een afgeleide index-tabel + (`content_index`) voor snelle lijst-queries en taxonomie-lookups, **rebuildbaar uit git**. Drop het, replay `git log`, terug. +- Voordeel: "SAMA meets git" verhaal blijft kloppen. `sama-meets-git-cms.md` blijft waarheid. Audit-trail = `git log content/`. +- Nadeel: complexer dan podman's directe SQLite-writes. Trager bij grote sites (>10k posts). Niet relevant op onze schaal. + +### B. SQLite-canon (1-op-1 podman-port) + +- `content/*.md` wordt eenmalig geïmporteerd naar `sx_documents` + `posts` tabellen, daarna read-only. +- Editor schrijft uitsluitend naar SQLite. Git-history van content stopt op het migratie-commit. +- Voordeel: minimale afwijking van podman's code. Sneller te porten. +- Nadeel: tdd.md verliest "elke content-edit = commit" — kern van het product per memory. + +**Beslissing 2026-05-11: B (SQLite-canon) + git-als-audit-mirror. Locked.** + +--- + +## Locked decisions (2026-05-11) + +### Storage canon: **B (SQLite-canon)** + git-als-audit-mirror +- **Canoniek:** `sx_documents` tabel in `c13_database` (bun:sqlite). Editor reads/writes hier; live-preview en alle render-paden lezen hier. +- **Audit-mirror:** elke save → 1 multi-path commit met `content/{slug}.md` (afgeleide markdown-projectie) + `content/{slug}.sxdoc.json` (canonical JSON-tree). Zo blijft `git log content/` de menselijk-leesbare audit-trail; "elke save = een commit" uit `sama-meets-git-cms.md` blijft waar — de **canoniciteit** ligt nu in SQLite, het **bewijs** in git. +- **Recovery:** SQLite-corruptie? Drop tabel, replay van `*.sxdoc.json`. +- **Initial migration:** eenmalig `scripts/migrate_content_to_sxdoc.ts` leest huidige `content/**/*.md` → parseert naar `SxDocument` → schrijft SQLite + emit één migratie-commit met alle nieuwe `.sxdoc.json` ernaast. + +### Parser laag: **c31** · Render laag: **c51** +- `c31_sxdoc_parse.ts` (HTML → SxDocument) + sibling `c31_sxdoc_parse.test.ts`. + Reden: `content/sama/modeled.md` is expliciet — *"every external input has a parser in a c31_* model"*. HTML strings vanuit de editor/migratie zijn external input → c31. +- `c51_render_sxdoc.ts` (SxDocument → HTML) + sibling `c51_render_sxdoc.test.ts`. + Reden: `content/sama/architecture.md` picking-order regel 4 — *"Does it produce HTML? Yes → c51"*. sxToHtml produceert HTML. +- **Correctie t.o.v. eerder plan + research-migration:** parser/renderer waren foutief op c32 geplaatst (research keek alleen naar verifier-hard-rule "c32 vereist sibling-test", maar canon-docs sturen anders). Tests blijven (c31 sibling is informationally verplicht via Modeled; c51 idem voor goed onderhoud al staat het niet hard in de verifier). + +### Commit-vorm: **één multi-path commit per save** +- `c14_git` krijgt nieuwe `commitFiles(paths: Array<{path, body}>)` naast bestaande `commitFile`. +- Eén commit → atomic rollback van die SHA herstelt beide bestanden. + +--- + +## Werkwijze (build-discipline per file-landing) + +Elke file-write moet alle vier SAMA-axes passeren vóór de volgende file landt. Geen pile-up van violations. + +| Axis | Wat dat afdwingt | Hoe we dat afdwingen | +|---|---|---| +| **Sorted** | c1*/c3* mogen niet relatief upward importeren | Bottom-up bouwen: c1 → c3 → c2 → c5. Nooit import naar hogere laag. | +| **Architecture** | prefix ∈ {11, 13, 14, 21, 31, 32, 51} | Layer-toewijzing vóór tik. I/O? → c14. Logic+transform? → c32. Pure types/registry? → c31. | +| **Modeled** | c32_*.ts vereist sibling .test.ts (hard); c31 = info-only | **c32 source + test landen in dezelfde edit-batch**, nooit los. Test heeft ≥1 `expect()` per `test(...)`-body. | +| **Atomic** | ≤700 LOC per file; geen placeholder tests | `wc -l` checken vóór commit. Splits gebudgetteerd (client/render per block-kind; shortcodes registry+substitute). | + +### Niet-verifier-afgedwongen SAMA-canon (per `content/sama/*.md`) + +- **Flat `src/`** — geen subdirs server-side. Client onder `src/client/**.ts` (buiten verifier-glob). +- **Geen barrel re-exports** (`atomic.md`). +- **c31/c32 importeren geen I/O-modules** (sharp, fs, bun:sqlite, fetch) — verifier ziet alleen relative imports, dus dit is persoonlijke discipline. +- **One concept per file** — types apart van parser apart van renderer. + +### Verificatie-cadans + +Na **elke** file-landing (niet alleen aan eind van fase): + +``` +bun test src/c32_sama_verify.test.ts # verifier zelf groen? +bun test src/.test.ts # nieuwe sibling-test groen? +wc -l src/c**.ts # geen file > 700 +``` + +Aan einde van elke fase: +``` +bun test # alles groen +bun run src/c11_server.ts & # boot smoke +curl localhost:3000/health # 200 +``` + +### Anti-patronen (expliciet verboden) + +- **"Het werkt, test komt later"** voor c32 — source en test landen samen of niet. +- **Refactoren van lower-layer code in een higher-layer fase** — bv. `c14_git.commitFiles` toevoegen tijdens Fase 1 omdat het "handig" is. Lower-layer changes horen bij de fase waar de caller landt. +- **Sub-folders onder `src/`** server-side om "het netter te organiseren". Flatten is SAMA-canon. +- **Improviseren over layer-toewijzing** — als je twijfelt over c31 vs c32, default naar c32 (sibling-test = vangnet). + +--- + +## Layer-correcties uit research-migration.md + +Plan.md zat op drie plaatsen fout volgens de SAMA-rules in `c32_sama_verify.ts`. Gecorrigeerd: + +| Was | Wordt | Reden | +|---|---|---| +| `c31_image_resize.ts` | `c14_image_resize.ts` | sharp doet I/O — c14 verplicht | +| `c31_ai_edit_block.ts` | `c14_openrouter.ts` (HTTP) + `c32_ai_edit_block.ts` (validate/transform) | OpenRouter HTTP = c14; orchestratie + sibling-test = c32 | +| `c31_sxdoc_parse.ts` | `c32_sxdoc_parse.ts` | logica, geen pure types — c32 vereist sibling-test | +| `c31_sxdoc_render.ts` | `c32_sxdoc_render.ts` | idem | + +### Atomic-700 splits gebudgetteerd + +| File | LOC bij directe port | Splits | +|---|---|---| +| `sx-editor/src/client/render.ts` | 775 | over Atomic-700; split per block-kind onder `src/client/blocks/render-{p,h,list,quote,code,img,html,shortcode}.ts` + één `src/client/render.ts`-dispatch ≤200 LOC | +| `sx-filter/src/shortcodes.ts` | 650 | krap; pak meteen split langs `c31_shortcodes_registry.ts` (built-ins) + `c32_shortcodes_substitute.ts` (HTML-rewriter met regio-skip) | + +### Tests-zijn-siblings rule (was niet geëxpliciteerd) + +Podman's `sx-editor/tests/unit.test.ts` shape **incompatibel**. Elke `cXX_*.test.ts` moet als sibling naast `cXX_*.ts` staan onder `src/`. Bestaande tdd.md tests doen dit al correct. + +### Client-side placement: `src/client/**.ts` + +Geen verifier-impact (alleen `cXX_*.ts` wordt gescand). Relatieve imports naar `../c31_sxdoc.ts` werken vanuit hier. Bun.build bundelt uit `src/client/`. Geen nieuwe top-level dir. + +### Verboden subdirs onder `src/` + +Podman's `sxdoc/`, `core/`, `db/`, `client/blocks/` mag niet onder `src/` blijven bij server-port. Dat geldt **niet** voor `src/client/` (die staat buiten verifier-scope). Server-code flat houden. + +--- + +## SAMA-mapping — podman-stuk → tdd.md cXX-laag + +SAMA-conventie (per memory): cXX_*.ts, `c1X` = data/I-O, `c2X` = handlers/app, `c3X` = pure logic, `c5X` = render. Lower layer never imports higher. + +| Podman | tdd.md (nieuw) | SAMA-laag | Wat het doet | +|---|---|---|---| +| `sx-data/sx.db` schema | `c13_database.ts` (extend) | c1 | tabellen `sx_documents`, `media`, `content_index`, `api_keys` | +| `sx-editor/src/sxdoc/types.ts` | `c31_sxdoc.ts` | c3 | `SxDocument`, `Block`, helpers — pure types/registry | +| `sx-editor/src/sxdoc/html-to-sx.ts` | `c31_sxdoc_parse.ts` (+ sibling `.test.ts`) | c3 | HTML → SxDocument (parser = c31 per Modeled.md) | +| `sx-editor/src/sxdoc/sx-to-html.ts` | `c51_render_sxdoc.ts` (+ sibling `.test.ts`) | c5 | SxDocument → HTML (produces HTML = c51 per Architecture.md) | +| `sx-editor/src/sxdoc/db.ts` | `c13_database.ts` extend (saveDocument/loadDocument/listDocuments/deleteDocument) | c1 | SQLite read/write (canon-B); bun:sqlite = c13, niet c14 | +| `sx-editor/src/upload.ts` + sharp resize | `c14_media.ts` + `c14_image_resize.ts` | c1 | upload, on-disk store, sharp transforms (sharp = I/O) | +| `sx-editor/src/ai.ts` (OpenRouter) | `c14_openrouter.ts` + `c32_ai_edit_block.ts` | c1 + c3 | HTTP-call in c14; validate + transform in c32 met sibling-test | +| `sx-editor/src/templates.ts` (list/edit shells) | `c51_render_admin.ts` | c5 | admin-list + edit-page chrome | +| `sx-editor/src/routes.ts` (urlForPage/Post) | bestaande `c31_site_config.ts` extend | c3 | routes.yaml-equivalent — wij hebben al routes-config | +| `sx-editor/src/client/blockeditor.ts` + `slashmenu.ts` + `blocks/*` | `client/` (TS bundle) → served door `c21_handlers_edit.ts` | client | block-editor JS, slash-menu, AI ✨, autosave | +| `sx-editor/src/build.ts` (Bun.build serve) | `c14_client_bundle.ts` | c1 | bundle TS-client → ESM, cache in geheugen | +| `sx-filter/src/shortcodes.ts` (650 LOC — over 700 binnen 1 add) | **split**: `c31_shortcodes_registry.ts` (built-ins, namen, args) + `c32_shortcodes_substitute.ts` (HTML-rewriter met meta/script-skip) + verplichte `.test.ts` op de c32 | c3 | parsing/substitutie, voorkomt Atomic-700 violation | +| `sx-filter/src/admin.ts` (admin-button injectie) | bestaande edit-flow heeft al login-gate | c2 | n.v.t. — wij hebben echte auth | +| `sx-content/src/render.ts` (Handlebars renderer) | `c51_render_theme.ts` | c5 | Ghost-compat theme renderer; **geen Handlebars-dep** — pure TS template-helpers | +| `sx-content/src/sitemap.ts` | `c51_render_sitemap.ts` | c5 | sitemap.xml + RSS | +| `sx-content/src/images.ts` | onderdeel van `c14_media.ts` boven | c1 | path-routed /content/images/* | +| `sx-themes/syntax/*.hbs` partials | `theme/*.html` of `c51_render_theme_partials.ts` | c5 | Ghost-look, maar als TS template-helpers | + +### Nieuwe handlers (c21) + +- `c21_handlers_admin_list.ts` — `/admin/` lijst van pages+posts +- `c21_handlers_admin_edit.ts` — `/admin/edit/{type}/{slug}` (block-editor) +- `c21_handlers_admin_new.ts` — `/admin/new` +- `c21_handlers_admin_upload.ts` — `/admin/upload` +- `c21_handlers_admin_ai.ts` — `/admin/ai/edit-block` +- `c21_handlers_admin_preview.ts` — `/admin/preview` (live render) +- `c21_handlers_content.ts` — public render dispatcher (post/page/tag/author) +- `c21_handlers_sitemap.ts` — `/sitemap.xml`, `/blog/rss/` +- `c21_handlers_media.ts` — `/content/images/*` + +Bestaande `c21_handlers_edit.ts` wordt **vervangen** door `c21_handlers_admin_edit.ts` (block-editor i.p.v. textarea). + +--- + +## Content-migratie + +Bestaande tdd.md content: +``` +content/home.md +content/blog/*.md (9 posts) +content/sama/*.md (5 pages) +content/games/*/ (2 games — multi-file) +content/guides/*.md (3 pages) +content/git-history/* (commit-meta JSON) +``` + +Migratie-strategie (canon B, SQLite + git-mirror): + +1. **Eenmalig script** `scripts/migrate_content_to_sxdoc.ts` (loopt lokaal, niet in container). +2. Voor elke `.md`: lees frontmatter (titel, tags, status), parseer body → `SxDocument` via `c32_sxdoc_parse`, **insert** in `sx_documents` tabel, schrijf óók `*.sxdoc.json` ernaast voor git-mirror. +3. `home.md` → slug `_home` (matcht podman's special `_home` slug). +4. Games (`content/games/*/`) blijven multi-file — buiten CMS-scope, blijven via `c31_games.ts` gerenderd. +5. `git-history/` is geen content — geen migratie nodig. +6. Eén batch-commit: "Migrate: content → sxdoc (SQLite-canon + git-mirror)" met alle `*.sxdoc.json` toevoegingen. + +Public URLs blijven gelijk (deze zijn al via `c31_site_config` gerouteerd). De Ghost-style `/blog/{primary_tag}/{slug}/` permalink is optioneel en gaat door de redirects-laag die we al hebben. + +--- + +## Fasering + +Per memory: bypass-pacing / JOLO is OK voor scopes die in één run passen. Dit is een dagen-werk port, dus ik fasering aanhouden met deploy + verify per fase. + +### Fase 0 — beslissing + scaffolding (afgerond 2026-05-11) +- [x] Plan vastleggen (dit document). +- [x] Storage-canon bevestigd: **B (SQLite-canon + git-audit-mirror)**. +- [x] Parser/render laag bevestigd: **c32**. +- [x] Commit-vorm bevestigd: **één multi-path commit per save**. +- [x] Research-migration onderzoek afgerond → `research-migration.md`. +- [x] Layer-correcties verwerkt in mapping-tabel. +- [ ] `plan.md` committen (wacht op user-go). + +### Fase 1 — sxdoc-fundament (in uitvoering 2026-05-11) +- [x] `c31_sxdoc.ts` — types only (geen sibling-test verplicht) +- [x] `c31_sxdoc_parse.ts` (HTML→tree, port van podman `html-to-sx.ts`) + sibling `c31_sxdoc_parse.test.ts` +- [x] `c51_render_sxdoc.ts` (tree→HTML, port van podman `sx-to-html.ts`) + sibling `c51_render_sxdoc.test.ts` +- [x] Skip typed marketing blocks — niet nodig voor tdd.md content (~600 LOC bespaard). +- [x] `c13_database.ts` extend: `sx_documents` tabel + saveDocument/loadDocument/listDocuments/deleteDocument +- [x] `package.json`: `node-html-parser` toegevoegd +- [x] `bun install` — node-html-parser@7.1.0 binnen +- [x] `bun test src/c31_sxdoc_parse.test.ts src/c51_render_sxdoc.test.ts` — 53/53 ✓ +- [x] `bun test src/c32_sama_verify.test.ts` — 10/10 ✓ (verifier zelf groen) +- [x] `bun test` (full suite) — 120/120 ✓ (67 pre-Fase-1 + 53 nieuwe) +- [x] `wc -l` op nieuwe files — hoogste 327 LOC (c31_sxdoc_parse), c13_database 390 LOC; allemaal < 700 +- [x] `bun run src/c11_server.ts` boot-smoke OK — `/` en `/sama` beide 200 +- `c14_client_bundle.ts` (Bun.build memoised) — komt pas in Fase 2 +- Geen route-impact — alles puur unit-getest, niets aan de live site veranderd. + +**Fase 1 gates passed 2026-05-11. Sxdoc-fundament SAMA-canon compliant en groen.** + +### Fase 2 — admin-UI + +**2a — server-side CRUD (afgerond 2026-05-11):** +- [x] `c31_admin_validation.ts` + sibling test (14/14 groen) — parser/validator per Modeled.md +- [x] `c51_render_admin.ts` — list + edit form + login/non-admin walls +- [x] `c21_handlers_admin.ts` — adminListHandler, adminNewHandler, adminEditHandler, adminDeleteHandler (één bestand i.p.v. plan-spec 4; matcht bestaande `c21_handlers_agents`/`c21_handlers_auth` pattern, 218 LOC) +- [x] `c21_app.ts` routes: /admin, /admin/new, /admin/edit/:type/:slug, /admin/delete/:type/:slug +- [x] Boot-smoke: anonymous → 401, login-wall rendert ✓ +- ⚠ `c21_app.ts` is nu 702 LOC (Atomic-grens 700 overschreden door 2 regels). Vraagt aparte split-refactor — c21_handlers_projects.ts, c21_handlers_api_agents.ts, c21_handlers_webhook.ts uit het inline-deel halen. + +**2b — client-side block editor (afgerond 2026-05-11):** +- [x] `c14_client_bundle.ts` — Bun.build memoised + ETag, 72 LOC +- [x] `src/client/blockeditor.ts` — hydratie + state + autosave + raw-mode toggle, 336 LOC +- [x] `src/client/slashmenu.ts` — filterable popup met arrows/enter/escape, 161 LOC +- [x] `src/client/blocks.ts` — per-block-kind renderers (p, h, ul, ol, quote, code, img, hr, html, shortcode), inline marks parser, slash-trigger, 393 LOC (één file ipv `blocks/*` — onder Atomic-700) +- [x] `c51_render_admin.ts` — neemt nu SxDocument als input, projecteert naar textarea-HTML + embedt `