page:docs-specs-platform-gaps-30-testing
Spec 30 — Testing (WP-TEST-*) reference
Date: 2026-09-27 · Status: Gate-B draft · Binding decisions: D7 (the testing bar), constrained by D4 (spec only, no implementation), D11 (testing follows migration + coverage in the roadmap), D13 (mobile deferred — no mobile lanes anywhere below), and amendment D17–D21 (CI/publish isolation from public pipelines; private Artifact Registry) (docs/specs/platform-gaps/00-intent.md:19-46).
Spec 30 — Testing (WP-TEST-*)
Date: 2026-09-27 · Status: Gate-B draft · Binding decisions: **D7** (the testing bar), constrained by **D4** (spec only, no implementation), **D11** (testing follows migration + coverage in the roadmap), **D13** (mobile deferred — no mobile lanes anywhere below), and amendment **D17–D21** (CI/publish isolation from public pipelines; private Artifact Registry) (docs/specs/platform-gaps/00-intent.md:19-46).
Inputs consumed in full: the Gate-A-approved adoption proposal research/synthesis-adoption-proposal.md (planned 9-WP inventory §4.3; survivor register §3 rows TEST1→Q-01 and TEST2→Q-08); dossiers R1 research/fabric-current-state.md (test/CI reality §4, spec inputs 4–6, 13), R2 research/t3code-architecture.md (testing patterns §3.1, §14, spec input 11); sibling specs 10-t3code-migration.md (lane WPs, §10 cross-refs to Q-07/Q-09) and 15-private-isolation-rename.md (§7 obligations on this spec: floors attach to renamed packages; whole-platform e2e installs from Artifact Registry, never npmjs). Baseline document: MATURITY-MATRIX.md (grading doctrine at MATURITY-MATRIX.md:3, levels L0–L4 at :13-17), refreshed by R1 §4. Citations: fabric: paths relative to C:/work/fabric at HEAD 5e46b998e; t3: paths relative to the MIT t3code snapshot at C:/Users/tmusk/AppData/Local/Temp/claude/C--work-company/368e0399-fd11-4960-8b4d-b4dcf1d79c43/scratchpad/t3code.
**ID convention.** Work packages here are WP-TEST-01 … WP-TEST-09. The short form Q-<n> used by the Gate-A proposal and by specs 10/15/20/40/50 refers to the same packages one-to-one (Q-03 ≡ WP-TEST-03, etc.). The count matches the proposal's planned 9 (synthesis-adoption-proposal.md §4.3); no WPs were added or removed — the sibling specs already cross-reference Q-01/Q-02/Q-04/Q-05/Q-07/Q-09 by these IDs (spec 10 §10; spec 15 §7), so preserving the inventory keeps the Gate-B compliance matrix (R-03) one-to-one. Cross-spec references use the sibling conventions: WP-T3-nn (spec 10), WP-RN-nn (spec 15), F-nn (spec 20), and WP-PKG-nn ≡ P-nn of the Gate-A inventory (spec 40).
---
1. Scope, the D7 bar, and invariants
D7 fixes the bar in three parts, all mandatory:
- **(a) Live per-harness e2e matrix** — Claude Code, Codex, Cursor, Grok Build, OpenCode, Antigravity, each driven end-to-end per OS (§3, WP-TEST-03).
- **(b) Whole-platform e2e** — packaged install → create run → dispatch → agent session → journal → UI assertions, as one product test (§4, WP-TEST-04/05).
- **(c) Unit floors + CI gating** — every package's suite wired into CI with agreed floors (§5, WP-TEST-01/02/06).
An adversarial correctness suite is explicitly **not** part of the bar but is recommended (D7 parenthetical; TEST2); it is WP-TEST-08 and is labeled recommended, not gating.
Binding invariants:
1. **Private-pipeline isolation (D17/D19/D21).** Every workflow, secret, and registry interaction this spec creates lives in the private fabric repo only. No work package touches the public a5c-ai/babysitter repo's workflows or publishing. Harness login/subscription secrets (§3.4) are fabric repo/environment secrets and are never mirrored anywhere public. No install step in any e2e lane ever resolves @fabricstack/* packages from public npmjs (spec 15 §7 obligation on this spec; guards G1–G6 apply to test fixtures too). Where a lane uses remote resolution at all, it comes only from the GCP Artifact Registry npm repo (15-private-isolation-rename.md §4.1); PR-gating deterministic lanes are stricter still — they install the PR's own workspace-packed tarballs (or a per-run local-registry fixture seeded exclusively from the PR build, §4.1.1), so no remote registry is consulted for @fabricstack/* at all and the check gates the code in the PR, not the last published train. True AR resolution is exercised where it means something: the nightly live lanes and the release train. 2. **Names follow spec 15.** Floors, workflow steps, and workspace filters attach to whatever names are current at execution time; the RN-1 rename sweep (WP-RN-05) updates them mechanically, and new suites written after RN-1 use @fabricstack/* names from birth. Pre-rename script names (test:kip, test:kradle, …) are cited below as they exist at HEAD (fabric:package.json:63,72-74,83,95,101). 3. **Retired surfaces are not tested into permanence.** The WP-T3-27 retirement list (7 genty device shells, desktop scaffold, web-app after parity) receives no new floors; genty/web-app's existing Playwright gate (fabric:.github/workflows/ci.yml:190-193 region, step "adapters webui browser e2e") is *kept green until* WP-T3-25's parity bar consumes it, then retires with the shell. 4. **Testing debt vs functionality debt.** Where a behavior under test does not exist yet (journal checksum verification, hash chain — C1/H1, fabric:packages/babysitter-sdk/src/storage/journal.ts:151-161), the *behavior* is a spec-20 WP (F-01…F-15) and the *suite* lands with it. This spec only owns suites for behavior that exists or arrives via spec 10 WPs, plus the recommended adversarial suite (WP-TEST-08) that lands after F-01/F-04. 5. **Mobile deferred (D13).** No iOS/Android lanes, simulators, or device farms in any WP below. 6. **No implementation in this run (D4).**
---
2. Baseline: what actually gates today (evidence)
2.1 PR gates (`fabric:.github/workflows/ci.yml`, jobs at :15, :54, :319, :372, :459 — R1 §4.1)
docs-quality; test (SDK lint, test:sdk at :129-132, architecture gate :114-117, tasks/hooks/trust-core/policy/tools-adapter tests :171-183, triggers unit/e2e/coverage, genty-web-app Playwright :190-193, genty-core/platform tests); gateway-node-engine (Node matrix + engine floor, :319); workspace-coverage (transport + cloud, :372-457); observer-dashboard (build + test:observer, :459-505).
2.2 Release gates (`fabric:.github/workflows/publish.yml` validate jobs at :203, :276, :457, :517, :578 — R1 §4.2)
Adds kradle-core node:test suite (publish.yml:565-568), kradle deployed-stack live e2e + smoke (publish.yml:423-424), cloud release verification, GKE deploys.
2.3 Live harness matrix (`fabric:.github/workflows/live-stack.yml` — R1 §4.3)
The strongest suite fabric has, and the seed for D7(a). Facts this spec builds on:
- **Agent map of 14 ids** (
live-stack.yml:159-174): claude, codex, pi, gemini, copilot, cursor, hermes, opencode, openclaw, omp, droid, amp, genty, antigravity. **Nogrokentry** (R1 §3.1 — zero grok code repo-wide). - **Model axis** (
live-stack.yml:175-182), **four interaction modes** (:183-188), **two install modes with fixed per-lane artifact assertions** —adapters-events,transport-adapter-trace,provider-trace-redacted, trace ids; BP addsplugin-command-transcript(:189-192). - **OS is an input, not an axis**: one
inputs.osvalue applies to the entire dispatch (live-stack.yml:17-21,runs-on: ${{ inputs.os || 'ubuntu-latest' }}at:44); push runs are ubuntu-only. Per-OS coverage exists only as separately dispatched runs and inlive-stack-published.ymldocumented-install lanes (3 OS — R1 §4.5). - Four lane jobs consume computed matrices
vanilla_ni/vanilla_interactive/bp_interactive/bp_bridged(live-stack.yml:327,549,565,581). - **Status vs the six D7(a) harnesses** (R1 §4.3;
MATURITY-MATRIX.md:64-79): Claude Code GREEN, Codex GREEN, Cursor ⛔ (#562), OpenCode ⛔ (Bun server lifecycle), Antigravity ⛔ (#945), Grok Build absent entirely. Hermes-Windows blocked (#856) — a Tier-2 concern.
2.4 The gating holes (R1 §4.4, spec input 6; `MATURITY-MATRIX.md` rows)
Ungated suites (script/files exist, no workflow invokes them): test:kip (kip-sdk 114 test files; only the single-file kip-inv12-golden-digest.yml:71 gate exists), test:kradle (PR CI — gated only in publish.yml), test:atlas-catalog-contracts (fabric:package.json:63) + atlas's 31 in-package tests, test:library/test:shared (fabric:package.json:73-74), genty-runtime (19)/genty-cli (10)/genty-ui (18) (MATURITY-MATRIX.md:46-49), genty-tui's 40 tests (only verify:release runs, :48), kradle-web 30 spec files (:54), adapters-tools 6 tests (:39), apps/commander 51 test files + Playwright with **zero** workflow references (R1 §3.9), observer-dashboard Playwright config present but not invoked (fabric:packages/observer-dashboard/playwright.config.ts, R1 §3.8 INFERRED-ungated). Zero-test packages: adapters-config, adapters-launch, kradle-sdk, kradle-cli (MATURITY-MATRIX.md:40,53). Empty file: fabric:packages/adapters/harness-mock/src/multi-execution.ts (0 lines, R1 §3.1).
2.5 Whole-platform gap (R1 §4.5, spec input 5)
No lane asserts *packaged install → create run → dispatch → agent session → journal → UI*. The UI ends (observer-dashboard, kradle-web, commander) are never asserted inside any live lane; live-stack asserts artifacts and traces, not UI state.
---
3. D7(a) — the live per-harness e2e matrix (design for WP-TEST-03)
3.1 Tier structure and lane inventory
**Tier-1 (the D7(a) six, release-gating):** each driven end-to-end through its spec-10 drive lane, per OS.
| # | Harness | Drive lane (spec 10) | Today | OS coverage target | CI auth mode (§3.4) |
|---|---|---|---|---|---|
| 1 | Claude Code | Claude SDK lane (WP-T3-10) | GREEN (R1 §4.3) | ubuntu + macos + windows | subscription OAuth token, API-key fallback |
| 2 | Codex | Codex app-server lane (WP-T3-11) | GREEN | ubuntu + macos + windows | seeded auth.json (ChatGPT subscription), API-key fallback |
| 3 | Cursor | ACP lane (WP-T3-07) | ⛔ #562 | ubuntu + macos + windows | seeded cursor-agent login token |
| 4 | Grok Build | ACP lane — net-new (WP-T3-08) | absent (R1 §3.1) | per vendor-support declaration (§3.3) | xAI login token |
| 5 | OpenCode | OpenCode HTTP lane (WP-T3-12) | ⛔ | ubuntu + macos + windows | opencode auth config with provider key |
| 6 | Antigravity | ACP lane + managed installer (WP-T3-09) | ⛔ #945 | per vendor-support declaration (§3.3) | seeded per-instance profile (OAuth loopback flow pre-baked) |
**Tier-2 (fabric's extras, per the spec-10 §3.2 lane decisions — scheduled, non-release-gating):** the retained-subprocess lanes gemini, copilot, pi, omp (≡ oh-my-pi, one harness per spec 10 §3.2), openclaw, hermes, droid, amp (8 lanes, all already in the agent map live-stack.yml:159-174), plus genty via the ACP-agent lane (WP-T3-18). **qwen is exempt by decision**: codec-only, no live-stack obligation (spec 10 §3.2 row +1) — the exemption is recorded in the matrix config, not silently omitted. The babysitter and adapters-remote codecs are lane infrastructure, not harnesses, and get no agent-map rows (spec 10 §3.1/§3.2 disposition).
**Pre/post-migration staging.** The matrix machinery (workflows, OS axis, auth, flake policy) is built against the *current* adapters immediately — Claude Code and Codex are green today and become per-OS-gated first. As each spec-10 lane WP lands, its lane re-points at the new drive path (each lane WP already carries "live-stack lane green" acceptance — WP-T3-07 AC1, WP-T3-08 AC1, WP-T3-09 AC2, WP-T3-10 AC1, WP-T3-11 AC1, WP-T3-12 AC1). Turning the red lanes green is jointly owned: infrastructure and gating here (WP-TEST-03), lane implementations in spec 10, red-lane remediation tracked as F-18 (spec 20). D7(a) is satisfied when all six Tier-1 lanes are green per their OS-support declarations.
3.2 Workflow layout (concrete)
1. **live-stack.yml gains OS as a matrix axis.** The single-valued inputs.os (live-stack.yml:17-21,44) is replaced in the matrix generator (Compute Matrix job, :130-233) by an os field per combination; the four lane jobs (:327,549,565,581) set runs-on: ${{ matrix.os }}. Dispatch entries become {agent, model, mode, install, live, os}; push defaults stay ubuntu-only (cost discipline; the per-OS bar is carried by the scheduled tier, not by push). 2. **New workflow harness-matrix.yml** (scheduled + dispatch) is the D7(a) gate: - **Nightly**: Tier-1 — 6 harnesses × declared OS set × {vanilla-NI, bp-bridged-hooks} on each harness's canonical model, native auth (§3.4). Upper bound 6×3×2 = 36 lanes/night; the vendor-support declarations (§3.3) reduce this to the real set. Tier-2 — 9 lanes, ubuntu, vanilla-NI. - **Weekly**: Tier-2 × declared OS set (hermes-Windows stays declared-blocked under #856 until resolved). - Reuses the live-stack runner packages (fabric:packages/adapters/cli/tests/live-stack/*, scripts fabric:package.json:131-138) and the fixed artifact assertions (live-stack.yml:189-192); it is a scheduler/gater over the same scenario engine, not a second harness driver. 3. **Release gating**: publish.yml gains a **freshness gate** job before the validate fan-out: the most recent scheduled harness-matrix.yml Tier-1 run must be ≤24h old and every Tier-1 lane must be green or in valid quarantine (§3.5). A stale or red matrix blocks the train. (PRs are *not* gated on live lanes — determinism at PR time comes from §4's mock lanes and §5's unit gates.) 4. **live-stack-published.yml retargets to the private train**: post-RN-1 it validates the AR-published @fabricstack/* blueprint/plugin install lanes (documented-install pattern, 3 OS — R1 §4.5) with .npmrc scope routing per spec 15 §4.1. Validation of the public @a5c-ai/* plugin remains a public-repo concern and is out of scope (D17). 5. **Existing triage/gardening workflows** (failure-triage-live-stack.yml, live-stack-gardening.yml — R1 §4.3) extend to harness-matrix.yml runs; gardening produces the pass-rate ledger the flake policy consumes (§3.5).
3.3 OS-support declarations
A committed config e2e-tests/harness-os-support.json declares, per harness × OS: supported | unsupported-by-vendor (requires a citation URL/evidence string) | blocked (requires an issue reference, e.g. hermes-Windows #856) | fixture-blocked (the vendor supports the OS but the **CI fixture** — runner image, credential seeding, installer path — is not yet workable; requires an open issue reference **and** an expiresAt ≤14 days with max 2 renewals, the §3.5.4 quarantine clock applied verbatim). fixture-blocked is deliberately distinct from unsupported-by-vendor: the latter is reserved for genuine vendor non-support with evidence and may never be used to paper over a fixture problem — a cell whose harness runs on that OS outside CI must use fixture-blocked, keeping the debt visible and time-boxed. Lanes generate **only** for supported cells; unsupported-by-vendor renders as — in reporting (mirroring the MATURITY-MATRIX.md:21 marker legend), never as a silent skip; blocked cells count as red for D7(a) purposes until re-declared; fixture-blocked cells render with their issue + expiry, count as red for the D7(a) exit, and an entry that expires unresolved (or exhausts its renewals) reverts to gating-red exactly like a §3.5.4 escalation. Claude Code, Codex, Cursor, OpenCode are declared 3-OS from the start (Claude/Codex already proven 3-OS green — MATURITY-MATRIX.md:58); Grok Build and Antigravity cells are established as an acceptance item of WP-T3-08/WP-T3-09 respectively and recorded here.
3.4 Auth and fixture strategy per harness (subscriptions/logins in CI, private-repo secret model)
All harness credentials live in a dedicated GitHub **environment live-harnesses** of the private fabric repo (D17: never mirrored; D21: no public pipeline can reference them). Naming convention HARNESS_<ID>_<KIND>. Each lane's setup step materializes credentials into an **ephemeral, lane-scoped HOME** (per-lane temp dir torn down in always()), never into the runner's real profile. The existing per-lane redaction artifact (provider-trace-redacted, live-stack.yml:190) is extended with a **canary check**: a step greps every uploaded artifact for the credential material's fingerprints and fails the lane on a hit.
| Harness | Mechanism (evidence) | CI auth fixture |
|---|---|---|
| Claude Code | @anthropic-ai/claude-agent-sdk in-process (t3:apps/server/src/provider/Layers/ClaudeAdapter.ts:5-27) | HARNESS_CLAUDE_OAUTH_TOKEN (subscription) exported for the SDK-spawned claude executable; ANTHROPIC_API_KEY fallback for API-metered lanes. Usage limits read from SDKRateLimitInfo (t3:apps/server/src/provider/Layers/claudeUsageLimits.ts) feed §3.5 quota-exhaustion classification |
| Codex | codex app-server JSON-RPC stdio (t3:apps/server/src/provider/Layers/codexLaunchArgs.ts:13) | HARNESS_CODEX_AUTH_JSON seeded into a lane-scoped $CODEX_HOME/auth.json (ChatGPT subscription session); OPENAI_API_KEY fallback. WP-T3-04's probe example is the lane preflight (t3:packages/effect-codex-app-server/test/examples/codex-app-server-probe.ts) |
| Cursor | cursor-agent … acp (t3:apps/server/src/provider/acp/CursorAcpSupport.ts:56-60) | HARNESS_CURSOR_TOKEN seeded into cursor-agent's config dir; the lane pins the installed binary version and **never invokes Cursor's updater** (WP-T3-07 AC4; t3:docs/internals/providers.md, updates section) |
| Grok Build | grok --permission-mode <m> agent stdio (t3:apps/server/src/provider/acp/GrokAcpSupport.ts:33-41) | HARNESS_GROK_TOKEN (xAI login). Lane preflight uses the no-side-effect probe rule (WP-T3-08 AC4; t3:docs/internals/providers.md, setup section) |
| OpenCode | fabric-managed server per thread via @opencode-ai/sdk/v2 (t3:apps/server/src/provider/Layers/OpenCodeAdapter.ts:32) | HARNESS_OPENCODE_AUTH — opencode auth config carrying the provider key; teardown asserts zero orphan opencode processes (WP-T3-12 AC2 mirrors this in unit form) |
| Antigravity | fabric-downloaded release, identity-verified antigravity-acp/protocolVersion 1 (t3:apps/server/src/provider/AntigravityInstallation.ts:484-502) | HARNESS_ANTIGRAVITY_PROFILE — a pre-authenticated per-instance profile archive seeded into the lane HOME (the OAuth loopback flow of t3:apps/server/src/provider/AntigravityAuth.ts is exercised in WP-T3-09's integration test, not per-lane); installer version leases pin the release per run |
Cross-cutting rules: (1) **secret rotation** is owned per credential with owner + expiry recorded in e2e-tests/harness-os-support.json's sibling e2e-tests/harness-credentials.md (names and owners only — never values); (2) an auth failure classifies as auth-expired (infra class, §3.5), not product red, via the triage workflow's failure taxonomy; (3) model-proxy lanes keep today's provider API-key envs (OPENAI_API_KEY, GOOGLE_CLOUD_*, ANTHROPIC_API_KEY — live-stack.yml:176-181) for the model axis; native-auth lanes are additive, not a replacement.
3.5 Flake policy and quarantine rules (binding for all live lanes)
1. **Failure classes**: red (deterministic product failure), product-flake (nondeterministic product behavior), infra-flake (runner/network provisioning), auth-expired, quota-exhausted (provider rate/credit ceilings — the #485-class). Classification is written by the triage workflow into the run summary; unclassified failures default to red. 2. **Retries**: live lanes get exactly **one** automatic re-run of the failed lane job. Unit/integration suites get **zero** retries — a flaky unit test is a bug, fixed or quarantined, never re-run to green. 3. **Flake threshold**: a lane whose trailing 14 scheduled runs contain both passes and failures with pass-rate <90% is flaky; the gardening workflow opens/updates its quarantine proposal automatically. 4. **Quarantine mechanics**: committed file e2e-tests/quarantine.json; entry = {laneId, issue, class, enteredAt, expiresAt (≤14 days), renewals}. Quarantined lanes **still run** (signal preserved) but do not gate the §3.2.3 freshness check. Exit requires **5 consecutive scheduled greens**. Maximum **2 renewals**; a third expiry escalates: the lane reverts to gating-red and the release train blocks until the owning issue resolves or the harness's OS cell is re-declared (§3.3) with justification. 5. **No timeout-inflation as flake treatment**: raising per-test or per-lane timeouts is not an accepted flake fix; waits must convert to receipt/drain waits (§6.1, WP-TEST-07). Lane wall-clock ceilings exist for cost control (60 min per lane job), not as synchronization. 6. **Quarantine cannot satisfy D7(a)**: the D7(a) exit criterion counts a quarantined Tier-1 lane as not-green.
---
4. D7(b) — whole-platform e2e as a product test (design for WP-TEST-04/05)
4.1 The chain, stated once
**Packaged install → create run → dispatch → agent session → journal → observer/UI assertions.** Every stage asserts at the user-observable level (a product test, not a wiring test):
1. **Packaged install** on a clean runner. Install forms tie to spec 40, staged by availability (priorities per synthesis-adoption-proposal.md §1.5, C → A → B): - **Form C (first)**: npm i -g @fabricstack/orchestrator @fabricstack/harness-cli, **sourced per lane so each lane gates what it should**. *PR-gating product-mock lane*: installs the PR's own code, never a published train — every @fabricstack/* package is built and npm packed from the PR workspace, and the global install resolves from those tarballs into a clean prefix; where transitive @fabricstack/* ranges demand a registry shape, a per-run local-registry fixture (loopback-only, e.g. verdaccio on 127.0.0.1, torn down in always()) is seeded exclusively with the PR-built tarballs. The seeding bypasses npm publish — it uploads each tarball directly via the local registry's API (or npm pack + a registry-side import) — precisely because guard G1 pins every @fabricstack/* publishConfig.registry to the Artifact Registry URL, making npm publish unusable against any non-AR registry. No remote registry is consulted for @fabricstack/* at PR time. *Nightly product-live lane and the release train*: resolve **only** from Artifact Registry via committed .npmrc scope routing (spec 15 §4.1; needs WP-RN-03/05/07 + WP-PKG-01 exports maps) — the true published-train resolution proof. Both lanes assert no registry.npmjs.org fetch occurred for @fabricstack/* (npm debug-log grep — the runtime counterpart of guards G1–G3). - **Form A (when WP-PKG-03/04 ship)**: install.sh/install.ps1 channel-aware checksum-verified install of the single-node archive; the same downstream stages run unchanged — proving the product, not the transport. - **Cluster profile (Form B)**: the existing kradle deployed-stack e2e + smoke (fabric:.github/workflows/publish.yml:423-424) is extended with stage 5–6 assertions below (chart consumption productized by WP-PKG-09/10). 2. **Create run**: via the orchestrator CLI (today babysitter/babysitter-sdk bins; post-RN-1 orchestrator — spec 15 §3.1 rows 1–2) in the single-node profile; via kradle dispatch (agent-dispatch-controller) in the cluster profile. 3. **Dispatch → agent session**: - **Deterministic lane (PR-gating)**: driven by the **ACP mock agent** (t3:apps/server/scripts/acp-mock-agent.ts, landed by WP-T3-03 AC3) once the ACP lane exists; until then by the existing mock harness (fabric:packages/adapters/harness-mock, whose multi-execution.ts WP-TEST-02 fills). Scripted transcript, zero external credentials, runs on every PR touching the packaging/run/journal/UI paths. - **Live lane (scheduled)**: same chain with a real harness (Claude Code — the strongest green lane, R1 §4.3), nightly alongside §3.2's Tier-1. 4. **Journal**: a run journal exists under the runs dir (.a5c/runs today, .fabric/ post-RN-3 — spec 15 §3.5), its event sequence validates (fabric:packages/babysitter-sdk/src/runtime/replay/effectIndex.ts:144-185 sequence validation), it replays, and kanban source discovery finds the run (fabric:packages/adapters/gateway/src/kanban/lib/source-discovery.ts:12-34). Journal sovereignty holds per WP-T3-16 once the environment server exists (checksum *verification* remains F-01 — invariant §1.4). 5. **Observer/UI assertions**: Playwright (the pattern already proven in-repo by the genty-web-app browser e2e, ci.yml:190-193, and configured-but-ungated in observer-dashboard — fabric:packages/observer-dashboard/playwright.config.ts) asserts against the running stack: the run appears in the observer dashboard, status transitions render, journal events stream into the UI. Cluster profile adds kradle-web org-admin assertions (org's run visible, approval surfaces render). Post-WP-T3-25, console-web run/thread views join the assertion set; commander joins only if F-17 wires it live (mock-only today by design — fabric:apps/commander/README.md, R1 §3.9). 6. **Teardown assertions**: no orphan processes, no stray state outside the versioned runtime layout.
4.2 Workflow placement
New workflow platform-e2e.yml: job product-mock (PR-gating, deterministic, ubuntu + windows — the two OSes with the most path-handling divergence; installs the PR's packed tarballs / seeded local registry per §4.1.1, so the required check gates the code in the PR) and job product-live (nightly, 3 OS, Form C resolved from Artifact Registry; Form A added when WP-PKG-04 lands). The cluster profile lives where its stack lives: the publish.yml e2e extension (§4.1.1 Form B). platform-e2e.yml is a **required PR check** for its product-mock job once stable for 14 days (staged enforcement to avoid day-one gate churn).
---
5. D7(c) — unit floors and full CI gating (design for WP-TEST-01/02/06)
5.1 Orphan-suite wiring map (WP-TEST-01) — every row is mandatory
| Suite (HEAD name) | Volume | Wired into (concrete job/step) | Floor at wiring |
|---|---|---|---|
test:kip (fabric:package.json:95) | 114 test files (R1 §3.5) | **new** ci.yml job memory-catalog (kip is too heavy for the test job's critical path); mirrored in a new publish.yml validate_memory step. kip-inv12-golden-digest.yml (single file at :71) retires into the full suite, keeping the golden-digest test as a named step | suite green; floor per §5.3 |
test:kradle (fabric:package.json:101) | 76 node:test files (MATURITY-MATRIX.md:52) | ci.yml job memory-catalog gains step "kradle-core tests" (PR gating; publish.yml :565-568 already gates releases) | suite green |
test:atlas-catalog-contracts (fabric:package.json:63) + atlas in-package tests | 13 contract files + 31 in-package (R1 §3.6) | ci.yml test job, after the existing atlas build steps (ci.yml:135-138 region) | suite green |
test:library + test:shared (fabric:package.json:73-74) | process-library suites (R1 §3.4) | ci.yml test job, adjacent to test:sdk (:129-132) | suite green |
genty-runtime (19) / genty-cli (10) / genty-ui (18) (MATURITY-MATRIX.md:46-49) | 47 files | ci.yml test job steps beside the existing genty-core/platform steps; genty-ui's 18 files are additionally the WP-T3-24 parity bar (Q-09) | suite green |
genty-tui unit run (40 tests; only verify:release gated today — MATURITY-MATRIX.md:48) | 40 files | ci.yml test job: add a real npm test -w @a5c-ai/genty-tui step beside verify:release (ci.yml:160-163) | suite green |
| kradle-web specs | 30 files (MATURITY-MATRIX.md:54) | new workspace script test:kradle-web; ci.yml memory-catalog job + publish.yml validate_kradle step | suite green; these same specs are WP-T3-22 AC3's local bar |
| adapters-tools | 6 files (MATURITY-MATRIX.md:39) | added to the adapters package-tests enumeration consumed by ci.yml/publish.yml (the enumeration that today omits it) | suite green |
| commander unit tests | 51 files, zero workflow refs (R1 §3.9) | new ci.yml job ops-console (unit portion; Playwright is WP-TEST-05). Runs regardless of the F-17 live-wiring decision — the mock-backend contract tests are valuable as frozen-contract regression either way | suite green |
Rule: a wired suite is invoked **by name** in a workflow step (the MATURITY-MATRIX.md:3 doctrine — in-tree specs no workflow invokes do not count), on PRs (ci.yml) and releases (publish.yml), with the same command a developer runs locally.
5.2 Zero-test package minimum suites (WP-TEST-02)
| Package | Minimum suite (concrete) |
|---|---|
adapters-config (0 tests — MATURITY-MATRIX.md:40) | config load/parse/merge/validation tests over every exported entry point; malformed-input rejection cases; ≥12 tests |
adapters-launch (0 tests) | launch-arg construction per supported target + spawn smoke against a stub executable; failure-path (missing binary) typed error; ≥8 tests |
kradle-sdk (0 tests — MATURITY-MATRIX.md:53; spec 15 §3.2 notes "rename does not launder that; Q-02 owns floors") | client construction, auth header injection, typed request/response round-trip against a stub server, error taxonomy; ≥15 tests |
kradle-cli (0 tests) | command parse matrix + exit-code smoke per subcommand; ≥10 tests |
adapters/harness-mock — src/multi-execution.ts is **0 lines** (R1 §3.1) | the file is either implemented (multi-execution scenarios in the mock harness, with tests driving §4.1.3's deterministic lane) or deleted with the export removed and the decision recorded in the WP closing note; an empty exported module may not survive |
Every minimum suite runs under the standard workspace test script and is picked up by the §5.1 job map automatically.
5.3 Floors: mechanism and numbers (WP-TEST-06)
- **Floor registry**: committed
testing/floors.json— per workspace:{testFiles, lineCoveragePct}. Enforced by a newscripts/check-test-floors.cjsrun in theci.ymltestjob and in everypublish.ymlvalidate job (same pattern as the existing guardrail scriptscheck-binary-renames.cjs/check-architecture-boundaries.cjs,fabric:package.json:65-67). - **Ratchet discipline**: floors only move up.
testFilesfloor = the measured count at wiring time (baselines at HEAD: babysitter-sdk 173 · adapters 353 · genty 251 · kradle 130 · kip-sdk 114 · observer-dashboard 105 · commander 51 · atlas 31 · trust-core 8 — R1 §4.4);lineCoveragePctfloor = measured-at-wiring rounded **down** to the nearest 5, raised as suites grow. Deleting tests below floor fails CI; a deliberate reduction requires editingfloors.jsonin the same PR, which is review-visible by construction. - **Named minimum raises** (explicit, not ratchet-derived): trust-core is security-critical with only 8 test files (R1 §3.7) — floor set to **25** test files covering chain verification, revocation, expiry, and root-kind paths (the H9/F-08 fix adds its own tests on top); the four §5.2 packages get the stated per-package minimums.
- **Coverage tooling**: vitest
--coverage(v8) for vitest workspaces;node:testworkspaces (kradle-core) use--experimental-test-coverageoutput mapped into the same floors file schema. - **
MATURITY-MATRIX.mdrefresh**: WP-TEST-06 regenerates the matrix's "Automated test signals" claims from the floors registry + workflow greps, replacing the 2026-08-13 snapshot with a generated section so the matrix cannot silently rot again.
5.4 Per-test timeout policy (binding, all suites)
1. **Config-owned timeouts only**: unit configs testTimeout ≤ 5000ms; integration configs ≤ 120000ms; live-lane scenario deadlines stay in the live-stack runner. Inline per-test timeout raises ({ timeout: n } above the config cap, test.setTimeout) are forbidden. 2. **No real-time sleeps in tests**: await new Promise(r => setTimeout(r, n)) and equivalent sleep helpers are banned in test files. Allowed waits: fake timers; receipt waits and queue-drain waits (§6.1); event/promise conditions; a single shared bounded-poll helper that requires an explicit deadline and reason string. Enforcement: a lint rule + scripts/check-no-sleeps-in-tests.cjs grep gate wired beside the floors check (the doctrine imported from t3code — tests wait on async milestones instead of sleeping: drainable workers + runtime receipts, t3:docs/internals/overview.md:86-92; enforcement-by-custom-lint precedent: t3:oxlint-plugin-t3code, R2 §2). 3. **Timeout ≠ synchronization**: per §3.5.5, raising timeouts is never an accepted flake fix.
---
6. Testing-pattern adoptions from t3code (design for WP-TEST-07/09)
6.1 Receipts and queue-drain instead of sleeps
- **Typed runtime receipts**: t3code's test-only
RuntimeReceiptBus(production layer a no-op) lets tests await async milestones (t3:docs/internals/overview.md:89-92). It arrives in the environment server via WP-T3-19 (AC7). WP-TEST-07 additionally lands a **plain-TS receipts utility** (B4-compliant — no Effect) insidepackages/babysitter-sdk/src/testing/for fabric-core async suites (journal commit milestones, hook dispatch completion, background-effect settlement — today's sleep-prone spots). No new package is minted, keeping spec 15 §3 closed under its own matrix (same rule spec 10 §2 row 11 applies to cherry-picked modules). - **DrainableWorker**:
t3:packages/shared/src/DrainableWorker.tsis cherry-picked into its adopters by spec 10 (rows land underapps/environment/src/shared/,packages/client-runtime/src/shared/,infra/relay/src/shared/— spec 10 §2 row 11); WP-TEST-07 owns converting fabric's existing sleep-based waits in the suites it touches to drain waits and wiring the §5.4.2 ban so regressions cannot land.
6.2 ACP mock agent as the deterministic protocol fixture
t3:apps/server/scripts/acp-mock-agent.ts (scripted ACP agent) lands with WP-T3-03 (AC3: initialize → session/new → prompt-turn → session/update in CI). WP-TEST-07 promotes it from a package-local script to the **shared deterministic driver**: transcript-file-driven (recorded session/update sequences replayed byte-stable), used by (a) the ACP lane runtime suite (WP-T3-06 AC1), (b) the fabric-as-ACP-agent conformance suite inverted as a mock *client* (WP-T3-18 AC1), and (c) the §4.1.3 deterministic whole-platform lane.
6.3 Release smoke
t3:scripts/release-smoke.ts (R2 §14) is adapted as scripts/release-smoke.mjs for the fabric train: after WP-RN-07's AR publish, a clean-machine job installs @fabricstack/orchestrator-sdk + @fabricstack/harness-cli from Artifact Registry via scope routing only, runs each §3.1-listed bin's --version/--help, creates and replays a trivial run. This *is* the "published-consumer validation job" WP-RN-07's test requirements name — WP-TEST-07 supplies the suite; WP-RN-07 wires it into publish-npm-private.yml.
6.4 Migration parity fixtures (Q-09 anchor set)
Spec 10 defines three parity bars and names this spec their gate (spec 10 §10): AgentEvent projection goldens (WP-T3-14 AC1), gateway-frame↔contracts-RPC equivalence (WP-T3-21 AC1, over the §4.5 mapping table of spec 10), genty-ui hook-API parity (WP-T3-24 AC1, the 18 genty-ui test files). WP-TEST-09 owns the **shared fixture infrastructure**: the recorded-transcript store format (one directory per lane under packages/adapters/core/src/projection/__tests__/fixtures/, per WP-T3-14's layout), golden-diff runner, and the CI jobs that keep all three suites gating from the moment their anchor WPs land until the migration completes. Its groundwork additionally owns the **legacy-codec baseline corpus**: WP-T3-13 AC2's before/after golden diff needs "before" recordings that can only exist while the legacy codecs still run, so WP-TEST-09 captures that corpus during Phase 2 at a pinned pre-lane commit, before any codec is retired or migrated (WP-TEST-09 AC6).
---
7. Sequencing
- **Immediately (no migration dependency)**: WP-TEST-01, WP-TEST-02, WP-TEST-06 (unit floors are pure debt-payment on today's tree); WP-TEST-03 stage 1 (OS axis,
harness-matrix.yml, auth model, flake policy — over the current green lanes). - **As soon as a deterministic driver exists (WP-TEST-02 first, WP-T3-03's ACP mock agent thereafter)**: WP-TEST-04's PR-gating
product-mocklane — its packed-tarball install (§4.1.1) needs no registry, so PR gating does not wait on Artifact Registry provisioning or the AR-published train (it does assume the Phase-1 rename has landed, which precedes its Phase-2 activation). **After WP-RN-03/05/07**: WP-TEST-04's AR-resolvingproduct-livelane and release-train Form C; WP-TEST-07's release smoke. - **Lane-coupled**: WP-TEST-03's Grok/Cursor/OpenCode/Antigravity green-ness arrives with WP-T3-07/08/09/12 (+F-18); WP-TEST-09 activates per anchor WP (T3-14/21/24), with its Phase-2 groundwork (fixture store + legacy-codec baseline corpus, AC6) completing before any codec retirement; WP-TEST-07's receipts-in-environment rides WP-T3-19.
- **Last**: WP-TEST-08 (recommended tier), after F-01/F-04 create the behavior to attack.
- D11 is honored: nothing here blocks migration WPs; spec 50's R-01 owns the merged graph.
---
8. Work packages
WP-TEST-01: CI gating of every orphaned suite (TEST1 residual — D7(c))
- **Objective**: Wire all §5.1 orphan suites into PR and release CI by name, per the wiring map:
test:kip(114 files),test:kradle(PR-side),test:atlas-catalog-contracts+ atlas's 31 tests,test:library/test:shared, genty-runtime/cli/ui (47), genty-tui unit run (40), kradle-web (30 specs), adapters-tools (6), commander unit tests (51) — so that zero test suites exist that no workflow invokes (closing R1 §4.4 items 1–8). - **Affected paths**:
fabric:.github/workflows/ci.yml(testjob:54gains library/shared/genty/tui/atlas-contract steps; new jobsmemory-catalogandops-console);fabric:.github/workflows/publish.yml(newvalidate_memorystep block;validate_kradle:517gains kradle-web;validate_core:203gains library/shared);fabric:.github/workflows/kip-inv12-golden-digest.yml(retired into the full-suite job, golden-digest kept as a named step);fabric:package.json(newtest:kradle-webscript; adapters test-enumeration extended topackages/adapters/tools);fabric:packages/kradle/web/package.json(test script). - **Acceptance criteria**: (1) for every §5.1 row, a workflow grep finds the suite invoked by name in both a PR-gating and a release-gating job (the TEST1 verification inverted — R1 §2 TEST1 evidence method); (2) all wired suites green at merge; any red test found during wiring is fixed or quarantined via an entry referencing an issue — never deleted or skipped silently; (3)
kip-inv12-golden-digest.ymlno longer exists as a standalone single-file gate and the golden-digest test still runs; (4) total PR wall-clock impact measured and recorded: the new jobs run in parallel totest(:54), keeping PR latency increase ≤ the slowest new job; (5) suites keep gating after RN-1 renames (workspace filters keyed by directory, not scope name, or updated in the WP-RN-05 sweep — spec 15 §7). - **Test requirements**: the wiring itself is verified by a meta-test extending
scripts/check-test-floors.cjs(WP-TEST-06): every workspace with ≥1 test file must be reachable from at least one workflow step (negative fixture: an orphan workspace trips it). - **Dependencies**: none to start; WP-TEST-06 for the meta-gate; coordinates with WP-RN-05 (name sweep).
- **Size**: L
WP-TEST-02: Minimum suites for zero-test packages + mock-harness completion
- **Objective**: adapters-config, adapters-launch, kradle-sdk, kradle-cli each gain the §5.2 minimum suite (≥12/≥8/≥15/≥10 tests respectively, entry-point-complete); the empty
multi-execution.ts(0 lines) is implemented-with-tests or removed-with-record; no L0/L2-with-zero-tests workspace remains outside the retirement list. - **Affected paths**:
fabric:packages/adapters/config/(newsrc/__tests__/, vitest config);fabric:packages/adapters/launch/(same);fabric:packages/kradle/sdk/,fabric:packages/kradle/cli/(new test dirs +testscripts);fabric:packages/adapters/harness-mock/src/multi-execution.ts(fill or delete);testing/floors.json(four new rows at the §5.2 minimums). - **Acceptance criteria**: (1) each of the four packages' suites runs via its workspace
testscript and is picked up by the §5.1 job map (proof: the WP-TEST-06 orphan meta-gate passes with the new suites present, and would fail if a suite were unplugged); (2) per-package test counts meet the §5.2 minimums and every exported entry point is imported by at least one test (export-coverage assertion in each suite); (3)multi-execution.tsis non-empty-and-tested or absent-and-unexported, with the decision recorded in the WP closing note; if implemented, the §4.1.3 deterministic lane consumes it; (4)MATURITY-MATRIX.mdrows for the four packages move off L0/L2-zero-tests in the WP-TEST-06 regenerated matrix. - **Test requirements**: the minimum suites themselves; export-coverage assertions; floors entries active in
check-test-floors.cjs. - **Dependencies**: WP-TEST-06 (floors registry exists); none from spec 10.
- **Size**: M
WP-TEST-03: Live per-harness e2e matrix to D7(a) — six harnesses per OS
- **Objective**: Build §3 in full: OS becomes a matrix axis in
live-stack.yml; newharness-matrix.ymlschedules Tier-1 (the six D7(a) harnesses × declared OS × {vanilla-NI, bp-bridged-hooks}, native auth) nightly and Tier-2 (the 9 extra lanes per spec 10 §3.2: gemini, copilot, pi, omp, openclaw, hermes, droid, amp subprocess + genty ACP-agent; qwen exempt-by-record) nightly-ubuntu/weekly-per-OS; per-harness auth fixtures per §3.4 under the private-repo secret model; OS-support declarations per §3.3; flake policy + quarantine per §3.5;publish.ymlfreshness gate;live-stack-published.ymlretargeted at the AR-published train. - **Affected paths**:
fabric:.github/workflows/live-stack.yml(:17-21,44os-input removal; matrix generator:130-233gainsosper combination and agrokagent-map row once WP-T3-08 lands; lane jobs:327,549,565,581runs-onfrom matrix); newfabric:.github/workflows/harness-matrix.yml;fabric:.github/workflows/publish.yml(freshness-gate job before validate fan-out);fabric:.github/workflows/live-stack-published.yml(AR retarget);fabric:.github/workflows/{failure-triage-live-stack,live-stack-gardening}.yml(extend to the new workflow; pass-rate ledger); newe2e-tests/harness-os-support.json,e2e-tests/quarantine.json,e2e-tests/harness-credentials.md; GitHub environmentlive-harnesses(repo settings; secret names per §3.4). - **Acceptance criteria**: (1) a dispatched
harness-matrix.ymlrun executes lanes on ubuntu, macos, and windows **within one run** (OS-as-axis proven); (2) Claude Code and Codex Tier-1 lanes green on all three OSes under native auth (subscription/login secrets), with the credential-canary artifact scan passing (§3.4) — at the stage-1/M1 gate, an individual not-yet-green OS cell may satisfy this criterion **only** as afixture-blockeddeclaration per §3.3 (open issue + unexpiredexpiresAt, ≤14 days, max 2 renewals; neverunsupported-by-vendor), and an expired or renewal-exhausted fixture-blocked cell reverts to gating-red, so the escape is time-boxed rather than a silent waiver; (3) Cursor, OpenCode, Antigravity, Grok lanes exist in the matrix, re-point at their spec-10 lanes as those land, and their per-OS cells carry declarations per §3.3 — no silent skips anywhere (every non-running cell isunsupported-by-vendorwith evidence,blockedwith an issue, orfixture-blockedwith an open issue and unexpiredexpiresAt); (4) the flake machinery works end-to-end: a seeded flaky fixture lane crosses the <90%/14-run threshold, gets auto-proposed for quarantine, stops gating while still running, and exits after 5 greens (rehearsed with a synthetic lane); (5)publish.ymlblocks on a stale (>24h) or red Tier-1 matrix (negative rehearsal recorded); (6) **D7(a) exit**: all six Tier-1 harnesses green per their OS-support declarations, quarantine-free — jointly delivered with WP-T3-07/08/09/10/11/12 and F-18, tracked as this WP's completion gate; (7) no workflow or secret this WP creates references the public repo or public npmjs (D17/D21 audit note; install steps resolve from AR only). - **Test requirements**: matrix-generator unit tests (os axis, grok row, exemption records); quarantine lifecycle rehearsal; declaration-state validation tests (a
fixture-blockedcell missing its issue orexpiresAtfails config validation; an expired or renewal-exhausted entry flips the cell to gating-red); freshness-gate negative rehearsal; per-harness auth preflights (probe steps per §3.4) withauth-expiredclassification tests in the triage workflow. - **Dependencies**: stage 1 (axis/workflow/auth/flake over current lanes): none. Full six-green: WP-T3-07, WP-T3-08 (Grok — net-new, mandatory), WP-T3-09, WP-T3-10, WP-T3-11, WP-T3-12; F-18 (spec 20) for red-lane remediation; WP-RN-03/07 for the AR retarget of
live-stack-published.yml. - **Size**: L
WP-TEST-04: Whole-platform e2e — packaged install → run → dispatch → session → journal → UI (D7(b))
- **Objective**: Build the §4 product test as
platform-e2e.yml: deterministic mock-agent lane (PR-gating, ubuntu + windows, installing the PR's own workspace-packed tarballs / seeded local registry per §4.1.1 so the check gates the code it runs on) and nightly live lane (3 OS, Form C npm-from-AR — the published-train resolution proof), Form A (installer) when WP-PKG-03/04 ship, with the cluster profile landing as an extension of the deployed-stack e2e (publish.yml:423-424). No such lane exists today (R1 spec input 5) — this WP creates the only test that exercises the platform as one product. - **Affected paths**: new
fabric:.github/workflows/platform-e2e.yml; newe2e-tests/platform/suite (install-stage drivers: PR tarball pack + local-registry seeding, Form C AR routing, Form A installer; run-creation, dispatch, journal assertions, Playwright UI assertions);fabric:.github/workflows/publish.yml(cluster-profile UI-assertion extension to the existing e2e+smoke steps at:423-424);.npmrcfixtures — AR scope routing (per spec 15 §4.1) for the live lane, loopback local-registry routing for the mock lane; observer-dashboard Playwright project consumed as a library of assertions (config atfabric:packages/observer-dashboard/playwright.config.ts). - **Acceptance criteria**: (1) the deterministic lane completes the full six-stage chain of §4.1 on a clean runner with **zero external credentials and zero remote-registry access for
@fabricstack/***: packed-tarball install (every@fabricstack/*packagenpm packed from the PR workspace into a clean prefix, or the per-run loopback local-registry fixture seeded only from the PR build — §4.1.1) → CLI run creation → dispatch → mock agent session (ACP mock agent per §6.2, or the WP-TEST-02 mock harness pre-ACP) → journal exists+validates+replays+is-discovered (effectIndex.ts:144-185;source-discovery.ts:12-34) → observer UI shows the run with streamed events (Playwright assertions); (2) install-provenance proofs per lane: theproduct-mockinstall demonstrably resolves every@fabricstack/*package to a tarball packed from the PR's own checkout (the pack step stamps the PR head SHA into each packed manifest — e.g. agitHead/build-metadata field — and a post-install assertion verifies that SHA in the installed tree, so a stale-train install fails the check), and npm logs for **both** lanes show noregistry.npmjs.orgfetch for@fabricstack/*(runtime guard complementing G1–G3, spec 15 §4.3); (3) the nightly live lane passes the same chain with Claude Code on 3 OSes, resolving@fabricstack/***only** from Artifact Registry via the committed.npmrcscope routing — the AR-resolution obligation of spec 15 §7 is carried by this lane and by the §6.3 release smoke, never by the PR lane; (4) when WP-PKG-04 lands, the Form A path (checksum-verified installer → same stages 2–6) runs green with **no changes to stages 2–6** (product-not-transport proof); (5) the cluster profile asserts stage 5–6 (run visible in kradle-web org views + observer) inside the deployed-stack e2e; (6)product-mockbecomes a required PR check after 14 stable days (staged enforcement recorded); (7) teardown leaves no orphan processes or out-of-layout state (assertion inalways()). - **Test requirements**: the platform suite itself; a chaos case (kill the agent mid-session → run reaches a terminal failure state, journal remains valid, UI shows the failure — no hangs); journal-replay determinism assertion; Playwright assertion library shared with WP-TEST-05.
- **Dependencies**: WP-RN-03/05/07 + WP-PKG-01 (AR-resolving
product-livelane and release-train Form C only — theproduct-mocktarball lane needs just the workspace build plus a deterministic driver, so PR gating activates without waiting on AR); WP-T3-03 (ACP mock agent) or WP-TEST-02 (mock harness) for the deterministic driver; WP-TEST-05 (UI assertion harnesses); WP-PKG-03/04 (Form A stage); WP-PKG-09/10 (cluster chart consumption); WP-T3-16 (journal sovereignty, once the environment server participates). - **Size**: L
WP-TEST-05: UI e2e gating — observer, kradle-web, ops-console, and the parity handoff
- **Objective**: Every UI surface with an e2e harness is CI-gated: observer-dashboard's existing-but-uninvoked Playwright e2e (R1 §3.8) enters CI; commander's Playwright suite runs against its seeded mock backend (contracts-regression value independent of the F-17 disposition); kradle-web browser-level smoke joins the cluster profile; the genty-web-app Playwright gate (
ci.yml:190-193) is maintained until WP-T3-25's parity bar consumes it, then hands off to the console-web parity suite (WP-T3-25 AC1) without a coverage gap. - **Affected paths**:
fabric:.github/workflows/ci.ymlobserver-dashboardjob (:459) gains a Playwright step invoking the in-package config (packages/observer-dashboard/playwright.config.ts,e2e/);ops-consolejob (from WP-TEST-01) gains the commander Playwright step (fabric:apps/commanderin-package Playwright — R1 §3.9);publish.ymlvalidate_observer_and_compiler:578mirrors the observer e2e; the §4 Playwright assertion library shared frome2e-tests/platform/; a handoff checklist item in WP-T3-25/27's closing notes referencing this WP's gate inventory. - **Acceptance criteria**: (1) observer-dashboard Playwright runs headless in PR CI and gates (INFERRED-ungated status in R1 §6 resolved to gated-confirmed); (2) commander Playwright green in CI against the mock backend; if F-17 later wires it live, the same suite re-points at the live profile without rewriting scenarios; (3) kradle-web smoke assertions run inside the cluster-profile e2e (with WP-TEST-04 AC5); (4) at no commit between now and WP-T3-27 is the repo without a gating browser e2e for the primary run/thread UX: genty-web-app's gate is removed **only** in the same PR that makes console-web's parity suite (WP-T3-25 AC1) a required check (gap-free handoff, verifiable in the PR diff); (5) UI e2e suites obey the §5.4 timeout policy (no sleep-based waits; Playwright auto-waiting + explicit expect conditions only).
- **Test requirements**: the wired suites; a headless-CI stability burn-in (14 consecutive green scheduled runs before required-check promotion, per §3.5 discipline).
- **Dependencies**: WP-TEST-01 (
ops-consolejob exists); handoff leg: WP-T3-25, WP-T3-27. Independent of the rename train. - **Size**: M
WP-TEST-06: Floors registry, coverage gates, timeout policy — the D7(c) enforcement spine
- **Objective**: Land
testing/floors.json+scripts/check-test-floors.cjs(ratchet-only floors: test-file count + line-coverage per §5.3, baselines from R1 §4.4, trust-core raised to 25), the orphan-workspace meta-gate (every workspace with tests must be workflow-reachable), the §5.4 per-test timeout policy (config caps, inline-raise ban, real-sleep ban withscripts/check-no-sleeps-in-tests.cjs+ lint rule), and the regeneratedMATURITY-MATRIX.mdso maturity claims derive from the registry instead of a hand-updated snapshot (stale since 2026-08-13 —MATURITY-MATRIX.md:5). - **Affected paths**: new
testing/floors.json; newscripts/check-test-floors.cjs,scripts/check-no-sleeps-in-tests.cjs;fabric:.github/workflows/ci.ymltestjob + everypublish.ymlvalidate job (floor + sleep-ban steps, beside the existing guardrails atci.yml:99-102,114-117); root and per-package vitest configs (testTimeoutcaps; coverage reporters); kradle-core node:test invocation (publish.yml:565-568) gains--experimental-test-coveragemapping;MATURITY-MATRIX.md(generated section + generator script). - **Acceptance criteria**: (1) floors active for every workspace in the §5.1/§5.2 maps; negative fixtures prove each gate: deleting a test file below floor fails CI, a coverage drop below floor fails, an orphan workspace fails, an inline
{ timeout: 300000 }fails, asetTimeout-sleep in a test file fails; (2) floors are ratchet-only: the checker rejects afloors.jsonedit that lowers a value unless the PR carries afloors-reductionlabel (review-visible escape hatch, usage logged); (3) trust-core floor of 25 test files is met (tests added under this WP cover chain verify/revoke/expiry paths that exist today; therequiredRootKindtests ride F-08); (4) regeneratedMATURITY-MATRIX.mdmatches reality by construction (generator re-run in CI, diff must be empty); (5) the full gate set adds ≤90s to thetestjob's wall-clock (measured, recorded). - **Test requirements**: negative-fixture suite for all five gates; generator idempotence test; coverage-schema mapping tests for both vitest and node:test outputs.
- **Dependencies**: none (first mover of part (c)); consumed by WP-TEST-01/02.
- **Size**: M
WP-TEST-07: t3code test-pattern adoption — receipts, queue-drain, mock agent, release smoke
- **Objective**: Import the four t3code patterns as working fabric infrastructure per §6: (a) plain-TS runtime receipts in
packages/babysitter-sdk/src/testing/(B4-compliant; production no-op — pattern pert3:docs/internals/overview.md:89-92) and adoption of the environment server's receipt bus (WP-T3-19 AC7) in its suites; (b) DrainableWorker drain-waits replacing sleep-based waits in the suites this WP converts (source modules land via spec 10 §2 row 11); (c) the ACP mock agent (t3:apps/server/scripts/acp-mock-agent.ts, landed by WP-T3-03) promoted to the shared transcript-driven deterministic driver used by lane suites, the ACP-agent conformance suite, and WP-TEST-04's mock lane; (d)scripts/release-smoke.mjsadapted fromt3:scripts/release-smoke.tsas the AR published-consumer smoke WP-RN-07 wires. - **Affected paths**: new
fabric:packages/babysitter-sdk/src/testing/{receipts.ts,drain.ts}(+ exports entry limited to test consumption); sleep-wait conversions inpackages/babysitter-sdksuites (hook dispatcher, background effects, session cleanup — today's real-time-wait hotspots per the R1 §2 H5/M5/M6 cluster's test surroundings); mock-agent transcript format + fixtures underpackages/acp/test tree (post-WP-T3-03 home); newscripts/release-smoke.mjs;fabric:.github/workflows/publish-npm-private.yml(smoke step — file created by WP-RN-07). - **Acceptance criteria**: (1) receipts utility used by ≥5 previously sleep-based babysitter-sdk tests, with the sleeps gone and the §5.4.2 ban green over the package; (2) drain-wait pattern demonstrated in at least the gateway run-manager suite (fan-out settlement without sleeps); (3) the mock agent replays a committed transcript byte-stably across two consecutive CI runs (determinism proof) and is consumed by WP-TEST-04's mock lane; (4) release smoke passes against a real staging AR publish: scope-routed install, bin
--versionsweep over the spec 15 §3.1 bin column, trivial run created and replayed; (5) zero new packages minted (spec 15 §3 closure preserved); (6) no Effect import enterspackages/babysitter-sdk(B4 boundary check). - **Test requirements**: the converted suites themselves; transcript determinism test; smoke rehearsal on a staging train (link in closing note).
- **Dependencies**: WP-T3-03 (mock agent lands), WP-T3-19 (environment receipts — that leg only), WP-RN-07 (smoke wiring); the receipts/drain/babysitter-sdk legs have no migration dependency and start immediately.
- **Size**: M
WP-TEST-08: Adversarial ledger suite — recommended, explicitly outside the D7 bar (TEST2)
- **Objective**: Attack-style tests for journal integrity once F-01/F-04 land the behavior: tamper (mutate an event payload → load fails checksum verification), truncation (delete tail files → hash-chain/head-checksum detection fires), value-drift (change an effect arg value → replay refuses the stale result instead of silently reusing it — the H2 class), plus signature tamper against F-02's signed events. Extends the only existing tamper precedent (
fabric:packages/babysitter-sdk/src/breakpoints/__tests__/enforce-signed-breakpoint.test.ts— R1 §2 TEST2) from breakpoints to the whole ledger. - **Affected paths**: new
fabric:packages/babysitter-sdk/src/storage/__tests__/adversarial/(tamper/truncate/drift/signature suites overstorage/journal.ts+runtime/replay/effectIndex.ts); fixture corpus of valid journals with scripted corruptions;testing/floors.json(suite counted once landed). - **Acceptance criteria**: (1) every attack class above is **proven to fail pre-fix behavior** — and because this WP executes in Phase 5, after F-01/F-02/F-04 have already merged (spec 50 places them in Phases 1–2), the proof is executed against the **pinned pre-fix commit** of the class's owning F-WP: the first parent of the merge commit that landed that F-WP (recorded in the F-WP's closing note where present; otherwise recovered deterministically from git history), checked out in a throwaway worktree where the class's test is run and its **failing** output archived as evidence in this WP's closing note, alongside the same test **passing** at HEAD; a class whose per-class test already rode its F-WP's own PR train (e.g. F-01's tamper suite, green at M1 — spec 50 M1 exit 6) may cite that train's recorded red→green transition instead of re-executing the pinned-commit run; (2) corruption fixtures are generated, not hand-frozen (generator script committed), so canonicalization changes (S6/F-01) regenerate them; (3) the suite is CI-gated like any other (§5.1 rule) once landed; (4) the spec-level label "recommended, not part of the D7 bar" is preserved in the suite README and the R-03 compliance matrix row.
- **Test requirements**: the adversarial suites; generator determinism test; the pinned-commit worktree harness (checkout → run class test → capture failing evidence) rehearsed against F-01's pinned pre-fix commit and the transcript attached to the closing note.
- **Dependencies**: F-01, F-02, F-04 (spec 20) — hard dependencies; nothing in this WP blocks anything else.
- **Size**: M
WP-TEST-09: Migration parity suites — projection goldens, frame↔RPC equivalence, hook-API parity
- **Objective**: Own the shared fixture infrastructure and CI gating for the three spec-10 parity bars (spec 10 §10 names this WP their anchor): (a) AgentEvent projection golden tests — recorded native transcripts per lane → expected
AgentEventsequences over the **full 67-member union** (all 67 members extend BaseEvent — 49 defined in events.ts plus 18 in events-control.ts; the dossier's 49 counted only events.ts —fabric:packages/adapters/core/src/events.ts:402-411; golden scope covers all 67; WP-T3-14 AC1); (b) gateway-frame↔contracts-RPC equivalence over the spec 10 §4.5 mapping table — same stimulus, equivalent observable result on old frame vs new RPC (WP-T3-21 AC1); (c) genty-ui hook-API parity — the 18 genty-ui test files green against the WP-T3-24 compat layer with only import-path changes (WP-T3-24 AC1); plus (d) the **legacy-codec baseline corpus** as Phase-2 groundwork: native transcripts and legacy-path projected goldens for every retain-behind-contract codec in WP-T3-13's disposition table, captured at a pinned pre-lane commit **before any codec is retired or migrated** — the "before" side of WP-T3-13 AC2's before/after golden diff, which no other WP owns capturing. These suites are the mechanical proof that D6's projection direction and D10's break-freely policy destroyed no consumer semantics. - **Affected paths**: fixture store + golden-diff runner under
fabric:packages/adapters/core/src/projection/__tests__/fixtures/(layout per WP-T3-14), including alegacy-baseline/subtree with a manifest pinning the capture commit SHA; equivalence harness underfabric:packages/adapters/gateway/src/__tests__/equivalence/(retired with the gateway at WP-T3-21 completion, suites migrating to the contracts side); genty-ui suite wiring (already CI-entering via WP-TEST-01);fabric:.github/workflows/ci.yml(parity steps activate per anchor WP; marked required once their anchor lands). - **Acceptance criteria**: (1) fixture format documented and shared: one transcript corpus drives both the projection goldens and the lane unit suites of WP-T3-07…13 (no per-WP fixture dialects); (2) each parity suite becomes a required check in the same PR that lands its anchor WP (T3-14/21/24) and stays required until the corresponding legacy surface is deleted (codecs at WP-T3-13,
protocol/v1.tsat WP-T3-21, genty-ui protocol copy at WP-T3-27); (3) golden updates require agolden-updatePR label with the regenerating command recorded (no silent re-baselining); (4) at WP-T3-21 completion, the equivalence suite's assertions convert to contracts-side regression tests rather than being deleted with the gateway (survival recorded in the WP closing note); (5) all three suites obey §5.4 (transcript-replay, no sleeps); (6) the legacy-codec baseline corpus exists **before** the first codec retirement/migration merge (WP-T3-07/10/11/12/13): captured during Phase 2 at a pinned pre-lane commit whose SHA is committed in the corpus manifest, covering every retain-behind-contract codec of WP-T3-13's disposition table (the 12 retained lanes), each entry a recorded native transcript plus its legacy-path projectedAgentEventgolden — WP-T3-13 AC2's "before" side reads exclusively from this corpus (the cross-reference recorded in both WPs' closing notes), and the golden-diff runner rejects a "before" fixture whose manifest SHA postdates the first lane-WP merge, so the baseline cannot be silently re-recorded post-migration. - **Test requirements**: fixture-format round-trip test; golden-diff runner unit tests; corpus-manifest pin check (manifest SHA ordering vs lane-WP merge commits, negative fixture included); a seeded intentional-regression rehearsal per suite (mutated projection/RPC/hook behavior is caught).
- **Dependencies**: WP-T3-14 (leg a), WP-T3-21 (leg b), WP-T3-24 (leg c); WP-TEST-01 (genty-ui suite wired); fixture-infrastructure groundwork has no dependency; the AC6 corpus capture is Phase-2 groundwork that must complete before WP-T3-07/10/11/12/13 begin retiring or migrating codecs (spec 50 already schedules "WP-TEST-09 fixtures beside Phase 2").
- **Size**: M
---
9. Cross-spec consistency (D21 checklist for Gate B)
| Obligation | Honored by |
|---|---|
| Spec 15 §7: floors attach to renamed packages; whole-platform e2e installs from Artifact Registry, never npmjs | Invariant §1.1/§1.2; WP-TEST-01 AC5; WP-TEST-04 AC2/AC3 (AR resolution proven in the live lane and release train; the PR mock lane installs PR-packed tarballs and touches no remote registry for @fabricstack/* at all — strictly stronger than the never-npmjs obligation) |
| Spec 10 §10: Q-07 seeded by WP-T3-03/19; Q-09 anchored by WP-T3-14/21/24; WP-T3-13 AC2's "before" goldens supplied by the Phase-2 legacy-codec baseline corpus | WP-TEST-07 dependencies; WP-TEST-09 acceptance (AC6)/dependencies |
| Spec 10 lane WPs carry "live-stack lane green" acceptance; this spec supplies matrix + gate | §3.1 staging; WP-TEST-03 AC3/AC6 |
Spec 20: F-18 (red lanes) joint ownership; F-17 (commander) disposition independent of its suites entering CI; F-01/F-02/F-04 precede the adversarial suite; TEST1→Q-01, TEST2→Q-08 absorbed (synthesis-adoption-proposal.md §3) | WP-TEST-03, WP-TEST-01/05, WP-TEST-08 |
| Spec 40: install forms — WP-PKG-01/03/04 (Forms C/A) and WP-PKG-09/10 (Form B) consumed by the whole-platform test; release smoke feeds WP-RN-07/WP-PKG-11 trains | WP-TEST-04 §4.1; WP-TEST-07 AC4 |
| D17/D19/D21: no public-repo or public-npm touchpoints anywhere in testing CI; harness secrets private-only | Invariant §1.1; WP-TEST-03 AC7 |
| D13: no mobile lanes | Invariant §1.5 |
| Intent acceptance criterion 3 (WP format) and 5 (no placeholder text) | §8 throughout |
Work-package count in this spec: **9** (WP-TEST-01 … WP-TEST-09).