Agentic AI Atlasby a5c.ai
OverviewWikiGraphFor AgentsEdgesSearchWorkspace
/
GitHubDocsDiscord
iiRecord
Agentic AI Atlas · Spec 30 — Testing (WP-TEST-*)
page:docs-specs-platform-gaps-30-testinga5c.ai
Search record views/
Record · tabs

Available views

II.Record viewspp. 1 - 1
overviewarticlejsongraph
III.Related pagespp. 1 - 1
II.
Page reference

page:docs-specs-platform-gaps-30-testing

Reading · 42 min

Spec 30 — Testing (WP-TEST-*) reference

Date: 2026-09-27 · Status: Gate-B draft · Binding decisions: D7 (the testing bar), constrained by D4 (spec only, no implementation), D11 (testing follows migration + coverage in the roadmap), D13 (mobile deferred — no mobile lanes anywhere below), and amendment D17–D21 (CI/publish isolation from public pipelines; private Artifact Registry) (docs/specs/platform-gaps/00-intent.md:19-46).

Pagewiki/docs/specs/platform-gaps/30-testing.mdOutgoing · 0Incoming · 1

Spec 30 — Testing (WP-TEST-*)

Date: 2026-09-27 · Status: Gate-B draft · Binding decisions: **D7** (the testing bar), constrained by **D4** (spec only, no implementation), **D11** (testing follows migration + coverage in the roadmap), **D13** (mobile deferred — no mobile lanes anywhere below), and amendment **D17–D21** (CI/publish isolation from public pipelines; private Artifact Registry) (docs/specs/platform-gaps/00-intent.md:19-46).

Inputs consumed in full: the Gate-A-approved adoption proposal research/synthesis-adoption-proposal.md (planned 9-WP inventory §4.3; survivor register §3 rows TEST1→Q-01 and TEST2→Q-08); dossiers R1 research/fabric-current-state.md (test/CI reality §4, spec inputs 4–6, 13), R2 research/t3code-architecture.md (testing patterns §3.1, §14, spec input 11); sibling specs 10-t3code-migration.md (lane WPs, §10 cross-refs to Q-07/Q-09) and 15-private-isolation-rename.md (§7 obligations on this spec: floors attach to renamed packages; whole-platform e2e installs from Artifact Registry, never npmjs). Baseline document: MATURITY-MATRIX.md (grading doctrine at MATURITY-MATRIX.md:3, levels L0–L4 at :13-17), refreshed by R1 §4. Citations: fabric: paths relative to C:/work/fabric at HEAD 5e46b998e; t3: paths relative to the MIT t3code snapshot at C:/Users/tmusk/AppData/Local/Temp/claude/C--work-company/368e0399-fd11-4960-8b4d-b4dcf1d79c43/scratchpad/t3code.

**ID convention.** Work packages here are WP-TEST-01 … WP-TEST-09. The short form Q-<n> used by the Gate-A proposal and by specs 10/15/20/40/50 refers to the same packages one-to-one (Q-03 ≡ WP-TEST-03, etc.). The count matches the proposal's planned 9 (synthesis-adoption-proposal.md §4.3); no WPs were added or removed — the sibling specs already cross-reference Q-01/Q-02/Q-04/Q-05/Q-07/Q-09 by these IDs (spec 10 §10; spec 15 §7), so preserving the inventory keeps the Gate-B compliance matrix (R-03) one-to-one. Cross-spec references use the sibling conventions: WP-T3-nn (spec 10), WP-RN-nn (spec 15), F-nn (spec 20), and WP-PKG-nn ≡ P-nn of the Gate-A inventory (spec 40).

---

1. Scope, the D7 bar, and invariants

D7 fixes the bar in three parts, all mandatory:

  • **(a) Live per-harness e2e matrix** — Claude Code, Codex, Cursor, Grok Build, OpenCode, Antigravity, each driven end-to-end per OS (§3, WP-TEST-03).
  • **(b) Whole-platform e2e** — packaged install → create run → dispatch → agent session → journal → UI assertions, as one product test (§4, WP-TEST-04/05).
  • **(c) Unit floors + CI gating** — every package's suite wired into CI with agreed floors (§5, WP-TEST-01/02/06).

An adversarial correctness suite is explicitly **not** part of the bar but is recommended (D7 parenthetical; TEST2); it is WP-TEST-08 and is labeled recommended, not gating.

Binding invariants:

1. **Private-pipeline isolation (D17/D19/D21).** Every workflow, secret, and registry interaction this spec creates lives in the private fabric repo only. No work package touches the public a5c-ai/babysitter repo's workflows or publishing. Harness login/subscription secrets (§3.4) are fabric repo/environment secrets and are never mirrored anywhere public. No install step in any e2e lane ever resolves @fabricstack/* packages from public npmjs (spec 15 §7 obligation on this spec; guards G1–G6 apply to test fixtures too). Where a lane uses remote resolution at all, it comes only from the GCP Artifact Registry npm repo (15-private-isolation-rename.md §4.1); PR-gating deterministic lanes are stricter still — they install the PR's own workspace-packed tarballs (or a per-run local-registry fixture seeded exclusively from the PR build, §4.1.1), so no remote registry is consulted for @fabricstack/* at all and the check gates the code in the PR, not the last published train. True AR resolution is exercised where it means something: the nightly live lanes and the release train. 2. **Names follow spec 15.** Floors, workflow steps, and workspace filters attach to whatever names are current at execution time; the RN-1 rename sweep (WP-RN-05) updates them mechanically, and new suites written after RN-1 use @fabricstack/* names from birth. Pre-rename script names (test:kip, test:kradle, …) are cited below as they exist at HEAD (fabric:package.json:63,72-74,83,95,101). 3. **Retired surfaces are not tested into permanence.** The WP-T3-27 retirement list (7 genty device shells, desktop scaffold, web-app after parity) receives no new floors; genty/web-app's existing Playwright gate (fabric:.github/workflows/ci.yml:190-193 region, step "adapters webui browser e2e") is *kept green until* WP-T3-25's parity bar consumes it, then retires with the shell. 4. **Testing debt vs functionality debt.** Where a behavior under test does not exist yet (journal checksum verification, hash chain — C1/H1, fabric:packages/babysitter-sdk/src/storage/journal.ts:151-161), the *behavior* is a spec-20 WP (F-01…F-15) and the *suite* lands with it. This spec only owns suites for behavior that exists or arrives via spec 10 WPs, plus the recommended adversarial suite (WP-TEST-08) that lands after F-01/F-04. 5. **Mobile deferred (D13).** No iOS/Android lanes, simulators, or device farms in any WP below. 6. **No implementation in this run (D4).**

---

2. Baseline: what actually gates today (evidence)

2.1 PR gates (`fabric:.github/workflows/ci.yml`, jobs at :15, :54, :319, :372, :459 — R1 §4.1)

docs-quality; test (SDK lint, test:sdk at :129-132, architecture gate :114-117, tasks/hooks/trust-core/policy/tools-adapter tests :171-183, triggers unit/e2e/coverage, genty-web-app Playwright :190-193, genty-core/platform tests); gateway-node-engine (Node matrix + engine floor, :319); workspace-coverage (transport + cloud, :372-457); observer-dashboard (build + test:observer, :459-505).

2.2 Release gates (`fabric:.github/workflows/publish.yml` validate jobs at :203, :276, :457, :517, :578 — R1 §4.2)

Adds kradle-core node:test suite (publish.yml:565-568), kradle deployed-stack live e2e + smoke (publish.yml:423-424), cloud release verification, GKE deploys.

2.3 Live harness matrix (`fabric:.github/workflows/live-stack.yml` — R1 §4.3)

The strongest suite fabric has, and the seed for D7(a). Facts this spec builds on:

  • **Agent map of 14 ids** (live-stack.yml:159-174): claude, codex, pi, gemini, copilot, cursor, hermes, opencode, openclaw, omp, droid, amp, genty, antigravity. **No grok entry** (R1 §3.1 — zero grok code repo-wide).
  • **Model axis** (live-stack.yml:175-182), **four interaction modes** (:183-188), **two install modes with fixed per-lane artifact assertions** — adapters-events, transport-adapter-trace, provider-trace-redacted, trace ids; BP adds plugin-command-transcript (:189-192).
  • **OS is an input, not an axis**: one inputs.os value applies to the entire dispatch (live-stack.yml:17-21, runs-on: ${{ inputs.os || 'ubuntu-latest' }} at :44); push runs are ubuntu-only. Per-OS coverage exists only as separately dispatched runs and in live-stack-published.yml documented-install lanes (3 OS — R1 §4.5).
  • Four lane jobs consume computed matrices vanilla_ni / vanilla_interactive / bp_interactive / bp_bridged (live-stack.yml:327,549,565,581).
  • **Status vs the six D7(a) harnesses** (R1 §4.3; MATURITY-MATRIX.md:64-79): Claude Code GREEN, Codex GREEN, Cursor ⛔ (#562), OpenCode ⛔ (Bun server lifecycle), Antigravity ⛔ (#945), Grok Build absent entirely. Hermes-Windows blocked (#856) — a Tier-2 concern.

2.4 The gating holes (R1 §4.4, spec input 6; `MATURITY-MATRIX.md` rows)

Ungated suites (script/files exist, no workflow invokes them): test:kip (kip-sdk 114 test files; only the single-file kip-inv12-golden-digest.yml:71 gate exists), test:kradle (PR CI — gated only in publish.yml), test:atlas-catalog-contracts (fabric:package.json:63) + atlas's 31 in-package tests, test:library/test:shared (fabric:package.json:73-74), genty-runtime (19)/genty-cli (10)/genty-ui (18) (MATURITY-MATRIX.md:46-49), genty-tui's 40 tests (only verify:release runs, :48), kradle-web 30 spec files (:54), adapters-tools 6 tests (:39), apps/commander 51 test files + Playwright with **zero** workflow references (R1 §3.9), observer-dashboard Playwright config present but not invoked (fabric:packages/observer-dashboard/playwright.config.ts, R1 §3.8 INFERRED-ungated). Zero-test packages: adapters-config, adapters-launch, kradle-sdk, kradle-cli (MATURITY-MATRIX.md:40,53). Empty file: fabric:packages/adapters/harness-mock/src/multi-execution.ts (0 lines, R1 §3.1).

2.5 Whole-platform gap (R1 §4.5, spec input 5)

No lane asserts *packaged install → create run → dispatch → agent session → journal → UI*. The UI ends (observer-dashboard, kradle-web, commander) are never asserted inside any live lane; live-stack asserts artifacts and traces, not UI state.

---

3. D7(a) — the live per-harness e2e matrix (design for WP-TEST-03)

3.1 Tier structure and lane inventory

**Tier-1 (the D7(a) six, release-gating):** each driven end-to-end through its spec-10 drive lane, per OS.

#HarnessDrive lane (spec 10)TodayOS coverage targetCI auth mode (§3.4)
1Claude CodeClaude SDK lane (WP-T3-10)GREEN (R1 §4.3)ubuntu + macos + windowssubscription OAuth token, API-key fallback
2CodexCodex app-server lane (WP-T3-11)GREENubuntu + macos + windowsseeded auth.json (ChatGPT subscription), API-key fallback
3CursorACP lane (WP-T3-07)⛔ #562ubuntu + macos + windowsseeded cursor-agent login token
4Grok BuildACP lane — net-new (WP-T3-08)absent (R1 §3.1)per vendor-support declaration (§3.3)xAI login token
5OpenCodeOpenCode HTTP lane (WP-T3-12)⛔ubuntu + macos + windowsopencode auth config with provider key
6AntigravityACP lane + managed installer (WP-T3-09)⛔ #945per vendor-support declaration (§3.3)seeded per-instance profile (OAuth loopback flow pre-baked)

**Tier-2 (fabric's extras, per the spec-10 §3.2 lane decisions — scheduled, non-release-gating):** the retained-subprocess lanes gemini, copilot, pi, omp (≡ oh-my-pi, one harness per spec 10 §3.2), openclaw, hermes, droid, amp (8 lanes, all already in the agent map live-stack.yml:159-174), plus genty via the ACP-agent lane (WP-T3-18). **qwen is exempt by decision**: codec-only, no live-stack obligation (spec 10 §3.2 row +1) — the exemption is recorded in the matrix config, not silently omitted. The babysitter and adapters-remote codecs are lane infrastructure, not harnesses, and get no agent-map rows (spec 10 §3.1/§3.2 disposition).

**Pre/post-migration staging.** The matrix machinery (workflows, OS axis, auth, flake policy) is built against the *current* adapters immediately — Claude Code and Codex are green today and become per-OS-gated first. As each spec-10 lane WP lands, its lane re-points at the new drive path (each lane WP already carries "live-stack lane green" acceptance — WP-T3-07 AC1, WP-T3-08 AC1, WP-T3-09 AC2, WP-T3-10 AC1, WP-T3-11 AC1, WP-T3-12 AC1). Turning the red lanes green is jointly owned: infrastructure and gating here (WP-TEST-03), lane implementations in spec 10, red-lane remediation tracked as F-18 (spec 20). D7(a) is satisfied when all six Tier-1 lanes are green per their OS-support declarations.

3.2 Workflow layout (concrete)

1. **live-stack.yml gains OS as a matrix axis.** The single-valued inputs.os (live-stack.yml:17-21,44) is replaced in the matrix generator (Compute Matrix job, :130-233) by an os field per combination; the four lane jobs (:327,549,565,581) set runs-on: ${{ matrix.os }}. Dispatch entries become {agent, model, mode, install, live, os}; push defaults stay ubuntu-only (cost discipline; the per-OS bar is carried by the scheduled tier, not by push). 2. **New workflow harness-matrix.yml** (scheduled + dispatch) is the D7(a) gate: - **Nightly**: Tier-1 — 6 harnesses × declared OS set × {vanilla-NI, bp-bridged-hooks} on each harness's canonical model, native auth (§3.4). Upper bound 6×3×2 = 36 lanes/night; the vendor-support declarations (§3.3) reduce this to the real set. Tier-2 — 9 lanes, ubuntu, vanilla-NI. - **Weekly**: Tier-2 × declared OS set (hermes-Windows stays declared-blocked under #856 until resolved). - Reuses the live-stack runner packages (fabric:packages/adapters/cli/tests/live-stack/*, scripts fabric:package.json:131-138) and the fixed artifact assertions (live-stack.yml:189-192); it is a scheduler/gater over the same scenario engine, not a second harness driver. 3. **Release gating**: publish.yml gains a **freshness gate** job before the validate fan-out: the most recent scheduled harness-matrix.yml Tier-1 run must be ≤24h old and every Tier-1 lane must be green or in valid quarantine (§3.5). A stale or red matrix blocks the train. (PRs are *not* gated on live lanes — determinism at PR time comes from §4's mock lanes and §5's unit gates.) 4. **live-stack-published.yml retargets to the private train**: post-RN-1 it validates the AR-published @fabricstack/* blueprint/plugin install lanes (documented-install pattern, 3 OS — R1 §4.5) with .npmrc scope routing per spec 15 §4.1. Validation of the public @a5c-ai/* plugin remains a public-repo concern and is out of scope (D17). 5. **Existing triage/gardening workflows** (failure-triage-live-stack.yml, live-stack-gardening.yml — R1 §4.3) extend to harness-matrix.yml runs; gardening produces the pass-rate ledger the flake policy consumes (§3.5).

3.3 OS-support declarations

A committed config e2e-tests/harness-os-support.json declares, per harness × OS: supported | unsupported-by-vendor (requires a citation URL/evidence string) | blocked (requires an issue reference, e.g. hermes-Windows #856) | fixture-blocked (the vendor supports the OS but the **CI fixture** — runner image, credential seeding, installer path — is not yet workable; requires an open issue reference **and** an expiresAt ≤14 days with max 2 renewals, the §3.5.4 quarantine clock applied verbatim). fixture-blocked is deliberately distinct from unsupported-by-vendor: the latter is reserved for genuine vendor non-support with evidence and may never be used to paper over a fixture problem — a cell whose harness runs on that OS outside CI must use fixture-blocked, keeping the debt visible and time-boxed. Lanes generate **only** for supported cells; unsupported-by-vendor renders as — in reporting (mirroring the MATURITY-MATRIX.md:21 marker legend), never as a silent skip; blocked cells count as red for D7(a) purposes until re-declared; fixture-blocked cells render with their issue + expiry, count as red for the D7(a) exit, and an entry that expires unresolved (or exhausts its renewals) reverts to gating-red exactly like a §3.5.4 escalation. Claude Code, Codex, Cursor, OpenCode are declared 3-OS from the start (Claude/Codex already proven 3-OS green — MATURITY-MATRIX.md:58); Grok Build and Antigravity cells are established as an acceptance item of WP-T3-08/WP-T3-09 respectively and recorded here.

3.4 Auth and fixture strategy per harness (subscriptions/logins in CI, private-repo secret model)

All harness credentials live in a dedicated GitHub **environment live-harnesses** of the private fabric repo (D17: never mirrored; D21: no public pipeline can reference them). Naming convention HARNESS_<ID>_<KIND>. Each lane's setup step materializes credentials into an **ephemeral, lane-scoped HOME** (per-lane temp dir torn down in always()), never into the runner's real profile. The existing per-lane redaction artifact (provider-trace-redacted, live-stack.yml:190) is extended with a **canary check**: a step greps every uploaded artifact for the credential material's fingerprints and fails the lane on a hit.

HarnessMechanism (evidence)CI auth fixture
Claude Code@anthropic-ai/claude-agent-sdk in-process (t3:apps/server/src/provider/Layers/ClaudeAdapter.ts:5-27)HARNESS_CLAUDE_OAUTH_TOKEN (subscription) exported for the SDK-spawned claude executable; ANTHROPIC_API_KEY fallback for API-metered lanes. Usage limits read from SDKRateLimitInfo (t3:apps/server/src/provider/Layers/claudeUsageLimits.ts) feed §3.5 quota-exhaustion classification
Codexcodex app-server JSON-RPC stdio (t3:apps/server/src/provider/Layers/codexLaunchArgs.ts:13)HARNESS_CODEX_AUTH_JSON seeded into a lane-scoped $CODEX_HOME/auth.json (ChatGPT subscription session); OPENAI_API_KEY fallback. WP-T3-04's probe example is the lane preflight (t3:packages/effect-codex-app-server/test/examples/codex-app-server-probe.ts)
Cursorcursor-agent … acp (t3:apps/server/src/provider/acp/CursorAcpSupport.ts:56-60)HARNESS_CURSOR_TOKEN seeded into cursor-agent's config dir; the lane pins the installed binary version and **never invokes Cursor's updater** (WP-T3-07 AC4; t3:docs/internals/providers.md, updates section)
Grok Buildgrok --permission-mode <m> agent stdio (t3:apps/server/src/provider/acp/GrokAcpSupport.ts:33-41)HARNESS_GROK_TOKEN (xAI login). Lane preflight uses the no-side-effect probe rule (WP-T3-08 AC4; t3:docs/internals/providers.md, setup section)
OpenCodefabric-managed server per thread via @opencode-ai/sdk/v2 (t3:apps/server/src/provider/Layers/OpenCodeAdapter.ts:32)HARNESS_OPENCODE_AUTH — opencode auth config carrying the provider key; teardown asserts zero orphan opencode processes (WP-T3-12 AC2 mirrors this in unit form)
Antigravityfabric-downloaded release, identity-verified antigravity-acp/protocolVersion 1 (t3:apps/server/src/provider/AntigravityInstallation.ts:484-502)HARNESS_ANTIGRAVITY_PROFILE — a pre-authenticated per-instance profile archive seeded into the lane HOME (the OAuth loopback flow of t3:apps/server/src/provider/AntigravityAuth.ts is exercised in WP-T3-09's integration test, not per-lane); installer version leases pin the release per run

Cross-cutting rules: (1) **secret rotation** is owned per credential with owner + expiry recorded in e2e-tests/harness-os-support.json's sibling e2e-tests/harness-credentials.md (names and owners only — never values); (2) an auth failure classifies as auth-expired (infra class, §3.5), not product red, via the triage workflow's failure taxonomy; (3) model-proxy lanes keep today's provider API-key envs (OPENAI_API_KEY, GOOGLE_CLOUD_*, ANTHROPIC_API_KEY — live-stack.yml:176-181) for the model axis; native-auth lanes are additive, not a replacement.

3.5 Flake policy and quarantine rules (binding for all live lanes)

1. **Failure classes**: red (deterministic product failure), product-flake (nondeterministic product behavior), infra-flake (runner/network provisioning), auth-expired, quota-exhausted (provider rate/credit ceilings — the #485-class). Classification is written by the triage workflow into the run summary; unclassified failures default to red. 2. **Retries**: live lanes get exactly **one** automatic re-run of the failed lane job. Unit/integration suites get **zero** retries — a flaky unit test is a bug, fixed or quarantined, never re-run to green. 3. **Flake threshold**: a lane whose trailing 14 scheduled runs contain both passes and failures with pass-rate <90% is flaky; the gardening workflow opens/updates its quarantine proposal automatically. 4. **Quarantine mechanics**: committed file e2e-tests/quarantine.json; entry = {laneId, issue, class, enteredAt, expiresAt (≤14 days), renewals}. Quarantined lanes **still run** (signal preserved) but do not gate the §3.2.3 freshness check. Exit requires **5 consecutive scheduled greens**. Maximum **2 renewals**; a third expiry escalates: the lane reverts to gating-red and the release train blocks until the owning issue resolves or the harness's OS cell is re-declared (§3.3) with justification. 5. **No timeout-inflation as flake treatment**: raising per-test or per-lane timeouts is not an accepted flake fix; waits must convert to receipt/drain waits (§6.1, WP-TEST-07). Lane wall-clock ceilings exist for cost control (60 min per lane job), not as synchronization. 6. **Quarantine cannot satisfy D7(a)**: the D7(a) exit criterion counts a quarantined Tier-1 lane as not-green.

---

4. D7(b) — whole-platform e2e as a product test (design for WP-TEST-04/05)

4.1 The chain, stated once

**Packaged install → create run → dispatch → agent session → journal → observer/UI assertions.** Every stage asserts at the user-observable level (a product test, not a wiring test):

1. **Packaged install** on a clean runner. Install forms tie to spec 40, staged by availability (priorities per synthesis-adoption-proposal.md §1.5, C → A → B): - **Form C (first)**: npm i -g @fabricstack/orchestrator @fabricstack/harness-cli, **sourced per lane so each lane gates what it should**. *PR-gating product-mock lane*: installs the PR's own code, never a published train — every @fabricstack/* package is built and npm packed from the PR workspace, and the global install resolves from those tarballs into a clean prefix; where transitive @fabricstack/* ranges demand a registry shape, a per-run local-registry fixture (loopback-only, e.g. verdaccio on 127.0.0.1, torn down in always()) is seeded exclusively with the PR-built tarballs. The seeding bypasses npm publish — it uploads each tarball directly via the local registry's API (or npm pack + a registry-side import) — precisely because guard G1 pins every @fabricstack/* publishConfig.registry to the Artifact Registry URL, making npm publish unusable against any non-AR registry. No remote registry is consulted for @fabricstack/* at PR time. *Nightly product-live lane and the release train*: resolve **only** from Artifact Registry via committed .npmrc scope routing (spec 15 §4.1; needs WP-RN-03/05/07 + WP-PKG-01 exports maps) — the true published-train resolution proof. Both lanes assert no registry.npmjs.org fetch occurred for @fabricstack/* (npm debug-log grep — the runtime counterpart of guards G1–G3). - **Form A (when WP-PKG-03/04 ship)**: install.sh/install.ps1 channel-aware checksum-verified install of the single-node archive; the same downstream stages run unchanged — proving the product, not the transport. - **Cluster profile (Form B)**: the existing kradle deployed-stack e2e + smoke (fabric:.github/workflows/publish.yml:423-424) is extended with stage 5–6 assertions below (chart consumption productized by WP-PKG-09/10). 2. **Create run**: via the orchestrator CLI (today babysitter/babysitter-sdk bins; post-RN-1 orchestrator — spec 15 §3.1 rows 1–2) in the single-node profile; via kradle dispatch (agent-dispatch-controller) in the cluster profile. 3. **Dispatch → agent session**: - **Deterministic lane (PR-gating)**: driven by the **ACP mock agent** (t3:apps/server/scripts/acp-mock-agent.ts, landed by WP-T3-03 AC3) once the ACP lane exists; until then by the existing mock harness (fabric:packages/adapters/harness-mock, whose multi-execution.ts WP-TEST-02 fills). Scripted transcript, zero external credentials, runs on every PR touching the packaging/run/journal/UI paths. - **Live lane (scheduled)**: same chain with a real harness (Claude Code — the strongest green lane, R1 §4.3), nightly alongside §3.2's Tier-1. 4. **Journal**: a run journal exists under the runs dir (.a5c/runs today, .fabric/ post-RN-3 — spec 15 §3.5), its event sequence validates (fabric:packages/babysitter-sdk/src/runtime/replay/effectIndex.ts:144-185 sequence validation), it replays, and kanban source discovery finds the run (fabric:packages/adapters/gateway/src/kanban/lib/source-discovery.ts:12-34). Journal sovereignty holds per WP-T3-16 once the environment server exists (checksum *verification* remains F-01 — invariant §1.4). 5. **Observer/UI assertions**: Playwright (the pattern already proven in-repo by the genty-web-app browser e2e, ci.yml:190-193, and configured-but-ungated in observer-dashboard — fabric:packages/observer-dashboard/playwright.config.ts) asserts against the running stack: the run appears in the observer dashboard, status transitions render, journal events stream into the UI. Cluster profile adds kradle-web org-admin assertions (org's run visible, approval surfaces render). Post-WP-T3-25, console-web run/thread views join the assertion set; commander joins only if F-17 wires it live (mock-only today by design — fabric:apps/commander/README.md, R1 §3.9). 6. **Teardown assertions**: no orphan processes, no stray state outside the versioned runtime layout.

4.2 Workflow placement

New workflow platform-e2e.yml: job product-mock (PR-gating, deterministic, ubuntu + windows — the two OSes with the most path-handling divergence; installs the PR's packed tarballs / seeded local registry per §4.1.1, so the required check gates the code in the PR) and job product-live (nightly, 3 OS, Form C resolved from Artifact Registry; Form A added when WP-PKG-04 lands). The cluster profile lives where its stack lives: the publish.yml e2e extension (§4.1.1 Form B). platform-e2e.yml is a **required PR check** for its product-mock job once stable for 14 days (staged enforcement to avoid day-one gate churn).

---

5. D7(c) — unit floors and full CI gating (design for WP-TEST-01/02/06)

5.1 Orphan-suite wiring map (WP-TEST-01) — every row is mandatory

Suite (HEAD name)VolumeWired into (concrete job/step)Floor at wiring
test:kip (fabric:package.json:95)114 test files (R1 §3.5)**new** ci.yml job memory-catalog (kip is too heavy for the test job's critical path); mirrored in a new publish.yml validate_memory step. kip-inv12-golden-digest.yml (single file at :71) retires into the full suite, keeping the golden-digest test as a named stepsuite green; floor per §5.3
test:kradle (fabric:package.json:101)76 node:test files (MATURITY-MATRIX.md:52)ci.yml job memory-catalog gains step "kradle-core tests" (PR gating; publish.yml :565-568 already gates releases)suite green
test:atlas-catalog-contracts (fabric:package.json:63) + atlas in-package tests13 contract files + 31 in-package (R1 §3.6)ci.yml test job, after the existing atlas build steps (ci.yml:135-138 region)suite green
test:library + test:shared (fabric:package.json:73-74)process-library suites (R1 §3.4)ci.yml test job, adjacent to test:sdk (:129-132)suite green
genty-runtime (19) / genty-cli (10) / genty-ui (18) (MATURITY-MATRIX.md:46-49)47 filesci.yml test job steps beside the existing genty-core/platform steps; genty-ui's 18 files are additionally the WP-T3-24 parity bar (Q-09)suite green
genty-tui unit run (40 tests; only verify:release gated today — MATURITY-MATRIX.md:48)40 filesci.yml test job: add a real npm test -w @a5c-ai/genty-tui step beside verify:release (ci.yml:160-163)suite green
kradle-web specs30 files (MATURITY-MATRIX.md:54)new workspace script test:kradle-web; ci.yml memory-catalog job + publish.yml validate_kradle stepsuite green; these same specs are WP-T3-22 AC3's local bar
adapters-tools6 files (MATURITY-MATRIX.md:39)added to the adapters package-tests enumeration consumed by ci.yml/publish.yml (the enumeration that today omits it)suite green
commander unit tests51 files, zero workflow refs (R1 §3.9)new ci.yml job ops-console (unit portion; Playwright is WP-TEST-05). Runs regardless of the F-17 live-wiring decision — the mock-backend contract tests are valuable as frozen-contract regression either waysuite green

Rule: a wired suite is invoked **by name** in a workflow step (the MATURITY-MATRIX.md:3 doctrine — in-tree specs no workflow invokes do not count), on PRs (ci.yml) and releases (publish.yml), with the same command a developer runs locally.

5.2 Zero-test package minimum suites (WP-TEST-02)

PackageMinimum suite (concrete)
adapters-config (0 tests — MATURITY-MATRIX.md:40)config load/parse/merge/validation tests over every exported entry point; malformed-input rejection cases; ≥12 tests
adapters-launch (0 tests)launch-arg construction per supported target + spawn smoke against a stub executable; failure-path (missing binary) typed error; ≥8 tests
kradle-sdk (0 tests — MATURITY-MATRIX.md:53; spec 15 §3.2 notes "rename does not launder that; Q-02 owns floors")client construction, auth header injection, typed request/response round-trip against a stub server, error taxonomy; ≥15 tests
kradle-cli (0 tests)command parse matrix + exit-code smoke per subcommand; ≥10 tests
adapters/harness-mock — src/multi-execution.ts is **0 lines** (R1 §3.1)the file is either implemented (multi-execution scenarios in the mock harness, with tests driving §4.1.3's deterministic lane) or deleted with the export removed and the decision recorded in the WP closing note; an empty exported module may not survive

Every minimum suite runs under the standard workspace test script and is picked up by the §5.1 job map automatically.

5.3 Floors: mechanism and numbers (WP-TEST-06)

  • **Floor registry**: committed testing/floors.json — per workspace: {testFiles, lineCoveragePct}. Enforced by a new scripts/check-test-floors.cjs run in the ci.yml test job and in every publish.yml validate job (same pattern as the existing guardrail scripts check-binary-renames.cjs / check-architecture-boundaries.cjs, fabric:package.json:65-67).
  • **Ratchet discipline**: floors only move up. testFiles floor = the measured count at wiring time (baselines at HEAD: babysitter-sdk 173 · adapters 353 · genty 251 · kradle 130 · kip-sdk 114 · observer-dashboard 105 · commander 51 · atlas 31 · trust-core 8 — R1 §4.4); lineCoveragePct floor = measured-at-wiring rounded **down** to the nearest 5, raised as suites grow. Deleting tests below floor fails CI; a deliberate reduction requires editing floors.json in the same PR, which is review-visible by construction.
  • **Named minimum raises** (explicit, not ratchet-derived): trust-core is security-critical with only 8 test files (R1 §3.7) — floor set to **25** test files covering chain verification, revocation, expiry, and root-kind paths (the H9/F-08 fix adds its own tests on top); the four §5.2 packages get the stated per-package minimums.
  • **Coverage tooling**: vitest --coverage (v8) for vitest workspaces; node:test workspaces (kradle-core) use --experimental-test-coverage output mapped into the same floors file schema.
  • **MATURITY-MATRIX.md refresh**: WP-TEST-06 regenerates the matrix's "Automated test signals" claims from the floors registry + workflow greps, replacing the 2026-08-13 snapshot with a generated section so the matrix cannot silently rot again.

5.4 Per-test timeout policy (binding, all suites)

1. **Config-owned timeouts only**: unit configs testTimeout ≤ 5000ms; integration configs ≤ 120000ms; live-lane scenario deadlines stay in the live-stack runner. Inline per-test timeout raises ({ timeout: n } above the config cap, test.setTimeout) are forbidden. 2. **No real-time sleeps in tests**: await new Promise(r => setTimeout(r, n)) and equivalent sleep helpers are banned in test files. Allowed waits: fake timers; receipt waits and queue-drain waits (§6.1); event/promise conditions; a single shared bounded-poll helper that requires an explicit deadline and reason string. Enforcement: a lint rule + scripts/check-no-sleeps-in-tests.cjs grep gate wired beside the floors check (the doctrine imported from t3code — tests wait on async milestones instead of sleeping: drainable workers + runtime receipts, t3:docs/internals/overview.md:86-92; enforcement-by-custom-lint precedent: t3:oxlint-plugin-t3code, R2 §2). 3. **Timeout ≠ synchronization**: per §3.5.5, raising timeouts is never an accepted flake fix.

---

6. Testing-pattern adoptions from t3code (design for WP-TEST-07/09)

6.1 Receipts and queue-drain instead of sleeps

  • **Typed runtime receipts**: t3code's test-only RuntimeReceiptBus (production layer a no-op) lets tests await async milestones (t3:docs/internals/overview.md:89-92). It arrives in the environment server via WP-T3-19 (AC7). WP-TEST-07 additionally lands a **plain-TS receipts utility** (B4-compliant — no Effect) inside packages/babysitter-sdk/src/testing/ for fabric-core async suites (journal commit milestones, hook dispatch completion, background-effect settlement — today's sleep-prone spots). No new package is minted, keeping spec 15 §3 closed under its own matrix (same rule spec 10 §2 row 11 applies to cherry-picked modules).
  • **DrainableWorker**: t3:packages/shared/src/DrainableWorker.ts is cherry-picked into its adopters by spec 10 (rows land under apps/environment/src/shared/, packages/client-runtime/src/shared/, infra/relay/src/shared/ — spec 10 §2 row 11); WP-TEST-07 owns converting fabric's existing sleep-based waits in the suites it touches to drain waits and wiring the §5.4.2 ban so regressions cannot land.

6.2 ACP mock agent as the deterministic protocol fixture

t3:apps/server/scripts/acp-mock-agent.ts (scripted ACP agent) lands with WP-T3-03 (AC3: initialize → session/new → prompt-turn → session/update in CI). WP-TEST-07 promotes it from a package-local script to the **shared deterministic driver**: transcript-file-driven (recorded session/update sequences replayed byte-stable), used by (a) the ACP lane runtime suite (WP-T3-06 AC1), (b) the fabric-as-ACP-agent conformance suite inverted as a mock *client* (WP-T3-18 AC1), and (c) the §4.1.3 deterministic whole-platform lane.

6.3 Release smoke

t3:scripts/release-smoke.ts (R2 §14) is adapted as scripts/release-smoke.mjs for the fabric train: after WP-RN-07's AR publish, a clean-machine job installs @fabricstack/orchestrator-sdk + @fabricstack/harness-cli from Artifact Registry via scope routing only, runs each §3.1-listed bin's --version/--help, creates and replays a trivial run. This *is* the "published-consumer validation job" WP-RN-07's test requirements name — WP-TEST-07 supplies the suite; WP-RN-07 wires it into publish-npm-private.yml.

6.4 Migration parity fixtures (Q-09 anchor set)

Spec 10 defines three parity bars and names this spec their gate (spec 10 §10): AgentEvent projection goldens (WP-T3-14 AC1), gateway-frame↔contracts-RPC equivalence (WP-T3-21 AC1, over the §4.5 mapping table of spec 10), genty-ui hook-API parity (WP-T3-24 AC1, the 18 genty-ui test files). WP-TEST-09 owns the **shared fixture infrastructure**: the recorded-transcript store format (one directory per lane under packages/adapters/core/src/projection/__tests__/fixtures/, per WP-T3-14's layout), golden-diff runner, and the CI jobs that keep all three suites gating from the moment their anchor WPs land until the migration completes. Its groundwork additionally owns the **legacy-codec baseline corpus**: WP-T3-13 AC2's before/after golden diff needs "before" recordings that can only exist while the legacy codecs still run, so WP-TEST-09 captures that corpus during Phase 2 at a pinned pre-lane commit, before any codec is retired or migrated (WP-TEST-09 AC6).

---

7. Sequencing

  • **Immediately (no migration dependency)**: WP-TEST-01, WP-TEST-02, WP-TEST-06 (unit floors are pure debt-payment on today's tree); WP-TEST-03 stage 1 (OS axis, harness-matrix.yml, auth model, flake policy — over the current green lanes).
  • **As soon as a deterministic driver exists (WP-TEST-02 first, WP-T3-03's ACP mock agent thereafter)**: WP-TEST-04's PR-gating product-mock lane — its packed-tarball install (§4.1.1) needs no registry, so PR gating does not wait on Artifact Registry provisioning or the AR-published train (it does assume the Phase-1 rename has landed, which precedes its Phase-2 activation). **After WP-RN-03/05/07**: WP-TEST-04's AR-resolving product-live lane and release-train Form C; WP-TEST-07's release smoke.
  • **Lane-coupled**: WP-TEST-03's Grok/Cursor/OpenCode/Antigravity green-ness arrives with WP-T3-07/08/09/12 (+F-18); WP-TEST-09 activates per anchor WP (T3-14/21/24), with its Phase-2 groundwork (fixture store + legacy-codec baseline corpus, AC6) completing before any codec retirement; WP-TEST-07's receipts-in-environment rides WP-T3-19.
  • **Last**: WP-TEST-08 (recommended tier), after F-01/F-04 create the behavior to attack.
  • D11 is honored: nothing here blocks migration WPs; spec 50's R-01 owns the merged graph.

---

8. Work packages

WP-TEST-01: CI gating of every orphaned suite (TEST1 residual — D7(c))

  • **Objective**: Wire all §5.1 orphan suites into PR and release CI by name, per the wiring map: test:kip (114 files), test:kradle (PR-side), test:atlas-catalog-contracts + atlas's 31 tests, test:library/test:shared, genty-runtime/cli/ui (47), genty-tui unit run (40), kradle-web (30 specs), adapters-tools (6), commander unit tests (51) — so that zero test suites exist that no workflow invokes (closing R1 §4.4 items 1–8).
  • **Affected paths**: fabric:.github/workflows/ci.yml (test job :54 gains library/shared/genty/tui/atlas-contract steps; new jobs memory-catalog and ops-console); fabric:.github/workflows/publish.yml (new validate_memory step block; validate_kradle:517 gains kradle-web; validate_core:203 gains library/shared); fabric:.github/workflows/kip-inv12-golden-digest.yml (retired into the full-suite job, golden-digest kept as a named step); fabric:package.json (new test:kradle-web script; adapters test-enumeration extended to packages/adapters/tools); fabric:packages/kradle/web/package.json (test script).
  • **Acceptance criteria**: (1) for every §5.1 row, a workflow grep finds the suite invoked by name in both a PR-gating and a release-gating job (the TEST1 verification inverted — R1 §2 TEST1 evidence method); (2) all wired suites green at merge; any red test found during wiring is fixed or quarantined via an entry referencing an issue — never deleted or skipped silently; (3) kip-inv12-golden-digest.yml no longer exists as a standalone single-file gate and the golden-digest test still runs; (4) total PR wall-clock impact measured and recorded: the new jobs run in parallel to test (:54), keeping PR latency increase ≤ the slowest new job; (5) suites keep gating after RN-1 renames (workspace filters keyed by directory, not scope name, or updated in the WP-RN-05 sweep — spec 15 §7).
  • **Test requirements**: the wiring itself is verified by a meta-test extending scripts/check-test-floors.cjs (WP-TEST-06): every workspace with ≥1 test file must be reachable from at least one workflow step (negative fixture: an orphan workspace trips it).
  • **Dependencies**: none to start; WP-TEST-06 for the meta-gate; coordinates with WP-RN-05 (name sweep).
  • **Size**: L

WP-TEST-02: Minimum suites for zero-test packages + mock-harness completion

  • **Objective**: adapters-config, adapters-launch, kradle-sdk, kradle-cli each gain the §5.2 minimum suite (≥12/≥8/≥15/≥10 tests respectively, entry-point-complete); the empty multi-execution.ts (0 lines) is implemented-with-tests or removed-with-record; no L0/L2-with-zero-tests workspace remains outside the retirement list.
  • **Affected paths**: fabric:packages/adapters/config/ (new src/__tests__/, vitest config); fabric:packages/adapters/launch/ (same); fabric:packages/kradle/sdk/, fabric:packages/kradle/cli/ (new test dirs + test scripts); fabric:packages/adapters/harness-mock/src/multi-execution.ts (fill or delete); testing/floors.json (four new rows at the §5.2 minimums).
  • **Acceptance criteria**: (1) each of the four packages' suites runs via its workspace test script and is picked up by the §5.1 job map (proof: the WP-TEST-06 orphan meta-gate passes with the new suites present, and would fail if a suite were unplugged); (2) per-package test counts meet the §5.2 minimums and every exported entry point is imported by at least one test (export-coverage assertion in each suite); (3) multi-execution.ts is non-empty-and-tested or absent-and-unexported, with the decision recorded in the WP closing note; if implemented, the §4.1.3 deterministic lane consumes it; (4) MATURITY-MATRIX.md rows for the four packages move off L0/L2-zero-tests in the WP-TEST-06 regenerated matrix.
  • **Test requirements**: the minimum suites themselves; export-coverage assertions; floors entries active in check-test-floors.cjs.
  • **Dependencies**: WP-TEST-06 (floors registry exists); none from spec 10.
  • **Size**: M

WP-TEST-03: Live per-harness e2e matrix to D7(a) — six harnesses per OS

  • **Objective**: Build §3 in full: OS becomes a matrix axis in live-stack.yml; new harness-matrix.yml schedules Tier-1 (the six D7(a) harnesses × declared OS × {vanilla-NI, bp-bridged-hooks}, native auth) nightly and Tier-2 (the 9 extra lanes per spec 10 §3.2: gemini, copilot, pi, omp, openclaw, hermes, droid, amp subprocess + genty ACP-agent; qwen exempt-by-record) nightly-ubuntu/weekly-per-OS; per-harness auth fixtures per §3.4 under the private-repo secret model; OS-support declarations per §3.3; flake policy + quarantine per §3.5; publish.yml freshness gate; live-stack-published.yml retargeted at the AR-published train.
  • **Affected paths**: fabric:.github/workflows/live-stack.yml (:17-21,44 os-input removal; matrix generator :130-233 gains os per combination and a grok agent-map row once WP-T3-08 lands; lane jobs :327,549,565,581 runs-on from matrix); new fabric:.github/workflows/harness-matrix.yml; fabric:.github/workflows/publish.yml (freshness-gate job before validate fan-out); fabric:.github/workflows/live-stack-published.yml (AR retarget); fabric:.github/workflows/{failure-triage-live-stack,live-stack-gardening}.yml (extend to the new workflow; pass-rate ledger); new e2e-tests/harness-os-support.json, e2e-tests/quarantine.json, e2e-tests/harness-credentials.md; GitHub environment live-harnesses (repo settings; secret names per §3.4).
  • **Acceptance criteria**: (1) a dispatched harness-matrix.yml run executes lanes on ubuntu, macos, and windows **within one run** (OS-as-axis proven); (2) Claude Code and Codex Tier-1 lanes green on all three OSes under native auth (subscription/login secrets), with the credential-canary artifact scan passing (§3.4) — at the stage-1/M1 gate, an individual not-yet-green OS cell may satisfy this criterion **only** as a fixture-blocked declaration per §3.3 (open issue + unexpired expiresAt, ≤14 days, max 2 renewals; never unsupported-by-vendor), and an expired or renewal-exhausted fixture-blocked cell reverts to gating-red, so the escape is time-boxed rather than a silent waiver; (3) Cursor, OpenCode, Antigravity, Grok lanes exist in the matrix, re-point at their spec-10 lanes as those land, and their per-OS cells carry declarations per §3.3 — no silent skips anywhere (every non-running cell is unsupported-by-vendor with evidence, blocked with an issue, or fixture-blocked with an open issue and unexpired expiresAt); (4) the flake machinery works end-to-end: a seeded flaky fixture lane crosses the <90%/14-run threshold, gets auto-proposed for quarantine, stops gating while still running, and exits after 5 greens (rehearsed with a synthetic lane); (5) publish.yml blocks on a stale (>24h) or red Tier-1 matrix (negative rehearsal recorded); (6) **D7(a) exit**: all six Tier-1 harnesses green per their OS-support declarations, quarantine-free — jointly delivered with WP-T3-07/08/09/10/11/12 and F-18, tracked as this WP's completion gate; (7) no workflow or secret this WP creates references the public repo or public npmjs (D17/D21 audit note; install steps resolve from AR only).
  • **Test requirements**: matrix-generator unit tests (os axis, grok row, exemption records); quarantine lifecycle rehearsal; declaration-state validation tests (a fixture-blocked cell missing its issue or expiresAt fails config validation; an expired or renewal-exhausted entry flips the cell to gating-red); freshness-gate negative rehearsal; per-harness auth preflights (probe steps per §3.4) with auth-expired classification tests in the triage workflow.
  • **Dependencies**: stage 1 (axis/workflow/auth/flake over current lanes): none. Full six-green: WP-T3-07, WP-T3-08 (Grok — net-new, mandatory), WP-T3-09, WP-T3-10, WP-T3-11, WP-T3-12; F-18 (spec 20) for red-lane remediation; WP-RN-03/07 for the AR retarget of live-stack-published.yml.
  • **Size**: L

WP-TEST-04: Whole-platform e2e — packaged install → run → dispatch → session → journal → UI (D7(b))

  • **Objective**: Build the §4 product test as platform-e2e.yml: deterministic mock-agent lane (PR-gating, ubuntu + windows, installing the PR's own workspace-packed tarballs / seeded local registry per §4.1.1 so the check gates the code it runs on) and nightly live lane (3 OS, Form C npm-from-AR — the published-train resolution proof), Form A (installer) when WP-PKG-03/04 ship, with the cluster profile landing as an extension of the deployed-stack e2e (publish.yml:423-424). No such lane exists today (R1 spec input 5) — this WP creates the only test that exercises the platform as one product.
  • **Affected paths**: new fabric:.github/workflows/platform-e2e.yml; new e2e-tests/platform/ suite (install-stage drivers: PR tarball pack + local-registry seeding, Form C AR routing, Form A installer; run-creation, dispatch, journal assertions, Playwright UI assertions); fabric:.github/workflows/publish.yml (cluster-profile UI-assertion extension to the existing e2e+smoke steps at :423-424); .npmrc fixtures — AR scope routing (per spec 15 §4.1) for the live lane, loopback local-registry routing for the mock lane; observer-dashboard Playwright project consumed as a library of assertions (config at fabric:packages/observer-dashboard/playwright.config.ts).
  • **Acceptance criteria**: (1) the deterministic lane completes the full six-stage chain of §4.1 on a clean runner with **zero external credentials and zero remote-registry access for @fabricstack/***: packed-tarball install (every @fabricstack/* package npm packed from the PR workspace into a clean prefix, or the per-run loopback local-registry fixture seeded only from the PR build — §4.1.1) → CLI run creation → dispatch → mock agent session (ACP mock agent per §6.2, or the WP-TEST-02 mock harness pre-ACP) → journal exists+validates+replays+is-discovered (effectIndex.ts:144-185; source-discovery.ts:12-34) → observer UI shows the run with streamed events (Playwright assertions); (2) install-provenance proofs per lane: the product-mock install demonstrably resolves every @fabricstack/* package to a tarball packed from the PR's own checkout (the pack step stamps the PR head SHA into each packed manifest — e.g. a gitHead/build-metadata field — and a post-install assertion verifies that SHA in the installed tree, so a stale-train install fails the check), and npm logs for **both** lanes show no registry.npmjs.org fetch for @fabricstack/* (runtime guard complementing G1–G3, spec 15 §4.3); (3) the nightly live lane passes the same chain with Claude Code on 3 OSes, resolving @fabricstack/* **only** from Artifact Registry via the committed .npmrc scope routing — the AR-resolution obligation of spec 15 §7 is carried by this lane and by the §6.3 release smoke, never by the PR lane; (4) when WP-PKG-04 lands, the Form A path (checksum-verified installer → same stages 2–6) runs green with **no changes to stages 2–6** (product-not-transport proof); (5) the cluster profile asserts stage 5–6 (run visible in kradle-web org views + observer) inside the deployed-stack e2e; (6) product-mock becomes a required PR check after 14 stable days (staged enforcement recorded); (7) teardown leaves no orphan processes or out-of-layout state (assertion in always()).
  • **Test requirements**: the platform suite itself; a chaos case (kill the agent mid-session → run reaches a terminal failure state, journal remains valid, UI shows the failure — no hangs); journal-replay determinism assertion; Playwright assertion library shared with WP-TEST-05.
  • **Dependencies**: WP-RN-03/05/07 + WP-PKG-01 (AR-resolving product-live lane and release-train Form C only — the product-mock tarball lane needs just the workspace build plus a deterministic driver, so PR gating activates without waiting on AR); WP-T3-03 (ACP mock agent) or WP-TEST-02 (mock harness) for the deterministic driver; WP-TEST-05 (UI assertion harnesses); WP-PKG-03/04 (Form A stage); WP-PKG-09/10 (cluster chart consumption); WP-T3-16 (journal sovereignty, once the environment server participates).
  • **Size**: L

WP-TEST-05: UI e2e gating — observer, kradle-web, ops-console, and the parity handoff

  • **Objective**: Every UI surface with an e2e harness is CI-gated: observer-dashboard's existing-but-uninvoked Playwright e2e (R1 §3.8) enters CI; commander's Playwright suite runs against its seeded mock backend (contracts-regression value independent of the F-17 disposition); kradle-web browser-level smoke joins the cluster profile; the genty-web-app Playwright gate (ci.yml:190-193) is maintained until WP-T3-25's parity bar consumes it, then hands off to the console-web parity suite (WP-T3-25 AC1) without a coverage gap.
  • **Affected paths**: fabric:.github/workflows/ci.yml observer-dashboard job (:459) gains a Playwright step invoking the in-package config (packages/observer-dashboard/playwright.config.ts, e2e/); ops-console job (from WP-TEST-01) gains the commander Playwright step (fabric:apps/commander in-package Playwright — R1 §3.9); publish.yml validate_observer_and_compiler:578 mirrors the observer e2e; the §4 Playwright assertion library shared from e2e-tests/platform/; a handoff checklist item in WP-T3-25/27's closing notes referencing this WP's gate inventory.
  • **Acceptance criteria**: (1) observer-dashboard Playwright runs headless in PR CI and gates (INFERRED-ungated status in R1 §6 resolved to gated-confirmed); (2) commander Playwright green in CI against the mock backend; if F-17 later wires it live, the same suite re-points at the live profile without rewriting scenarios; (3) kradle-web smoke assertions run inside the cluster-profile e2e (with WP-TEST-04 AC5); (4) at no commit between now and WP-T3-27 is the repo without a gating browser e2e for the primary run/thread UX: genty-web-app's gate is removed **only** in the same PR that makes console-web's parity suite (WP-T3-25 AC1) a required check (gap-free handoff, verifiable in the PR diff); (5) UI e2e suites obey the §5.4 timeout policy (no sleep-based waits; Playwright auto-waiting + explicit expect conditions only).
  • **Test requirements**: the wired suites; a headless-CI stability burn-in (14 consecutive green scheduled runs before required-check promotion, per §3.5 discipline).
  • **Dependencies**: WP-TEST-01 (ops-console job exists); handoff leg: WP-T3-25, WP-T3-27. Independent of the rename train.
  • **Size**: M

WP-TEST-06: Floors registry, coverage gates, timeout policy — the D7(c) enforcement spine

  • **Objective**: Land testing/floors.json + scripts/check-test-floors.cjs (ratchet-only floors: test-file count + line-coverage per §5.3, baselines from R1 §4.4, trust-core raised to 25), the orphan-workspace meta-gate (every workspace with tests must be workflow-reachable), the §5.4 per-test timeout policy (config caps, inline-raise ban, real-sleep ban with scripts/check-no-sleeps-in-tests.cjs + lint rule), and the regenerated MATURITY-MATRIX.md so maturity claims derive from the registry instead of a hand-updated snapshot (stale since 2026-08-13 — MATURITY-MATRIX.md:5).
  • **Affected paths**: new testing/floors.json; new scripts/check-test-floors.cjs, scripts/check-no-sleeps-in-tests.cjs; fabric:.github/workflows/ci.yml test job + every publish.yml validate job (floor + sleep-ban steps, beside the existing guardrails at ci.yml:99-102,114-117); root and per-package vitest configs (testTimeout caps; coverage reporters); kradle-core node:test invocation (publish.yml:565-568) gains --experimental-test-coverage mapping; MATURITY-MATRIX.md (generated section + generator script).
  • **Acceptance criteria**: (1) floors active for every workspace in the §5.1/§5.2 maps; negative fixtures prove each gate: deleting a test file below floor fails CI, a coverage drop below floor fails, an orphan workspace fails, an inline { timeout: 300000 } fails, a setTimeout-sleep in a test file fails; (2) floors are ratchet-only: the checker rejects a floors.json edit that lowers a value unless the PR carries a floors-reduction label (review-visible escape hatch, usage logged); (3) trust-core floor of 25 test files is met (tests added under this WP cover chain verify/revoke/expiry paths that exist today; the requiredRootKind tests ride F-08); (4) regenerated MATURITY-MATRIX.md matches reality by construction (generator re-run in CI, diff must be empty); (5) the full gate set adds ≤90s to the test job's wall-clock (measured, recorded).
  • **Test requirements**: negative-fixture suite for all five gates; generator idempotence test; coverage-schema mapping tests for both vitest and node:test outputs.
  • **Dependencies**: none (first mover of part (c)); consumed by WP-TEST-01/02.
  • **Size**: M

WP-TEST-07: t3code test-pattern adoption — receipts, queue-drain, mock agent, release smoke

  • **Objective**: Import the four t3code patterns as working fabric infrastructure per §6: (a) plain-TS runtime receipts in packages/babysitter-sdk/src/testing/ (B4-compliant; production no-op — pattern per t3:docs/internals/overview.md:89-92) and adoption of the environment server's receipt bus (WP-T3-19 AC7) in its suites; (b) DrainableWorker drain-waits replacing sleep-based waits in the suites this WP converts (source modules land via spec 10 §2 row 11); (c) the ACP mock agent (t3:apps/server/scripts/acp-mock-agent.ts, landed by WP-T3-03) promoted to the shared transcript-driven deterministic driver used by lane suites, the ACP-agent conformance suite, and WP-TEST-04's mock lane; (d) scripts/release-smoke.mjs adapted from t3:scripts/release-smoke.ts as the AR published-consumer smoke WP-RN-07 wires.
  • **Affected paths**: new fabric:packages/babysitter-sdk/src/testing/{receipts.ts,drain.ts} (+ exports entry limited to test consumption); sleep-wait conversions in packages/babysitter-sdk suites (hook dispatcher, background effects, session cleanup — today's real-time-wait hotspots per the R1 §2 H5/M5/M6 cluster's test surroundings); mock-agent transcript format + fixtures under packages/acp/ test tree (post-WP-T3-03 home); new scripts/release-smoke.mjs; fabric:.github/workflows/publish-npm-private.yml (smoke step — file created by WP-RN-07).
  • **Acceptance criteria**: (1) receipts utility used by ≥5 previously sleep-based babysitter-sdk tests, with the sleeps gone and the §5.4.2 ban green over the package; (2) drain-wait pattern demonstrated in at least the gateway run-manager suite (fan-out settlement without sleeps); (3) the mock agent replays a committed transcript byte-stably across two consecutive CI runs (determinism proof) and is consumed by WP-TEST-04's mock lane; (4) release smoke passes against a real staging AR publish: scope-routed install, bin --version sweep over the spec 15 §3.1 bin column, trivial run created and replayed; (5) zero new packages minted (spec 15 §3 closure preserved); (6) no Effect import enters packages/babysitter-sdk (B4 boundary check).
  • **Test requirements**: the converted suites themselves; transcript determinism test; smoke rehearsal on a staging train (link in closing note).
  • **Dependencies**: WP-T3-03 (mock agent lands), WP-T3-19 (environment receipts — that leg only), WP-RN-07 (smoke wiring); the receipts/drain/babysitter-sdk legs have no migration dependency and start immediately.
  • **Size**: M

WP-TEST-08: Adversarial ledger suite — recommended, explicitly outside the D7 bar (TEST2)

  • **Objective**: Attack-style tests for journal integrity once F-01/F-04 land the behavior: tamper (mutate an event payload → load fails checksum verification), truncation (delete tail files → hash-chain/head-checksum detection fires), value-drift (change an effect arg value → replay refuses the stale result instead of silently reusing it — the H2 class), plus signature tamper against F-02's signed events. Extends the only existing tamper precedent (fabric:packages/babysitter-sdk/src/breakpoints/__tests__/enforce-signed-breakpoint.test.ts — R1 §2 TEST2) from breakpoints to the whole ledger.
  • **Affected paths**: new fabric:packages/babysitter-sdk/src/storage/__tests__/adversarial/ (tamper/truncate/drift/signature suites over storage/journal.ts + runtime/replay/effectIndex.ts); fixture corpus of valid journals with scripted corruptions; testing/floors.json (suite counted once landed).
  • **Acceptance criteria**: (1) every attack class above is **proven to fail pre-fix behavior** — and because this WP executes in Phase 5, after F-01/F-02/F-04 have already merged (spec 50 places them in Phases 1–2), the proof is executed against the **pinned pre-fix commit** of the class's owning F-WP: the first parent of the merge commit that landed that F-WP (recorded in the F-WP's closing note where present; otherwise recovered deterministically from git history), checked out in a throwaway worktree where the class's test is run and its **failing** output archived as evidence in this WP's closing note, alongside the same test **passing** at HEAD; a class whose per-class test already rode its F-WP's own PR train (e.g. F-01's tamper suite, green at M1 — spec 50 M1 exit 6) may cite that train's recorded red→green transition instead of re-executing the pinned-commit run; (2) corruption fixtures are generated, not hand-frozen (generator script committed), so canonicalization changes (S6/F-01) regenerate them; (3) the suite is CI-gated like any other (§5.1 rule) once landed; (4) the spec-level label "recommended, not part of the D7 bar" is preserved in the suite README and the R-03 compliance matrix row.
  • **Test requirements**: the adversarial suites; generator determinism test; the pinned-commit worktree harness (checkout → run class test → capture failing evidence) rehearsed against F-01's pinned pre-fix commit and the transcript attached to the closing note.
  • **Dependencies**: F-01, F-02, F-04 (spec 20) — hard dependencies; nothing in this WP blocks anything else.
  • **Size**: M

WP-TEST-09: Migration parity suites — projection goldens, frame↔RPC equivalence, hook-API parity

  • **Objective**: Own the shared fixture infrastructure and CI gating for the three spec-10 parity bars (spec 10 §10 names this WP their anchor): (a) AgentEvent projection golden tests — recorded native transcripts per lane → expected AgentEvent sequences over the **full 67-member union** (all 67 members extend BaseEvent — 49 defined in events.ts plus 18 in events-control.ts; the dossier's 49 counted only events.ts — fabric:packages/adapters/core/src/events.ts:402-411; golden scope covers all 67; WP-T3-14 AC1); (b) gateway-frame↔contracts-RPC equivalence over the spec 10 §4.5 mapping table — same stimulus, equivalent observable result on old frame vs new RPC (WP-T3-21 AC1); (c) genty-ui hook-API parity — the 18 genty-ui test files green against the WP-T3-24 compat layer with only import-path changes (WP-T3-24 AC1); plus (d) the **legacy-codec baseline corpus** as Phase-2 groundwork: native transcripts and legacy-path projected goldens for every retain-behind-contract codec in WP-T3-13's disposition table, captured at a pinned pre-lane commit **before any codec is retired or migrated** — the "before" side of WP-T3-13 AC2's before/after golden diff, which no other WP owns capturing. These suites are the mechanical proof that D6's projection direction and D10's break-freely policy destroyed no consumer semantics.
  • **Affected paths**: fixture store + golden-diff runner under fabric:packages/adapters/core/src/projection/__tests__/fixtures/ (layout per WP-T3-14), including a legacy-baseline/ subtree with a manifest pinning the capture commit SHA; equivalence harness under fabric:packages/adapters/gateway/src/__tests__/equivalence/ (retired with the gateway at WP-T3-21 completion, suites migrating to the contracts side); genty-ui suite wiring (already CI-entering via WP-TEST-01); fabric:.github/workflows/ci.yml (parity steps activate per anchor WP; marked required once their anchor lands).
  • **Acceptance criteria**: (1) fixture format documented and shared: one transcript corpus drives both the projection goldens and the lane unit suites of WP-T3-07…13 (no per-WP fixture dialects); (2) each parity suite becomes a required check in the same PR that lands its anchor WP (T3-14/21/24) and stays required until the corresponding legacy surface is deleted (codecs at WP-T3-13, protocol/v1.ts at WP-T3-21, genty-ui protocol copy at WP-T3-27); (3) golden updates require a golden-update PR label with the regenerating command recorded (no silent re-baselining); (4) at WP-T3-21 completion, the equivalence suite's assertions convert to contracts-side regression tests rather than being deleted with the gateway (survival recorded in the WP closing note); (5) all three suites obey §5.4 (transcript-replay, no sleeps); (6) the legacy-codec baseline corpus exists **before** the first codec retirement/migration merge (WP-T3-07/10/11/12/13): captured during Phase 2 at a pinned pre-lane commit whose SHA is committed in the corpus manifest, covering every retain-behind-contract codec of WP-T3-13's disposition table (the 12 retained lanes), each entry a recorded native transcript plus its legacy-path projected AgentEvent golden — WP-T3-13 AC2's "before" side reads exclusively from this corpus (the cross-reference recorded in both WPs' closing notes), and the golden-diff runner rejects a "before" fixture whose manifest SHA postdates the first lane-WP merge, so the baseline cannot be silently re-recorded post-migration.
  • **Test requirements**: fixture-format round-trip test; golden-diff runner unit tests; corpus-manifest pin check (manifest SHA ordering vs lane-WP merge commits, negative fixture included); a seeded intentional-regression rehearsal per suite (mutated projection/RPC/hook behavior is caught).
  • **Dependencies**: WP-T3-14 (leg a), WP-T3-21 (leg b), WP-T3-24 (leg c); WP-TEST-01 (genty-ui suite wired); fixture-infrastructure groundwork has no dependency; the AC6 corpus capture is Phase-2 groundwork that must complete before WP-T3-07/10/11/12/13 begin retiring or migrating codecs (spec 50 already schedules "WP-TEST-09 fixtures beside Phase 2").
  • **Size**: M

---

9. Cross-spec consistency (D21 checklist for Gate B)

ObligationHonored by
Spec 15 §7: floors attach to renamed packages; whole-platform e2e installs from Artifact Registry, never npmjsInvariant §1.1/§1.2; WP-TEST-01 AC5; WP-TEST-04 AC2/AC3 (AR resolution proven in the live lane and release train; the PR mock lane installs PR-packed tarballs and touches no remote registry for @fabricstack/* at all — strictly stronger than the never-npmjs obligation)
Spec 10 §10: Q-07 seeded by WP-T3-03/19; Q-09 anchored by WP-T3-14/21/24; WP-T3-13 AC2's "before" goldens supplied by the Phase-2 legacy-codec baseline corpusWP-TEST-07 dependencies; WP-TEST-09 acceptance (AC6)/dependencies
Spec 10 lane WPs carry "live-stack lane green" acceptance; this spec supplies matrix + gate§3.1 staging; WP-TEST-03 AC3/AC6
Spec 20: F-18 (red lanes) joint ownership; F-17 (commander) disposition independent of its suites entering CI; F-01/F-02/F-04 precede the adversarial suite; TEST1→Q-01, TEST2→Q-08 absorbed (synthesis-adoption-proposal.md §3)WP-TEST-03, WP-TEST-01/05, WP-TEST-08
Spec 40: install forms — WP-PKG-01/03/04 (Forms C/A) and WP-PKG-09/10 (Form B) consumed by the whole-platform test; release smoke feeds WP-RN-07/WP-PKG-11 trainsWP-TEST-04 §4.1; WP-TEST-07 AC4
D17/D19/D21: no public-repo or public-npm touchpoints anywhere in testing CI; harness secrets private-onlyInvariant §1.1; WP-TEST-03 AC7
D13: no mobile lanesInvariant §1.5
Intent acceptance criterion 3 (WP format) and 5 (no placeholder text)§8 throughout

Work-package count in this spec: **9** (WP-TEST-01 … WP-TEST-09).

Article source

The article body is owned directly by this record.

Related pages

No related wiki pages for this record.

Shortcuts

Open overview
Open JSON
Open graph