Agentic AI Atlasby a5c.ai
OverviewWikiGraphFor AgentsEdgesSearchWorkspace
/
GitHubDocsDiscord
iiRecord
Agentic AI Atlas · Spec 30 — Testing (WP-TEST-*)
page:docs-specs-platform-gaps-30-testinga5c.ai
Search record views/
Record · tabs

Available views

II.Record viewspp. 1 - 1
overviewarticlejsongraph
II.
Page JSON

page:docs-specs-platform-gaps-30-testing

Structured · live

Spec 30 — Testing (WP-TEST-*) json

Inspect the normalized record payload exactly as the atlas UI reads it.

File · wiki/docs/specs/platform-gaps/30-testing.mdCluster · wiki
Record JSON
{
  "id": "page:docs-specs-platform-gaps-30-testing",
  "_kind": "Page",
  "_file": "wiki/docs/specs/platform-gaps/30-testing.md",
  "_cluster": "wiki",
  "attributes": {
    "nodeKind": "Page",
    "sourcePath": "docs/specs/platform-gaps/30-testing.md",
    "sourceKind": "repo-docs",
    "title": "Spec 30 — Testing (WP-TEST-*)",
    "displayName": "Spec 30 — Testing (WP-TEST-*)",
    "slug": "docs/specs/platform-gaps/30-testing",
    "articlePath": "wiki/docs/specs/platform-gaps/30-testing.md",
    "article": "\n# Spec 30 — Testing (WP-TEST-*)\n\nDate: 2026-09-27 · Status: Gate-B draft · Binding decisions: **D7** (the testing bar), constrained by **D4** (spec only, no implementation), **D11** (testing follows migration + coverage in the roadmap), **D13** (mobile deferred — no mobile lanes anywhere below), and amendment **D17–D21** (CI/publish isolation from public pipelines; private Artifact Registry) (`docs/specs/platform-gaps/00-intent.md:19-46`).\n\nInputs consumed in full: the Gate-A-approved adoption proposal `research/synthesis-adoption-proposal.md` (planned 9-WP inventory §4.3; survivor register §3 rows TEST1→Q-01 and TEST2→Q-08); dossiers R1 `research/fabric-current-state.md` (test/CI reality §4, spec inputs 4–6, 13), R2 `research/t3code-architecture.md` (testing patterns §3.1, §14, spec input 11); sibling specs `10-t3code-migration.md` (lane WPs, §10 cross-refs to Q-07/Q-09) and `15-private-isolation-rename.md` (§7 obligations on this spec: floors attach to renamed packages; whole-platform e2e installs from Artifact Registry, never npmjs). Baseline document: `MATURITY-MATRIX.md` (grading doctrine at `MATURITY-MATRIX.md:3`, levels L0–L4 at `:13-17`), refreshed by R1 §4. Citations: `fabric:` paths relative to `C:/work/fabric` at HEAD `5e46b998e`; `t3:` paths relative to the MIT t3code snapshot at `C:/Users/tmusk/AppData/Local/Temp/claude/C--work-company/368e0399-fd11-4960-8b4d-b4dcf1d79c43/scratchpad/t3code`.\n\n**ID convention.** Work packages here are `WP-TEST-01 … WP-TEST-09`. The short form `Q-<n>` used by the Gate-A proposal and by specs 10/15/20/40/50 refers to the same packages one-to-one (`Q-03` ≡ `WP-TEST-03`, etc.). The count matches the proposal's planned 9 (`synthesis-adoption-proposal.md` §4.3); no WPs were added or removed — the sibling specs already cross-reference Q-01/Q-02/Q-04/Q-05/Q-07/Q-09 by these IDs (spec 10 §10; spec 15 §7), so preserving the inventory keeps the Gate-B compliance matrix (R-03) one-to-one. Cross-spec references use the sibling conventions: `WP-T3-nn` (spec 10), `WP-RN-nn` (spec 15), `F-nn` (spec 20), and `WP-PKG-nn` ≡ `P-nn` of the Gate-A inventory (spec 40).\n\n---\n\n## 1. Scope, the D7 bar, and invariants\n\nD7 fixes the bar in three parts, all mandatory:\n\n- **(a) Live per-harness e2e matrix** — Claude Code, Codex, Cursor, Grok Build, OpenCode, Antigravity, each driven end-to-end per OS (§3, WP-TEST-03).\n- **(b) Whole-platform e2e** — packaged install → create run → dispatch → agent session → journal → UI assertions, as one product test (§4, WP-TEST-04/05).\n- **(c) Unit floors + CI gating** — every package's suite wired into CI with agreed floors (§5, WP-TEST-01/02/06).\n\nAn adversarial correctness suite is explicitly **not** part of the bar but is recommended (D7 parenthetical; TEST2); it is WP-TEST-08 and is labeled recommended, not gating.\n\nBinding invariants:\n\n1. **Private-pipeline isolation (D17/D19/D21).** Every workflow, secret, and registry interaction this spec creates lives in the private fabric repo only. No work package touches the public `a5c-ai/babysitter` repo's workflows or publishing. Harness login/subscription secrets (§3.4) are fabric repo/environment secrets and are never mirrored anywhere public. No install step in any e2e lane ever resolves `@fabricstack/*` packages from public npmjs (spec 15 §7 obligation on this spec; guards G1–G6 apply to test fixtures too). Where a lane uses remote resolution at all, it comes only from the GCP Artifact Registry npm repo (`15-private-isolation-rename.md` §4.1); PR-gating deterministic lanes are stricter still — they install the PR's own workspace-packed tarballs (or a per-run local-registry fixture seeded exclusively from the PR build, §4.1.1), so no remote registry is consulted for `@fabricstack/*` at all and the check gates the code in the PR, not the last published train. True AR resolution is exercised where it means something: the nightly live lanes and the release train.\n2. **Names follow spec 15.** Floors, workflow steps, and workspace filters attach to whatever names are current at execution time; the RN-1 rename sweep (WP-RN-05) updates them mechanically, and new suites written after RN-1 use `@fabricstack/*` names from birth. Pre-rename script names (`test:kip`, `test:kradle`, …) are cited below as they exist at HEAD (`fabric:package.json:63,72-74,83,95,101`).\n3. **Retired surfaces are not tested into permanence.** The WP-T3-27 retirement list (7 genty device shells, desktop scaffold, web-app after parity) receives no new floors; `genty/web-app`'s existing Playwright gate (`fabric:.github/workflows/ci.yml:190-193` region, step \"adapters webui browser e2e\") is *kept green until* WP-T3-25's parity bar consumes it, then retires with the shell.\n4. **Testing debt vs functionality debt.** Where a behavior under test does not exist yet (journal checksum verification, hash chain — C1/H1, `fabric:packages/babysitter-sdk/src/storage/journal.ts:151-161`), the *behavior* is a spec-20 WP (F-01…F-15) and the *suite* lands with it. This spec only owns suites for behavior that exists or arrives via spec 10 WPs, plus the recommended adversarial suite (WP-TEST-08) that lands after F-01/F-04.\n5. **Mobile deferred (D13).** No iOS/Android lanes, simulators, or device farms in any WP below.\n6. **No implementation in this run (D4).**\n\n---\n\n## 2. Baseline: what actually gates today (evidence)\n\n### 2.1 PR gates (`fabric:.github/workflows/ci.yml`, jobs at :15, :54, :319, :372, :459 — R1 §4.1)\n\n`docs-quality`; `test` (SDK lint, `test:sdk` at `:129-132`, architecture gate `:114-117`, tasks/hooks/trust-core/policy/tools-adapter tests `:171-183`, triggers unit/e2e/coverage, genty-web-app Playwright `:190-193`, genty-core/platform tests); `gateway-node-engine` (Node matrix + engine floor, `:319`); `workspace-coverage` (transport + cloud, `:372-457`); `observer-dashboard` (build + `test:observer`, `:459-505`).\n\n### 2.2 Release gates (`fabric:.github/workflows/publish.yml` validate jobs at :203, :276, :457, :517, :578 — R1 §4.2)\n\nAdds kradle-core node:test suite (`publish.yml:565-568`), kradle deployed-stack live e2e + smoke (`publish.yml:423-424`), cloud release verification, GKE deploys.\n\n### 2.3 Live harness matrix (`fabric:.github/workflows/live-stack.yml` — R1 §4.3)\n\nThe strongest suite fabric has, and the seed for D7(a). Facts this spec builds on:\n\n- **Agent map of 14 ids** (`live-stack.yml:159-174`): claude, codex, pi, gemini, copilot, cursor, hermes, opencode, openclaw, omp, droid, amp, genty, antigravity. **No `grok` entry** (R1 §3.1 — zero grok code repo-wide).\n- **Model axis** (`live-stack.yml:175-182`), **four interaction modes** (`:183-188`), **two install modes with fixed per-lane artifact assertions** — `adapters-events`, `transport-adapter-trace`, `provider-trace-redacted`, trace ids; BP adds `plugin-command-transcript` (`:189-192`).\n- **OS is an input, not an axis**: one `inputs.os` value applies to the entire dispatch (`live-stack.yml:17-21`, `runs-on: ${{ inputs.os || 'ubuntu-latest' }}` at `:44`); push runs are ubuntu-only. Per-OS coverage exists only as separately dispatched runs and in `live-stack-published.yml` documented-install lanes (3 OS — R1 §4.5).\n- Four lane jobs consume computed matrices `vanilla_ni` / `vanilla_interactive` / `bp_interactive` / `bp_bridged` (`live-stack.yml:327,549,565,581`).\n- **Status vs the six D7(a) harnesses** (R1 §4.3; `MATURITY-MATRIX.md:64-79`): Claude Code GREEN, Codex GREEN, Cursor ⛔ (#562), OpenCode ⛔ (Bun server lifecycle), Antigravity ⛔ (#945), Grok Build absent entirely. Hermes-Windows blocked (#856) — a Tier-2 concern.\n\n### 2.4 The gating holes (R1 §4.4, spec input 6; `MATURITY-MATRIX.md` rows)\n\nUngated suites (script/files exist, no workflow invokes them): `test:kip` (kip-sdk 114 test files; only the single-file `kip-inv12-golden-digest.yml:71` gate exists), `test:kradle` (PR CI — gated only in publish.yml), `test:atlas-catalog-contracts` (`fabric:package.json:63`) + atlas's 31 in-package tests, `test:library`/`test:shared` (`fabric:package.json:73-74`), genty-runtime (19)/genty-cli (10)/genty-ui (18) (`MATURITY-MATRIX.md:46-49`), genty-tui's 40 tests (only `verify:release` runs, `:48`), kradle-web 30 spec files (`:54`), adapters-tools 6 tests (`:39`), apps/commander 51 test files + Playwright with **zero** workflow references (R1 §3.9), observer-dashboard Playwright config present but not invoked (`fabric:packages/observer-dashboard/playwright.config.ts`, R1 §3.8 INFERRED-ungated). Zero-test packages: adapters-config, adapters-launch, kradle-sdk, kradle-cli (`MATURITY-MATRIX.md:40,53`). Empty file: `fabric:packages/adapters/harness-mock/src/multi-execution.ts` (0 lines, R1 §3.1).\n\n### 2.5 Whole-platform gap (R1 §4.5, spec input 5)\n\nNo lane asserts *packaged install → create run → dispatch → agent session → journal → UI*. The UI ends (observer-dashboard, kradle-web, commander) are never asserted inside any live lane; live-stack asserts artifacts and traces, not UI state.\n\n---\n\n## 3. D7(a) — the live per-harness e2e matrix (design for WP-TEST-03)\n\n### 3.1 Tier structure and lane inventory\n\n**Tier-1 (the D7(a) six, release-gating):** each driven end-to-end through its spec-10 drive lane, per OS.\n\n| # | Harness | Drive lane (spec 10) | Today | OS coverage target | CI auth mode (§3.4) |\n|---|---|---|---|---|---|\n| 1 | Claude Code | Claude SDK lane (WP-T3-10) | GREEN (R1 §4.3) | ubuntu + macos + windows | subscription OAuth token, API-key fallback |\n| 2 | Codex | Codex app-server lane (WP-T3-11) | GREEN | ubuntu + macos + windows | seeded `auth.json` (ChatGPT subscription), API-key fallback |\n| 3 | Cursor | ACP lane (WP-T3-07) | ⛔ #562 | ubuntu + macos + windows | seeded `cursor-agent` login token |\n| 4 | Grok Build | ACP lane — net-new (WP-T3-08) | absent (R1 §3.1) | per vendor-support declaration (§3.3) | xAI login token |\n| 5 | OpenCode | OpenCode HTTP lane (WP-T3-12) | ⛔ | ubuntu + macos + windows | opencode auth config with provider key |\n| 6 | Antigravity | ACP lane + managed installer (WP-T3-09) | ⛔ #945 | per vendor-support declaration (§3.3) | seeded per-instance profile (OAuth loopback flow pre-baked) |\n\n**Tier-2 (fabric's extras, per the spec-10 §3.2 lane decisions — scheduled, non-release-gating):** the retained-subprocess lanes `gemini`, `copilot`, `pi`, `omp` (≡ oh-my-pi, one harness per spec 10 §3.2), `openclaw`, `hermes`, `droid`, `amp` (8 lanes, all already in the agent map `live-stack.yml:159-174`), plus `genty` via the ACP-agent lane (WP-T3-18). **`qwen` is exempt by decision**: codec-only, no live-stack obligation (spec 10 §3.2 row +1) — the exemption is recorded in the matrix config, not silently omitted. The `babysitter` and `adapters-remote` codecs are lane infrastructure, not harnesses, and get no agent-map rows (spec 10 §3.1/§3.2 disposition).\n\n**Pre/post-migration staging.** The matrix machinery (workflows, OS axis, auth, flake policy) is built against the *current* adapters immediately — Claude Code and Codex are green today and become per-OS-gated first. As each spec-10 lane WP lands, its lane re-points at the new drive path (each lane WP already carries \"live-stack lane green\" acceptance — WP-T3-07 AC1, WP-T3-08 AC1, WP-T3-09 AC2, WP-T3-10 AC1, WP-T3-11 AC1, WP-T3-12 AC1). Turning the red lanes green is jointly owned: infrastructure and gating here (WP-TEST-03), lane implementations in spec 10, red-lane remediation tracked as F-18 (spec 20). D7(a) is satisfied when all six Tier-1 lanes are green per their OS-support declarations.\n\n### 3.2 Workflow layout (concrete)\n\n1. **`live-stack.yml` gains OS as a matrix axis.** The single-valued `inputs.os` (`live-stack.yml:17-21,44`) is replaced in the matrix generator (Compute Matrix job, `:130-233`) by an `os` field per combination; the four lane jobs (`:327,549,565,581`) set `runs-on: ${{ matrix.os }}`. Dispatch entries become `{agent, model, mode, install, live, os}`; push defaults stay ubuntu-only (cost discipline; the per-OS bar is carried by the scheduled tier, not by push).\n2. **New workflow `harness-matrix.yml`** (scheduled + dispatch) is the D7(a) gate:\n   - **Nightly**: Tier-1 — 6 harnesses × declared OS set × {vanilla-NI, bp-bridged-hooks} on each harness's canonical model, native auth (§3.4). Upper bound 6×3×2 = 36 lanes/night; the vendor-support declarations (§3.3) reduce this to the real set. Tier-2 — 9 lanes, ubuntu, vanilla-NI.\n   - **Weekly**: Tier-2 × declared OS set (hermes-Windows stays declared-blocked under #856 until resolved).\n   - Reuses the live-stack runner packages (`fabric:packages/adapters/cli/tests/live-stack/*`, scripts `fabric:package.json:131-138`) and the fixed artifact assertions (`live-stack.yml:189-192`); it is a scheduler/gater over the same scenario engine, not a second harness driver.\n3. **Release gating**: `publish.yml` gains a **freshness gate** job before the validate fan-out: the most recent scheduled `harness-matrix.yml` Tier-1 run must be ≤24h old and every Tier-1 lane must be green or in valid quarantine (§3.5). A stale or red matrix blocks the train. (PRs are *not* gated on live lanes — determinism at PR time comes from §4's mock lanes and §5's unit gates.)\n4. **`live-stack-published.yml` retargets to the private train**: post-RN-1 it validates the AR-published `@fabricstack/*` blueprint/plugin install lanes (documented-install pattern, 3 OS — R1 §4.5) with `.npmrc` scope routing per spec 15 §4.1. Validation of the public `@a5c-ai/*` plugin remains a public-repo concern and is out of scope (D17).\n5. **Existing triage/gardening workflows** (`failure-triage-live-stack.yml`, `live-stack-gardening.yml` — R1 §4.3) extend to `harness-matrix.yml` runs; gardening produces the pass-rate ledger the flake policy consumes (§3.5).\n\n### 3.3 OS-support declarations\n\nA committed config `e2e-tests/harness-os-support.json` declares, per harness × OS: `supported` | `unsupported-by-vendor` (requires a citation URL/evidence string) | `blocked` (requires an issue reference, e.g. hermes-Windows #856) | `fixture-blocked` (the vendor supports the OS but the **CI fixture** — runner image, credential seeding, installer path — is not yet workable; requires an open issue reference **and** an `expiresAt` ≤14 days with max 2 renewals, the §3.5.4 quarantine clock applied verbatim). `fixture-blocked` is deliberately distinct from `unsupported-by-vendor`: the latter is reserved for genuine vendor non-support with evidence and may never be used to paper over a fixture problem — a cell whose harness runs on that OS outside CI must use `fixture-blocked`, keeping the debt visible and time-boxed. Lanes generate **only** for `supported` cells; `unsupported-by-vendor` renders as `—` in reporting (mirroring the `MATURITY-MATRIX.md:21` marker legend), never as a silent skip; `blocked` cells count as red for D7(a) purposes until re-declared; `fixture-blocked` cells render with their issue + expiry, count as red for the D7(a) exit, and an entry that expires unresolved (or exhausts its renewals) reverts to gating-red exactly like a §3.5.4 escalation. Claude Code, Codex, Cursor, OpenCode are declared 3-OS from the start (Claude/Codex already proven 3-OS green — `MATURITY-MATRIX.md:58`); Grok Build and Antigravity cells are established as an acceptance item of WP-T3-08/WP-T3-09 respectively and recorded here.\n\n### 3.4 Auth and fixture strategy per harness (subscriptions/logins in CI, private-repo secret model)\n\nAll harness credentials live in a dedicated GitHub **environment `live-harnesses`** of the private fabric repo (D17: never mirrored; D21: no public pipeline can reference them). Naming convention `HARNESS_<ID>_<KIND>`. Each lane's setup step materializes credentials into an **ephemeral, lane-scoped HOME** (per-lane temp dir torn down in `always()`), never into the runner's real profile. The existing per-lane redaction artifact (`provider-trace-redacted`, `live-stack.yml:190`) is extended with a **canary check**: a step greps every uploaded artifact for the credential material's fingerprints and fails the lane on a hit.\n\n| Harness | Mechanism (evidence) | CI auth fixture |\n|---|---|---|\n| Claude Code | `@anthropic-ai/claude-agent-sdk` in-process (`t3:apps/server/src/provider/Layers/ClaudeAdapter.ts:5-27`) | `HARNESS_CLAUDE_OAUTH_TOKEN` (subscription) exported for the SDK-spawned `claude` executable; `ANTHROPIC_API_KEY` fallback for API-metered lanes. Usage limits read from `SDKRateLimitInfo` (`t3:apps/server/src/provider/Layers/claudeUsageLimits.ts`) feed §3.5 quota-exhaustion classification |\n| Codex | `codex app-server` JSON-RPC stdio (`t3:apps/server/src/provider/Layers/codexLaunchArgs.ts:13`) | `HARNESS_CODEX_AUTH_JSON` seeded into a lane-scoped `$CODEX_HOME/auth.json` (ChatGPT subscription session); `OPENAI_API_KEY` fallback. WP-T3-04's probe example is the lane preflight (`t3:packages/effect-codex-app-server/test/examples/codex-app-server-probe.ts`) |\n| Cursor | `cursor-agent … acp` (`t3:apps/server/src/provider/acp/CursorAcpSupport.ts:56-60`) | `HARNESS_CURSOR_TOKEN` seeded into cursor-agent's config dir; the lane pins the installed binary version and **never invokes Cursor's updater** (WP-T3-07 AC4; `t3:docs/internals/providers.md`, updates section) |\n| Grok Build | `grok --permission-mode <m> agent stdio` (`t3:apps/server/src/provider/acp/GrokAcpSupport.ts:33-41`) | `HARNESS_GROK_TOKEN` (xAI login). Lane preflight uses the no-side-effect probe rule (WP-T3-08 AC4; `t3:docs/internals/providers.md`, setup section) |\n| OpenCode | fabric-managed server per thread via `@opencode-ai/sdk/v2` (`t3:apps/server/src/provider/Layers/OpenCodeAdapter.ts:32`) | `HARNESS_OPENCODE_AUTH` — opencode auth config carrying the provider key; teardown asserts zero orphan `opencode` processes (WP-T3-12 AC2 mirrors this in unit form) |\n| Antigravity | fabric-downloaded release, identity-verified `antigravity-acp`/protocolVersion 1 (`t3:apps/server/src/provider/AntigravityInstallation.ts:484-502`) | `HARNESS_ANTIGRAVITY_PROFILE` — a pre-authenticated per-instance profile archive seeded into the lane HOME (the OAuth loopback flow of `t3:apps/server/src/provider/AntigravityAuth.ts` is exercised in WP-T3-09's integration test, not per-lane); installer version leases pin the release per run |\n\nCross-cutting rules: (1) **secret rotation** is owned per credential with owner + expiry recorded in `e2e-tests/harness-os-support.json`'s sibling `e2e-tests/harness-credentials.md` (names and owners only — never values); (2) an auth failure classifies as `auth-expired` (infra class, §3.5), not product red, via the triage workflow's failure taxonomy; (3) model-proxy lanes keep today's provider API-key envs (`OPENAI_API_KEY`, `GOOGLE_CLOUD_*`, `ANTHROPIC_API_KEY` — `live-stack.yml:176-181`) for the model axis; native-auth lanes are additive, not a replacement.\n\n### 3.5 Flake policy and quarantine rules (binding for all live lanes)\n\n1. **Failure classes**: `red` (deterministic product failure), `product-flake` (nondeterministic product behavior), `infra-flake` (runner/network provisioning), `auth-expired`, `quota-exhausted` (provider rate/credit ceilings — the #485-class). Classification is written by the triage workflow into the run summary; unclassified failures default to `red`.\n2. **Retries**: live lanes get exactly **one** automatic re-run of the failed lane job. Unit/integration suites get **zero** retries — a flaky unit test is a bug, fixed or quarantined, never re-run to green.\n3. **Flake threshold**: a lane whose trailing 14 scheduled runs contain both passes and failures with pass-rate <90% is flaky; the gardening workflow opens/updates its quarantine proposal automatically.\n4. **Quarantine mechanics**: committed file `e2e-tests/quarantine.json`; entry = `{laneId, issue, class, enteredAt, expiresAt (≤14 days), renewals}`. Quarantined lanes **still run** (signal preserved) but do not gate the §3.2.3 freshness check. Exit requires **5 consecutive scheduled greens**. Maximum **2 renewals**; a third expiry escalates: the lane reverts to gating-red and the release train blocks until the owning issue resolves or the harness's OS cell is re-declared (§3.3) with justification.\n5. **No timeout-inflation as flake treatment**: raising per-test or per-lane timeouts is not an accepted flake fix; waits must convert to receipt/drain waits (§6.1, WP-TEST-07). Lane wall-clock ceilings exist for cost control (60 min per lane job), not as synchronization.\n6. **Quarantine cannot satisfy D7(a)**: the D7(a) exit criterion counts a quarantined Tier-1 lane as not-green.\n\n---\n\n## 4. D7(b) — whole-platform e2e as a product test (design for WP-TEST-04/05)\n\n### 4.1 The chain, stated once\n\n**Packaged install → create run → dispatch → agent session → journal → observer/UI assertions.** Every stage asserts at the user-observable level (a product test, not a wiring test):\n\n1. **Packaged install** on a clean runner. Install forms tie to spec 40, staged by availability (priorities per `synthesis-adoption-proposal.md` §1.5, C → A → B):\n   - **Form C (first)**: `npm i -g @fabricstack/orchestrator @fabricstack/harness-cli`, **sourced per lane so each lane gates what it should**. *PR-gating `product-mock` lane*: installs the PR's own code, never a published train — every `@fabricstack/*` package is built and `npm pack`ed from the PR workspace, and the global install resolves from those tarballs into a clean prefix; where transitive `@fabricstack/*` ranges demand a registry shape, a per-run local-registry fixture (loopback-only, e.g. verdaccio on `127.0.0.1`, torn down in `always()`) is seeded exclusively with the PR-built tarballs. The seeding bypasses `npm publish` — it uploads each tarball directly via the local registry's API (or `npm pack` + a registry-side import) — precisely because guard G1 pins every `@fabricstack/*` `publishConfig.registry` to the Artifact Registry URL, making `npm publish` unusable against any non-AR registry. No remote registry is consulted for `@fabricstack/*` at PR time. *Nightly `product-live` lane and the release train*: resolve **only** from Artifact Registry via committed `.npmrc` scope routing (spec 15 §4.1; needs WP-RN-03/05/07 + WP-PKG-01 exports maps) — the true published-train resolution proof. Both lanes assert no `registry.npmjs.org` fetch occurred for `@fabricstack/*` (npm debug-log grep — the runtime counterpart of guards G1–G3).\n   - **Form A (when WP-PKG-03/04 ship)**: `install.sh`/`install.ps1` channel-aware checksum-verified install of the single-node archive; the same downstream stages run unchanged — proving the product, not the transport.\n   - **Cluster profile (Form B)**: the existing kradle deployed-stack e2e + smoke (`fabric:.github/workflows/publish.yml:423-424`) is extended with stage 5–6 assertions below (chart consumption productized by WP-PKG-09/10).\n2. **Create run**: via the orchestrator CLI (today `babysitter`/`babysitter-sdk` bins; post-RN-1 `orchestrator` — spec 15 §3.1 rows 1–2) in the single-node profile; via kradle dispatch (`agent-dispatch-controller`) in the cluster profile.\n3. **Dispatch → agent session**:\n   - **Deterministic lane (PR-gating)**: driven by the **ACP mock agent** (`t3:apps/server/scripts/acp-mock-agent.ts`, landed by WP-T3-03 AC3) once the ACP lane exists; until then by the existing mock harness (`fabric:packages/adapters/harness-mock`, whose `multi-execution.ts` WP-TEST-02 fills). Scripted transcript, zero external credentials, runs on every PR touching the packaging/run/journal/UI paths.\n   - **Live lane (scheduled)**: same chain with a real harness (Claude Code — the strongest green lane, R1 §4.3), nightly alongside §3.2's Tier-1.\n4. **Journal**: a run journal exists under the runs dir (`.a5c/runs` today, `.fabric/` post-RN-3 — spec 15 §3.5), its event sequence validates (`fabric:packages/babysitter-sdk/src/runtime/replay/effectIndex.ts:144-185` sequence validation), it replays, and kanban source discovery finds the run (`fabric:packages/adapters/gateway/src/kanban/lib/source-discovery.ts:12-34`). Journal sovereignty holds per WP-T3-16 once the environment server exists (checksum *verification* remains F-01 — invariant §1.4).\n5. **Observer/UI assertions**: Playwright (the pattern already proven in-repo by the genty-web-app browser e2e, `ci.yml:190-193`, and configured-but-ungated in observer-dashboard — `fabric:packages/observer-dashboard/playwright.config.ts`) asserts against the running stack: the run appears in the observer dashboard, status transitions render, journal events stream into the UI. Cluster profile adds kradle-web org-admin assertions (org's run visible, approval surfaces render). Post-WP-T3-25, console-web run/thread views join the assertion set; commander joins only if F-17 wires it live (mock-only today by design — `fabric:apps/commander/README.md`, R1 §3.9).\n6. **Teardown assertions**: no orphan processes, no stray state outside the versioned runtime layout.\n\n### 4.2 Workflow placement\n\nNew workflow `platform-e2e.yml`: job `product-mock` (PR-gating, deterministic, ubuntu + windows — the two OSes with the most path-handling divergence; installs the PR's packed tarballs / seeded local registry per §4.1.1, so the required check gates the code in the PR) and job `product-live` (nightly, 3 OS, Form C resolved from Artifact Registry; Form A added when WP-PKG-04 lands). The cluster profile lives where its stack lives: the `publish.yml` e2e extension (§4.1.1 Form B). `platform-e2e.yml` is a **required PR check** for its `product-mock` job once stable for 14 days (staged enforcement to avoid day-one gate churn).\n\n---\n\n## 5. D7(c) — unit floors and full CI gating (design for WP-TEST-01/02/06)\n\n### 5.1 Orphan-suite wiring map (WP-TEST-01) — every row is mandatory\n\n| Suite (HEAD name) | Volume | Wired into (concrete job/step) | Floor at wiring |\n|---|---|---|---|\n| `test:kip` (`fabric:package.json:95`) | 114 test files (R1 §3.5) | **new** `ci.yml` job `memory-catalog` (kip is too heavy for the `test` job's critical path); mirrored in a new `publish.yml` `validate_memory` step. `kip-inv12-golden-digest.yml` (single file at `:71`) retires into the full suite, keeping the golden-digest test as a named step | suite green; floor per §5.3 |\n| `test:kradle` (`fabric:package.json:101`) | 76 node:test files (`MATURITY-MATRIX.md:52`) | `ci.yml` job `memory-catalog` gains step \"kradle-core tests\" (PR gating; publish.yml `:565-568` already gates releases) | suite green |\n| `test:atlas-catalog-contracts` (`fabric:package.json:63`) + atlas in-package tests | 13 contract files + 31 in-package (R1 §3.6) | `ci.yml` `test` job, after the existing atlas build steps (`ci.yml:135-138` region) | suite green |\n| `test:library` + `test:shared` (`fabric:package.json:73-74`) | process-library suites (R1 §3.4) | `ci.yml` `test` job, adjacent to `test:sdk` (`:129-132`) | suite green |\n| genty-runtime (19) / genty-cli (10) / genty-ui (18) (`MATURITY-MATRIX.md:46-49`) | 47 files | `ci.yml` `test` job steps beside the existing genty-core/platform steps; genty-ui's 18 files are additionally the WP-T3-24 parity bar (Q-09) | suite green |\n| genty-tui unit run (40 tests; only `verify:release` gated today — `MATURITY-MATRIX.md:48`) | 40 files | `ci.yml` `test` job: add a real `npm test -w @a5c-ai/genty-tui` step beside `verify:release` (`ci.yml:160-163`) | suite green |\n| kradle-web specs | 30 files (`MATURITY-MATRIX.md:54`) | new workspace script `test:kradle-web`; `ci.yml` `memory-catalog` job + `publish.yml` `validate_kradle` step | suite green; these same specs are WP-T3-22 AC3's local bar |\n| adapters-tools | 6 files (`MATURITY-MATRIX.md:39`) | added to the adapters package-tests enumeration consumed by `ci.yml`/`publish.yml` (the enumeration that today omits it) | suite green |\n| commander unit tests | 51 files, zero workflow refs (R1 §3.9) | new `ci.yml` job `ops-console` (unit portion; Playwright is WP-TEST-05). Runs regardless of the F-17 live-wiring decision — the mock-backend contract tests are valuable as frozen-contract regression either way | suite green |\n\nRule: a wired suite is invoked **by name** in a workflow step (the `MATURITY-MATRIX.md:3` doctrine — in-tree specs no workflow invokes do not count), on PRs (`ci.yml`) and releases (`publish.yml`), with the same command a developer runs locally.\n\n### 5.2 Zero-test package minimum suites (WP-TEST-02)\n\n| Package | Minimum suite (concrete) |\n|---|---|\n| `adapters-config` (0 tests — `MATURITY-MATRIX.md:40`) | config load/parse/merge/validation tests over every exported entry point; malformed-input rejection cases; ≥12 tests |\n| `adapters-launch` (0 tests) | launch-arg construction per supported target + spawn smoke against a stub executable; failure-path (missing binary) typed error; ≥8 tests |\n| `kradle-sdk` (0 tests — `MATURITY-MATRIX.md:53`; spec 15 §3.2 notes \"rename does not launder that; Q-02 owns floors\") | client construction, auth header injection, typed request/response round-trip against a stub server, error taxonomy; ≥15 tests |\n| `kradle-cli` (0 tests) | command parse matrix + exit-code smoke per subcommand; ≥10 tests |\n| `adapters/harness-mock` — `src/multi-execution.ts` is **0 lines** (R1 §3.1) | the file is either implemented (multi-execution scenarios in the mock harness, with tests driving §4.1.3's deterministic lane) or deleted with the export removed and the decision recorded in the WP closing note; an empty exported module may not survive |\n\nEvery minimum suite runs under the standard workspace `test` script and is picked up by the §5.1 job map automatically.\n\n### 5.3 Floors: mechanism and numbers (WP-TEST-06)\n\n- **Floor registry**: committed `testing/floors.json` — per workspace: `{testFiles, lineCoveragePct}`. Enforced by a new `scripts/check-test-floors.cjs` run in the `ci.yml` `test` job and in every `publish.yml` validate job (same pattern as the existing guardrail scripts `check-binary-renames.cjs` / `check-architecture-boundaries.cjs`, `fabric:package.json:65-67`).\n- **Ratchet discipline**: floors only move up. `testFiles` floor = the measured count at wiring time (baselines at HEAD: babysitter-sdk 173 · adapters 353 · genty 251 · kradle 130 · kip-sdk 114 · observer-dashboard 105 · commander 51 · atlas 31 · trust-core 8 — R1 §4.4); `lineCoveragePct` floor = measured-at-wiring rounded **down** to the nearest 5, raised as suites grow. Deleting tests below floor fails CI; a deliberate reduction requires editing `floors.json` in the same PR, which is review-visible by construction.\n- **Named minimum raises** (explicit, not ratchet-derived): trust-core is security-critical with only 8 test files (R1 §3.7) — floor set to **25** test files covering chain verification, revocation, expiry, and root-kind paths (the H9/F-08 fix adds its own tests on top); the four §5.2 packages get the stated per-package minimums.\n- **Coverage tooling**: vitest `--coverage` (v8) for vitest workspaces; `node:test` workspaces (kradle-core) use `--experimental-test-coverage` output mapped into the same floors file schema.\n- **`MATURITY-MATRIX.md` refresh**: WP-TEST-06 regenerates the matrix's \"Automated test signals\" claims from the floors registry + workflow greps, replacing the 2026-08-13 snapshot with a generated section so the matrix cannot silently rot again.\n\n### 5.4 Per-test timeout policy (binding, all suites)\n\n1. **Config-owned timeouts only**: unit configs `testTimeout ≤ 5000ms`; integration configs `≤ 120000ms`; live-lane scenario deadlines stay in the live-stack runner. Inline per-test timeout raises (`{ timeout: n }` above the config cap, `test.setTimeout`) are forbidden.\n2. **No real-time sleeps in tests**: `await new Promise(r => setTimeout(r, n))` and equivalent sleep helpers are banned in test files. Allowed waits: fake timers; receipt waits and queue-drain waits (§6.1); event/promise conditions; a single shared bounded-poll helper that requires an explicit deadline and reason string. Enforcement: a lint rule + `scripts/check-no-sleeps-in-tests.cjs` grep gate wired beside the floors check (the doctrine imported from t3code — tests wait on async milestones instead of sleeping: drainable workers + runtime receipts, `t3:docs/internals/overview.md:86-92`; enforcement-by-custom-lint precedent: `t3:oxlint-plugin-t3code`, R2 §2).\n3. **Timeout ≠ synchronization**: per §3.5.5, raising timeouts is never an accepted flake fix.\n\n---\n\n## 6. Testing-pattern adoptions from t3code (design for WP-TEST-07/09)\n\n### 6.1 Receipts and queue-drain instead of sleeps\n\n- **Typed runtime receipts**: t3code's test-only `RuntimeReceiptBus` (production layer a no-op) lets tests await async milestones (`t3:docs/internals/overview.md:89-92`). It arrives in the environment server via WP-T3-19 (AC7). WP-TEST-07 additionally lands a **plain-TS receipts utility** (B4-compliant — no Effect) inside `packages/babysitter-sdk/src/testing/` for fabric-core async suites (journal commit milestones, hook dispatch completion, background-effect settlement — today's sleep-prone spots). No new package is minted, keeping spec 15 §3 closed under its own matrix (same rule spec 10 §2 row 11 applies to cherry-picked modules).\n- **DrainableWorker**: `t3:packages/shared/src/DrainableWorker.ts` is cherry-picked into its adopters by spec 10 (rows land under `apps/environment/src/shared/`, `packages/client-runtime/src/shared/`, `infra/relay/src/shared/` — spec 10 §2 row 11); WP-TEST-07 owns converting fabric's existing sleep-based waits in the suites it touches to drain waits and wiring the §5.4.2 ban so regressions cannot land.\n\n### 6.2 ACP mock agent as the deterministic protocol fixture\n\n`t3:apps/server/scripts/acp-mock-agent.ts` (scripted ACP agent) lands with WP-T3-03 (AC3: initialize → session/new → prompt-turn → session/update in CI). WP-TEST-07 promotes it from a package-local script to the **shared deterministic driver**: transcript-file-driven (recorded `session/update` sequences replayed byte-stable), used by (a) the ACP lane runtime suite (WP-T3-06 AC1), (b) the fabric-as-ACP-agent conformance suite inverted as a mock *client* (WP-T3-18 AC1), and (c) the §4.1.3 deterministic whole-platform lane.\n\n### 6.3 Release smoke\n\n`t3:scripts/release-smoke.ts` (R2 §14) is adapted as `scripts/release-smoke.mjs` for the fabric train: after WP-RN-07's AR publish, a clean-machine job installs `@fabricstack/orchestrator-sdk` + `@fabricstack/harness-cli` from Artifact Registry via scope routing only, runs each §3.1-listed bin's `--version`/`--help`, creates and replays a trivial run. This *is* the \"published-consumer validation job\" WP-RN-07's test requirements name — WP-TEST-07 supplies the suite; WP-RN-07 wires it into `publish-npm-private.yml`.\n\n### 6.4 Migration parity fixtures (Q-09 anchor set)\n\nSpec 10 defines three parity bars and names this spec their gate (spec 10 §10): AgentEvent projection goldens (WP-T3-14 AC1), gateway-frame↔contracts-RPC equivalence (WP-T3-21 AC1, over the §4.5 mapping table of spec 10), genty-ui hook-API parity (WP-T3-24 AC1, the 18 genty-ui test files). WP-TEST-09 owns the **shared fixture infrastructure**: the recorded-transcript store format (one directory per lane under `packages/adapters/core/src/projection/__tests__/fixtures/`, per WP-T3-14's layout), golden-diff runner, and the CI jobs that keep all three suites gating from the moment their anchor WPs land until the migration completes. Its groundwork additionally owns the **legacy-codec baseline corpus**: WP-T3-13 AC2's before/after golden diff needs \"before\" recordings that can only exist while the legacy codecs still run, so WP-TEST-09 captures that corpus during Phase 2 at a pinned pre-lane commit, before any codec is retired or migrated (WP-TEST-09 AC6).\n\n---\n\n## 7. Sequencing\n\n- **Immediately (no migration dependency)**: WP-TEST-01, WP-TEST-02, WP-TEST-06 (unit floors are pure debt-payment on today's tree); WP-TEST-03 stage 1 (OS axis, `harness-matrix.yml`, auth model, flake policy — over the current green lanes).\n- **As soon as a deterministic driver exists (WP-TEST-02 first, WP-T3-03's ACP mock agent thereafter)**: WP-TEST-04's PR-gating `product-mock` lane — its packed-tarball install (§4.1.1) needs no registry, so PR gating does not wait on Artifact Registry provisioning or the AR-published train (it does assume the Phase-1 rename has landed, which precedes its Phase-2 activation). **After WP-RN-03/05/07**: WP-TEST-04's AR-resolving `product-live` lane and release-train Form C; WP-TEST-07's release smoke.\n- **Lane-coupled**: WP-TEST-03's Grok/Cursor/OpenCode/Antigravity green-ness arrives with WP-T3-07/08/09/12 (+F-18); WP-TEST-09 activates per anchor WP (T3-14/21/24), with its Phase-2 groundwork (fixture store + legacy-codec baseline corpus, AC6) completing before any codec retirement; WP-TEST-07's receipts-in-environment rides WP-T3-19.\n- **Last**: WP-TEST-08 (recommended tier), after F-01/F-04 create the behavior to attack.\n- D11 is honored: nothing here blocks migration WPs; spec 50's R-01 owns the merged graph.\n\n---\n\n## 8. Work packages\n\n### WP-TEST-01: CI gating of every orphaned suite (TEST1 residual — D7(c))\n- **Objective**: Wire all §5.1 orphan suites into PR and release CI by name, per the wiring map: `test:kip` (114 files), `test:kradle` (PR-side), `test:atlas-catalog-contracts` + atlas's 31 tests, `test:library`/`test:shared`, genty-runtime/cli/ui (47), genty-tui unit run (40), kradle-web (30 specs), adapters-tools (6), commander unit tests (51) — so that zero test suites exist that no workflow invokes (closing R1 §4.4 items 1–8).\n- **Affected paths**: `fabric:.github/workflows/ci.yml` (`test` job `:54` gains library/shared/genty/tui/atlas-contract steps; new jobs `memory-catalog` and `ops-console`); `fabric:.github/workflows/publish.yml` (new `validate_memory` step block; `validate_kradle:517` gains kradle-web; `validate_core:203` gains library/shared); `fabric:.github/workflows/kip-inv12-golden-digest.yml` (retired into the full-suite job, golden-digest kept as a named step); `fabric:package.json` (new `test:kradle-web` script; adapters test-enumeration extended to `packages/adapters/tools`); `fabric:packages/kradle/web/package.json` (test script).\n- **Acceptance criteria**: (1) for every §5.1 row, a workflow grep finds the suite invoked by name in both a PR-gating and a release-gating job (the TEST1 verification inverted — R1 §2 TEST1 evidence method); (2) all wired suites green at merge; any red test found during wiring is fixed or quarantined via an entry referencing an issue — never deleted or skipped silently; (3) `kip-inv12-golden-digest.yml` no longer exists as a standalone single-file gate and the golden-digest test still runs; (4) total PR wall-clock impact measured and recorded: the new jobs run in parallel to `test` (`:54`), keeping PR latency increase ≤ the slowest new job; (5) suites keep gating after RN-1 renames (workspace filters keyed by directory, not scope name, or updated in the WP-RN-05 sweep — spec 15 §7).\n- **Test requirements**: the wiring itself is verified by a meta-test extending `scripts/check-test-floors.cjs` (WP-TEST-06): every workspace with ≥1 test file must be reachable from at least one workflow step (negative fixture: an orphan workspace trips it).\n- **Dependencies**: none to start; WP-TEST-06 for the meta-gate; coordinates with WP-RN-05 (name sweep).\n- **Size**: L\n\n### WP-TEST-02: Minimum suites for zero-test packages + mock-harness completion\n- **Objective**: adapters-config, adapters-launch, kradle-sdk, kradle-cli each gain the §5.2 minimum suite (≥12/≥8/≥15/≥10 tests respectively, entry-point-complete); the empty `multi-execution.ts` (0 lines) is implemented-with-tests or removed-with-record; no L0/L2-with-zero-tests workspace remains outside the retirement list.\n- **Affected paths**: `fabric:packages/adapters/config/` (new `src/__tests__/`, vitest config); `fabric:packages/adapters/launch/` (same); `fabric:packages/kradle/sdk/`, `fabric:packages/kradle/cli/` (new test dirs + `test` scripts); `fabric:packages/adapters/harness-mock/src/multi-execution.ts` (fill or delete); `testing/floors.json` (four new rows at the §5.2 minimums).\n- **Acceptance criteria**: (1) each of the four packages' suites runs via its workspace `test` script and is picked up by the §5.1 job map (proof: the WP-TEST-06 orphan meta-gate passes with the new suites present, and would fail if a suite were unplugged); (2) per-package test counts meet the §5.2 minimums and every exported entry point is imported by at least one test (export-coverage assertion in each suite); (3) `multi-execution.ts` is non-empty-and-tested or absent-and-unexported, with the decision recorded in the WP closing note; if implemented, the §4.1.3 deterministic lane consumes it; (4) `MATURITY-MATRIX.md` rows for the four packages move off L0/L2-zero-tests in the WP-TEST-06 regenerated matrix.\n- **Test requirements**: the minimum suites themselves; export-coverage assertions; floors entries active in `check-test-floors.cjs`.\n- **Dependencies**: WP-TEST-06 (floors registry exists); none from spec 10.\n- **Size**: M\n\n### WP-TEST-03: Live per-harness e2e matrix to D7(a) — six harnesses per OS\n- **Objective**: Build §3 in full: OS becomes a matrix axis in `live-stack.yml`; new `harness-matrix.yml` schedules Tier-1 (the six D7(a) harnesses × declared OS × {vanilla-NI, bp-bridged-hooks}, native auth) nightly and Tier-2 (the 9 extra lanes per spec 10 §3.2: gemini, copilot, pi, omp, openclaw, hermes, droid, amp subprocess + genty ACP-agent; qwen exempt-by-record) nightly-ubuntu/weekly-per-OS; per-harness auth fixtures per §3.4 under the private-repo secret model; OS-support declarations per §3.3; flake policy + quarantine per §3.5; `publish.yml` freshness gate; `live-stack-published.yml` retargeted at the AR-published train.\n- **Affected paths**: `fabric:.github/workflows/live-stack.yml` (`:17-21,44` os-input removal; matrix generator `:130-233` gains `os` per combination and a `grok` agent-map row once WP-T3-08 lands; lane jobs `:327,549,565,581` `runs-on` from matrix); new `fabric:.github/workflows/harness-matrix.yml`; `fabric:.github/workflows/publish.yml` (freshness-gate job before validate fan-out); `fabric:.github/workflows/live-stack-published.yml` (AR retarget); `fabric:.github/workflows/{failure-triage-live-stack,live-stack-gardening}.yml` (extend to the new workflow; pass-rate ledger); new `e2e-tests/harness-os-support.json`, `e2e-tests/quarantine.json`, `e2e-tests/harness-credentials.md`; GitHub environment `live-harnesses` (repo settings; secret names per §3.4).\n- **Acceptance criteria**: (1) a dispatched `harness-matrix.yml` run executes lanes on ubuntu, macos, and windows **within one run** (OS-as-axis proven); (2) Claude Code and Codex Tier-1 lanes green on all three OSes under native auth (subscription/login secrets), with the credential-canary artifact scan passing (§3.4) — at the stage-1/M1 gate, an individual not-yet-green OS cell may satisfy this criterion **only** as a `fixture-blocked` declaration per §3.3 (open issue + unexpired `expiresAt`, ≤14 days, max 2 renewals; never `unsupported-by-vendor`), and an expired or renewal-exhausted fixture-blocked cell reverts to gating-red, so the escape is time-boxed rather than a silent waiver; (3) Cursor, OpenCode, Antigravity, Grok lanes exist in the matrix, re-point at their spec-10 lanes as those land, and their per-OS cells carry declarations per §3.3 — no silent skips anywhere (every non-running cell is `unsupported-by-vendor` with evidence, `blocked` with an issue, or `fixture-blocked` with an open issue and unexpired `expiresAt`); (4) the flake machinery works end-to-end: a seeded flaky fixture lane crosses the <90%/14-run threshold, gets auto-proposed for quarantine, stops gating while still running, and exits after 5 greens (rehearsed with a synthetic lane); (5) `publish.yml` blocks on a stale (>24h) or red Tier-1 matrix (negative rehearsal recorded); (6) **D7(a) exit**: all six Tier-1 harnesses green per their OS-support declarations, quarantine-free — jointly delivered with WP-T3-07/08/09/10/11/12 and F-18, tracked as this WP's completion gate; (7) no workflow or secret this WP creates references the public repo or public npmjs (D17/D21 audit note; install steps resolve from AR only).\n- **Test requirements**: matrix-generator unit tests (os axis, grok row, exemption records); quarantine lifecycle rehearsal; declaration-state validation tests (a `fixture-blocked` cell missing its issue or `expiresAt` fails config validation; an expired or renewal-exhausted entry flips the cell to gating-red); freshness-gate negative rehearsal; per-harness auth preflights (probe steps per §3.4) with `auth-expired` classification tests in the triage workflow.\n- **Dependencies**: stage 1 (axis/workflow/auth/flake over current lanes): none. Full six-green: WP-T3-07, WP-T3-08 (Grok — net-new, mandatory), WP-T3-09, WP-T3-10, WP-T3-11, WP-T3-12; F-18 (spec 20) for red-lane remediation; WP-RN-03/07 for the AR retarget of `live-stack-published.yml`.\n- **Size**: L\n\n### WP-TEST-04: Whole-platform e2e — packaged install → run → dispatch → session → journal → UI (D7(b))\n- **Objective**: Build the §4 product test as `platform-e2e.yml`: deterministic mock-agent lane (PR-gating, ubuntu + windows, installing the PR's own workspace-packed tarballs / seeded local registry per §4.1.1 so the check gates the code it runs on) and nightly live lane (3 OS, Form C npm-from-AR — the published-train resolution proof), Form A (installer) when WP-PKG-03/04 ship, with the cluster profile landing as an extension of the deployed-stack e2e (`publish.yml:423-424`). No such lane exists today (R1 spec input 5) — this WP creates the only test that exercises the platform as one product.\n- **Affected paths**: new `fabric:.github/workflows/platform-e2e.yml`; new `e2e-tests/platform/` suite (install-stage drivers: PR tarball pack + local-registry seeding, Form C AR routing, Form A installer; run-creation, dispatch, journal assertions, Playwright UI assertions); `fabric:.github/workflows/publish.yml` (cluster-profile UI-assertion extension to the existing e2e+smoke steps at `:423-424`); `.npmrc` fixtures — AR scope routing (per spec 15 §4.1) for the live lane, loopback local-registry routing for the mock lane; observer-dashboard Playwright project consumed as a library of assertions (config at `fabric:packages/observer-dashboard/playwright.config.ts`).\n- **Acceptance criteria**: (1) the deterministic lane completes the full six-stage chain of §4.1 on a clean runner with **zero external credentials and zero remote-registry access for `@fabricstack/*`**: packed-tarball install (every `@fabricstack/*` package `npm pack`ed from the PR workspace into a clean prefix, or the per-run loopback local-registry fixture seeded only from the PR build — §4.1.1) → CLI run creation → dispatch → mock agent session (ACP mock agent per §6.2, or the WP-TEST-02 mock harness pre-ACP) → journal exists+validates+replays+is-discovered (`effectIndex.ts:144-185`; `source-discovery.ts:12-34`) → observer UI shows the run with streamed events (Playwright assertions); (2) install-provenance proofs per lane: the `product-mock` install demonstrably resolves every `@fabricstack/*` package to a tarball packed from the PR's own checkout (the pack step stamps the PR head SHA into each packed manifest — e.g. a `gitHead`/build-metadata field — and a post-install assertion verifies that SHA in the installed tree, so a stale-train install fails the check), and npm logs for **both** lanes show no `registry.npmjs.org` fetch for `@fabricstack/*` (runtime guard complementing G1–G3, spec 15 §4.3); (3) the nightly live lane passes the same chain with Claude Code on 3 OSes, resolving `@fabricstack/*` **only** from Artifact Registry via the committed `.npmrc` scope routing — the AR-resolution obligation of spec 15 §7 is carried by this lane and by the §6.3 release smoke, never by the PR lane; (4) when WP-PKG-04 lands, the Form A path (checksum-verified installer → same stages 2–6) runs green with **no changes to stages 2–6** (product-not-transport proof); (5) the cluster profile asserts stage 5–6 (run visible in kradle-web org views + observer) inside the deployed-stack e2e; (6) `product-mock` becomes a required PR check after 14 stable days (staged enforcement recorded); (7) teardown leaves no orphan processes or out-of-layout state (assertion in `always()`).\n- **Test requirements**: the platform suite itself; a chaos case (kill the agent mid-session → run reaches a terminal failure state, journal remains valid, UI shows the failure — no hangs); journal-replay determinism assertion; Playwright assertion library shared with WP-TEST-05.\n- **Dependencies**: WP-RN-03/05/07 + WP-PKG-01 (AR-resolving `product-live` lane and release-train Form C only — the `product-mock` tarball lane needs just the workspace build plus a deterministic driver, so PR gating activates without waiting on AR); WP-T3-03 (ACP mock agent) or WP-TEST-02 (mock harness) for the deterministic driver; WP-TEST-05 (UI assertion harnesses); WP-PKG-03/04 (Form A stage); WP-PKG-09/10 (cluster chart consumption); WP-T3-16 (journal sovereignty, once the environment server participates).\n- **Size**: L\n\n### WP-TEST-05: UI e2e gating — observer, kradle-web, ops-console, and the parity handoff\n- **Objective**: Every UI surface with an e2e harness is CI-gated: observer-dashboard's existing-but-uninvoked Playwright e2e (R1 §3.8) enters CI; commander's Playwright suite runs against its seeded mock backend (contracts-regression value independent of the F-17 disposition); kradle-web browser-level smoke joins the cluster profile; the genty-web-app Playwright gate (`ci.yml:190-193`) is maintained until WP-T3-25's parity bar consumes it, then hands off to the console-web parity suite (WP-T3-25 AC1) without a coverage gap.\n- **Affected paths**: `fabric:.github/workflows/ci.yml` `observer-dashboard` job (`:459`) gains a Playwright step invoking the in-package config (`packages/observer-dashboard/playwright.config.ts`, `e2e/`); `ops-console` job (from WP-TEST-01) gains the commander Playwright step (`fabric:apps/commander` in-package Playwright — R1 §3.9); `publish.yml` `validate_observer_and_compiler:578` mirrors the observer e2e; the §4 Playwright assertion library shared from `e2e-tests/platform/`; a handoff checklist item in WP-T3-25/27's closing notes referencing this WP's gate inventory.\n- **Acceptance criteria**: (1) observer-dashboard Playwright runs headless in PR CI and gates (INFERRED-ungated status in R1 §6 resolved to gated-confirmed); (2) commander Playwright green in CI against the mock backend; if F-17 later wires it live, the same suite re-points at the live profile without rewriting scenarios; (3) kradle-web smoke assertions run inside the cluster-profile e2e (with WP-TEST-04 AC5); (4) at no commit between now and WP-T3-27 is the repo without a gating browser e2e for the primary run/thread UX: genty-web-app's gate is removed **only** in the same PR that makes console-web's parity suite (WP-T3-25 AC1) a required check (gap-free handoff, verifiable in the PR diff); (5) UI e2e suites obey the §5.4 timeout policy (no sleep-based waits; Playwright auto-waiting + explicit expect conditions only).\n- **Test requirements**: the wired suites; a headless-CI stability burn-in (14 consecutive green scheduled runs before required-check promotion, per §3.5 discipline).\n- **Dependencies**: WP-TEST-01 (`ops-console` job exists); handoff leg: WP-T3-25, WP-T3-27. Independent of the rename train.\n- **Size**: M\n\n### WP-TEST-06: Floors registry, coverage gates, timeout policy — the D7(c) enforcement spine\n- **Objective**: Land `testing/floors.json` + `scripts/check-test-floors.cjs` (ratchet-only floors: test-file count + line-coverage per §5.3, baselines from R1 §4.4, trust-core raised to 25), the orphan-workspace meta-gate (every workspace with tests must be workflow-reachable), the §5.4 per-test timeout policy (config caps, inline-raise ban, real-sleep ban with `scripts/check-no-sleeps-in-tests.cjs` + lint rule), and the regenerated `MATURITY-MATRIX.md` so maturity claims derive from the registry instead of a hand-updated snapshot (stale since 2026-08-13 — `MATURITY-MATRIX.md:5`).\n- **Affected paths**: new `testing/floors.json`; new `scripts/check-test-floors.cjs`, `scripts/check-no-sleeps-in-tests.cjs`; `fabric:.github/workflows/ci.yml` `test` job + every `publish.yml` validate job (floor + sleep-ban steps, beside the existing guardrails at `ci.yml:99-102,114-117`); root and per-package vitest configs (`testTimeout` caps; coverage reporters); kradle-core node:test invocation (`publish.yml:565-568`) gains `--experimental-test-coverage` mapping; `MATURITY-MATRIX.md` (generated section + generator script).\n- **Acceptance criteria**: (1) floors active for every workspace in the §5.1/§5.2 maps; negative fixtures prove each gate: deleting a test file below floor fails CI, a coverage drop below floor fails, an orphan workspace fails, an inline `{ timeout: 300000 }` fails, a `setTimeout`-sleep in a test file fails; (2) floors are ratchet-only: the checker rejects a `floors.json` edit that lowers a value unless the PR carries a `floors-reduction` label (review-visible escape hatch, usage logged); (3) trust-core floor of 25 test files is met (tests added under this WP cover chain verify/revoke/expiry paths that exist today; the `requiredRootKind` tests ride F-08); (4) regenerated `MATURITY-MATRIX.md` matches reality by construction (generator re-run in CI, diff must be empty); (5) the full gate set adds ≤90s to the `test` job's wall-clock (measured, recorded).\n- **Test requirements**: negative-fixture suite for all five gates; generator idempotence test; coverage-schema mapping tests for both vitest and node:test outputs.\n- **Dependencies**: none (first mover of part (c)); consumed by WP-TEST-01/02.\n- **Size**: M\n\n### WP-TEST-07: t3code test-pattern adoption — receipts, queue-drain, mock agent, release smoke\n- **Objective**: Import the four t3code patterns as working fabric infrastructure per §6: (a) plain-TS runtime receipts in `packages/babysitter-sdk/src/testing/` (B4-compliant; production no-op — pattern per `t3:docs/internals/overview.md:89-92`) and adoption of the environment server's receipt bus (WP-T3-19 AC7) in its suites; (b) DrainableWorker drain-waits replacing sleep-based waits in the suites this WP converts (source modules land via spec 10 §2 row 11); (c) the ACP mock agent (`t3:apps/server/scripts/acp-mock-agent.ts`, landed by WP-T3-03) promoted to the shared transcript-driven deterministic driver used by lane suites, the ACP-agent conformance suite, and WP-TEST-04's mock lane; (d) `scripts/release-smoke.mjs` adapted from `t3:scripts/release-smoke.ts` as the AR published-consumer smoke WP-RN-07 wires.\n- **Affected paths**: new `fabric:packages/babysitter-sdk/src/testing/{receipts.ts,drain.ts}` (+ exports entry limited to test consumption); sleep-wait conversions in `packages/babysitter-sdk` suites (hook dispatcher, background effects, session cleanup — today's real-time-wait hotspots per the R1 §2 H5/M5/M6 cluster's test surroundings); mock-agent transcript format + fixtures under `packages/acp/` test tree (post-WP-T3-03 home); new `scripts/release-smoke.mjs`; `fabric:.github/workflows/publish-npm-private.yml` (smoke step — file created by WP-RN-07).\n- **Acceptance criteria**: (1) receipts utility used by ≥5 previously sleep-based babysitter-sdk tests, with the sleeps gone and the §5.4.2 ban green over the package; (2) drain-wait pattern demonstrated in at least the gateway run-manager suite (fan-out settlement without sleeps); (3) the mock agent replays a committed transcript byte-stably across two consecutive CI runs (determinism proof) and is consumed by WP-TEST-04's mock lane; (4) release smoke passes against a real staging AR publish: scope-routed install, bin `--version` sweep over the spec 15 §3.1 bin column, trivial run created and replayed; (5) zero new packages minted (spec 15 §3 closure preserved); (6) no Effect import enters `packages/babysitter-sdk` (B4 boundary check).\n- **Test requirements**: the converted suites themselves; transcript determinism test; smoke rehearsal on a staging train (link in closing note).\n- **Dependencies**: WP-T3-03 (mock agent lands), WP-T3-19 (environment receipts — that leg only), WP-RN-07 (smoke wiring); the receipts/drain/babysitter-sdk legs have no migration dependency and start immediately.\n- **Size**: M\n\n### WP-TEST-08: Adversarial ledger suite — recommended, explicitly outside the D7 bar (TEST2)\n- **Objective**: Attack-style tests for journal integrity once F-01/F-04 land the behavior: tamper (mutate an event payload → load fails checksum verification), truncation (delete tail files → hash-chain/head-checksum detection fires), value-drift (change an effect arg value → replay refuses the stale result instead of silently reusing it — the H2 class), plus signature tamper against F-02's signed events. Extends the only existing tamper precedent (`fabric:packages/babysitter-sdk/src/breakpoints/__tests__/enforce-signed-breakpoint.test.ts` — R1 §2 TEST2) from breakpoints to the whole ledger.\n- **Affected paths**: new `fabric:packages/babysitter-sdk/src/storage/__tests__/adversarial/` (tamper/truncate/drift/signature suites over `storage/journal.ts` + `runtime/replay/effectIndex.ts`); fixture corpus of valid journals with scripted corruptions; `testing/floors.json` (suite counted once landed).\n- **Acceptance criteria**: (1) every attack class above is **proven to fail pre-fix behavior** — and because this WP executes in Phase 5, after F-01/F-02/F-04 have already merged (spec 50 places them in Phases 1–2), the proof is executed against the **pinned pre-fix commit** of the class's owning F-WP: the first parent of the merge commit that landed that F-WP (recorded in the F-WP's closing note where present; otherwise recovered deterministically from git history), checked out in a throwaway worktree where the class's test is run and its **failing** output archived as evidence in this WP's closing note, alongside the same test **passing** at HEAD; a class whose per-class test already rode its F-WP's own PR train (e.g. F-01's tamper suite, green at M1 — spec 50 M1 exit 6) may cite that train's recorded red→green transition instead of re-executing the pinned-commit run; (2) corruption fixtures are generated, not hand-frozen (generator script committed), so canonicalization changes (S6/F-01) regenerate them; (3) the suite is CI-gated like any other (§5.1 rule) once landed; (4) the spec-level label \"recommended, not part of the D7 bar\" is preserved in the suite README and the R-03 compliance matrix row.\n- **Test requirements**: the adversarial suites; generator determinism test; the pinned-commit worktree harness (checkout → run class test → capture failing evidence) rehearsed against F-01's pinned pre-fix commit and the transcript attached to the closing note.\n- **Dependencies**: F-01, F-02, F-04 (spec 20) — hard dependencies; nothing in this WP blocks anything else.\n- **Size**: M\n\n### WP-TEST-09: Migration parity suites — projection goldens, frame↔RPC equivalence, hook-API parity\n- **Objective**: Own the shared fixture infrastructure and CI gating for the three spec-10 parity bars (spec 10 §10 names this WP their anchor): (a) AgentEvent projection golden tests — recorded native transcripts per lane → expected `AgentEvent` sequences over the **full 67-member union** (all 67 members extend BaseEvent — 49 defined in events.ts plus 18 in events-control.ts; the dossier's 49 counted only events.ts — `fabric:packages/adapters/core/src/events.ts:402-411`; golden scope covers all 67; WP-T3-14 AC1); (b) gateway-frame↔contracts-RPC equivalence over the spec 10 §4.5 mapping table — same stimulus, equivalent observable result on old frame vs new RPC (WP-T3-21 AC1); (c) genty-ui hook-API parity — the 18 genty-ui test files green against the WP-T3-24 compat layer with only import-path changes (WP-T3-24 AC1); plus (d) the **legacy-codec baseline corpus** as Phase-2 groundwork: native transcripts and legacy-path projected goldens for every retain-behind-contract codec in WP-T3-13's disposition table, captured at a pinned pre-lane commit **before any codec is retired or migrated** — the \"before\" side of WP-T3-13 AC2's before/after golden diff, which no other WP owns capturing. These suites are the mechanical proof that D6's projection direction and D10's break-freely policy destroyed no consumer semantics.\n- **Affected paths**: fixture store + golden-diff runner under `fabric:packages/adapters/core/src/projection/__tests__/fixtures/` (layout per WP-T3-14), including a `legacy-baseline/` subtree with a manifest pinning the capture commit SHA; equivalence harness under `fabric:packages/adapters/gateway/src/__tests__/equivalence/` (retired with the gateway at WP-T3-21 completion, suites migrating to the contracts side); genty-ui suite wiring (already CI-entering via WP-TEST-01); `fabric:.github/workflows/ci.yml` (parity steps activate per anchor WP; marked required once their anchor lands).\n- **Acceptance criteria**: (1) fixture format documented and shared: one transcript corpus drives both the projection goldens and the lane unit suites of WP-T3-07…13 (no per-WP fixture dialects); (2) each parity suite becomes a required check in the same PR that lands its anchor WP (T3-14/21/24) and stays required until the corresponding legacy surface is deleted (codecs at WP-T3-13, `protocol/v1.ts` at WP-T3-21, genty-ui protocol copy at WP-T3-27); (3) golden updates require a `golden-update` PR label with the regenerating command recorded (no silent re-baselining); (4) at WP-T3-21 completion, the equivalence suite's assertions convert to contracts-side regression tests rather than being deleted with the gateway (survival recorded in the WP closing note); (5) all three suites obey §5.4 (transcript-replay, no sleeps); (6) the legacy-codec baseline corpus exists **before** the first codec retirement/migration merge (WP-T3-07/10/11/12/13): captured during Phase 2 at a pinned pre-lane commit whose SHA is committed in the corpus manifest, covering every retain-behind-contract codec of WP-T3-13's disposition table (the 12 retained lanes), each entry a recorded native transcript plus its legacy-path projected `AgentEvent` golden — WP-T3-13 AC2's \"before\" side reads exclusively from this corpus (the cross-reference recorded in both WPs' closing notes), and the golden-diff runner rejects a \"before\" fixture whose manifest SHA postdates the first lane-WP merge, so the baseline cannot be silently re-recorded post-migration.\n- **Test requirements**: fixture-format round-trip test; golden-diff runner unit tests; corpus-manifest pin check (manifest SHA ordering vs lane-WP merge commits, negative fixture included); a seeded intentional-regression rehearsal per suite (mutated projection/RPC/hook behavior is caught).\n- **Dependencies**: WP-T3-14 (leg a), WP-T3-21 (leg b), WP-T3-24 (leg c); WP-TEST-01 (genty-ui suite wired); fixture-infrastructure groundwork has no dependency; the AC6 corpus capture is Phase-2 groundwork that must complete before WP-T3-07/10/11/12/13 begin retiring or migrating codecs (spec 50 already schedules \"WP-TEST-09 fixtures beside Phase 2\").\n- **Size**: M\n\n---\n\n## 9. Cross-spec consistency (D21 checklist for Gate B)\n\n| Obligation | Honored by |\n|---|---|\n| Spec 15 §7: floors attach to renamed packages; whole-platform e2e installs from Artifact Registry, never npmjs | Invariant §1.1/§1.2; WP-TEST-01 AC5; WP-TEST-04 AC2/AC3 (AR resolution proven in the live lane and release train; the PR mock lane installs PR-packed tarballs and touches no remote registry for `@fabricstack/*` at all — strictly stronger than the never-npmjs obligation) |\n| Spec 10 §10: Q-07 seeded by WP-T3-03/19; Q-09 anchored by WP-T3-14/21/24; WP-T3-13 AC2's \"before\" goldens supplied by the Phase-2 legacy-codec baseline corpus | WP-TEST-07 dependencies; WP-TEST-09 acceptance (AC6)/dependencies |\n| Spec 10 lane WPs carry \"live-stack lane green\" acceptance; this spec supplies matrix + gate | §3.1 staging; WP-TEST-03 AC3/AC6 |\n| Spec 20: F-18 (red lanes) joint ownership; F-17 (commander) disposition independent of its suites entering CI; F-01/F-02/F-04 precede the adversarial suite; TEST1→Q-01, TEST2→Q-08 absorbed (`synthesis-adoption-proposal.md` §3) | WP-TEST-03, WP-TEST-01/05, WP-TEST-08 |\n| Spec 40: install forms — WP-PKG-01/03/04 (Forms C/A) and WP-PKG-09/10 (Form B) consumed by the whole-platform test; release smoke feeds WP-RN-07/WP-PKG-11 trains | WP-TEST-04 §4.1; WP-TEST-07 AC4 |\n| D17/D19/D21: no public-repo or public-npm touchpoints anywhere in testing CI; harness secrets private-only | Invariant §1.1; WP-TEST-03 AC7 |\n| D13: no mobile lanes | Invariant §1.5 |\n| Intent acceptance criterion 3 (WP format) and 5 (no placeholder text) | §8 throughout |\n\nWork-package count in this spec: **9** (WP-TEST-01 … WP-TEST-09).\n",
    "documents": []
  },
  "outgoingEdges": [],
  "incomingEdges": [
    {
      "from": "page:docs",
      "to": "page:docs-specs-platform-gaps-30-testing",
      "kind": "contains_page"
    }
  ]
}

Shortcuts

Back to overview
Open graph tab