{
  "id": "page:docs-testing-adapters-and-runtime-e2e",
  "_kind": "Page",
  "_file": "wiki/docs/testing/adapters-and-runtime-e2e.md",
  "_cluster": "wiki",
  "attributes": {
    "nodeKind": "Page",
    "sourcePath": "docs/testing/adapters-and-runtime-e2e.md",
    "sourceKind": "repo-docs",
    "title": "Agent Adapter And Runtime E2E",
    "displayName": "Agent Adapter And Runtime E2E",
    "slug": "docs/testing/adapters-and-runtime-e2e",
    "articlePath": "wiki/docs/testing/adapters-and-runtime-e2e.md",
    "article": "\n# Agent Adapter And Runtime E2E\n\nThis strategy covers runtime paths after setup is already satisfied. It separates adapters sessions, transport carriers, genty-core programmatic sessions, and `@a5c-ai/genty-platform` orchestration. Harness/plugin install coverage lives in [Harness And Plugin E2E](./harness-e2e.md), not in agent-platform runtime E2E.\n\n## Stack Scopes\n\n| Scope | Packages | No-model coverage | Model-backed coverage |\n| --- | --- | --- | --- |\n| Protocol and event contracts | `packages/adapters/core`, `packages/adapters/gateway`, `packages/transport-adapter` | Schema, event ordering, session lifecycle, error envelopes, reconnect behavior | Real event streams from Codex and Claude Code sessions match protocol contracts |\n| Adapter translation | `packages/adapters/adapters` | Prompt normalization, tool-call mapping, stop reasons, model selection, fallback behavior with mock adapters | Live Codex and Claude Code adapters translate real provider output into adapter events |\n| Transport runtime | `packages/transport-adapter` | Route matrix, proxy auth, protocol codecs, runtime env injection, passthrough forwarding, subprocess lifecycle, stream cancellation, timeout/error paths, metrics/cache snapshots | Transport-adapter carries traffic for a real external harness through adapters launch and for an genty-core-backed stream |\n| Agent-core bridge | `packages/genty/core` | Programmatic session creation, mock provider responses, cancellation, usage accounting | Agent-core invokes a real provider and returns events compatible with adapters and agent-platform |\n| Hooks muxes | `packages/adapters/hooks/*` | Adapter normalization, hook payload fixtures, CLI execution, approval/deny/error events | Real harness hook payloads from Codex and Claude Code normalize to the same hook contract |\n| Babysitter-agent runtime | `packages/genty/platform` | Seam contract, phase orchestration, planner/executor mocks, run journal state, task posting, selected backend | `agent-platform call/create-run/invoke` uses preinstalled or mocked backends; no harness install or plugin install steps are part of this E2E |\n| User surfaces | `packages/adapters/webui`, `packages/adapters/ui`, `packages/adapters/tui` | Playwright/Vitest against mock gateway and fixture sessions | Optional manual/live smoke against a model-backed gateway session |\n\n## No-Model Runtime Suite\n\nThe no-model runtime suite should be built first. It should include:\n\n- `transport-adapter` unit and E2E tests using local HTTP/subprocess fixtures for every supported exposed transport route.\n- `transport-adapter` runtime tests for `startTransportMuxRuntime`, `applyTransportMuxToHarnessEnv`, proxy auth, redacted env diffs, metrics, cache stats, passthrough path/query preservation, invalid JSON, and upstream failure mapping.\n- `adapters` launch-plan tests for proxy forced, proxy if-needed, native/no-proxy, and proxy-forbidden cases so launch coverage proves the transport-adapter decision seam.\n- `adapters` gateway/session tests using existing mock harness scenarios.\n- Adapter translation tests for Codex, Claude Code, and genty-core-style event streams.\n- `agent-platform` seam and orchestration tests with mocked planner/executor calls.\n- WebUI and TUI session tests using fixture transcripts and mock gateway responses.\n- Adapters plugin/session fixtures live in [Harness And Plugin E2E](./harness-e2e.md); this file consumes their event fixtures only as runtime compatibility inputs.\n\nCandidate command grouping:\n\n```bash\nnpm run test --workspace=@a5c-ai/transport-adapter\nnpm run test --workspace=@a5c-ai/comm-adapter\nnpm run test --workspace=@a5c-ai/adapters-codecs\nnpm run test --workspace=@a5c-ai/adapters-gateway\nnpm run test --workspace=@a5c-ai/genty-core\nnpm run test --workspace=@a5c-ai/genty-platform\nnpm run test:e2e --workspace=@a5c-ai/genty-web-app\n```\n\n## Transport-Adapter Coverage Matrix\n\nTransport-adapter coverage has to prove the proxy/runtime seam directly before it is used as evidence for adapters or agent-platform paths.\n\n### No-Model Transport Coverage\n\n| Surface | Tests to add or keep | Required artifacts |\n| --- | --- | --- |\n| Supported transport routes | Exercise `anthropic`, `openai-chat`, `openai-responses`, `google`, `bedrock-converse`, `azure-foundry`, `vertex-native`, and `passthrough` against fixture engines | Route transcript, response shape snapshot, invalid JSON/auth failure transcript |\n| Streaming and non-streaming codecs | Verify text deltas, final events, finish reasons, usage totals, and provider-specific envelopes | Streaming event transcript and non-streaming body snapshot |\n| Token and observability routes | Cover `count_tokens` success/unsupported behavior plus `/metrics` and `/cache/stats` | Token count transcript, metrics snapshot, cache stats snapshot |\n| Runtime env injection | Call `applyTransportMuxToHarnessEnv` for every exposed transport | Redacted env diff proving only expected vars changed |\n| Adapters launch seam | Cover `resolveLaunchPlan` for proxy forced, proxy if-needed, native/no-proxy, and forbidden proxy cases | Launch-plan JSON with `proxyNeeded`, `proxyReason`, and exposed transport |\n| Passthrough forwarding | Preserve path/query/body, inject upstream auth safely, and map upstream failures | Redacted upstream transcript and failure envelope |\n\n### Model-Backed Transport Coverage\n\n| Path | Valid stack | Assertion focus |\n| --- | --- | --- |\n| genty-core stream bridge | `genty-core` provider backend -> transport-adapter -> fixture consumer | Real or credential-gated stream deltas, final event, cancellation/timeout, and usage metadata survive the proxy layer |\n| External harness bridge | `adapters launch <harness> <provider> --with-proxy*` -> transport-adapter -> target provider | Harness receives proxy env, provider endpoint is not called directly by the harness, sentinel prompt completes, metrics show traffic |\n| Babysitter-agent precondition bridge | `agent-platform` external-harness path only when it delegates through adapters and the selected harness requires proxy translation | Transport-adapter artifacts are supporting evidence for the bridge, while Babysitter run creation/task posting remains asserted by agent-platform tests |\n\n### Invalid Transport Claims\n\n- Do not use transport-adapter tests to prove `babysitter harness:install` or `harness:install-plugin`.\n- Do not use transport-adapter tests to prove agent-native plugin manager support.\n- Do not use transport-adapter tests to prove hooks-adapter normalization.\n- Do not use transport-adapter tests to prove Babysitter journal terminal state unless a higher-level runtime test also asserts that state.\n\n\n### Live Install Modes\n\nThe `Publish` workflow runs external-harness live E2E through a workflow-owned install-mode axis:\n\n- `babysitter-plugin` generates plugin artifacts, installs the target with `adapters install`, installs the local Babysitter SDK, installs the Babysitter plugin for the harness, then launches through `adapters launch` with a `/babysitter:call` prompt.\n- `vanilla` installs the target with `adapters install`, launches through `adapters launch`, and uses a non-plugin prompt so it proves adapters/transport/provider behavior; the vanilla `agent-platform` rows use the `babysitter` adapter with `BABYSITTER_HARNESS=genty-core`.\n- Both modes use the same target mapping: `claude-code -> claude`, `codex -> codex`, `gemini-cli -> gemini`, `pi -> pi`, and vanilla-only `agent-platform -> babysitter`.\n\n## Model-Backed Runtime Suite\n\nThe model-backed suite should prove that real providers and real harnesses behave like the no-model contracts expect.\n\n| Test | Required real dependency | Assertion focus |\n| --- | --- | --- |\n| Transport-adapter + external harness through adapters | Claude Code or Codex-compatible harness, provider credential, and `adapters launch --with-proxy` or `--with-proxy-if-needed` | Launch starts transport-adapter, harness receives proxy env, sentinel traffic uses proxy routes, stream completes, metrics snapshot increments |\n| Transport-adapter + genty-core | Provider credential for genty-core backend | Agent-core deltas/final events travel through transport-adapter without adapter-only assumptions, including cancellation or timeout evidence |\n| Adapters + Codex adapter | Codex CLI or configured Codex runtime and OpenAI credential | Codex output maps to adapter protocol events, including final message and usage metadata when available |\n| Adapters + Claude Code adapter | Claude Code CLI plus Foundry/OpenAI credential through transport-adapter | Claude Code output maps to adapter protocol events while model traffic is proxied to GPT-5.5, including tool-call and stop metadata when available |\n| Babysitter-agent full run | Provider credentials or mocked backend already available | `agent-platform call/create-run` creates a bounded process, plans, emits a task, posts a result, completes, and records selected backend evidence without running installer commands |\n\nModel-backed runtime tests must upload redacted event logs, provider/harness version metadata, run IDs, and command durations.\n\n## Runtime Path Assertions\n\nRuntime tests must declare which entry path they exercise:\n\n| Path | Entry point | Valid backend combinations | Assertions |\n| --- | --- | --- | --- |\n| Adapters session | `adapters run <agent>` or `createClient().run` | Mock adapter, Claude, Codex, Gemini, Cursor, OpenCode, adapters `babysitter` adapter where registered | Session start/end, event ordering, provider/model config, runtime hooks, capability-gated plugin events |\n| Babysitter-agent internal runtime | `agent-platform call/create-run --harness genty-core` | genty-core backend with mocked or live model provider | Run creation, planning, task posting, terminal state, redacted model trace |\n| Babysitter-agent external-harness bridge | `agent-platform call/invoke --harness <external>` | Harness names mapped in `adapterHarnessMap`; excludes `pi` and `genty-core` | Adapters mapped events, session ID, result, selected harness, no install commands |\n| Transport runtime | transport-adapter around genty-core or adapters-launched external harness traffic | Local route fixture, genty-core stream, external harness stream | Route/codec contract, proxy auth, env injection, launch proxy decision, framing, reconnect, cancellation, timeout, backpressure, metrics/cache artifact |\n\nDo not fold plugin setup into the agent-platform runtime assertions. If a runtime job needs an installed external harness or plugin, that is a precondition supplied by a setup job and recorded separately.\n\n## Adapter-Specific Assertions\n\nAdapter tests should assert behavior that package-local unit tests cannot prove alone:\n\n- A session can be started, observed, cancelled, and resumed through the adapter boundary.\n- Tool-call, text-delta, final-message, usage, and error events preserve ordering and session IDs.\n- Adapter-specific errors are normalized before they cross gateway or transport boundaries.\n- Model selection is explicit and recorded in the session state.\n- Credential absence is detected before provider calls are attempted.\n- Mock and live event streams conform to the same protocol fixtures.\n\n## Babysitter-Agent Whole-System Assertions\n\nWhole-system tests for `@a5c-ai/genty-platform` should cover:\n\n- process loading and validation,\n- run creation,\n- session binding,\n- planning phase output shape,\n- task effect emission,\n- task result posting,\n- journal rebuild/repair compatibility,\n- terminal run state,\n- artifact and log redaction.\n\nThe no-model version should use mocks for planner and executor behavior. The model-backed version should use the smallest possible bounded process and real model credentials or a preconfigured external harness. It must not execute `harness:install` or `harness:install-plugin` as part of the agent-platform runtime test.\n\n## Hooks-Adapter Assertions\n\nHooks-adapter tests should cover both adapter-local behavior and end-to-end event compatibility:\n\n- each adapter normalizes raw harness hook payloads into the shared hook contract,\n- CLI execution preserves stdin/stdout/stderr boundaries and exit codes,\n- approval, denial, timeout, and malformed-payload cases are fixture-backed,\n- Codex and Claude Code live hook payloads can be redacted and replayed as no-model fixtures,\n- adapters UI and TUI approval surfaces consume the same normalized hook events.\n\nHooks-adapter live coverage should not be promoted until the no-model fixture suite covers the same event types.\n",
    "documents": []
  },
  "outgoingEdges": [],
  "incomingEdges": [
    {
      "from": "page:docs-testing",
      "to": "page:docs-testing-adapters-and-runtime-e2e",
      "kind": "contains_page"
    }
  ]
}