II.
Page reference
Reading · 8 minpage:docs-testing-adapters-and-runtime-e2e
Agent Adapter And Runtime E2E reference
This strategy covers runtime paths after setup is already satisfied. It separates adapters sessions, transport carriers, genty-core programmatic sessions, and @a5c-ai/genty-platform orchestration. Harness/plugin install coverage lives in Harness And Plugin E2E(./harness-e2e.md), not in agent-platform runtime E2E.
Agent Adapter And Runtime E2E
This strategy covers runtime paths after setup is already satisfied. It separates adapters sessions, transport carriers, genty-core programmatic sessions, and @a5c-ai/genty-platform orchestration. Harness/plugin install coverage lives in Harness And Plugin E2E, not in agent-platform runtime E2E.
Stack Scopes
| Scope | Packages | No-model coverage | Model-backed coverage |
|---|---|---|---|
| Protocol and event contracts | packages/adapters/core, packages/adapters/gateway, packages/transport-adapter | Schema, event ordering, session lifecycle, error envelopes, reconnect behavior | Real event streams from Codex and Claude Code sessions match protocol contracts |
| Adapter translation | packages/adapters/adapters | Prompt normalization, tool-call mapping, stop reasons, model selection, fallback behavior with mock adapters | Live Codex and Claude Code adapters translate real provider output into adapter events |
| Transport runtime | packages/transport-adapter | Route matrix, proxy auth, protocol codecs, runtime env injection, passthrough forwarding, subprocess lifecycle, stream cancellation, timeout/error paths, metrics/cache snapshots | Transport-adapter carries traffic for a real external harness through adapters launch and for an genty-core-backed stream |
| Agent-core bridge | packages/genty/core | Programmatic session creation, mock provider responses, cancellation, usage accounting | Agent-core invokes a real provider and returns events compatible with adapters and agent-platform |
| Hooks muxes | packages/adapters/hooks/* | Adapter normalization, hook payload fixtures, CLI execution, approval/deny/error events | Real harness hook payloads from Codex and Claude Code normalize to the same hook contract |
| Babysitter-agent runtime | packages/genty/platform | Seam contract, phase orchestration, planner/executor mocks, run journal state, task posting, selected backend | agent-platform call/create-run/invoke uses preinstalled or mocked backends; no harness install or plugin install steps are part of this E2E |
| User surfaces | packages/adapters/webui, packages/adapters/ui, packages/adapters/tui | Playwright/Vitest against mock gateway and fixture sessions | Optional manual/live smoke against a model-backed gateway session |
No-Model Runtime Suite
The no-model runtime suite should be built first. It should include:
transport-adapterunit and E2E tests using local HTTP/subprocess fixtures for every supported exposed transport route.transport-adapterruntime tests forstartTransportMuxRuntime,applyTransportMuxToHarnessEnv, proxy auth, redacted env diffs, metrics, cache stats, passthrough path/query preservation, invalid JSON, and upstream failure mapping.adapterslaunch-plan tests for proxy forced, proxy if-needed, native/no-proxy, and proxy-forbidden cases so launch coverage proves the transport-adapter decision seam.adaptersgateway/session tests using existing mock harness scenarios.- Adapter translation tests for Codex, Claude Code, and genty-core-style event streams.
agent-platformseam and orchestration tests with mocked planner/executor calls.- WebUI and TUI session tests using fixture transcripts and mock gateway responses.
- Adapters plugin/session fixtures live in Harness And Plugin E2E; this file consumes their event fixtures only as runtime compatibility inputs.
Candidate command grouping:
npm run test --workspace=@a5c-ai/transport-adapter
npm run test --workspace=@a5c-ai/comm-adapter
npm run test --workspace=@a5c-ai/adapters-codecs
npm run test --workspace=@a5c-ai/adapters-gateway
npm run test --workspace=@a5c-ai/genty-core
npm run test --workspace=@a5c-ai/genty-platform
npm run test:e2e --workspace=@a5c-ai/genty-web-appTransport-Adapter Coverage Matrix
Transport-adapter coverage has to prove the proxy/runtime seam directly before it is used as evidence for adapters or agent-platform paths.
No-Model Transport Coverage
| Surface | Tests to add or keep | Required artifacts |
|---|---|---|
| Supported transport routes | Exercise anthropic, openai-chat, openai-responses, google, bedrock-converse, azure-foundry, vertex-native, and passthrough against fixture engines | Route transcript, response shape snapshot, invalid JSON/auth failure transcript |
| Streaming and non-streaming codecs | Verify text deltas, final events, finish reasons, usage totals, and provider-specific envelopes | Streaming event transcript and non-streaming body snapshot |
| Token and observability routes | Cover count_tokens success/unsupported behavior plus /metrics and /cache/stats | Token count transcript, metrics snapshot, cache stats snapshot |
| Runtime env injection | Call applyTransportMuxToHarnessEnv for every exposed transport | Redacted env diff proving only expected vars changed |
| Adapters launch seam | Cover resolveLaunchPlan for proxy forced, proxy if-needed, native/no-proxy, and forbidden proxy cases | Launch-plan JSON with proxyNeeded, proxyReason, and exposed transport |
| Passthrough forwarding | Preserve path/query/body, inject upstream auth safely, and map upstream failures | Redacted upstream transcript and failure envelope |
Model-Backed Transport Coverage
| Path | Valid stack | Assertion focus |
|---|---|---|
| genty-core stream bridge | genty-core provider backend -> transport-adapter -> fixture consumer | Real or credential-gated stream deltas, final event, cancellation/timeout, and usage metadata survive the proxy layer |
| External harness bridge | adapters launch <harness> <provider> --with-proxy* -> transport-adapter -> target provider | Harness receives proxy env, provider endpoint is not called directly by the harness, sentinel prompt completes, metrics show traffic |
| Babysitter-agent precondition bridge | agent-platform external-harness path only when it delegates through adapters and the selected harness requires proxy translation | Transport-adapter artifacts are supporting evidence for the bridge, while Babysitter run creation/task posting remains asserted by agent-platform tests |
Invalid Transport Claims
- Do not use transport-adapter tests to prove
babysitter harness:installorharness:install-plugin. - Do not use transport-adapter tests to prove agent-native plugin manager support.
- Do not use transport-adapter tests to prove hooks-adapter normalization.
- Do not use transport-adapter tests to prove Babysitter journal terminal state unless a higher-level runtime test also asserts that state.
Live Install Modes
The Publish workflow runs external-harness live E2E through a workflow-owned install-mode axis:
babysitter-plugingenerates plugin artifacts, installs the target withadapters install, installs the local Babysitter SDK, installs the Babysitter plugin for the harness, then launches throughadapters launchwith a/babysitter:callprompt.vanillainstalls the target withadapters install, launches throughadapters launch, and uses a non-plugin prompt so it proves adapters/transport/provider behavior; the vanillaagent-platformrows use thebabysitteradapter withBABYSITTER_HARNESS=genty-core.- Both modes use the same target mapping:
claude-code -> claude,codex -> codex,gemini-cli -> gemini,pi -> pi, and vanilla-onlyagent-platform -> babysitter.
Model-Backed Runtime Suite
The model-backed suite should prove that real providers and real harnesses behave like the no-model contracts expect.
| Test | Required real dependency | Assertion focus |
|---|---|---|
| Transport-adapter + external harness through adapters | Claude Code or Codex-compatible harness, provider credential, and adapters launch --with-proxy or --with-proxy-if-needed | Launch starts transport-adapter, harness receives proxy env, sentinel traffic uses proxy routes, stream completes, metrics snapshot increments |
| Transport-adapter + genty-core | Provider credential for genty-core backend | Agent-core deltas/final events travel through transport-adapter without adapter-only assumptions, including cancellation or timeout evidence |
| Adapters + Codex adapter | Codex CLI or configured Codex runtime and OpenAI credential | Codex output maps to adapter protocol events, including final message and usage metadata when available |
| Adapters + Claude Code adapter | Claude Code CLI plus Foundry/OpenAI credential through transport-adapter | Claude Code output maps to adapter protocol events while model traffic is proxied to GPT-5.5, including tool-call and stop metadata when available |
| Babysitter-agent full run | Provider credentials or mocked backend already available | agent-platform call/create-run creates a bounded process, plans, emits a task, posts a result, completes, and records selected backend evidence without running installer commands |
Model-backed runtime tests must upload redacted event logs, provider/harness version metadata, run IDs, and command durations.
Runtime Path Assertions
Runtime tests must declare which entry path they exercise:
| Path | Entry point | Valid backend combinations | Assertions |
|---|---|---|---|
| Adapters session | adapters run <agent> or createClient().run | Mock adapter, Claude, Codex, Gemini, Cursor, OpenCode, adapters babysitter adapter where registered | Session start/end, event ordering, provider/model config, runtime hooks, capability-gated plugin events |
| Babysitter-agent internal runtime | agent-platform call/create-run --harness genty-core | genty-core backend with mocked or live model provider | Run creation, planning, task posting, terminal state, redacted model trace |
| Babysitter-agent external-harness bridge | agent-platform call/invoke --harness <external> | Harness names mapped in adapterHarnessMap; excludes pi and genty-core | Adapters mapped events, session ID, result, selected harness, no install commands |
| Transport runtime | transport-adapter around genty-core or adapters-launched external harness traffic | Local route fixture, genty-core stream, external harness stream | Route/codec contract, proxy auth, env injection, launch proxy decision, framing, reconnect, cancellation, timeout, backpressure, metrics/cache artifact |
Do not fold plugin setup into the agent-platform runtime assertions. If a runtime job needs an installed external harness or plugin, that is a precondition supplied by a setup job and recorded separately.
Adapter-Specific Assertions
Adapter tests should assert behavior that package-local unit tests cannot prove alone:
- A session can be started, observed, cancelled, and resumed through the adapter boundary.
- Tool-call, text-delta, final-message, usage, and error events preserve ordering and session IDs.
- Adapter-specific errors are normalized before they cross gateway or transport boundaries.
- Model selection is explicit and recorded in the session state.
- Credential absence is detected before provider calls are attempted.
- Mock and live event streams conform to the same protocol fixtures.
Babysitter-Agent Whole-System Assertions
Whole-system tests for @a5c-ai/genty-platform should cover:
- process loading and validation,
- run creation,
- session binding,
- planning phase output shape,
- task effect emission,
- task result posting,
- journal rebuild/repair compatibility,
- terminal run state,
- artifact and log redaction.
The no-model version should use mocks for planner and executor behavior. The model-backed version should use the smallest possible bounded process and real model credentials or a preconfigured external harness. It must not execute harness:install or harness:install-plugin as part of the agent-platform runtime test.
Hooks-Adapter Assertions
Hooks-adapter tests should cover both adapter-local behavior and end-to-end event compatibility:
- each adapter normalizes raw harness hook payloads into the shared hook contract,
- CLI execution preserves stdin/stdout/stderr boundaries and exit codes,
- approval, denial, timeout, and malformed-payload cases are fixture-backed,
- Codex and Claude Code live hook payloads can be redacted and replayed as no-model fixtures,
- adapters UI and TUI approval surfaces consume the same normalized hook events.
Hooks-adapter live coverage should not be promoted until the no-model fixture suite covers the same event types.