{
  "id": "page:docs-research-g0-rt-live-validation-runbook",
  "_kind": "Page",
  "_file": "wiki/docs/research/g0-rt-live-validation-runbook.md",
  "_cluster": "wiki",
  "attributes": {
    "nodeKind": "Page",
    "sourcePath": "docs/research/g0-rt-live-validation-runbook.md",
    "sourceKind": "repo-docs",
    "title": "G0-RT — Live-cluster validation runbook (kradle realtime avatar agent)",
    "displayName": "G0-RT — Live-cluster validation runbook (kradle realtime avatar agent)",
    "slug": "docs/research/g0-rt-live-validation-runbook",
    "articlePath": "wiki/docs/research/g0-rt-live-validation-runbook.md",
    "article": "\n# G0-RT — Live-cluster validation runbook (kradle realtime avatar agent)\n\n> **Status:** Operator runbook (manual / cluster). Everything in the realtime-avatar harness that is\n> *headlessly* verifiable is built, tested, and green (see [`realtime-agent-gaps.md`](./realtime-agent-gaps.md) —\n> G0–G17 DONE (impl)). **G0-RT is the one thing a CI host cannot prove:** booting the real\n> `jitsi-agent-sidecar` into a live Jitsi room and confirming it (1) joins, (2) publishes the avatar\n> **video** + TTS **audio** tracks, (3) drives the avatar from the fast-path tools, and (4) honors the\n> **media-governance bridge** (G13) on a consequential tool — with `JitsiMeeting.status` populating.\n> This runbook is the step-by-step to do that on a cluster. Every field/env/route name below is cited\n> from the code so the manifests apply verbatim.\n\n---\n\n## 0. What this validates (and what it does not)\n\n| Validate (live) | Maps to |\n|---|---|\n| Sidecar Job is created + scheduled + `connect()`s to the room | dispatch→sidecar Job, `bin/sidecar.mjs` |\n| Avatar renders + a **video** track is published (`setEffect`) | G1 / G2 live-publish residual |\n| TTS **audio** track is published + lipsync rides the shared clock | G3 / G4 live-publish residual |\n| `set_expression`/`play_gesture` (fast path) visibly drive the avatar | G8 live |\n| `share_surface` (governed) → approval breakpoint → only-then socket write | **G13** governance round-trip |\n| `JitsiMeeting.status.media.agentTracks` / `session.agents` / `governanceRuns` populate | G12 live-pop residual |\n| Real noVNC RFB against a live websockify/VNC server shows a desktop | G7 real-VNC residual |\n\n**Out of scope here:** real-GPU perceptual quality (X2/X3), A/V sync drift measurement (X1/X5 — benchmark\nseparately), telephony/SIP (G18), and the `@a5c-ai/voice-adapter` TS/LiveKit + MCP async-callback delivery\n(separate follow-up). Those are tracked in the gaps doc; do not block G0-RT on them.\n\n---\n\n## 1. Prerequisites\n\n1. **Cluster access** to the kradle cluster (`kubectl` context set), and the **kradle control plane**\n   deployed (controllers + web/BFF). Org namespace convention is `kradle-org-<org>`\n   (`adapters-client.js:551`).\n2. **A reachable Jitsi deployment** the sidecar can join (the `JitsiMeetProvider` the meeting references),\n   and the JWT secret the BFF signs room tokens with (`KRADLE_JITSI_JWT_SECRET`,\n   `mcp-server.js:717`).\n3. **The agent image** (the harness wrapper) available at `KRADLE_AGENT_IMAGE` in the **web/controller pod\n   env** — this is the load-bearing env var; if unset it falls back to `ghcr.io/a5c-ai/adapters:latest`\n   (`adapters-client.js:514`). It must be in the pod's env range or the Job pulls the wrong image\n   (see Landmines).\n4. **The sidecar image** `kradle/jitsi-agent-sidecar:<tag>` built from\n   `packages/kradle/jitsi-agent-sidecar` and pushed to a registry the cluster can pull. The Job sets the\n   sidecar container image from `jitsi.sidecarImage || 'kradle/jitsi-agent-sidecar:latest'`\n   (`adapters-client.js:146`). **The image must ship a real Chromium with GPU** (`--headless=new`; old\n   headless breaks `captureStream`, see design doc D.3) — set `PUPPETEER_EXECUTABLE_PATH`/`CHROMIUM_PATH`\n   (`config.js`) and request a GPU/SwiftShader-capable node.\n5. **A license-clean avatar GLB URL** the operator supplies via `avatarModelUrl` — the sidecar **never**\n   auto-loads a CC-BY-NC asset (avatar defaults to the primitive placeholder; see\n   `src/browser/avatar.js`). Host your own RPM/GLB and point `avatarModelUrl` at it.\n6. **The model-provider Secret** for the agent container exists in `kradle-org-<org>` and is referenced as\n   `modelSecretName` — it is mounted `optional:false` (`adapters-client.js:508-510`); a missing Secret\n   means the agent pod never starts.\n\n### Build + push the sidecar image\n\n```bash\n# from repo root; do NOT run npm install on Windows (lockfile pollution) — build the image in CI/Linux\ndocker build -t <registry>/kradle/jitsi-agent-sidecar:g0rt packages/kradle/jitsi-agent-sidecar\ndocker push <registry>/kradle/jitsi-agent-sidecar:g0rt\n```\n\n---\n\n## 2. The declarative flow (create-stack → appearance/voice → meeting → dispatch)\n\nAll resources are kradle CRDs (`kradle.a5c.ai/...`); apply them into the org. **Order matters**: the\n`AgentAppearance` + `AgentVoiceProfile` must exist before the `AgentStack` validates, because the G9\nvalidator resolves `avatarRef` against `AgentAppearance` and **fails hard** if `capabilities.video ===\n'publish'` and the appearance is missing (`agent-stack-controller.js:252-259`).\n\n### 2.1 AgentAppearance (the avatar identity — G11 fields)\n\n`AgentAppearance.spec` requires `organizationRef`; the avatar fields consumed by the sidecar are\n`renderer`, `avatarModelUrl`, `visemeSet`, `defaultMood`, `defaultView`\n(`resource-model.js:36`, threaded at `jitsi-agent-bridge.js:116-124`).\n\n```yaml\napiVersion: kradle.a5c.ai/v1\nkind: AgentAppearance\nmetadata:\n  name: support-avatar\n  namespace: kradle-org-acme\nspec:\n  organizationRef: acme\n  renderer: talkinghead            # JITSI_AVATAR_RENDERER\n  avatarModelUrl: https://cdn.acme.example/avatars/support.glb   # JITSI_AVATAR_MODEL_URL (operator-hosted, license-clean)\n  visemeSet: oculus                # JITSI_AVATAR_VISEME_SET  (oculus | arkit)\n  defaultMood: neutral             # JITSI_AVATAR_DEFAULT_MOOD\n  defaultView: upper               # JITSI_AVATAR_DEFAULT_VIEW\n```\n\n### 2.2 AgentVoiceProfile (TTS identity)\n\n`AgentVoiceProfile.spec` requires `organizationRef` + `ttsProvider` (`resource-model.js:37`); voice fields\nflow to `JITSI_TTS_PROVIDER`/`JITSI_TTS_VOICE`/`JITSI_TTS_SPEED` (`adapters-client.js:128-133`).\n\n```yaml\napiVersion: kradle.a5c.ai/v1\nkind: AgentVoiceProfile\nmetadata:\n  name: support-voice\n  namespace: kradle-org-acme\nspec:\n  organizationRef: acme\n  ttsProvider: azure               # JITSI_TTS_PROVIDER  (a real provider w/ creds in the model secret)\n  ttsConfig:\n    voice: en-US-JennyNeural       # JITSI_TTS_VOICE\n    speed: \"1.0\"                    # JITSI_TTS_SPEED\n```\n\n> **Visemes** for lipsync (X1) are best driven by an Azure-style provider that emits `VisemeReceived`\n> timing — the sidecar lipsync branch activates only when the TTS descriptor carries\n> `visemes`+`vtimes` (`puppeteer-jitsi-client.js` publishAudio). A provider without visemes still\n> publishes audio; the mouth just won't sync.\n\n### 2.3 AgentStack (declare the video capability — G9)\n\nThe G9 validator (`agent-stack-controller.js:234-284`) requires, for a publishing avatar agent:\n`jitsiCapability: true`, a `jitsiMeetingProviderRef`, `role: participant` (an **observer cannot publish video**, the CRD enum is observer|participant|moderator,\n`:250`), `capabilities.video: publish` **with** a resolvable `avatarRef` (`:252-259`), valid `tools` (subset\nof the JITSI_TOOLS set, `:14-34`), and `governedTools ⊆ tools` (`:261`).\n\n```yaml\napiVersion: kradle.a5c.ai/v1\nkind: AgentStack\nmetadata:\n  name: support-avatar-stack\n  namespace: kradle-org-acme\nspec:\n  organizationRef: acme\n  jitsiCapability: true\n  jitsiMeetingProviderRef: default-jitsi      # REQUIRED — your JitsiMeetProvider\n  jitsiConfig:\n    role: participant\n    avatarRef: support-avatar                  # -> AgentAppearance above\n    capabilities:\n      video: publish\n      audio: both\n      chat: write\n      screenshare: send\n    tools:\n      - set_expression\n      - play_gesture\n      - set_posture\n      - look_at\n      - set_view\n      - publish_video\n      - draw_canvas\n      - share_surface\n      - send_video_metadata\n    governedTools:                             # MUST be a subset of tools (G9) -> these route through G13\n      - draw_canvas\n      - share_surface\n      - send_video_metadata\n    tts:                                        # optional inline; AgentVoiceProfile via dispatch identity wins\n      provider: azure\n      voice: en-US-JennyNeural\n```\n\nConfirm the stack went **Ready**:\n\n```bash\nkubectl -n kradle-org-acme get agentstack support-avatar-stack -o jsonpath='{.status.conditions[?(@.type==\"JitsiCapabilityReady\")]}{\"\\n\"}'\n# expect: status:\"True\". If \"False\"/InvalidJitsiCapability, read .message — it lists the exact violation\n# (observer-cannot-publish, avatarRef unresolved, governedTools⊄tools, unknown tool).\n```\n\n### 2.4 JitsiMeeting (the room)\n\n`JitsiMeeting.spec`: `organizationRef`, `providerRef`, `roomId`, `displayName`, `ttlMinutes`,\n`participants.invited`, `roomConfig` (`jitsi-meeting-controller.js:92-100`). Create via the BFF route so the\nroom URL + JWT signing path are exercised:\n\n```bash\ncurl -sS -X POST \"$KRADLE_WEB/api/orgs/acme/jitsi/meetings\" \\\n  -H 'content-type: application/json' \\\n  -d '{\"name\":\"support-demo\",\"displayName\":\"Support demo\",\"ttlMinutes\":30,\"providerRef\":\"default-jitsi\"}'\n# POST /api/orgs/[org]/jitsi/meetings -> createMeetingResource -> applyJitsiResource (meetings/route.js:17-25).\n# roomId defaults to \"<name>-<org>\" if unset (jitsi-service.js:81). Note the returned meeting name (meetingRef).\n```\n\n### 2.5 Dispatch the agent into the meeting\n\n```bash\ncurl -sS -X POST \"$KRADLE_WEB/api/orgs/acme/agents/dispatch\" \\\n  -H 'content-type: application/json' \\\n  -d '{\"agentStack\":\"support-avatar-stack\",\"meetingRef\":\"support-demo-acme\",\"input\":{\"title\":\"Join and greet the room\"}}'\n# dispatch/route.js:7-54 -> controller.dispatchAgent({...meetingRef}) -> prepareMeetingContext\n# (jitsi-agent-bridge.js:62-136) resolves avatar+voice and threads meetingContext -> createAgentJob.\n```\n\nThis creates a `batch/v1` Job `kradle-agent-<runId>` in `kradle-org-acme` with the agent container **and**\nthe `jitsi-agent-sidecar` container (`adapters-client.js:546-577`, sidecar at `:102-158`). The sidecar gets\nthe full `JITSI_*` env (room/jwt/role + `JITSI_VIDEO_MODE=publish` + the `JITSI_AVATAR_*` + `JITSI_TTS_*`\nvars) and an `agent-socket` emptyDir mounted at `/tmp` shared with the agent container; the MCP video tools\nwrite to `AGENT_SOCKET_PATH=/tmp/jitsi-agent.sock` (`adapters-client.js:67,125,481`).\n\n---\n\n## 3. Verification checkpoints\n\n### C1 — Job scheduled + both containers up\n```bash\nkubectl -n kradle-org-acme get job -l kradle.a5c.ai/component=agent-run\nkubectl -n kradle-org-acme get pods -l kradle.a5c.ai/run=<runId>\n# Both 'agent-run' and 'jitsi-agent-sidecar' containers should be Running (not ImagePullBackOff / CreateContainerConfigError).\n```\n**If the Job is missing entirely**, check the org-scope landmine (§4.1) and the controller logs.\n\n### C2 — Sidecar joined the room\n```bash\nkubectl -n kradle-org-acme logs <pod> -c jitsi-agent-sidecar | grep -iE \"connected|roomId|connect\"\n# bin/sidecar.mjs boots loadConfig -> createPuppeteerJitsiClient -> ipc.start() -> runtime.start()\n# -> jitsi.connect(); runtime broadcasts {type:'connected', roomId, participants} (runtime.js:43-48).\n```\nOpen the room URL (`.status.roomUrl`) in a browser as a human participant — you should see the agent\ntile join.\n\n### C3 — Avatar video + TTS audio published\n- In the room, the agent tile should show the **rendered avatar** (not a black/fake tile) and you should\n  **hear** TTS when the agent speaks. The video track is published via `captureStream → setEffect`\n  (`src/browser/publish-effect.js`); audio via the shared Web-Audio graph.\n- Inspect: `kubectl ... logs -c jitsi-agent-sidecar | grep -iE \"avatar|kradleAvatarBoot|setEffect|publishAudio\"`.\n  A failed avatar boot is surfaced (not swallowed) on `window.__kradleAvatarBoot.error` and as\n  `[kradle-avatar] avatar injection failed` (`puppeteer-jitsi-client.js`); a lipsync fault surfaces as\n  `[kradle-avatar] lipsync publish failed` (no silent degrade-to-tone).\n\n### C4 — Fast-path tools drive the avatar\nCall the MCP video tools (via the agent loop or the kradle MCP server) and watch the tile:\n```\nkradle_set_expression { mood: \"happy\" }   # -> /tmp/jitsi-agent.sock action set_expression -> avatar.setMood\nkradle_play_gesture   { gesture: \"wave\" }  # -> play_gesture -> avatar.playGesture\n```\nThese are the **fast lane** — no governance, immediate (`mcp-server.js:644-659` → direct `{socketPath,command}`).\n\n### C5 — Governed tool round-trip (G13, the key check)\n```\nkradle_share_surface { surface: \"browser\", url: \"https://slides.example/deck\" }\n```\nExpected sequence:\n1. The MCP tool returns a **governance descriptor** (`governed:true`, `correlationId`, `filler`,\n   `policy.decision:\"require-approval\"`, **no `command`**) — `mcp-server.js` `governedToolDescriptor`.\n2. The bridge/driver creates a governed run; the agent speaks the `filler`; an **owner approval\n   breakpoint** (`auth.share-surface`) is raised.\n3. **Only after approval** does the run emit `{status:\"approved\", socketPath, command:{action:\"share_surface\",...}}`\n   and the socket write reaches the sidecar → the surface composites into the video.\n4. A **policy hard-deny** (e.g. `surface: \"system\"`) returns `denied` and **never** reaches the socket.\nConfirm a denied/unapproved `share_surface` produces **no** visible surface change.\n\n### C6 — JitsiMeeting.status populates\n```bash\nkubectl -n kradle-org-acme get jitsimeeting support-demo-acme -o jsonpath='{.status.media}{\"\\n\"}{.status.session}{\"\\n\"}{.status.governanceRuns}{\"\\n\"}'\n# media.agentTracks (published tracks), session.agents (the dispatched agent), governanceRuns (the G13 runs).\n# NOTE: live governanceRuns population is itself a documented follow-up (G13 bridge live-pop) — if empty,\n# that is the known residual, not a C5 failure; C5 is judged by the socket-write/approval behavior above.\n```\n\n### C7 — (optional) real noVNC / screen-share (G7)\nPoint `share_surface` at a **VNC websocket** (`url: \"wss://<websockify-host>/...\"`) backed by a live\nwebsockify→VNC server. The sidecar lazily loads noVNC RFB and composites the desktop. (Headless verify only\never used a synthetic source; this is the real-VNC residual.)\n\n---\n\n## 4. Known live landmines (check these first when something fails)\n\n### 4.1 `withOrgScope` must not inject `organizationRef` into the Job\nThe dispatch Job is `batch/v1` (not a kradle CRD). `withOrgScope` skips non-`kradle.a5c.ai/*` resources\n(`kubernetes-controller.js:378-384`); if a regression reintroduces injection, the Kubernetes strict decoder\nrejects `spec.organizationRef` and **silently drops every Job** (the run exists, no Job appears). Symptom:\nC1 shows a run but **no Job**. (Memory: this exact bug shipped once.)\n\n### 4.2 `KRADLE_AGENT_IMAGE` must be in the controller/web pod env range\nThe agent image resolves to `config.image || process.env.KRADLE_AGENT_IMAGE || 'ghcr.io/a5c-ai/adapters:latest'`\n(`adapters-client.js:514`). If the controller pod doesn't carry `KRADLE_AGENT_IMAGE`, the Job pulls the\npublic default — wrong/older harness. Symptom: agent behaves unexpectedly or `ImagePullBackOff`. Set it in\nthe **web-pod secret env**, not just your shell.\n\n### 4.3 Workspace = `emptyDir`, not a PVC, on AKS\nPer-run workspaces use `workspace.ephemeral:true` → `emptyDir` (`adapters-client.js:486-495`,\n`agent-dispatch-controller.js:442-451`). On AKS the disk CSI provisioning timed out and left pods\n**Pending**; emptyDir binds instantly. If you switch to `pvcName`, expect provisioning latency. Symptom:\npod stuck `Pending` with a volume-attach event.\n\n### 4.4 Avatar is **required** for `video: publish` — fail-hard, no fallback\nIf `capabilities.video === 'publish'` and `avatarRef` doesn't resolve, validation/dispatch **throws**\n(`agent-stack-controller.js:256-258`, `jitsi-agent-bridge.js:91-92`) — there is no silent fallback (repo\nrule). Apply the `AgentAppearance` **before** the stack.\n\n### 4.5 Model-provider Secret is `optional:false`\nA missing `modelSecretName` Secret in `kradle-org-<org>` means the **agent** container never starts\n(`adapters-client.js:508-510`). Symptom: `CreateContainerConfigError` on the agent container while the\nsidecar may be Running.\n\n### 4.6 claude-code adapter needs the proxy `apiBase`\nIf the agent adapter is claude-code over the gateway, the launcher must receive the transport endpoint\n(`AGENT_MUX_TRANSPORT_ENDPOINT`, `adapters-client.js:471`) / `--api-base` — otherwise the launcher makes no\ncompletion engine and 404s (memory: resolved). Verify the transport binding endpoint is threaded.\n\n### 4.7 Headless Chromium + GPU\nOld headless Chromium breaks `mediaDevices`/`captureStream` (design doc D.3). Use `--headless=new` and a\nreal/SwiftShader GPU; set `PUPPETEER_EXECUTABLE_PATH` or `CHROMIUM_PATH` (`config.js`). Symptom: avatar tile\nblack or no video track.\n\n---\n\n## 5. Pass / fail criteria\n\nG0-RT **passes** when, in a live room: C1–C4 hold (Job up, sidecar joined, avatar A/V published, fast tools\nvisibly drive the avatar) **and** C5 holds (a governed `share_surface` is gated by an owner approval and the\nsocket write happens only on approval; a hard-deny never reaches the sidecar). C6 populating is desirable\nbut its `governanceRuns` live-population is a tracked follow-up; C7 (real noVNC) is optional.\n\nRecord the run: the `runId`, the Job name, the sidecar logs for C2/C3, a screenshot of the agent tile for\nC3/C4, and the approval/deny outcomes for C5. File any failure against the specific landmine in §4.\n\n---\n\n## 6. After a green G0-RT\n\nFlip the live-publish residuals in [`realtime-agent-gaps.md`](./realtime-agent-gaps.md): G1/G2/G3/G4\n`live-publish pending` → done, G12 `governanceRuns` live-pop (if C6 populated), and note G0-RT verified with\nthe recorded evidence. The remaining open items are then only the larger follow-ups: the\n`@a5c-ai/voice-adapter` TS/LiveKit package + MCP async-callback delivery, and G18 (LiveKit/SIP telephony).\n",
    "documents": []
  },
  "outgoingEdges": [],
  "incomingEdges": [
    {
      "from": "page:docs",
      "to": "page:docs-research-g0-rt-live-validation-runbook",
      "kind": "contains_page"
    }
  ]
}