{
  "id": "page:docs-reference-repos-sero-labs-sero-sero-concepts-and-opportunities",
  "_kind": "Page",
  "_file": "wiki/docs/reference-repos/sero-labs/sero/sero-concepts-and-opportunities.md",
  "_cluster": "wiki",
  "attributes": {
    "nodeKind": "Page",
    "sourcePath": "docs/reference-repos/sero-labs/sero/sero-concepts-and-opportunities.md",
    "sourceKind": "repo-docs",
    "title": "Sero Architecture Concepts — What We Can Learn & Build",
    "displayName": "Sero Architecture Concepts — What We Can Learn & Build",
    "slug": "docs/reference-repos/sero-labs/sero/sero-concepts-and-opportunities",
    "articlePath": "wiki/docs/reference-repos/sero-labs/sero/sero-concepts-and-opportunities.md",
    "article": "\n# Sero Architecture Concepts — What We Can Learn & Build\n\n## The Big Ideas\n\n### 1. Agent-as-Workspace (not Agent-in-Sidebar)\n\nSero's fundamental insight: **put the agent inside the workspace, not the workspace inside the agent.** Traditional agents live in a chat tab. Sero makes the agent the ambient intelligence of the entire development environment — it sees files, branches, running apps, terminal output, and browser content as native context, not injected snippets.\n\n**What this means for us:**\nOur genty/desktop-app is currently page-routed (Home → Sessions → Agents → Kanban). Each page is a standalone view. Sero's model is a **split-pane persistent workspace** where chat, terminal, file tree, and browser coexist simultaneously — the agent's context IS the workspace layout.\n\n**What to build:**\n- Redesign genty/desktop-app as a **mosaic workspace** (react-mosaic or allotment) with persistent split panes\n- Chat pane is always visible alongside the tool output pane\n- File explorer pane shows the project the agent is working in\n- Terminal pane shows agent's bash commands running in real-time\n- When the agent takes a screenshot, it appears inline in the conversation — not as a separate page\n\n### 2. Self-Extending Plugin Paradigm\n\nSero's most powerful concept: **the agent builds its own plugins on demand.** A user says \"I need a tool to analyze my test coverage\" → the agent scaffolds a plugin with manifest + tool implementation + optional React UI panel → it's live immediately via Module Federation, no restart.\n\nThis is fundamentally different from our plugin system. Our plugins are:\n- Created by developers, installed via marketplace\n- Static once installed — require CLI commands to modify\n- Separate from the agent's reasoning flow\n\nSero's plugins are:\n- Created BY the agent during a conversation\n- Live-reloadable (Module Federation)\n- Iteratively refined (\"extend this tool to also show trends\")\n- Persistent — once created, available in all future sessions\n\n**What to build:**\n- Add a `createPlugin` agentic tool to genty/core that lets the agent scaffold a plugin directory during a conversation\n- Add hot-reload to genty/desktop-app and genty/web-app via Vite's HMR or Module Federation\n- The agent builds the plugin, loads it, and the user immediately sees a new panel/tool without restarting\n- Store agent-created plugins in `.genty/plugins/` per project — they persist across sessions\n- The agent can later `extendPlugin` to add capabilities\n\n### 3. Content vs Details: Separating LLM Context from UI Rendering\n\nPi's tool response pattern elegantly solves a problem we haven't addressed:\n\n```\nTool returns:\n  content: \"Temperature is 72°F, sunny\"      ← goes to LLM context\n  details: {temp: 72, forecast: [...]}         ← goes to UI rendering\n```\n\nThe LLM sees clean text. The UI renders rich structured data. Neither pollutes the other.\n\n**What this means for us:**\nOur tool results currently serialize everything as text for the LLM. If a tool returns a table, the LLM gets the table as text AND the UI shows the same text. There's no separation between \"what the model needs to reason about\" and \"what the user needs to see.\"\n\n**What to build:**\n- Extend the tool result type in genty/core to include `content` (for LLM) + `details` (for UI)\n- Update EventCards in genty/ui to render `details` as rich UI (charts, tables, images, interactive widgets)\n- The LLM gets concise summaries; the user gets full visual richness\n- This is particularly powerful for: test results, performance metrics, deployment status, code coverage\n\n### 4. Session as Tree (not Linear Chat)\n\nPi represents conversation history as a **tree, not a list.** Each message is a node with a parent pointer. Users can fork (`/fork`) to explore alternative approaches without losing the original path. The LLM sees a linearized path from root to current leaf.\n\n**What this means for us:**\nOur sessions are linear — a flat list of messages. If the agent goes down a wrong path, you either undo manually or start over. There's no concept of \"try this approach\" and \"try that approach\" as branches.\n\n**What to build:**\n- Add `parentId` to session messages in sdk/session\n- Add `/fork` and `/clone` commands to the conversation\n- Visualize the session tree in genty/ui (like a git graph — nodes with branches)\n- Enable \"switch to branch 2\" mid-conversation\n- Context compaction can summarize entire dead-end branches\n\n### 5. Hooks as Surgical Extension Points\n\nPi's hooks system lets extensions intercept the agent loop at precise points:\n- `beforeToolCall` / `afterToolCall` — intercept any tool execution\n- `message_start` / `message_end` — observe LLM reasoning\n- `turn_start` / `turn_end` — lifecycle boundaries\n\nReal uses: plan mode (intercept tool calls, show user, wait for approval), git checkpointing (auto-commit before file mutations), custom compaction triggers.\n\n**What this means for us:**\nWe already have hooks-adapter with PreToolUse/PostToolUse/Stop/SessionStart/SessionEnd events. But our hooks are primarily for cross-harness normalization — making different CLI agents behave consistently. Sero's hooks are about **composing agent behavior from interceptors.**\n\n**What to build:**\n- Expose hooks to agent-created plugins (not just installed harness plugins)\n- Let plugins register `beforeToolCall` handlers that can modify, approve, or block tool execution\n- This enables: approval gates per-tool, automatic git commits before file edits, logging/auditing, cost budgets, safety filters\n- These are different from our babysitter breakpoints — they're lightweight, inline, and composable\n\n### 6. Specialist Agents as Markdown Files\n\nSero's built-in agents (scout, reviewer, test-writer) are plain markdown files:\n```yaml\n---\nname: scout\ntools: [fetch, search]\nmodel: claude-opus\n---\n# Scout Agent\nYou are an expert researcher...\n```\n\nEditable, forkable, composable. Not code — just configuration + prompt.\n\n**What this means for us:**\nWe already have this pattern — `.claude/agents/` has markdown agent definitions. But we also have the richer AgentPersona/AgentSoul/AgentDefinition CRD system in Kradle. The opportunity is to **bridge these** — let markdown agent files work as lightweight definitions that can be promoted to full CRD personas when needed.\n\n**What to build:**\n- Support `.genty/agents/` directory with markdown agent specs (we partially have this via .claude/agents/)\n- Agent specs should compose: \"use scout to research, then reviewer to check, then test-writer to verify\"\n- Enable the agent to CREATE new specialist markdown files during a conversation: \"create a database-migration-specialist agent\"\n- These become reusable across projects\n\n### 7. Context Compaction with Git Checkpointing\n\nPi's compaction strategy: before summarizing old conversation turns, create a git commit. The agent can later reconstruct state via `git log` and `git diff`. This means compaction doesn't lose information — it just moves it from conversation context to git history.\n\n**What this means for us:**\nOur compaction in the SDK compresses conversation history but doesn't coordinate with external state. If the agent was mid-refactor when compaction hits, the conversation summary might lose crucial decisions.\n\n**What to build:**\n- Before compaction, auto-commit current changes with a descriptive message\n- Include the commit SHA in the compaction summary: \"Previous work committed at abc123\"\n- After compaction, the agent can run `git diff abc123..HEAD` to see what changed\n- This is a safety net — compaction becomes reversible\n\n### 8. Visual Reasoning Loop\n\nSero's browser isn't just for automation — it creates a **closed reasoning loop:**\n1. Agent writes code\n2. Code renders in browser\n3. Agent takes screenshot\n4. Agent sees the result visually\n5. Agent fixes issues based on what it sees\n6. Repeat\n\nWith `--annotate`, UI elements get numbered labels that the agent can reference.\n\n**What this means for us:**\nOur Puppeteer tool takes screenshots but doesn't feed them back into the agent's reasoning loop naturally. The agent takes a screenshot, gets base64 data, but there's no automatic \"look at this and tell me what's wrong\" flow.\n\n**What to build:**\n- Add a `visualInspect` tool that takes a screenshot + runs the agent's vision capabilities to describe what it sees\n- Integrate with the tool result `content/details` pattern: `content` = \"The button is misaligned\" / `details` = screenshot image\n- Add annotated screenshots (numbered element labels) for precise element references\n- Enable \"watch mode\" — agent monitors a running dev server and reacts to visual changes\n\n## Synthesis: The Meta-Pattern\n\nAll of Sero's concepts share a meta-pattern: **collapse the distance between the agent and the developer's actual context.**\n\n| Traditional Agent | Sero/Pi Pattern | Our Opportunity |\n|---|---|---|\n| Chat in a tab | Agent IS the workspace | genty/desktop-app as mosaic workspace |\n| Pre-built plugins | Agent builds plugins on demand | createPlugin agentic tool + hot reload |\n| Text-only tool results | Content (LLM) + Details (UI) | Extend tool result type |\n| Linear chat history | Tree-structured sessions | Fork/clone/branch conversations |\n| External hooks config | Composable hook interceptors | Plugin-registered beforeToolCall handlers |\n| Code-defined agents | Markdown agent specs | .genty/agents/ with live creation |\n| Lossy compaction | Git-checkpointed compaction | Auto-commit before compact |\n| Screenshot-then-describe | Visual reasoning loop | visualInspect + annotated screenshots |\n\nThe unifying principle: **the agent should be embedded in the developer's reality, not a separate tool they switch to.** Every concept above serves this goal — reducing context switches, maintaining continuity, and enabling the agent to see what the developer sees.\n",
    "documents": []
  },
  "outgoingEdges": [],
  "incomingEdges": [
    {
      "from": "page:docs-reference-repos",
      "to": "page:docs-reference-repos-sero-labs-sero-sero-concepts-and-opportunities",
      "kind": "contains_page"
    }
  ]
}