II.
Page overview
Reference · livepage:docs-release-pipeline
Continuous Release Pipeline overview
Inspect the raw attributes, linked wiki pages, and inbound or outbound graph edges for page:docs-release-pipeline.
Attributes
nodeKind
Page
sourcePath
docs/release-pipeline.md
sourceKind
repo-docs
title
Continuous Release Pipeline
displayName
Continuous Release Pipeline
slug
docs/release-pipeline
articlePath
wiki/docs/release-pipeline.md
article
# Continuous Release Pipeline
The private product repository `a5c-ai/fabric` publishes its 43 `@fabricstack/*` packages to
exactly one registry — the GCP Artifact Registry npm repository
`https://us-central1-npm.pkg.dev/boot-464019/fabric-npm/` (spec 15 §4, WP-RN-03/RN-04/RN-07) —
under immutable `fabric/<branch>/v<version>` tags. The public repository `a5c-ai/babysitter`
keeps its own `@a5c-ai/*` names, `babysitter/<branch>/v*` tags and public-npm publishing, and is
never written to by anything described here (spec 15 §1.2; `docs/DIVERGENCE-POLICY.md`).
The operator procedures — normal train, dry-run rehearsal, half-published-train recovery,
dist-tag repair, public-namespace invariant checks — are in
[docs/RELEASE-RUNBOOK.md](RELEASE-RUNBOOK.md). This page is the model those procedures encode.
## Single authoritative release version (FIX-001)
The branch release workflow (`.github/workflows/publish-npm-private.yml`) is the **sole owner**
of npm publication and channel promotion. One immutable release version flows through the whole
pipeline; nothing downstream re-derives it.
The versioning posture itself — lockstep train, exact-pinned internal dependencies, majors for
breaking trains, and why per-package versioning was rejected — is recorded once in
[docs/VERSIONING.md](VERSIONING.md).
| Stage | Where the version comes from |
| --- | --- |
| Resolution | `node scripts/release-version.cjs resolve --branch <b> --sha <short-sha>` in `resolve_release`, exactly once. `main` releases `X.Y.Z` from the root manifest; `staging`/`develop` release `X.Y.(Z+1)-<branch>.<short-sha>`. Any other branch is a hard error. The resolved identity carries the version, the channel dist-tag, the `fabric/<branch>/v<version>` tag and the registry. |
| Publication checkout | `publish_candidates` re-validates the identity against the resolver for its own checkout (`release-version.cjs verify --write release-version.json`); every helper reads that plan. |
| Manifest synchronization | `scripts/sync-workspace-versions.mjs --version <releaseVersion>`, immediately verified by `release-version.cjs verify-manifests` and by `scripts/ci/publish-isolation-guard.cjs` over the synchronized tree (G1). |
| Publication | `scripts/publish-package-from-tag.mjs` fails unless the workspace manifest version, the release plan / `RELEASE_VERSION` / `--release-version`, and the tag version all agree, and unless the manifest pins the private registry (G1). It publishes under the candidate dist-tag (see FIX-010 below), never a channel, and only with the explicit opt-in `FABRIC_RELEASE_PUBLISH=1`; `--dry-run` rehearses with `npm publish --dry-run`. |
| Publication order | Derived from the package dependency graph: `scripts/release-matrix.cjs --group all-publishable --format waves` — the same waves the train builds in. |
| Channel assertion | `release-version.cjs assert-channel-tags` runs `preflight` before publishing (no backward channel movement), `final` over the candidate dist-tag after publication (every package published), and `final` over the channel after promotion. Every query names the private registry explicitly. |
| Tagging | `.github/workflows/release-tags.yml` **accepts** the version as an input and validates it against the resolver. The tag is `fabric/<branch>/v<releaseVersion>` — the name contains the exact published version and nothing else — annotated with `fabric-release-*` provenance, created on this repository only, after promotion and the channel assertion. |
`.github/workflows/publish-packages-from-tag.yml` is a **manual/recovery path only**. It derives
the exact version from a `fabric/<branch>/v<version>` tag (any other namespace, including the
public repository's, is rejected), refuses to run for a tag the publish workflow already
published from (no second dist-tag mutation), synchronizes every manifest to that version on
every channel, publishes in dependency order — skipping, but candidate-tagging, any package whose
exact version already exists, so a half-published train completes idempotently — validates, and
only then promotes and asserts the channel tags.
Re-running a completed release is idempotent: nothing is re-published, the tag is not moved, and
the channel assertion passes unchanged. See `docs/release-incident-2026-08-13.md` for the incident
this model prevents.
## Candidate publication and validated promotion (FIX-010)
Publishing a package is **not** releasing it. Publication writes a non-production candidate
dist-tag; a release channel moves only after the exact version has been installed back from
Artifact Registry and exercised.
| Stage | Contract |
| --- | --- |
| Candidate dist-tag | `node scripts/release-promotion.cjs candidate-tag --version <v>` → `candidate-<v>`. `scripts/publish-package-from-tag.mjs` publishes every package under it and never writes `latest`, `staging` or `develop`. |
| Published-consumer validation | `publish_candidates` → `published_consumer_validation`, which calls `.github/workflows/live-stack-published.yml` with the EXACT release version and the candidate dist-tag. Mutable inputs (dist-tags, ranges, partial versions) are rejected by `release-promotion.cjs assert-exact-version`. |
| Clean-consumer checks | `scripts/verify-published-release.mjs --version <v>` installs every public package at `@<v>` into a throwaway project outside the repository whose own `.npmrc` routes `@fabricstack` to Artifact Registry, imports every root and exported runtime subpath, and smokes every declared bin. Consumer surfaces come from `scripts/lib/package-surface.cjs`, shared with the pre-publication FIX-011 gate. The same job then performs the spec 15 §3.1 global-install smoke (WP-RN-06 AC2): `@fabricstack/orchestrator` + `@fabricstack/harness-cli`, then the hooks CLI and the harness SDK, with no `EEXIST` and exactly the §3.1 bins. |
| Live-stack execution | The representative published live-stack lanes install the §3.1 CLIs globally from Artifact Registry at the same exact version and assert it with `release-promotion.cjs assert-installed`. |
| Evidence | `release-promotion.cjs record-validation` writes `validation.json` (required checks: `package-install`, `root-import`, `subpath-import`, `bin-smoke`, `live-stack`) and uploads it as the `published-consumer-validation` artifact — on failure too, for incident review. |
| Promotion | `promote_release_channel` runs only when the validation job succeeded, and `release-promotion.cjs promote` independently refuses without evidence naming this exact version with every required check successful. It moves every package's channel tag (naming the private registry on every `npm dist-tag add`) and `assert_release_channel_tags` re-asserts the channel over the full inventory. |
| Failure behaviour | No channel tag is touched. The candidate stays installable as `<pkg>@<version>` and under `candidate-<version>` for diagnosis. |
| Recovery | `workflow_dispatch` on `live-stack-published.yml` takes the same exact-version input; `.github/workflows/publish-packages-from-tag.yml` follows the identical candidate → validate → promote sequence from the immutable tag. |
## Packed-artifact gate before publication (FIX-011)
Both publish paths call `.github/workflows/release-artifact-verifier.yml` with the full
43-package matrix and `allow_known_failures: 'false'`, and the one publishing job
(`publish_candidates` / `publish_non_plugin_packages`) both `needs` it and checks its result.
Per package: build → exact tarball → packed-manifest scope/registry gate (G6) → surfaces, bins,
shebangs → fresh install without workspace links (internal dependencies from tarballs packed in
the same run, the private-scope routing line in the consumer's `.npmrc`) → root and subpath
imports → consumer typecheck → exports exactness → bin smoke → the package's own
`verify:release`.
## Package inventory and release matrices
There is **one** authoritative answer to "which packages does this repository
publish", and every build, test, version, and publish matrix derives from it.
Hand-maintained package lists are what let one hooks leaf sit unpublished and unbuilt for an
entire release line (FIX-005), so they are no longer permitted in release surfaces.
| Layer | File | Contract |
| --- | --- | --- |
| Inventory | `scripts/lib/publishable-packages.cjs` | `listPublishablePackages(repoRoot)` returns every **workspace member** manifest — resolved from the root `package.json` `workspaces` globs — where `private !== true`, the name is in the private scope **and** `publishConfig.registry` is exactly the Artifact Registry URL (spec 15 §4.3 G1), sorted by package name. It throws on a duplicate name or directory, an unreadable workspace manifest, or an unsupported glob shape — the inventory never silently drops a package. Run it directly (`node scripts/lib/publishable-packages.cjs`) to print the inventory as JSON. |
| Groups | `scripts/lib/release-matrix.cjs` | Named selectors over the inventory. `all-publishable` selects everything; `hooks-leaves` selects `packages/adapters/hooks/adapter-*`. `node scripts/release-matrix.cjs --list-groups` prints them with their descriptions. A group that resolves to zero packages is a hard error. |
| CLI | `scripts/release-matrix.cjs` | `--group <id> [--format matrix\|workspaces\|json\|waves]`. `waves` emits dependency-ordered publication waves (one wave per line, space-separated) and is valid only for `all-publishable`; both release workflows build and publish from it. The CLI is read-only: it never builds, publishes, or contacts the registry. |
| Enforcement | `scripts/check-package-metadata.cjs` (`npm run verify:metadata`) | Compares the inventory against `docs/generated/package-plugin-docs-coverage.json`, against the two release surfaces (`RELEASE_MATRIX_SURFACES`: `.github/workflows/publish-npm-private.yml` and `.github/workflows/publish-packages-from-tag.yml`), and against each package's declared direct runtime dependencies. |
### How a new package joins the train
1. Add the workspace under a directory matched by the root `workspaces` globs, named
`@fabricstack/<name>`, with `private` unset (or `false`) and
`publishConfig.registry` set to the Artifact Registry URL (no `access`), and `git add` the
manifest. It is now in the inventory — nothing else declares it, and the publish-isolation
guard refuses any other registry.
2. Declare every bare runtime import in the package's own `dependencies` / `optionalDependencies` / peer contract; pin internal dependencies to the exact train version (`docs/VERSIONING.md`).
3. Give the package a docs home and regenerate `docs/generated/package-plugin-docs-coverage.json`.
4. Nothing to add to any workflow: both release surfaces derive their build and publication
waves from the inventory, and `npm run verify:metadata` proves the coverage.
5. Run `npm run verify:metadata && npm run guard:packages && npm run test:release-tooling`.
### Dependency ownership model
A public package **owns** every module it imports at runtime. `scripts/lib/dependency-ownership.cjs`, run by `verify:metadata`, enforces exactly this rule:
> Every non-type runtime import in a public package's shipped sources must be satisfied by that package's **own** `dependencies`, `optionalDependencies`, or declared peer contract.
- **Scope.** Shipped runtime sources only — `src/`, `bin/`, `lib/`. Test suites, fixtures, examples, docs, dev scripts, `*.test.*`, `*.config.*`, and `*.d.ts` are excluded, because they do not run in a consumer installation.
- **`devDependencies` do not satisfy runtime ownership.** Inside the monorepo, workspace hoisting silently supplies an undeclared package, so the workspace build passes while the published tarball is broken (FIX-002, FIX-006).
- **Type-only imports are not runtime dependencies**; declaration closure is validated separately by the packed-artifact typecheck.
- **Optional native capabilities use exactly one ownership model: an optional peer dependency, with an explicit tested contract** (`node-pty` in `@fabricstack/harness-core` is the reference case).
- **Internal workspace dependencies pin the exact release version** and are rewritten by `sync-workspace-versions.mjs`, which is why publication order must come from the dependency graph.
## Workflow Overview
- `.github/workflows/publish-npm-private.yml` is the **only** branch publication workflow. It runs on pushes to `main`, `staging`, and `develop`, is guarded by the `publish-npm-private-<branch>` concurrency group, and owns validation, build, candidate publication, published-consumer validation, channel promotion and tagging for every package in the inventory. A `workflow_dispatch` rehearses (dry run) unless `dry_run=false`.
- `.github/workflows/release-tags.yml` creates the immutable `fabric/<branch>/v<version>` tag from the version the train hands it. It never derives a version of its own.
- `.github/workflows/publish-packages-from-tag.yml` is a manual/recovery path only (see above).
- `.github/workflows/live-stack-published.yml` is the published-consumer validation lane, called with the exact release version before any channel moves.
- `.github/workflows/release-artifact-verifier.yml` is the reusable packed-artifact gate both publish paths call.
- `.github/workflows/fix-npm-tags.yml` is the operator dist-tag repair (audit / explicit set), dry run by default.
- `.github/workflows/publish.yml` is the GKE **deploy** pipeline of the product; it publishes nothing to any npm registry.
- Branch → channel mapping is fixed in `scripts/lib/release-version.cjs`: `main` → `latest`, `staging` → `staging`, `develop` → `develop`. Any other branch is a hard error.
- Ordering constraints fall out of the dependency-graph waves emitted by `scripts/release-matrix.cjs --group all-publishable --format waves`; nothing is maintained by hand.
## Ownership Matrix
- `.github/workflows/publish-npm-private.yml`: resolves the one release version, gates on the packed-artifact matrix, synchronizes and verifies every manifest, builds and publishes every package under the candidate dist-tag in dependency-graph order, validates the exact published version from a clean consumer, promotes the channel, asserts it, and tags.
- `scripts/release-version.cjs` / `scripts/lib/release-version.cjs`: the release version source of truth (version, channel, tag, registry).
- `scripts/sync-workspace-versions.mjs --version <releaseVersion>`: writes that one version into every workspace manifest; `release-version.cjs verify-manifests` proves it landed.
- `scripts/publish-package-from-tag.mjs`: the only `npm publish` caller. It runs each package's `verify:release` gate, hard-fails on any version or registry disagreement, and cannot write a channel dist-tag.
- `scripts/release-promotion.cjs`: the only channel dist-tag mutation in the pipeline, gated on recorded validation evidence for that exact version.
- `scripts/fix-broken-latest-tags.mjs`: operator dist-tag repair over the inventory; dry run unless `--fix`.
- `scripts/bump-version.mjs` and `scripts/rollback-release.sh` are legacy helpers retained for manual use. **No workflow invokes them**, and neither is a version source of truth for a release.
## Secrets & Permissions
- Publication, promotion and dist-tag repair authenticate to GCP through Workload Identity Federation (`google-github-actions/auth@v2` with the organization variables `GCP_WORKLOAD_IDENTITY_PROVIDER` / `GCP_SERVICE_ACCOUNT`; `id-token: write`) and refresh a short-lived Artifact Registry token into `$HOME/.npmrc` with `google-artifactregistry-auth`. The committed `.npmrc` carries scope routing only and is asserted unchanged after every refresh. No long-lived registry credential exists anywhere.
- The CI service account holds `roles/artifactregistry.writer` on `fabric-npm` only; humans hold `reader` (`docs/specs/platform-gaps/phase0/ar-provisioning.md`).
- `GITHUB_TOKEN` **must** retain `contents: write` in the branch train so `release-tags.yml` can push the annotated tag. The pipeline does **not** push a version-bump commit — manifest synchronization happens in the publication checkout and is never committed back.
- There is **no npmjs token** in this repository and no workflow may reference one (spec 15 §4.3 G3, enforced by `scripts/ci/publish-isolation-guard.cjs`). The organization-level `NPM_TOKEN` restriction is a pending owner action (`docs/specs/platform-gaps/phase0/g3-npm-token-restriction.md`).
## Guardrails
- Publish isolation G1–G6 (spec 15 §4.3) are hard on every PR and in every release job: manifest pinning, no public-registry publish context or credential in any workflow, packed-manifest scope gate, single-bin policy; `node scripts/ci/publish-isolation-guard.cjs --self-test` proves each red on a fixture.
- A release version is never re-derived downstream, a channel dist-tag is never written by a publish step, a channel never moves backward (`assert-channel-tags preflight`), and a tag is never re-pointed (`ensure-tag`).
- Every `npm view` / `npm dist-tag` / `npm publish` of the train names the private registry explicitly on top of the committed scope routing.
- The public-push guard (`scripts/ci/public-push-guard.cjs`) refuses any configuration that could write to `a5c-ai/babysitter`.
## Rollback
Rolling back a *release* is not the same as rolling back a *channel*.
- **Channel rollback.** Publication is immutable; the only reversible surface is the dist-tag. Move the channel back to a previously validated exact version with `scripts/fix-broken-latest-tags.mjs --set <channel>=<version>` (or `fix-npm-tags.yml`), then re-run `release-version.cjs assert-channel-tags --mode final` over the whole inventory. Approved operational action; procedure in `docs/RELEASE-RUNBOOK.md` § 5. Never unpublish or republish an existing version; recover forward through a new version.
- **Half-published train.** Recover from the immutable tag with `publish-packages-from-tag.yml` (`docs/RELEASE-RUNBOOK.md` § 4); already-published packages are candidate-tagged, not re-published.
- **Git tag.** Release tags are immutable records; a wrong tag is superseded by a new version, never moved. `scripts/rollback-release.sh` predates the `fabric/<branch>/v*` naming and is not part of any workflow.
- Document every rollback action — command, actor, timestamp, result — in the incident ticket. The worked example is `docs/release-incident-2026-08-13.md`.
## Channel Behavior
- `main`, `staging`, and `develop` all publish through the same `publish-npm-private.yml` contract and differ only in the resolved version shape and channel dist-tag: `main` publishes `X.Y.Z` to `latest`; `staging` and `develop` publish `X.Y.(Z+1)-<branch>.<short-sha>` to `staging` / `develop`.
- Every channel synchronizes **every** manifest to the release version before publishing.
- Every channel publishes to the candidate dist-tag first and moves its channel tag only after published-consumer validation of that exact version succeeds.
- Every channel creates an immutable annotated `fabric/<branch>/v<version>` tag through `release-tags.yml`, named for the exact published version, after promotion.
## Operational Checklist
1. Tabletop the channel-rollback and half-published-train procedures in `docs/RELEASE-RUNBOOK.md` quarterly (Release Eng + Security).
2. When adding or removing a package from the train, follow [How a new package joins the train](#how-a-new-package-joins-the-train); `npm run verify:metadata` is the check that proves coverage.
3. After the first train that publishes `@fabricstack/harness-hooks-agent-runtime`, close the FIX-005 follow-up (`docs/RELEASE-RUNBOOK.md` § 7.1).
documents
[]
Outgoing edges
None.
Incoming edges
contains_page1
- page:docs·PageBabysitter Docs