Two independent defects made every documented 'first API call' fail:
1. FICTIONAL MODEL IDS. `zen-omni` and `zen-coder` appear ZERO times in the
live catalog (api.hanzo.ai/v1/models, 440 models). Every SDK sample -
python, typescript, go, cpp, swift, kotlin - told the customer to call a
model that does not exist, so even a correct request 400s. Corrected to the
real ids: zen5 and zen5-coder.
2. DEAD HOST + STALE PINNED MODEL. llm.hanzo.ai is retired and 301s (a POST
returns the literal string 'Permanent Redirect', not a completion), and
`claude-sonnet-4-5-20250929` is long superseded. Both removed: the host is
gone from customer-facing docs entirely rather than documented as retired,
and pinned upstream ids give way to our own current zen5.
50 files. Verified against the live catalog, not assumed.
The docs App CR + pods pull ghcr.io/hanzoai/docs (public), but deploy-docs.yml
built+pushed+patched ghcr.io/hanzo-docs/docs (private) → every operator deploy
ErrImagePull'd (kept alive only by a manual crane-copy). Converge both on the
canonical public ghcr.io/hanzoai/docs per the Hanzo registry rule: CI push target
now equals the CR pull source. Immutable SHA tag only (drop floating :latest).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Close the loop the docs deploy was missing: push -> build -> TEST -> deploy ->
re-index, no human in it.
- Test gate validates the export (index.html, docs/index.html, docs/services/
index.html, >=50 pages, /docs carries nav) BEFORE anything ships — exactly the
empty-export and trailingSlash failures that reached prod before.
- Deploy is operator-driven: patch App/docs .spec.image.tag to the immutable
built SHA; hanzo-operator reconciles. No floating tag in the cluster, no CI
kubectl on Deployments. In-cluster runner SA scoped by k8s/docs-cd-rbac.yaml
to get+patch ONLY app/docs (patch deploy = denied).
- Re-index posts to /v1/index-docs so search + zen-coder-flash chat answer from
the shipped content (best-effort, never blocks).
- Rips the Cloudflare Pages + docs-router worker path: docs.hanzo.ai serves
from the in-cluster App via the universe file-provider route. One way.
The native docs pod CrashLoopBackOff'd: the 0.4.1 static binary has no -404 flag
(flags are -port/-root/-s3-*/-spa), so '/static ... -404 404.html' printed usage and
exited. Serve with '/static -port 3000 -root /public'. Image pulls + starts clean now.
The cross-org push to ghcr.io/hanzoai/docs failed ('permission_denied: The requested
installation does not exist') — the workflow GITHUB_TOKEN can only write packages in
its OWN org (hanzo-docs), not hanzoai. Push to ghcr.io/hanzo-docs/docs instead (same
org as the repo — conventional, self-contained) and point the docs-landing Deployment
at it. Base pull (hanzoai/static:0.4.1, now public) already works; this clears the push.
The native docs image build failed at 'ghcr.io/hanzoai/static:v0.3.1: not found'.
hanzoai/static is public, but there is no v0.3.1 tag — the real tags are 0.3.0 /
0.4.0 / 0.4.1 / latest (no 'v' prefix). Pin the base to 0.4.1 (newest multi-arch
stable) in both the repo Dockerfile and the CI Dockerfile.serve. With this + the bun
fix + public bases, ghcr.io/hanzoai/docs finally builds with real content for the
docs-landing native serve.
Migrate docs.hanzo.ai to the native stack: after the export build, package apps/docs/out
into hanzoai/static and push ghcr.io/hanzoai/docs:{latest,sha} — built on the self-hosted
hanzo-docs-build runner (docker present; 'no GitHub builders' = no GitHub-HOSTED runners).
The in-cluster docs-landing Deployment behind hanzoai/ingress serves docs.hanzo.ai; no
Cloudflare Pages, no docs-router worker (now continue-on-error — being retired; its deploy
token lacks Workers perms anyway). Mirrors the proven deploy-liquid-docs image pattern.
Same recipe generalizes to every doc site: static→hanzoai/static, SPA→hanzoai/spa, behind
hanzoai/ingress via the operator. Pairs with the bun fix that stopped the image /public
from being empty.
Second blocker on the frozen docs deploy: apps/docs 'build:pre' runs
'bun ./scripts/gen-services-nav.ts && bun ./scripts/pre-build.ts', but the
containerized hanzo-docs-build scale-set runner has no bun preinstalled and the
workflow never set it up — so 'next build' never even started (instant exit 1, no
output, right after 'cache miss, executing'). The sibling deploys (deploy-bot-docs
etc.) already 'uses: oven-sh/setup-bun@v2'; match them. Together with the
best-effort swap fix, this unblocks the whole docs build → the Jul-8 card redesign
finally ships to docs.hanzo.ai.
docs.hanzo.ai has been frozen on pre-Jul-6 content for ~2 weeks: every deploy since
failed at 'Extend swap for large build' (exit 255) — the containerized
hanzo-docs-build scale-set runner can't sudo a swapfile (fallocate fails), which
blocked the build + Cloudflare Pages deploy entirely. So the Jul-8 'Stripe-style
Browse by product' redesign + every doc since never shipped.
Make the swap step best-effort: continue-on-error + '|| true' on each op + a dd
fallback + 'free -h'. The build already caps Node heap at 16 GB and has a 60-min
timeout, so swap is a bonus, not a hard requirement. This unblocks the whole
backlog of committed-but-unshipped docs content.
ARC scale-set runners are org-scoped: the hanzoai pool can never pick up
hanzo-docs jobs, so every deploy queued forever. Point all workflows at the
new hanzo-docs-build-linux-amd64 set (universe infra/k8s/arc/
values-hanzo-docs-build-amd64.yaml).
* docs: redirect /docs/{startups,deploy} → real pages (QA 404 fix)
getting-started references /docs/startups + /docs/deploy but those slugs 404
(the mdx pages didn't survive an earlier merge). Redirect to the pages that
carry the content: getting-started ($5/startup on-ramp) + services/paas (deploy).
* docs: unified docs architecture — hub + hanzo-docs/<team> submodule pattern
Ratify docs.hanzo.ai as the single unified-build hub and define the one way
content joins it, by kind:
- authored -> content/docs/<section> (in-hub or hanzo-docs/<team> submodule)
- generated -> openapi + ZAP SDK generators, gitignored, never hand-written
- ported -> sync-project-docs mirror + upstream attribution
Changes:
- ADR as a rendered docs page (content/docs/contributing/docs-architecture.mdx)
+ nav entry; LLM.md canonical model + dedup/rollout debt.
- Wire hanzo-docs/studio-docs as the reference exemplar: git submodule at
content/docs/studio/ (was a single tracked file). deploy-docs.yml checkout
now fetches submodules recursively.
- Security: cf-pages-deploy.mjs no longer hard-codes a Cloudflare API token;
it is required from env/KMS and fails fast (token must be rotated).
Standalone-vs-hub criteria, API-ref-from-openapi, and SDK-slot wiring are all
documented in the ADR. Aggregation proven: 38 API-ref pages generated from 36
openapi specs; studio section resolves from the submodule on disk.
---------
Co-authored-by: hanzo-dev <dev@hanzo.ai>
The docs site fanned out into one CF Pages project per DOCS_SECTION, stitched by
an edge worker whose *.pages.dev origins had gone stale — so every
/docs/projects/* and /docs/services/{iam,kms,platform}/* page returned 530, and
those products were never actually merged into docs.hanzo.ai. Collapse it: one
unified static export → one hanzo-docs Pages project → the router passes every
path straight through.
- source.config.ts: build the whole doc set in one pass (core + every service
incl. iam/kms/platform + every synced OSS project). The next.config
ProjectDocsFallback + remarkPassthroughUnknownJsx neutralise foreign MDX from
ported docs, so a single build is safe.
- workers/docs-router: pass-through to the single hanzo-docs origin — nothing to
stitch, no stale section origin to 530.
- OpenAPI: /docs/openapi/<svc> reference pages are generated at build time from
the specs (gen-openapi-pages.ts wired into pre-build.ts; specs synced by
sync-openapi.sh). The runtime <APIPage> is export-disabled (deep operation
slugs break prerender); static per-service pages export cleanly and can't 530.
The openapi index links to the generated reference pages.
- public/_redirects: alias the console product "Docs" deep-links that live under
/docs/services or /docs/projects so console → docs always resolves.
- deploy-docs.yml: one full export (NEXT_EXPORT=1), ships content + router
together. Remove the four dead per-section deploy workflows.
Verified: full export builds green in one pass under Turbopack (1628 pages) and a
crawl of every internal link from / and /docs resolves 200 — 1320/1320, zero 404,
zero 530. iam/kms/platform/projects all 200 and reachable in nav.
All 14 hanzo docs deploy-*.yml used runs-on: [self-hosted, linux, amd64] — a label triplet the ARC scale-set runners (which advertise only their set-name hanzo-build-linux-amd64) do NOT carry, so every docs deploy queued until the next push cancelled it. Result: docs.hanzo.ai served a stale build for ~1 week; merged doc changes (redirects, new product pages) never deployed. Aligns these with the repo's already-correct lint/test/release workflows. deploy-liquid-docs.yml intentionally not touched.
Co-authored-by: Hanzo AI <ai@hanzo.ai>
The Infisical KMS routes (/api/v1/auth/universal-auth/login,
/api/v3/secrets/raw) are decommissioned and 404. Switch every Pages
deploy workflow to the canonical Hanzo KMS API: machine-identity
login at /v1/kms/auth/login, then read CF Pages creds from
orgs/hanzo/secrets/deploy/{CLOUDFLARE_API_TOKEN,CLOUDFLARE_ACCOUNT_ID}?env=prod.
Identity is the KMS_CLIENT_ID/KMS_CLIENT_SECRET repo secrets.
Replaces non-canonical scale-set / org-prefixed labels with the
existing labels every arcd host registers with. Matches evo for
amd64 and spark for arm64. No new labels added.
deploy-base-docs, deploy-zap-docs, deploy-zen-docs, and deploy-zt-docs
referenced steps.kms.outputs but had no KMS fetch step, causing empty
CF credentials and deploy failures.
Deploys dev-docs to Cloudflare Pages on push to main/dev.
Triggers on changes to apps/dev-docs/, packages/, or pnpm-lock.yaml.
Follows the same pattern as deploy-base-docs.yml.
The docs.hanzo.ai domain was deactivated because DNS didn't point to
hanzo-docs.pages.dev. This workflow now updates both the DNS record
and reactivates the Pages custom domain.
ZAP and ZT docs workflows used npx wrangler without installing it,
causing "wrangler: not found" errors. Zen docs used pnpm install -g.
Standardize all workflows to use pnpm dlx wrangler, matching the
main docs deploy workflow pattern.
Daily GitHub Actions workflow (04:30 UTC) fetches live pricing from
pricing.hanzo.ai and patches static fallback values in models.ts and
pricing.ts. Creates a changeset on change so existing release.yml
handles npm publish automatically. Also supports repository_dispatch
and manual workflow_dispatch triggers.
z
