studio/LLM.md

Hanzo Studio

Overview

The most powerful and modular visual AI engine and application.

Tech Stack

• Language: Python

Build & Run

uv sync
uv run pytest

Structure

studio/
  CODEOWNERS
  CONTRIBUTING.md
  Dockerfile
  LICENSE
  QUANTIZATION.md
  README.md
  alembic.ini
  alembic_db/
  api_server/
  app/
  blueprints/
  branding/
  comfyui.log
  comfyui.prev.log
  comfyui.prev2.log

Key Files

• README.md -- Project documentation
• pyproject.toml -- Python project config
• Dockerfile -- Container build (applies branding/ then runs main.py)
• hanzo.yml + .github/workflows/cicd.yml -- canonical CI/CD (hanzoai/ci); builds ghcr.io/hanzoai/studio, rolls the studio operator Service CR at studio.hanzo.ai.

Custom-node packs ("hanzo packages")

The popular ComfyUI packs are vendored as first-class hanzo packages, pinned and reproducible. One and only one way to add/bump them:

• Manifest: custom_nodes/hanzo-packs.txt — <dir> <git_url> <commit_sha> <spdx> lines (immutable SHAs). This is the tracked source of truth; the cloned pack sources are gitignored (vendored at build).
• Deps: custom_nodes-requirements.txt — hand-resolved, only what's not already in requirements.txt. Installed with the studio torch(cu130)/NumPy 2.x/ Transformers 5.x stack locked so packs can never downgrade it.
• Installer: scripts/install_custom_nodes.sh clones each pack at its pin and installs deps; the lock is derived from the live env (correct for both the CPU image and the GB10 cu130 venv). ultralytics is installed --no-deps (else it pulls non-headless opencv). Local (GB10): STUDIO_PY=.venv/bin/python STUDIO_PIP="uv pip install --python .venv/bin/python" scripts/install_custom_nodes.sh.
• Image: Dockerfile runs the installer after COPY . .; .dockerignore re-includes only the manifest + installer (custom_nodes/* / scripts/* form). So git push origin main → CI builds ghcr.io/hanzoai/studio with the packs → studio.hanzo.ai gets them on the next image build.

Compat shim — why packs work here at all. This fork renamed ComfyUI's core packages (comfy→studio, comfy_extras→studio_extras, comfy/comfy_types→ studio/node_types, and the Comfy*→Studio* API classes; see scripts/rename_modules.py). studio_compat/ is one meta-path finder that lazily aliases every comfy* import to its studio* counterpart and mirrors ComfyExtension/io.ComfyNode→Studio* — so upstream packs run unmodified, no forks/renames. It is the phase-4 redirect rename_modules.py deferred, done with the 3.12 find_spec API. Activated by import studio_compat at the top of main.py. nodes.py also honors the upstream comfy_entrypoint V3 name.

Restart the local worker only when idle: scripts/studio-service-swap.sh polls /queue and restarts on the first empty window. Bad packs are isolated by the loader (studio still boots); genuinely-incompatible ones go to custom_nodes_quarantine/ (e.g. was-node-suite: hard numba dep vs NumPy 2.5).

Multi-tenant + IAM (cloud: studio.hanzo.ai)

Standalone app; console.hanzo.ai link-outs land here already authenticated via shared Hanzo IAM (standard OIDC code flow). No embedding.

• Auth (middleware/iam_auth_middleware.py): local JWKS JWT validation against ${STUDIO_IAM_URL}/v1/iam/.well-known/jwks — signature + exp + iss + aud (client_id hanzo-studio). Org from the owner claim (organization fallback) → request["iam_user"]["org_id"]. Anonymous mode: off by default locally (loopback bypass); enable in cloud with STUDIO_ENABLE_IAM_AUTH=true (the one auth switch — this is the "STUDIO_AUTH=off default" behavior). Browser flow: unauth HTML → IAM authorize (code+PKCE) → /callback exchanges the code, sets an httpOnly session cookie (no frontend changes). verify_jwt(...) is a pure, unit-tested function.
• Tenancy (folder_paths.py, app/user_manager.py): with STUDIO_MULTI_TENANT=true, userdata is user/orgs/{org}/user/{user}/… and outputs are output/orgs/{org}/…. get_public_user_directory(user, org) is org-rooted (fixed the bug where org userdata always resolved to None). Execution outputs are scoped via folder_paths.set_execution_org(), bound by the prompt worker from the queue item's org_id.
• GPU federation (middleware/{worker_client,prompt_router,compute_config, visor_client}.py): /v1/workers/register (heartbeat), /v1/workers (list), /v1/worker/execute (in-cluster push fast path). Worker↔coordinator trust via X-Worker-Token (STUDIO_WORKER_TOKEN, KMS-sourced).
• Per-GPU gpu-jobs queue (middleware/{gpu_dispatch,studio_home}.py, server.py) — the ONE submit path onto a GPU. A model-less coordinator's POST /prompt enqueues to the cloud tasks engine (POST {STUDIO_CLOUD_API_URL}/v1/tasks/namespaces/gpu-jobs/activities, user bearer → org-scoped); a BYO worker (hanzo gpu connect) claims + runs it. Targeting: the Home "Run on " chip sets X-Target-GPU: <identity>; dispatch_if_worker enqueues on that machine's own lane taskQueue="gpu:<identity>" (namespace stays gpu-jobs), else the shared gpu-jobs lane any worker drains — X-Target-GPU rides through _queue_prompt so fix/compose/video/rerun all honor it. Authoritative visible queue: /v1/render-queue + /v1/nodes read the tasks engine's activity list — REAL claiming identity, status SCHEDULED/STARTED/COMPLETED/FAILED → queued/running/done/failed, lastHeartbeatTime liveness, per-node depth — not a local shadow; render_jobs.json survives only as a ~2s pre-claim placeholder. No hidden runs: in --worker-mode a direct /prompt POST is refused 403 unless it carries X-Worker-Token — the only accepted execution is a claimed job handed in over POST /v1/worker/execute (worker_client.reject_untrusted_worker_submit, one policy gate). Legacy push (prompt_router) is OFF unless STUDIO_LEGACY_PUSH_ROUTER=1. Tests: middleware_test/{gpu_dispatch_test,worker_seam_test}.py.
• Durable render queue (middleware/tasks_queue.py, docs/federation.md): SqlitePromptQueue — crash-durable render queue in a single SQLite file (stdlib sqlite3, WAL, zero external processes). STUDIO_QUEUE_DB=<path> → server.py swaps PromptQueue for it (precedence STUDIO_QUEUE_DB > STUDIO_PERSIST_QUEUE > memory; default local behavior unchanged). The PromptQueue seam maps to a durable job state machine: put→INSERT (pending, idempotent on prompt_id), get→claim next pending under BEGIN IMMEDIATE (exactly-once, leased), task_done→done | retry/failed. Crash-recovery: a lease + reap returns an abandoned claim to pending (on every get(), a heartbeat thread, and on boot) so another process re-runs it — idempotent (SaveImage suffixes increment). Multiple Studio processes on the same DB file compete safely. Tested: middleware_test/tasks_queue_test.py (incl. crash-recovery across instances). Future (federation.md §6): a remote queue backend served by Hanzo Tasks + a Go/unified-binary (HIP-0106) migration.
• Stacks (middleware/stacks_store.py, middleware/studio_home.py): Procreate- style gallery folders — a per-org SQLite store (orgs/<org>/stacks.db, WAL, every mutation under BEGIN IMMEDIATE, the same tasks_queue.py pattern) that is ONLY an organizational layer; assets stay the source of truth (library.json + files). Schema = the spec's stacks + stack_assets join; UNIQUE(asset_id) ⇒ one Stack per asset. Tenancy is structural (one DB file per org → cross-org 404s by construction). REST (/v1 house style): GET/POST /v1/stacks, GET/PATCH/DELETE /v1/stacks/:id (?mode=only|images), POST|DELETE /v1/stacks/:id/assets, PATCH /v1/stacks/:id/order, POST /v1/stacks/:id/unstack, GET/PATCH /v1/stacks/settings; registered via injected resolvers (org/owner/workspace + the library soft-deleter) so the module stays decoupled. Remove ≠ delete (returns to the gallery); Delete-Stack defaults to preserving; "…and Images" maps to the library soft-delete (status=deleted, files kept). Generation integration: fix/compose/rerun/template accept a stack id and the ingest point (_index_upload → stacks_store.absorb) assigns membership when outputs land (ALL outputs of a multi-output job); a per-org auto_stack_batches setting auto-creates "<prompt> — <date>" Stacks. UI is the vanilla-JS home (studio_home.html): layered cards, drag/multi-select/mobile-tap parity (no DnD ever required), a11y labels + live region. Tested: studio_home_test/test_stacks.py (routes, tenancy, one-per-asset, order persistence, delete/unstack, ingest→absorb).
• Engine selector (middleware/engine_selector.py): /v1/engines lists execution targets for the org (local + registered compute_config workers) and PUT /v1/engines/default sets a per-org default (stored on ComputeProfile.default_engine). prompt_router.route_prompt routes to the chosen worker; local = unchanged. Leased cloud machines (Visor GET /v1/machines) are a future engine class (interface stubbed, not built). Frontend picker is a TODO. Tested: middleware_test/engine_selector_test.py.
• Chat Copilot (middleware/copilot.py + branding/hanzo-copilot.{js,css}): a sidebar chat that reads and MUTATES the workflow graph in natural language. Backend POST /v1/copilot/chat proxies one OpenAI tool-calling round to a chat endpoint (STUDIO_COPILOT_URL; default = local engine 127.0.0.1:1234 if its port is open, else api.hanzo.ai; STUDIO_COPILOT_MODEL default zen; token from STUDIO_COPILOT_TOKEN→STUDIO_COMMERCE_TOKEN→request bearer). It returns {reply, ops}; add_node ops are validated against nodes.NODE_CLASS_MAPPINGS server-side so the model can't invent node types. The frontend applies ops against app.graph (LiteGraph) — op schema: add_node(id?/pos?) · set_widget · connect(slots by name or index) · set_prompt · move_node · delete_node · layout (columnar tidy) · queue. Every op is guarded (unknown type/slot = skip + note, never throw). Injected into the prebuilt frontend by branding/patch_copilot.py (copies the bundle into static/, CSS <link> in <head>, deferred <script> before </body>; idempotent, wired as step 9 of apply-branding.sh) — survives frontend upgrades. window.HanzoCopilot.applyOps is exposed for tests. Tab registered via app.extensionManager.registerSidebarTab (id hanzo-copilot). Tested: middleware_test/copilot_test.py (op-validation + mocked tool loop) and branding/copilot_smoke.py (headless: applier mutates a mock graph + panel renders). Live end-to-end needs one server restart to register the route.
• Identity user menu + session API (middleware/session.py + branding/hanzo-identity.{js,css}): the top-right user menu, driven entirely by the IAM-validated token. Backend (thin, /v1 house style, registered in add_routes so it works auth-on and auth-off): GET /v1/session → {user,email,org,orgs[],project,projects[],console_url,iam_url,authenticated} read from request["iam_user"]; POST /v1/session/org sets the active org (validated against membership → studio_active_org cookie); POST /v1/session/project sets the workflow-workspace project scope; GET|POST /logout clears the session + selection cookies and bounces to the IAM end-session → /login. Org switching is real, one hook: the auth middleware applies session.resolve_org(...) to a COPY of the validated user (never the cached dict) so iam_user["org_id"] — the id every tenancy/output path already reads — reflects the switch; membership comes from _orgs_from_claims (owner + organizations/orgs/groups, list or CSV or {name} dicts, home first, deduped). Frontend is a SPA-agnostic fixed pill + dropdown (avatar/initials, user, org switcher, project switcher, Console link → console.hanzo.ai, Logout); 401 → "Sign in with Hanzo" → /login. Injected by branding/patch_identity.py (step 10 of apply-branding.sh, same seam as the copilot). window.HanzoIdentity is exposed for tests. Tested: middleware_test/session_test.py (resolve/orgs/routes/logout, real aiohttp app) + scratchpad/studio-iam/test_identity.py (headless: pill, dropdown, org switch re-scope, signed-out). The /v1/session routes need one server restart to register (static + de-Comfy load without restart).
• De-Comfy display strings (branding/patch_destring.py, step 11 — runs LAST): the systematic English brand-word sweep the curated phrase list in patch_frontend.py structurally can't reach. Rule 1: whole-word ComfyUI → Hanzo Studio only where it reads as a display word (preceded by open-quote/space/>/(, followed by close-quote/space/sentence-punct) — that boundary skips every code use in the bundle (ComfyUIStarted, .ComfyUI, =ComfyUI, the /^(ComfyUI-.../ regex, dotted i18n keys .category.ComfyUI) and hyphenated product names (ComfyUI-Manager). Rule 2: curated bare-Comfy values as exact key: value`` pairs so the Settings dialog keys (Comfy, Comfy-Desktop, Comfy.ColorPalette) are preserved while the displayed section header becomes Studio / Studio Desktop (this is the "Settings must not say Comfy" fix) + Comfy Cloud → Hanzo Cloud. Scope: English only — files above LOCALE_RATIO (0.15%) non-ASCII are skipped (all 33 locale bundles are ≥0.31%; English bundles ~0.02%) and each match needs a pure-ASCII window, so the non-English main-*/nodeDefs-*/commands-* chunks are left as-is (noted, out of scope). Idempotent. Verified: served English display ComfyUI 63→0; a few example filesystem paths (Documents/ComfyUI) and the external comfy.org service labels (Comfy API Key, Comfy Node Registry, behind disabled API-nodes) are intentionally left. patch_frontend.py keeps its orthogonal concern (URLs/domains + Comfy-Org → Hanzo AI).

Office documents (the non-GPU pipeline)

Studio's creative side is ComfyUI graphs → GPU. Office documents are the orthogonal subsystem: structured content (a template's typed fields, filled by a person OR by the AI) → a pure-Python renderer → a real .pdf/.docx/.xlsx/.pptx/.md. No graph, no model, no GPU. NOT bolted onto graphs; its own module, store, endpoints, and Home tab.

• Renderers (middleware/doc_render.py, pure — no HTTP/storage/org): one neutral IR per archetype (doc prose blocks · sheet sheets · deck slides) and EXACTLY ONE renderer per format. Adding a template = one build(fields)->(kind,ir); adding a format = one renderer. Deps imported lazily so the module loads stdlib-only. PDF prefers WeasyPrint (full-CSS, monochrome print stylesheet); when its native pango/cairo stack is absent (the light CI unit venv, model-less pods) a built-in pure-Python PDF writer (base-14 Helvetica, WinAnsi/cp1252, auto-pagination, /Info title) produces a valid %PDF — so every format renders with just the wheels. DOCX/XLSX/PPTX = python-docx / openpyxl / python-pptx (real, always). Native libs in the Dockerfile (libpango/…).
• Catalog + AI-fill + store + REST (middleware/documents.py): ONE declarative CATALOG of 22 common templates (Documents 15 → pdf/docx/md · Spreadsheets 4 → xlsx/pdf/docx/md · Presentations 3 → pptx/pdf/md). Each template's typed fields drive the form, the AI-fill JSON, AND the renderer inputs (one schema, three uses). Registered from within add_studio_home_routes with the SAME injected tenancy resolvers as stacks_store (org_of/owner_of). Per-org store orgs/<org>/documents/ (atomic index.json + <id>/doc.json), doc.json is the source of truth, files rendered on demand at download. Endpoints (/v1 house style): GET /v1/documents (grouped catalog), POST /v1/documents/generate {templateId, description?|fields?}, GET /v1/documents/library, GET|PATCH|DELETE /v1/documents/{id}, GET /v1/documents/{id}/download?format=.
• AI-fill is a LAYERED ENHANCEMENT — the structured-form path renders a valid document with NO LLM, so a broken/unauthorized gateway can NEVER block creation (422 needFields → the UI reveals the prefilled form). It reuses the SAME working token the shipped Chat uses: the browser sends window.HZ.pk (the publishable pk- key _publishable/ _hz_config_script emit) as the bearer to /v1/documents/generate; the backend forwards it, fail-closed, in order: (1) a server-side gateway key (STUDIO_DOC_TOKEN→ STUDIO_COPILOT_TOKEN→STUDIO_COMMERCE_TOKEN); (2) else the caller's own token, forwarded ONLY when gateway-verifiable — a publishable pk- key or an asymmetric/JWKS JWT (RS/ES/PS/EdDSA). A symmetric HS256 hanzo.id session token is NEVER forwarded (the gateway rejects it: "unsupported signing method: HS256"), nor any secret sk-/hk- key. Model = STUDIO_DOC_MODEL→STUDIO_CHAT_MODEL→enso (matches window.HZ.model).
• UI (middleware/studio_home.html, vanilla JS, monochrome/Geist): a 📄 Documents Home tab — searchable server-driven catalog + "Your documents" grid with Download PDF/Word/Excel/PowerPoint. The docdlg dialog: describe in one line → Generate with AI (sends window.HZ.pk), or fill the short typed form → Generate; edit fields before/after; re-open a saved doc to edit (PATCH). The dead doc/sheet/paper composer KIND placeholders were replaced with one "Office documents" card that opens the tab.

Create composer + gateway auth (v0.18.1 — the intuitive first Create)

The "What should we create?" composer must PRODUCE something on the first ↑, and Chat must reach the gateway. Both were fixed here.

• Gateway auth is ONE fail-closed seam (middleware/gateway_auth.py): the canonical allowlist for handing a browser-edge bearer to api.hanzo.ai. ai_bearer(request, server_env=…) = a server key first (never in page source), else the caller's own token ONLY when the gateway can verify it — a publishable pk- key or an asymmetric/JWKS JWT (RS/ES/PS/EdDSA). A symmetric HS256 hanzo.id session token is NEVER forwarded (the gateway rejects it → "unsupported signing method: HS256" → the old 502), nor any secret sk-/hk-/rk- key. copilot.py._resolve_target now uses it (_SERVER_TOKEN_ENV=(STUDIO_COPILOT_TOKEN, STUDIO_COMMERCE_TOKEN)), so the editor-ops chat no longer 502s on the session token. The shipped Chat sidebar (shell.js) still streams enso directly from the gateway with window.HZ.pk when present; its copilot fallback now renders ONE friendly line on failure — never the raw backend error/JSON. (documents.py's AI-fill has an equivalent private copy; it should adopt gateway_auth once its concurrent review lands — the shared module is the going-forward one way.)
• Text-to-image is the default create lane (studio_home.py: _image_graph + dispatch_image + POST /v1/library/image): the proven Z-Image-Turbo graph (z_image_turbo_bf16 / qwen_3_4b lumina2 / ae vae; EmptySD3LatentImage → ModelSamplingAuraFlow shift 3 → KSampler res_multistep/simple 4 steps cfg 1.0 → SaveImage), lifted from the shipped blueprint, dispatched through the ONE funnel like video/fix. Words in → an image out; a model-less pod surfaces the real DispatchError, not a silent nothing.
• Composer routes ↑ by the selected model's modality (studio_home.html): default MODEL='zen-image' from hz_create_model — DECOUPLED from the chat model (hz_model), so an image pick never breaks the assistant (and shell.js chatModel() ignores a non-chat hz_model). composerPlan(): assistant model → hand to the ONE chat via window.HanzoChat.send (exposed by shell.js); no photo → text-to-image; 1 photo → fix; 2+ → compose; video model/lane → video. create() ALWAYS pulses a "Preparing…" live-bar state and, on failure, shows a friendly retry (prompt kept) — never a silent no-op. The code model (zen5-coder) is hidden from the creative composer; each model has a one-line description; the </> request-preview is gated behind Developers mode (body.hzdev, set by the shell's Developers pill).
• P2 friction fixes (studio_home.html + shell.css): a first-class Download on every tile AND in the viewer (PNG original · JPG/WebP client-side canvas convert, the popover flips up near the viewport bottom); the fixed header + sticky live-bar are now OPAQUE (were translucent → content scrolled under them); the ARCHIVED badge shows ONLY for truly deleted/flagged items (active work is unbadged); on mobile the ~30 filter pills collapse to one horizontally-scrollable row and the tab bar gets a scroll fade affordance. The Chat empty state (shell.js) shows an assistant identity + one line + 3 tappable starters.

Tests

tests-unit/{folder_paths_test,middleware_test,app_test,prompt_server_test,studio_home_test,documents_test} — run uv run pytest tests-unit/folder_paths_test tests-unit/middleware_test tests-unit/app_test tests-unit/prompt_server_test tests-unit/studio_home_test tests-unit/documents_test -q. The auth/tenancy work is covered by middleware_test/iam_auth_test.py, middleware_test/session_test.py and folder_paths_test/org_scoping_test.py. Office documents: documents_test/test_documents.py (catalog invariants, the full template×format render matrix asserted by magic numbers, AI-fill parse + fail-closed bearer allowlist, and the REST surface incl. per-org isolation on a real aiohttp app). Create composer + gateway auth: middleware_test/gateway_auth_test.py (the fail-closed allowlist — HS256/sk- refused, pk-/asymmetric forwarded, server-key precedence, session-cookie path), middleware_test/copilot_test.py::test_resolve_target_never_forwards_hs256_session_token, and studio_home_test/create_image_test.py (the Z-Image-Turbo graph shape, dispatch_image validation, and served-HTML invariants: default-model-is-image, text→image route, Create-always-shows-a-state, first-class Download, no ARCHIVED badge on active items, opaque bars, gated </>, collapsed mobile filters).