- Thumbnails grow to ~half the card width and a taller 4:5 crop (44% on mobile)
so the render is actually legible in Up next and History.
- Every card gets an always-visible × (fast path) beside the ⋯ overflow menu —
ONE shared handler, no second mechanism: a queued run cancels immediately
(POST /v1/queue/cancel); a finished result is a destructive soft-delete
(POST /v1/library/status status=deleted) and asks first with a small inline
confirm. History hides soft-deleted results (optimistic + reconciled from the
org's deleted set via /v1/library) so the delete persists across reloads.
When the item a user clicked was deduped/renamed/cleaned since their page loaded,
a fix/compose/rerun dispatch 400s "unknown asset: <path>" and the raw error left
them stuck. One self-heal wherever a dispatch happens:
- Legacy studio_home.html: healStale(j) — on an unknown-asset error, close any
open dialog, drop the stale selection/tray, re-fetch the library (load()), and
toast "That item changed — library refreshed, pick it again." Wired into the
fix + compose (sendRef), compose (sendCompose), and rerun (bulkFork) branches.
- studio-chat SPA rail: heal(e) — a stale run (unknown asset / not found / no such
job / already finished) on cancel or amend refreshes the queue and shows "That
run changed — refreshed, pick it again." instead of the raw error.
The studio-chat SPA's right rail becomes the full YouTube-watch-page surface,
shaped around generative flows (a card = a run in a flow):
- Up next: the live queue (GET /v1/queue/status joined to /v1/worklog) — pinned
"Next" card + vertical cards, thumb-left with an ETA/elapsed corner badge,
2-line prompt, node · time · status · position meta, per-card overflow menu.
Delete (POST /v1/queue/cancel) and Add references (file pick + drag-drop →
/upload/image → POST /v1/queue/amend) on each queued card.
- History: past runs newest-first (GET /v1/worklog +?q= search), output
thumbnail, "New" badge on just-landed renders, version pips for runs with
descendants → a walker over GET /v1/worklog/lineage.
- Rate 0-5 stars + a note ("What should the AI do differently next time?")
per run — the flywheel row {path, stars, note, ts} keyed by run id.
- Favorite (heart) → offer "turn this into a template" (name it → confirm what
it captures → POST /v1/templates with a captured flow).
- Stacks: drag one card and HOLD (~600ms) over another to form an iOS-style
folder; stacked-cards tile + count; click = folder-open reveal; unstack to
dissolve. Persisted per org.
- Filter chips (All · Queued · Done · Favorites · Stacks), resizable main|rail
and Up-next|History dividers, per-section collapse, active filter — all
persisted in localStorage keyed by org+user; "Reset layout" in the rail menu.
- Mobile (<=820px): single-scroll column, rail as the bottom section, composer
fixed to the viewport bottom with safe-area inset; example-prompt chips; a
44px account avatar; AA-contrast grays; real send-button state pair.
Middleware (all tenant-scoped via _org_of, one atomic per-org store each):
- GET/POST /v1/favorites (favorites.json) — the ONE favorites store; the legacy
studio_home.html favorite toggle migrated onto it too (was ratings.favorite).
- GET/POST /v1/stacks (stacks.json) — normalized, >=2 members, bounded.
- POST /v1/library/rate: carries `path` for the flywheel; favorite removed
(now its own store — one way).
- POST /v1/templates: accepts a `flow` payload, not only a node graph.
Tests: tests-unit/studio_home_test extended (favorites, stacks, rate stars+
notes+path, flow templates) — tenancy proven on each; added to the hanzo.yml
unit gate. web build green.
An interrupted mirror leaves bytes on disk with no library or worklog row;
the dedup branch skipped indexing, so re-ingesting repaired nothing. The
library upserts by path; the worklog now upserts by output. Recovery is a
re-POST of the same bytes.
Co-authored-by: Hanzo AI <ai@hanzo.ai>
The indexer skips them; _safe_asset refuses them (one gate for file serving,
fix sources, and every other path lookup). AppleDouble forks carry image
extensions, serve resource-fork bytes, and rendered 0x0 across the library.
Co-authored-by: Hanzo AI <ai@hanzo.ai>
The SPA called the gateway cross-origin; the session cookie is host-only, so
every signed-in user got 403 on their first message (401 once the cookie was
widened — the gateway takes bearers, not cookies). The bridge forwards the
caller's bearer server-side, the wallet pattern. VITE_AGENT_API still selects
direct mode.
Co-authored-by: Hanzo AI <ai@hanzo.ai>
On a phone (<=820px) the library rail was display:none, so a user could chat and
dispatch renders but never see them. Now the body stacks (chat fills, library
drops under it) and the rail shows as a compact scrollable bottom strip — but only
when there is something to show (jobs or assets set the `active` class), so the
empty chat state stays clean. Verified with Playwright at iPhone 14 (390x844):
empty = clean chat; with renders = 4-col strip under the composer, no overflow.
* upload: reject dotfiles and sniff magic bytes — the name lies, the bytes never do
AppleDouble forks (._foo.png) passed the extension check; 700+ poisoned a
library within an hour of the mirror going live. PNG/JPEG/WebP signatures
required; hidden names rejected.
* upload: size cap precedes the sniff; test payloads carry the full PNG signature
---------
Co-authored-by: Hanzo AI <ai@hanzo.ai>
The studio-chat frontend (web/, Vite on @hanzo/ai talking to POST /v1/agent) was
repointed + committed but never served. Build it in a node stage of the Dockerfile
(-> /app/web/dist) and serve it at GET /studio, with GET /studio/assets/{name} for
the hashed bundle (immutable) and a traversal guard. Falls back to the legacy
studio_home.html when the build is absent so /studio is never a dark page. This is
the decided cutover: studio.hanzo.ai/studio becomes the /v1/agent chat app.
An ingested render row has an empty prompt, so filtering Queue & History by
filename found nothing. Widen the /v1/worklog ?q= filter to also match kind,
output_prefix, output (the landed path), and refs — so a render is findable by
its filename or source image, not only by a typed prompt. Test covers filename,
ref, and prompt matches.
Co-authored-by: Hanzo AI <ai@hanzo.ai>
* Library ingest: POST /v1/library/upload — every render lands in the library
Tenant-scoped ingest so EVERY render is stored in studio.hanzo.ai, including
renders produced on a GPU node outside the job path (the node's mirror loop
POSTs here). Org via _org_of like every route.
Body: multipart (field image/file) OR raw bytes + ?name=, optional ?subpath=
(default renders). Writes orgs/<org>/output/<subpath>/<name> atomically
(tmp+replace, same pattern as the work log). Rejects traversal (basename-only
name + allowlisted subpath chars), caps size at 30MB, and skips a byte-identical
existing file (name+size match → 200 {ok, existed:true}); else 200 {ok, path}.
The manifest sidecar already indexes output/ — no other wiring.
Tests (studio_home_test): round-trip via GET /v1/library/file, dedup, tenant
isolation (org A upload never visible to org B, no cross-tenant overwrite),
subpath-traversal rejected, traversal-y name neutralized to basename, non-image
rejected, oversize rejected. 259 passed.
* Ship scripts/ whole (unbreak manifest sidecar) + index renders at ingest
(A) Regression: .dockerignore `scripts/*` + `!install_custom_nodes.sh` dropped
scripts/library_manifest.py from the image, so the build-manifest sidecar's
`[ -f ]` guard silently no-op'd and NOTHING got indexed into any org library.
Ship scripts/ whole again.
(B) Make indexing event-driven at the ingest point: POST /v1/library/upload, after
writing the asset, atomically appends the org library.json entry ({path, design?,
kind, role?, status:"draft", tags:[], updatedAt}) AND a worklog row ({kind:"render",
node, prompt:"", refs:[], parents:[], output_prefix:<path>, status:"done"}) — so
every mirrored render is instantly findable in Assets AND Queue & History with its
source node, filterable by run/flow, the moment it exists. Optional ?design= ?kind=
?role= ?node= with safe defaults (node→"upload"). Upserts by path (no dup on
re-render); the existed/dedup path skips indexing. The manifest sidecar stays the
sweeper for files that appear outside the route. _row_output resolves an ingested
render's output (its stored path) so it shows a thumbnail + opens its graph.
Tests: upload → library.json entry + worklog row appear (node, done, resolved
output), tenancy holds, dedup adds no second entry. 260 passed.
---------
Co-authored-by: Hanzo AI <ai@hanzo.ai>
cursor:pointer + title=Account with no handler. Tap now opens a menu:
identity, Manage account (hanzo.id), Sign out (/logout — existing route).
Co-authored-by: Hanzo AI <ai@hanzo.ai>
The orchestrator lives at POST /v1/agent (github.com/hanzoai/agent in the cloud
binary), not /v1/chat (which ai owns as its completions route). Use the canonical
`preset` selector (not the legacy `capability` alias) and thread conversationId so
the per-org history the orchestrator persists actually continues across turns.
VITE_CHAT_API → VITE_AGENT_API.
One shared shell asset (middleware/shell.{js,css}, served at /studio/shell.*)
is loaded by BOTH views — the Studio home <script>-includes it, and the Editor
gets a single tag injected into the engine index.html by server.get_root. It
renders one chrome in both: a Studio/Editor toggle, wallet chip, GPUs badge +
panel, and a dockable right chat sidebar backed by the ONE chat path
(/v1/copilot/chat). Same-origin cookies carry the IAM session, so login/org are
inherited — no auth is rebuilt.
New tenant-scoped routes (all via _org_of):
GET /studio/shell.js|css
GET /v1/workflow?id=<worklog id> graph from the item's output PNG (404 if
not owned / not landed) — Open in Editor
POST /v1/templates save {name, graph} → orgs/<org>/templates
GET /v1/templates list
GET /v1/templates/{name} fetch
POST /v1/templates/render queue through the ONE dispatch funnel
GET /v1/wallet display-only proxy of the user token to the
cloud finance balance → {org, balance, currency}
Studio home: "Advanced mode" link replaced by the shell toggle; dead chatfab CSS
removed; Templates tab lists org templates (Open in Editor + Render); work-item
context gets Open in Editor; favorite→template now writes a REAL template.
Mobile (verified 390x844 + 768x1024): chat collapses to a full-width bottom
sheet; the pill bar stays one compact line (icon-only Chat/GPUs) so all controls
stay reachable and clear the content; zero horizontal overflow.
Wallet is display + Top up link (pay.hanzo.ai) only — no card fields, no minting.
Tests extend studio_home_test for the new routes (tenant isolation + shape).
Co-authored-by: Hanzo AI <ai@hanzo.ai>
Liveness is the 1200s heartbeat; the cap only bounds a live-but-stuck render.
Observed: activities reaped at ~68m mid-render, queue re-ran the same job six
times, results stranded on the node.
Co-authored-by: Hanzo AI <ai@hanzo.ai>
Makes the unified queue honest about WHERE and WHEN, all server-computed and
tenant-scoped (an org sees only its own jobs' positions and its own nodes).
WHERE — node identity:
- gpu_dispatch reads whatever the gpu-jobs fleet publishes (_node_label prefers
name/hostname/label; home-lab labels spark/dbc/evo land there) — no parallel
registry invented. dispatch_if_worker records the target node (the single online
GPU is definitive; ambiguous 'gpu' with several) on the render_jobs row; _dispatch
copies it onto the work-log row. Local-core jobs are 'studio pod'.
- GET /v1/nodes: the org's GPU nodes (online/offline) + the pod, for header badges.
Fleet is token-scoped → only the caller's nodes.
WHEN — position + ETA (server-side, never client math over private data):
- worklog records queued/started/finished timestamps (mark_started when a job first
heads its lane; mark_finished from the output file mtime). Pure functions:
median_durations (rolling median render seconds per kind) and estimate_lanes
(position N of M per lane; wait ≈ jobs-ahead × medians + running remaining;
eta = wait + own median). Unknown kinds fall back to a default; labeled honestly.
- GET /v1/queue/status: this org's active jobs with {position, of, node, status,
elapsed_seconds, wait_seconds, remaining_seconds, eta_seconds} + medians. Reads
the org work log only; no fleet call on the hot path.
UI: 'Your work' rows and the BYO-GPU row show ⚙ node · #N/M · running <elapsed> ·
~Xm left / #N/M in line · ~X min est. Queue header shows per-node online/offline
badges + the median-render stat. Auto-refresh while the queue view is open (6s
positions, 20s nodes — no hammering); the running item's elapsed ticks each second
client-side; a completion toast fires when an open page sees a job land.
Tests (11 new): _node_label + online-node filtering; record-node; started/finished
+ median; median_durations per kind + ts fallback; estimate_lanes positions/ETA +
default-median. Route smoke: positions #1/2/3 with ETAs from the org's own median,
landed→finished drop, globex sees none of acme's queue, /v1/nodes maps fleet→badges.
Browser smoke: node+position+ETA render, elapsed ticks 0s→2s, median stat + pod
badge, zero console errors.
Co-authored-by: Hanzo AI <ai@hanzo.ai>
Cancel and amend any of the caller's OWN queued/running renders, resolved
server-side; a prompt_id that isn't the caller's 404s (never reveals it exists).
Cancel — POST /v1/queue/cancel {prompt_id}:
- local pod queue: delete a pending item OR interrupt a running one, org-scoped
(reuses the core prompt_queue primitives — one tenant can't touch another's).
- BYO-GPU lane: drop the render_jobs.json tracking row (per-org file → inherently
owned); a job already rendering on the worker may still finish (the pod can't
stop a remote process) — the UI says so honestly.
- marks the work-log row cancelled. _owns_job gates it: work-log OR org-stamped
local-queue item OR the org's gpu tracking file; else 404.
Amend — POST /v1/queue/amend {prompt_id, instruction, paths?, uploads?}:
- honest atomic swap (queued graphs are immutable): cancel the original, then
re-dispatch through the ONE funnel with the amended prompt + the original refs
(kept, read from the authoritative work-log row) + new library picks/uploads.
Records a fresh work-log row; position resets, shown honestly in the UI.
UI: Edit + Cancel on every queued/running work row and in the context panel;
Cancel on the BYO-GPU live row; the local 'Remove' now routes through the unified
cancel (dead delQueued removed). Amend reuses the Fix dialog in amend mode
(openAmend): prompt primed, current refs shown locked, add more via the same
dropzone + history picker. Errors surface in-dialog (no silent swallow).
Hardening: _queue_prompt tolerates a non-JSON downstream response (reports it
instead of 500-ing).
Tests: cancel org-scoped + gpu-row removal + cross-tenant not-owned; amend atomic
swap keeps original refs + adds new, unknown→NotOwned(404), terminal rejected (5).
Route smoke: amend swaps + re-queues, gpu cancel removes tracking, globex gets 404
on acme's job (cancel AND amend). Browser: Edit/Cancel on queued rows, amend dialog
primed with prompt + current ref, correct POST, zero console errors.
Co-authored-by: Hanzo AI <ai@hanzo.ai>
Adds a per-org work log — the ONE record of every render dispatched — as the
backbone for a queue/history that never loses a job and shows what produced it.
Backend (middleware/worklog.py, one JSON per org at orgs/<org>/output/worklog.json;
same pattern as library.json/render_jobs.json; no new DB, no backfill):
- The single dispatch funnel (dispatch_fix/dispatch_compose/rerun) records one row
at dispatch via _dispatch(): {id, ts, kind, prompt, refs, uploads, parents,
output_prefix, params, lane, status, events}. A FAILED dispatch is recorded too
(status=failed + reason) so the deployed 'nothing showed as queued' can't recur —
every dispatch is visible with real status, gpu-lane or local.
- GET /v1/worklog — org's dispatched renders, reverse-chron, ?q= over prompt+kind,
status enriched (done when the output lands). Server-resolved org only.
- GET /v1/worklog/lineage?path= — a library asset's version chain (ancestors it was
derived from + later fixes), a pure function over this org's rows.
- _landed_output matches SaveImage's exact <prefix>_<counter>_.png (not a loose
<prefix>*, which collapsed distinct lineage steps).
Frontend (studio_home.html):
- Queue & History → 'Your work': searchable, reverse-chron list with status badge +
lane + prompt + thumbnail; click → context panel (prompt, reference/result thumbs,
workflow family, params, process trail, and the Versions chain inline).
- 'Versions' affordance on every library card → the same lineage chain; click any
version to open it.
- 'Reuse context' / 'Fix this version' open the Fix dialog primed with the item's
prompt + references (openFixPath/primeFix) — reuse = continue from any version.
Tenant isolation: one log file per org; routes resolve org server-side; lineage is
pure over the caller's rows — cross-tenant leakage is impossible by construction.
Tests: worklog record/failed/status/cap, lineage chain + cycle-termination,
cross-tenant isolation (7). Route smoke: dispatch→done+output, 2-step lineage,
failed dispatch recorded with reason, globex sees none of acme's work. Browser
smoke: work list + status badge + context panel + Versions chain, zero console
errors.
Co-authored-by: Hanzo AI <ai@hanzo.ai>
Two user-reported live regressions on studio.hanzo.ai, root-caused at the
deployed image (sha-0d307dc, pre-#6):
1) Fix 'won't work / nothing queued' — the GPU-less prod pod fails /prompt
validation (models absent) or the router has no GPU worker; _queue_prompt
raised web.HTTPBadGateway(text=json.dumps(body)[:500]) — a raw 502 whose
TRUNCATED body the dialog couldn't parse, so the error was swallowed into an
empty 'Failed:'. Now _queue_prompt raises DispatchError carrying the REAL
reason (_prompt_error_message), which the fix/compose routes already return as
{"error"}. Every dialog/sender now SHOWS backend errors persistently and keeps
the dialog open on failure — no silent swallow anywhere on fix/compose paths.
2) The composer '+' attach button did nothing (no handler). Wired it to the
shared upload lane (POST /upload/image → org input dir, multi-tenant) with a
visible thumbnail; added drag-and-drop + paste to the Fix dialog AND the
composer, all through ONE shared helper (uploadImage/uploadMany/wireDrop/
wirePaste). Upload failures are shown, never swallowed. Reference tray now
holds library AND uploaded refs; a single uploaded photo drives a fix via the
new dispatch_fix(upload=) base (edit a fresh photo without landing it first).
Tests: text-only fix regression lock, uploaded-base fix + traversal refusal,
_prompt_error_message extraction, and a real-aiohttp test that a non-200 /prompt
surfaces the real reason (not a 502). Browser smoke (headless Chromium on the
served page): '+' opens the chooser and uploads, drag-drop adds a ref, text-only
Fix posts {path,instruction}, zero console errors.
Co-authored-by: Hanzo AI <ai@hanzo.ai>
The Fix action on a generated image becomes a reference-capable surface:
- base image (the clicked output) + up to 4 references
- drag-and-drop / click-to-upload image files as references, via the
existing core /upload/image route (org-scoped input dir — the same place
fix/compose already stage refs and the BYO-GPU collector ships from)
- add-from-history picker: attach past library generations as references
- no references -> single-image /v1/library/fix (unchanged); any reference
-> multi-input /v1/library/compose, base image first
Backend: dispatch_compose now accepts uploaded input-dir names (uploads[])
alongside library asset paths, validated as safe existing images; the 2-5
bound is on the combined reference count. One compose path, additive.
Tests: _resolve_uploads filtering, compose upload path + single-ref
rejection; corrected stale _STATUSES vocab (includes 'deleted').
Co-authored-by: Hanzo AI <ai@hanzo.ai>
Chat hits the cloud clients/chat orchestrator directly (credentials:'include'
carries the .hanzo.ai cookie cross-origin; handler replays it into the billed
completion). Engine calls stay same-origin. Needs gateway CORS to allow the
studio.hanzo.ai origin with credentials — infra config, no backend code.
Vite+React studio-chat: say what to create, the cloud /v1/chat orchestrator drives
studio's render tools, outputs land in the library. Increment 1 (own minimal UI
wired to /v1/chat create + library/queue/gpu). Served at /studio; the ComfyUI node
editor stays separate in studio-ui (was studio-frontend). Builds clean (npm-published
@hanzo/*).
MCP-over-HTTP (JSON-RPC 2.0) exposing the studio render pipeline as tools the
cloud /v1/chat 'create' capability drives via the tool registry. Reuses the ONE
dispatch path (studio_home.dispatch_fix/dispatch_compose, extracted from the
/v1/library/{fix,compose} routes) so an LLM-issued fix behaves identically to a
UI one. Registered per-org via cloud POST /v1/tools/servers. v0.17.3.
/studio: org-scoped asset gallery over library.json (status chips draft/
approved/queued/published), per-asset workflow provenance from the embedded
PNG graph, Re-run (exact graph, fresh seed) and chat-fix (instruction ->
Qwen-edit pass on that output) both through the ONE /prompt path with the
caller's own credentials. Pipelines tab reuses the 0.15.1 deep-link opener
for Edit/Run; STUDIO_HOME_DEFAULT=1 makes / land on /studio (SPA at
/?advanced=1). 5 unit tests.
Claude-Session: https://claude.ai/code/session_01Gq8suw7uuodAMPDRpo6iAB
A gallery/console/chat link lands you on the graph ready to render: sets the
session org (so a re-run dispatches to that org's BYO-GPU and outputs to its
gallery), loads the saved workflow, and optionally auto-queues. Injected as a
frontend shim (patch_deeplink.py, step 13 of apply-branding), guarded end-to-end.
/app/models is ephemeral overlay — every model vanished on pod restart
(CheckpointLoaderSimple: 'not in []'). Bake extra_model_paths.yaml into
the image registering /app/orgs/.models (PVC) as the default model store
for all model classes; main.py auto-loads it. Downloads land durable.
Serialized, SQLite-persisted queue fronting Hanzo Studio (ComfyUI :8188) that
turns a prompt into a real MP4 on the GB10. One GPU => one job at a time; a
single async worker drains an ordered queue and submits to Studio.
Two composable surfaces over one backend:
- OpenAI-style /v1/videos/generations (gateway-injected X-Org-Id identity, per
generated-second billing into a usage table).
- do-ai-compatible async /v1/videos + /{id} + /{id}/content (Bearer provider
key) — the exact Sora-style create->poll->download contract the cloud LLM
gateway's video client drives, so the ai registry adopts spark by pointing
the zen3-video provider at spark, no ai-side client change.
Zen video family public names (zen3-video/-fast/-pro); private Wan2.2 filenames
confined to zen_wan_workflow.py, never returned to callers. Includes the
systemd unit and studio_clean_restart.sh (breaks the Studio JIT/EADDRINUSE
crash-loop).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A GPU-less cloud pod validates ckpt_name & friends against ITS OWN (empty)
model list, rejecting every checkpoint-referencing graph with 'not in []'
before dispatch could ship it to the org's GPU box. Validation belongs where
execution happens: the dispatch attempt now runs first, and the worker
validates against its real node classes + models. Orgs with no online GPU
fall through to local validation unchanged.
BertModelWarper now delegates to the real BertModel.forward instead of
reimplementing internals whose private signatures drift between releases;
checkpoint weights remap bert.* -> bert.bert_model.* at load. The installer
force-checkouts each pin (discarding prior patches) then applies any
scripts/patches/<pack>-*.patch, so re-runs stay idempotent and the image
build gets the same fix the GB10 venv had by hand.
karma_manifest.py regenerates the ONE canonical library.json index of the
karma org's rendered assets (storefront + socials queue both read it).
studio_watchdog.sh keeps the local render backend on :8188 alive unattended;
paths derive from the script location (STUDIO/STUDIO_PY overridable).
A finished render becomes an `Asset` document — the DocType hanzoai/cloud
clients/content already declares (framework module "marketing") and already writes
itself when IT drives a render. Studio-initiated renders were the only ones with
no path into that record; this closes exactly that gap and adds NO second schema,
NO second lifecycle, and NO second storage plane. cloud is untouched.
POST /v1/framework/Asset (the already-live generic framework surface)
Same field shape clients/content writes (studio_render.go draftAsset): title,
kind, design, prompt, file, generator, workflow, source_prompt_id, render_params,
project, tags. `file` is the org-scoped output key (orgs/<org>/output/...) —
byte-identical to the key persistOrLink produces and the key Studio's own output
mirror lands, so the bytes are ALREADY persisted: this module uploads nothing and
holds no storage credential.
Written AS THE REQUESTING USER: the verified IAM access token rides to the prompt
worker on the queue's existing `sensitive` channel (never persisted into history)
and authenticates every write, so the Asset lands in that user's org with that
user's rights. No service account, no impersonation, no cross-org path.
Assets land in draft (the DocType default); the content lane's ONE state machine
(draft->in_review->approved->queued->published) owns every transition, and a human
does the promoting. Publishing is what a storefront shows, so an unattended render
can never push an image to karma.style.
Fire-and-forget on the same prompt-worker seam as billing: a render is never failed
because the lane was unreachable. Env-gated (STUDIO_CONTENT_PUBLISH); local dev
unaffected.
Add HanzoMusic (POST /v1/audio/music -> ACE-Step stereo AUDIO) and HanzoDub
(POST /v1/animate -> MuseTalk lip-sync: portrait IMAGE + driving AUDIO -> frames
+ audio). Fix HanzoImageTo3D to send the now-live `texture` flag (textured GLB
via SLAT + FlexiCubes) and offer obj. Correct HanzoTTS response_format to the
engine's actual wav/pcm. Drop the torchaudio dependency: studio_audio_to_wav_bytes
now uses the stdlib wave module, and studio_audio_to_data_url feeds the dub's
driving audio.
Ship user/default/workflows/hanzo-full-generative-pipeline.json: a prompt is
expanded by Chat and fans out to Image Gen -> (textured Image-to-3D AND WAN
Text-to-Video), plus TTS narration and ACE-Step Music -- every modality in one
graph, all on the native engine.
contract_check.py drives all 8 nodes over real HTTP against a stub engine that
enforces each Rust handler's request fields and the async job submit->poll->content
protocol.
Run Hanzo Studio AI workloads on the native-Rust Hanzo Engine instead of
in-process Python. Transport, config, and Studio<->engine type conversions
live in one client (client.py); each node is one endpoint, so a new engine
endpoint is a new small node with no framework change.
Nodes (category Hanzo/Engine): HanzoChat (SSE streaming, model dropdown from
/v1/models), HanzoImageGen (engine width/height/n schema), HanzoTTS
(/v1/audio/speech -> AUDIO), HanzoASR (/v1/audio/transcriptions -> text),
HanzoEngineRequest (generic /v1 escape hatch), HanzoVisionCaption,
HanzoTextToVideo, HanzoImageTo3D, HanzoSaveText.
Config: one way, HANZO_ENGINE_URL (default http://127.0.0.1:1234), /v1/* only.
Ships example user/default/workflows/hanzo-native-pipeline.json (LLM prompt ->
image prompt -> image gen -> save), e2e-verified against a CPU Qwen3-0.6B engine.
Uploaded images land in orgs/{org}/input on this GPU-less pod; the BYO
worker renders on its own disk and cannot read them, so LoadImage of an
uploaded photo failed. Collect every LoadImage-referenced file that
exists under the org input dir, base64-inline it into the studio.render
job input (capped, traversal-safe). The worker materializes them before
rendering (cloud cli/gpu.go). Files already on the worker are harmless to
resend.
The studio.render job input now carries this deployment's public base (STUDIO_PUBLIC_URL,
default studio.hanzo.ai) as uploadUrl, so the worker POSTs finished renders back to
/upload/output here — org derived from the user's token, landing in orgs/{org}/output.
A remote hanzod gpu-fleet worker renders a studio.render job on its own GPU
and POSTs each finished image to /upload/output with the user's IAM bearer.
The IAM middleware validates the token and derives org from owner/groups, so
the file lands in this org's orgs/{org}/output (PVC + S3 mirror -> gallery).
No S3/rclone credentials on the worker box; the session token is the only
credential. Replaces the old rclone/S3-creds output sync.
- verify_jwt accepts a set of first-party audiences (STUDIO_IAM_ALLOWED_AUDIENCES)
so a console/desktop/CLI-minted user token authorizes an org-scoped upload
without a per-service re-mint (org still comes from owner/groups, not aud).
- image_upload forwards the request so /upload/image|mask|output are org-scoped.
The bundled studio/sd1_tokenizer/vocab.json had the token 'comfy</w>' scrubbed
(49407 tokens vs canonical 49408) while merges.txt still contained the 'com fy'
merge, so CLIPTokenizer.from_pretrained raised 'Error while initializing BPE:
Token comfy</w> out of vocabulary' for ANY FLUX.1 CLIP load (DualCLIPLoader type
'flux'). Only ever hit now because FLUX.1-Fill is the first CLIP-L consumer here
(Qwen/Klein use other tokenizers). Restored the canonical openai/clip-vit-large-
patch14 vocab.json (49408 tokens). Required by couture_best.json.
The bottom Docs/GitHub/Console/Discord row had no .product a rule, so it fell
back to the browser default link color + underline. Match the card's muted
console tokens (#cdd2da, no underline, hover -> #eef0f3).
Task 2 — login/header look & feel to match console.hanzo.ai. The console is a
deliberately monochrome true-black system (no brand-purple; white primary
action; 9px radii; hairline #23262b borders), so the prior black+#8b5cf6 purple
login + identity pill did not read as the same product family. Restyled to
console's tokens:
- web/login.html: rebuilt as a console-style centered card (surface #050505,
1px #23262b, 9px radius) with the 5-path Hanzo H mark, 'Sign in with Hanzo'
as a white (#fbfcfd) primary button with near-black text — not the default
ComfyUI splash.
- branding/hanzo-identity.css: pill + dropdown restyled monochrome to match
console's OrgSwitcher/user menu (surface/border/text tokens, neutral avatar,
red only for logout). Bumped identity bundle v2 -> v3 (cache-bust).
Task 3 — Nodes 2.0 (Comfy.VueNodes.Enabled) shipped as the only node renderer.
New idempotent branding/patch_vuenodes.py (wired as apply-branding step 12):
forces the registered default !1 -> !0, sets type boolean -> hidden so the
Settings dialog row is gone (ComfyUI-native, like Bookmarks.V2), flips the
inline ??!1 read fallbacks to ??!0, and deletes the logo-menu nodes-2.0-toggle
item. Same hidden treatment for the sibling AutoScaleLayout so the whole
'Nodes 2.0' settings category collapses. Verified: forced on, no user toggle.
Locally (IAM off) /v1/session returns 200 with authenticated:false; the
menu now renders the functional Local User pill instead of a Sign in
button whose /login 404s locally. The real signed-out state is the cloud
401 (auth on, no token) -> Sign in. Verified unmocked against the live
backend. Bundle v2.
Add a comfy->studio import shim so upstream ComfyUI custom nodes run
unmodified against this fork's renamed core, then vendor the popular packs
(Impact-Pack/Subpack, IPAdapter_plus, controlnet_aux, Ultimate SD Upscale,
Frame-Interpolation, VideoHelperSuite, rgthree, Custom-Scripts, KJNodes,
cg-use-everywhere, SAM2, segment_anything) pinned to immutable commits and
reproducibly rebuilt in the image.
- studio_compat: one meta-path finder aliasing comfy*->studio* and mirroring
Comfy*->Studio* API classes (the deferred phase-4 of scripts/rename_modules.py,
done with the 3.12 find_spec API); activated by one import at top of main.py.
- nodes.py: honor the upstream `comfy_entrypoint` V3 extension name too.
- custom_nodes/hanzo-packs.txt: pinned pack manifest (SHAs + SPDX licenses).
- custom_nodes-requirements.txt + scripts/install_custom_nodes.sh: pack deps
with the cu130 torch / NumPy 2.x / Transformers 5.x stack locked (derived
from the live env, never downgraded); ultralytics installed --no-deps.
- Dockerfile/.dockerignore/.gitignore: image vendors the packs at build.
Live GB10 worker verified: node types 644 -> 1290 (+646), 0 import failures.
was-node-suite quarantined (hard numba dep incompatible with NumPy 2.5).
Top-right user menu driven by GET /v1/session (user, org switcher,
project scope, Console link, Logout). Org switching is real via one
hook: the auth middleware applies session.resolve_org to a COPY of the
validated user so iam_user["org_id"] -- the id every tenancy/output
path already reads -- reflects the selection; membership from
_orgs_from_claims (owner + organizations/orgs/groups). POST
/v1/session/{org,project} set validated cookies; /logout clears the
session + selection cookies and bounces to the IAM end-session -> /login.
Frontend is a SPA-agnostic pill + dropdown (branding/hanzo-identity.*),
injected like the copilot (patch_identity.py, apply-branding step 10).
patch_destring.py (step 11) sweeps English display 'ComfyUI' -> 'Hanzo
Studio' (boundary-safe: skips code identifiers, i18n keys, hyphenated
product names, and non-English locale bundles) and rebrands the Settings
section header value 'Comfy' -> 'Studio' / 'Comfy-Desktop' -> 'Studio
Desktop' while preserving the setting KEYS. Served English display
'ComfyUI' 63->0. Tests: middleware_test/session_test.py (12 new).
Backend POST /v1/copilot/chat (middleware/copilot.py, registered in server.py)
runs one OpenAI tool-calling round to a chat endpoint (STUDIO_COPILOT_URL,
default local engine :1234 else api.hanzo.ai) and returns {reply, ops}. add_node
ops validated against nodes.NODE_CLASS_MAPPINGS so the model can't invent nodes.
Frontend bundle (branding/hanzo-copilot.js + on-brand css) registers a Copilot
sidebar tab via app.extensionManager.registerSidebarTab and applies ops against
app.graph (LiteGraph): add/set_widget/connect/set_prompt/move/delete/layout/queue.
Every op guarded. Injected by branding/patch_copilot.py (step 9 of apply-branding.sh),
survives frontend upgrades. window.HanzoCopilot.applyOps exposed for tests.
Tests: middleware_test/copilot_test.py (13, op-validation + mocked tool loop),
branding/copilot_smoke.py (headless: applier mutates a mock graph + panel renders).
Live route needs one server restart to register.
On a successful prompt execution, emit exactly one fire-and-forget usage
event to Commerce meter-events (POST /v1/billing/meter-events): userId =
IAM org_id, value = artifacts produced, idempotency = prompt_id. Timeout +
retry-once; a billing failure only logs and is dropped — a render is never
blocked or failed for billing.
Replaces the execution-time-priced record_usage (which billed against the
STUDIO_ORG_ID env, not the real IAM tenant) with metered record_render:
Studio reports raw quantity, pricing lives on the meter in Commerce.
Env-gated on STUDIO_BILLING_URL + STUDIO_COMMERCE_TOKEN + STUDIO_BILLING_METER
(all absent locally = no-op, zero behavior change). check_balance left as-is.
Tests: tests-unit/middleware_test/billing_test.py — success shape, idempotency
key, timeout-doesn't-block + retry-once, 4xx-no-retry, 5xx-retry, unconfigured
and partial-config no-ops. Full middleware_test suite green (52 passed).
The SPA "Sign in with Hanzo" button hand-rolled an authorize URL
(IAM /login?redirect_uri=<studio root>), which the IAM default login app
(hanzo-console) rejected because the studio root is not in its allowed
redirect-URI list -- blocking all customer login.
Give the flow exactly one entry point: the button now links to the studio
server's own /login route, which runs the middleware's OIDC Authorization
Code flow (client_id=hanzo-studio, redirect_uri=<origin>/callback, PKCE +
signed state). /callback is already an allowed redirect URI, so login
completes and lands on /. Removes the client-side URL builder and the dead
{{IAM_URL}} templating.
Tests: /login is public; middleware exposes handle_login; unauth /login
builds the hanzo-studio authorize URL to /callback; post-login return path is /.
Co-authored-by: hanzo-dev <dev@hanzo.ai>
Move the workspace-tab seeder out of a hand-edited index.html and into a
dedicated branding patcher wired into apply-branding.sh, so it survives a
frontend-package reinstall and re-applies on every branding pass.
Root cause the seeder addresses: the frontend restores open tabs from
localStorage via getStorageValue/setStorageValue, whose clientId-suffixed
sessionStorage layer is read BEFORE plain localStorage. A previously-saved
1-tab shadow (Comfy.OpenWorkflowsPaths:<clientId>) masked the seed, and the
old guard was burned at v2 so existing profiles never re-seeded.
Fix: bump guard v2->v3 (existing profiles re-seed once) and purge the stale
sessionStorage shadows on re-seed so the fresh localStorage seed wins even on
a same-tab reload. Clean-profile boot now opens all 25 fashion workflows
(alongside ComfyUI's own default tab).
SqlitePromptQueue (middleware/tasks_queue.py): drop-in for PromptQueue backed
by one SQLite file (stdlib sqlite3, WAL, zero external processes). Opt in with
STUDIO_QUEUE_DB; precedence STUDIO_QUEUE_DB > STUDIO_PERSIST_QUEUE > memory so
local default is unchanged. The put/get/task_done seam maps to a durable job
state machine: idempotent submit (INSERT OR IGNORE on prompt_id), exactly-once
claim (BEGIN IMMEDIATE), retry-on-error, and crash-recovery via a lease + reap
that re-runs an abandoned claim (idempotent — SaveImage suffixes increment).
Multiple Studio processes on the same DB compete safely.
Engine selector (middleware/engine_selector.py): GET /v1/engines lists execution
targets for the org (local + registered compute_config workers), PUT
/v1/engines/default sets a per-org default stored on ComputeProfile; route_prompt
honors it (local = unchanged). Leased cloud machines are a future engine class
(interface stubbed). Frontend picker is a TODO.
docs/federation.md rewritten to the durable-queue reality with one future
paragraph (remote Tasks backend + Go/unified-binary HIP-0106 migration + leased
machines). Tests: middleware_test/{tasks_queue,engine_selector}_test.py, incl.
crash-recovery across queue instances.
ci@v1 defaults runs-on to [self-hosted,linux,amd64]; ARC scale sets are
matched by installation name, so the job sat queued with no runner. Pin the
runner to the hanzo build pool (same as cloud/release.yml).
- Remove .github/workflows/deploy.yml — the canonical build+test+deploy path
is cicd.yml (imports hanzoai/ci build.yml@v1, reads hanzo.yml). One way to
build. The old Docker workflow called hanzoai/.github docker-build.yml and
failed at 0s.
- prompt_queue_persist_test.py: drop the 4 diagnostic prints (T201); pytest
reports pass/fail and the standalone runner raises on failure.
- zen_video_adapter.py: remove unused json/asyncio imports (F401).
Root cause of the "Event loop stopped before Future completed" crash that
wiped the in-memory queue/history (seen 3x today): the fork's _graceful_shutdown
handler (added in a45a441) called event_loop.stop(). The server coroutine passed
to run_until_complete() never completes on its own, so stopping the loop makes
run_until_complete() raise RuntimeError; the surrounding try only caught
KeyboardInterrupt, so the process died non-zero with a traceback and discarded
the queue. The dirty exit also failed to release :8188 promptly, and a duplicate
transient/user `hanzo-studio.service` racing the canonical studio.service turned
that into an EADDRINUSE restart storm.
main.py:
- handler is now idempotent, flags the queue closed, snapshots it, then stops
the loop; the outer try treats RuntimeError('Event loop stopped before Future
completed.') as the expected graceful-exit path (logs "Server stopped by
shutdown signal", exits 0). Verified live in journald.
execution.py:
- opt-in crash-durable PromptQueue behind STUDIO_PERSIST_QUEUE=1. Pending +
in-flight prompts are snapshotted atomically to user/queue_snapshot.json on
every change and re-queued on boot. Best-effort: all I/O is guarded so it
never affects the render path; inert when the flag is unset.
ops/systemd/studio.service:
- enable STUDIO_PERSIST_QUEUE=1 on the single canonical unit.
scripts/studio-service-swap.sh:
- batch-safe reconcile+apply: waits for an empty queue, removes any stray
duplicate unit, reinstalls the canonical unit, restarts, health-checks.
docs/ops-crash-and-service.md:
- topology (one canonical unit), how to read crash history, the fix, persistence.
tests-unit/execution_test/prompt_queue_persist_test.py:
- round-trip, inert-when-disabled, and corrupt-row-skipping tests.
Auth (middleware/iam_auth_middleware.py): local JWKS JWT validation against
Hanzo IAM (signature + exp + iss + aud=hanzo-studio; org from the `owner`
claim). Standard OIDC Authorization Code flow with PKCE and a /callback
session cookie for the standalone studio.hanzo.ai app; anonymous localhost
bypass unchanged. verify_jwt() extracted as a pure, unit-tested function.
Tenancy (folder_paths.py, app/user_manager.py): make get_public_user_directory
org-rooted so multi-tenant userdata resolves under user/orgs/{org}/... (it
previously always returned None). Scope execution outputs to output/orgs/{org}/
via set_execution_org(), bound by the prompt worker from the queue item org_id.
Also block a token subject from escaping into a System User namespace.
Federation (middleware/{worker_client,prompt_router}.py, server.py): move the
worker registry + execute surface to /v1/workers/register, /v1/workers,
/v1/worker/execute; add X-Worker-Token coordinator trust. Outbound pull channel
for NAT'd local boxes (GB10) specced in docs/federation.md.
Deploy: hanzo.yml + .github/workflows/cicd.yml (hanzoai/ci reusable) build
ghcr.io/hanzoai/studio and roll the studio operator Service CR at
studio.hanzo.ai. PyJWT[crypto] pinned.
Tests: tests-unit/middleware_test/iam_auth_test.py (RS256 sign/verify, state
CSRF, path exemptions) + folder_paths_test/org_scoping_test.py; 140 passing
across the auth/tenancy/user-manager/prompt-server suites.
Anoeses-style PDP pattern: product_<design> = garment-only ghost presentation on pure white (base image); hover_<design> = model in dynamic pose on subtle white studio (rollover image). Workspace: index.html seeds Comfy.OpenWorkflowsPaths with all 25 fashion tabs on first load (set-once, localStorage); Comfy.Workflow.Persist on. Seeder added to apply-branding.sh replay path.
Native TextEncodeQwenImageEditPlus path wired against the live /object_info
schema: LoadImage -> Plus encoder (image1+vae) + VAEEncode -> KSampler
(30 steps, cfg 2.5, euler/simple) -> VAEDecode -> SaveImage swimwear/antje.
Models installed under models/{diffusion_models,text_encoders,vae} (merged
from the Qwen-Image-Edit-2511 diffusers shards, gitignored). 8 of Antje's
CAD drawings preloaded in input/ and enumerated by LoadImage.
- Sidebar logo: replace the inline ComfyLogo Vue component (top-left mark,
compiled into the JS bundle, missed by the file-asset pass) with the Hanzo
mark via branding/patch_sidebar_logo.py (keyed off the stable __name:`ComfyLogo`).
- Theme: branding/hanzo-theme.css forces a pure-black shell + Hanzo-purple
accents (AA contrast), injected as the last stylesheet; patch_theme.py also
rewrites the dark palette's canvas clear color to #000000 (CSS can't reach the
litegraph <canvas>).
- Favicon: make_progress_favicons.py renders the 10 progress frames with the
Hanzo mark + purple ring, fixing the tab reverting to the Comfy mark during
generations. Committed PNGs like favicon.ico (Docker needs no rasterizer).
- Tab title: rebrand the runtime ' - ComfyUI' title suffix the string pass missed.
- apply-branding.sh replays all of the above on frontend-package upgrades.
- Rename fork-owned update_comfyui{,_stable}.bat -> update{,_stable}.bat.
Black background, white text, opacity-based visual hierarchy,
pill-shaped CTA button, Inter font, feature cards with subtle
white/5-10% backgrounds. Removes red accent entirely.
- Compute profiles: per-org CPU/GPU config stored in user dir with
thread-safe CRUD and atomic file writes
- Prompt routing: forwards GPU prompts to registered workers, falls
back to local execution when no worker available
- Worker mode: --worker-mode flag runs Studio as headless executor
that registers with coordinator via heartbeats
- Visor integration: provision/terminate GPU VMs on AWS (T4/V100/A100/H100)
with shell-safe startup scripts
- Login page: unauthenticated browsers see branded login gate with
"Sign in with Hanzo" button instead of broken ComfyUI interface
- 6 new API endpoints under /api/compute/*
- Fix branding patcher to target actual upstream logo filenames
(comfy-logo-*.svg, not studio-logo-*.svg)
- Add more display string patterns to patch_frontend.py
- Add IAM authentication middleware (hanzo.id/api/get-account)
with localhost bypass and browser login redirect
- Add multi-tenant org-scoped storage (folder_paths.get_org_*)
with per-org output/input/temp/user/models directories
- Add CLI flags: --enable-iam-auth, --iam-url, --multi-tenant,
--org-id, --no-localhost-bypass
- Support env var toggles for Docker: STUDIO_ENABLE_IAM_AUTH,
STUDIO_MULTI_TENANT, STUDIO_ORG_ID, STUDIO_IAM_URL
- Integrate IAM user context into UserManager and server routes
- Remove bare 'Comfy-Org' and 'ComfyOrg' from plain text replacements
(was corrupting i18n keys like OpenComfy-OrgDiscord)
- Only replace Comfy-Org/ComfyOrg in multi-word display phrases
- Add regex patterns for quoted string value contexts
- Add standalone quoted value patterns ("ComfyUI" → "Hanzo Studio")
- Add Comfy Cloud → Hanzo Cloud display text replacement
- Replace all user-facing "ComfyUI" strings with "Hanzo Studio"
- Update Help menu links: Discord → discord.gg/hanzoai, GitHub → hanzoai/studio
- Replace comfy.org URLs with hanzo.ai equivalents
- Update support email to support@hanzo.ai
- Rewrite branding script to patch ALL JS bundles comprehensively
- Update default filename prefixes from ComfyUI to HanzoStudio
- Fix CI to trigger on main branch (not hanzo/main)
- Update pyproject.toml metadata (name, URLs)
- Replace ComfyUI logo with Hanzo geometric H mark
- Replace favicon with Hanzo H favicon
- Patch index.html title, PWA manifest, and JS bundles
- Update AboutPanel and OrgHeader references
- Branding applied as Docker build step after pip install
* chore: tune CodeRabbit config to limit review scope and disable for drafts
- Add tone_instructions to focus only on newly introduced issues
- Add global path_instructions entry to ignore pre-existing issues in moved/reformatted code
- Disable draft PR reviews (drafts: false) and add WIP title keywords
- Disable ruff tool to prevent linter-based outside-diff-range comments
Addresses feedback from maintainers about CodeRabbit flagging pre-existing
issues in code that was merely moved or de-indented (e.g., PR #12557),
which can discourage community contributions and cause scope creep.
Amp-Thread-ID: https://ampcode.com/threads/T-019c82de-0481-7253-ad42-20cb595bb1ba
* chore: add 'DO NOT MERGE' to ignore_title_keywords
Amp-Thread-ID: https://ampcode.com/threads/T-019c82de-0481-7253-ad42-20cb595bb1ba
On Windows, Python defaults to cp1252 encoding when no encoding is
specified. JSON files containing UTF-8 characters (e.g., non-ASCII
characters) cause UnicodeDecodeError when read with cp1252.
This fixes the error that occurs when loading blueprint subgraphs
on Windows systems.
https://claude.ai/code/session_014WHi3SL9Gzsi3U6kbSjbSb
Co-authored-by: Claude <noreply@anthropic.com>
Integrate comfy-aimdo 0.2 which takes a different approach to
installing the memory allocator hook. Instead of using the complicated
and buggy pytorch MemPool+CudaPluggableAlloctor, cuda is directly hooked
making the process much more transparent to both comfy and pytorch. As
far as pytorch knows, aimdo doesnt exist anymore, and just operates
behind the scenes.
Remove all the mempool setup stuff for dynamic_vram and bump the
comfy-aimdo version. Remove the allocator object from memory_management
and demote its use as an enablment check to a boolean flag.
Comfy-aimdo 0.2 also support the pytorch cuda async allocator, so
remove the dynamic_vram based force disablement of cuda_malloc and
just go back to the old settings of allocators based on command line
input.
Add 24 non-cloud essential blueprints from comfyui-wiki/Subgraph-Blueprints.
These cover common workflows: text/image/video generation, editing,
inpainting, outpainting, upscaling, depth maps, pose, captioning, and more.
Cloud-only blueprints (5) are excluded and will be added once
client-side distribution filtering lands.
Amp-Thread-ID: https://ampcode.com/threads/T-019c6f43-6212-7308-bea6-bfc35a486cbf
gemini-3-pro-image-preview nondeterministically returns image/jpeg
instead of image/png. get_image_from_response() hardcoded
get_parts_by_type(response, "image/png"), silently dropping JPEG
responses and falling back to torch.zeros (all-black output).
Add _mime_matches() helper using fnmatch for glob-style MIME matching.
Change get_image_from_response() to request "image/*" so any image
format returned by the API is correctly captured.
This check was far too broad and the dtype is not a reliable indicator
of wanting the requant (as QT returns the compute dtype as the dtype).
So explictly plumb whether fp8mm wants the requant or not.
* lora: add weight shape calculations.
This lets the loader know if a lora will change the shape of a weight
so it can take appropriate action.
* MPDynamic: force load flux img_in weight
This weight is a bit special, in that the lora changes its geometry.
This is rather unique, not handled by existing estimate and doesn't
work for either offloading or dynamic_vram.
Fix for dynamic_vram as a special case. Ideally we can fully precalculate
these lora geometry changes at load time, but just get these models
working first.
* lora_extract: Add a trange
If you bite off more than your GPU can chew, this kinda just hangs.
Give a rough indication of progress counting the weights in a trange.
* lora_extract: Support on-the-fly patching
Use the on-the-fly approach from the regular model saving logic for
lora extraction too. Switch off force_cast_weights accordingly.
This gets extraction working in dynamic vram while also supporting
extraction on GPU offloaded.
Get rid of the cat and unary negation and inplace add-cmul the two
halves of the rope. Precompute -sin once at the start of the model
rather than every transformer block.
This is slightly faster on both GPU and CPU bound setups.
The current behaviour of the default ModelPatcher is to .to a model
only if its fully loaded, which is how random non-leaf weights get
loaded in non-LowVRAM conditions.
The however means they never get loaded in dynamic_vram. In the
dynamic_vram case, force load them to the GPU.
* model_management: lazy-cache aimdo_tensor
These tensors cosntructed from aimdo-allocations are CPU expensive to
make on the pytorch side. Add a cache version that will be valid with
signature match to fast path past whatever torch is doing.
* dynamic_vram: Minimize fast path CPU work
Move as much as possible inside the not resident if block and cache
the formed weight and bias rather than the flat intermediates. In
extreme layer weight rates this adds up.
* Fix bypass dtype/device moving
* Force offloading mode for training
* training context var
* offloading implementation in training node
* fix wrong input type
* Support bypass load lora model, correct adapter/offloading handling
This was missing the stochastic rounding required for fp8 downcast
to be consistent with model_patcher.patch_weight_to_device.
Missed in testing as I spend too much time with quantized tensors
and overlooked the simpler ones.
If there are non-trivial python objects nested in the model_options, this
causes all sorts of issues. Traverse lists and dicts so clones can safely
overide settings and BYO objects but stop there on the deepclone.
* feat(api-nodes-Kling): add new models (V3, O3)
* remove storyboard from VideoToVideo node
* added check for total duration of storyboards
* fixed other small things
* updated display name for nodes
* added "fake" seed
* revert threaded model loader change
This change was only needed to get around the pytorch 2.7 mempool bugs,
and should have been reverted along with #12260. This fixes a different
memory leak where pytorch gets confused about cache emptying.
* load non comfy weights
* MPDynamic: Pre-generate the tensors for vbars
Apparently this is an expensive operation that slows down things.
* bump to aimdo 1.8
New features:
watermark limit feature
logging enhancements
-O2 build on linux
Torch has alignment enforcement when viewing with data type changes
but only relative to itself. Do all tensor constructions straight
off the memory-view individually so pytorch doesnt see an alignment
problem.
The is needed for handling misaligned safetensors weights, which are
reasonably common in third party models.
This limits usage of this safetensors loader to GPU compute only
as CPUs kernnel are very likely to bus error. But it works for
dynamic_vram, where we really dont want to take a deep copy and we
always use GPU copy_ which disentangles the misalignment.
* feat(comfy_api): add basic 3D Model file types
* update Tripo nodes to use File3DGLB
* update Rodin3D nodes to use File3DGLB
* address PR review feedback:
- Rename File3D parameter 'path' to 'source'
- Convert File3D.data property to get_data()
- Make .glb extension check case-insensitive in nodes_rodin.py
- Restrict SaveGLB node to only accept File3DGLB
* Fixed a bug in the Meshy Rig and Animation nodes
* Fix backward compatability
This is using a different layers weight with .to(). Change it to use
the ops caster if the original layer is a comfy weight so that it picks
up dynamic_vram and async_offload functionality in full.
Co-authored-by: Rattus <rattus128@gmail.com>
* mp: fix full dynamic unloading
This was not unloading dynamic models when requesting a full unload via
the unpatch() code path.
This was ok, i your workflow was all dynamic models but fails with big
VRAM leaks if you need to fully unload something for a regular ModelPatcher
It also fices the "unload models" button.
* mm: load models outside of Aimdo Mempool
In dynamic_vram mode, escape the Aimdo mempool and load into the regular
mempool. Use a dummy thread to do it.
This function has a dtype argument that allows the caller to set the
dtype in the cast. TIL Some models override this on weight casts, which
means its the highest priority.
Priority scheme is: argument > model dtype > state dict dtype
pinned memory was converted back to pinning the CPU side weight without
any changes. Fix the pinner to use the CPU weight and not the model defined
geometry. This will either save RAM or stop buffer overruns when the types
mismatch.
Fix the model defined weight caster to use the [ s.weight, s.bias ]
interpretation, as xfer_dest might be the flattened pin now. Fix the detection
of needing to cast to not be conditional on !pin.
- Change error type from 'invalid_prompt' to 'missing_node_type' for frontend detection
- Add extra_info with node_id, class_type, and node_title (from _meta.title)
- Improve user-facing message: 'Node X not found. The custom node may not be installed.'
Move count increment before isinstance(item, dict) check so that
non-dict output items (like text strings from PreviewAny node)
are included in outputs_count.
This aligns OSS Python with Cloud's Go implementation which uses
len(itemsArray) to count ALL items regardless of type.
Amp-Thread-ID: https://ampcode.com/threads/T-019c0bb5-14e0-744f-8808-1e57653f3ae3
Co-authored-by: Amp <amp@ampcode.com>
When a node is declared as dev-only, it doesn't show in the default UI
unless the dev mode is enabled in the settings. The intention is to
allow nodes related to unit testing to be included in ComfyUI
distributions without confusing the average user.
The code throughout is None safe to just skip the feature cache saving
step if none. Set it none in single frame use so qwen doesn't burn VRAM
on the unused cache.
* ops: introduce autopad for conv3d
This works around pytorch missing ability to causal pad as part of the
kernel and avoids massive weight duplications for padding.
* wan-vae: rework causal padding
This currently uses F.pad which takes a full deep copy and is liable to
be the VRAM peak. Instead, kick spatial padding back to the op and
consolidate the temporal padding with the cat for the cache.
* wan-vae: implement zero pad fast path
The WAN VAE is also QWEN where it is used single-image. These
convolutions are however zero padded 3d convolutions, which means the
VAE is actually just 2D down the last element of the conv weight in
the temporal dimension. Fast path this, to avoid adding zeros that
then just evaporate in convoluton math but cost computation.
* Disable timestep embed compression when inpainting
Spatial inpainting not compatible with the compression
* Reduce crossattn peak VRAM
* LTX2: Refactor forward function for better VRAM efficiency
- Add search_aliases for discoverability: resize, scale, dimensions, etc.
- Add node description for hover tooltip
- Add tooltips to all inputs explaining their behavior
- Reorder options: most common (scale dimensions) first, most technical (scale to multiple) last
Addresses user feedback that 'resize' search returned nothing useful and
options like 'match size' and 'scale to multiple' were not self-explanatory.
- Add search_aliases for discoverability: resize, scale, dimensions, etc.
- Add node description for hover tooltip
- Add tooltips to all inputs explaining their behavior
- Reorder options: most common (scale dimensions) first, most technical (scale to multiple) last
Addresses user feedback that 'resize' search returned nothing useful and
options like 'match size' and 'scale to multiple' were not self-explanatory.
* causal_video_ae: Remove attention ResNet
This attention_head_dim argument does not exist on this constructor so
this is dead code. Remove as generic attention mid VAE conflicts with
temporal roll.
* ltx-vae: consoldate causal/non-causal code paths
* ltx-vae: add cache rolling adder
* ltx-vae: use cached adder for resnet
* ltx-vae: Implement rolling VAE
Implement a temporal rolling VAE for the LTX2 VAE.
Usually when doing temporal rolling VAEs you can just chunk on time relying
on causality and cache behind you as you go. The LTX VAE is however
non-causal.
So go whole hog and implement per layer run ahead and backpressure between
the decoder layers using recursive state beween the layers.
Operations are ammended with temporal_cache_state{} which they can use to
hold any state then need for partial execution. Convolutions cache their
inputs behind the up to N-1 frames, and skip connections need to cache the
mismatch between convolution input and output that happens due to missing
future (non-causal) input.
Each call to run_up() processes a layer accross a range on input that
may or may not be complete. It goes depth first to process as much as
possible to try and digest frames to the final output ASAP. If layers run
out of input due to convolution losses, they simply return without action
effectively applying back-pressure to the earlier layers. As the earlier
layers do more work and caller deeper, the partial states are reconciled
and output continues to digest depth first as much as possible.
Chunking is done using a size quota rather than a fixed frame length and
any layer can initiate chunking, and multiple layers can chunk at different
granulatiries. This remove the old limitation of always having to process
1 latent frame to entirety and having to hold 8 full decoded frames as
the VRAM peak.
* re-init
* Update model_multitalk.py
* whitespace...
* Update model_multitalk.py
* remove print
* this is redundant
* remove import
* Restore preview functionality
* Move block_idx to transformer_options
* Remove LoopingSamplerCustomAdvanced
* Remove looping functionality, keep extension functionality
* Update model_multitalk.py
* Handle ref_attn_mask with separate patch to avoid having to always return q and k from self_attn
* Chunk attention map calculation for multiple speakers to reduce peak VRAM usage
* Update model_multitalk.py
* Add ModelPatch type back
* Fix for latest upstream
* Use DynamicCombo for cleaner node
Basically just so that single_speaker mode hides mask inputs and 2nd audio input
* Update nodes_wan.py
For LTX Audio VAE, remove normalization of audio during MEL spectrogram creation.
This aligs inference with training and prevents loud audio from being attenuated.
* In-progress autogrow validation fixes - properly looks at required/optional inputs, now working on the edge case that all inputs are optional and nothing is plugged in (should just be an empty dictionary passed into node)
* Allow autogrow to work with all inputs being optional
* Revert accidentally pushed changes to nodes_logic.py
Add 'advanced' boolean parameter to Input and WidgetInput base classes
and propagate to all typed Input subclasses (Boolean, Int, Float, String,
Combo, MultiCombo, Webcam, MultiType, MatchType, ImageCompare).
When set to True, the frontend will hide these inputs by default in a
collapsible 'Advanced Inputs' section in the right side panel, reducing
visual clutter for power-user options.
This enables nodes to expose advanced configuration options (like encoding
parameters, quality settings, etc.) without overwhelming typical users.
Frontend support: ComfyUI_frontend PR #7812
* feat: add CI container version bump automation
Adds a workflow that triggers on releases to create PRs in the
comfyui-ci-container repo, updating the ComfyUI version in the Dockerfile.
Supports both release events and manual workflow dispatch for testing.
* feat: add CI container version bump automation
Adds a workflow that triggers on releases to create PRs in the
comfyui-ci-container repo, updating the ComfyUI version in the Dockerfile.
Supports both release events and manual workflow dispatch for testing.
* ci: update CI container repository owner
* refactor: rename `update-ci-container.yaml` workflow to `update-ci-container.yml`
* Remove post-merge instructions from the CI container update workflow.
* api nodes: price badges moved to nodes code
* added price badges for 4 more node-packs
* added price badges for 10 more node-packs
* added new price badges for Omni STD mode
* add support for autogrow groups
* use full names for "widgets", "inputs" and "groups"
* add strict typing for JSONata rules
* add price badge for WanReferenceVideoApi node
* add support for DynamicCombo
* sync price badges changes (https://github.com/Comfy-Org/ComfyUI_frontend/pull/7900)
* sync badges for Vidu2 nodes
* fixed incorrect price for RecraftCrispUpscaleNode
* fixed incorrect price badges for LTXV nodes
* fixed price badge for MinimaxHailuoVideoNode
* fixed price badges for PixVerse nodes
Use vae.spacial_compression_encode() instead of directly accessing
downscale_ratio to handle both standard VAEs (int) and WAN VAEs (tuple).
Addresses reviewer feedback on PR #11259.
Co-authored-by: ChrisFab16 <christopher@fabritius.dk>
* Brought over minimal elements from PR 10045 to reproduce seed_assets and register_assets_system without adding anything to the DB or server routes yet, for now making everything sync (can introduce async once everything is cleaned up and brought over)
* Added db script to insert assets stuff, cleaned up some code; assets (models) now get added/rescanned
* Added support for 5 http endpoints for assets
* Replaced Optional with | None in schemas_in.py and schemas_out.py
* Remove two routes that will not be relevant yet in this PR: HEAD /api/assets/hash/<hash> and PUT /api/assets/<id>/preview
* Remove some functions the two deleted endpoints were using
* Don't show assets scan message upon calling /object_info endpoint
* removed unsued import to satisfy ruff
* Simplified hashing function tpye hint and _hash_file_obj
* Satisfied ruff
This logic was checking comfy_cast_weights, and going straight to
to the forward_comfy_cast_weights implementation without
attempting to downscale input to fp8 in the event comfy_cast_weights
is set.
The main reason comfy_cast_weights would be set would be for async
offload, which is not a good reason to nix FP8MM.
So instead, and together the underlying exclusions for FP8MM which
are:
* having a weight_function (usually LowVramPatch)
* force_cast_weights (compute dtype override)
* the weight is not Quantized
* the input is already quantized
* the model or layer has MM explictily disabled.
If you get past all of those exclusions, quantize the input tensor.
Then hand the new input, quantized or not off to
forward_comfy_cast_weights to handle it. If the weight is offloaded
but input is quantized you will get an offloaded MM8.
If the loader passes 1e32 as the usable memory size, it means force
the full load. This happens with CPU loads and a few other misc cases.
Removing the confusing number and just leave the other details.
* Support Combo outputs in a more sane way
* Remove test validate_inputs function on test node
* Make curr_prefix be a list of strings instead of string for easier parsing as keys get added to dynamic types
* Start to account for id prefixes from frontend, need to fix bug with nested dynamics
* Ensure inputs/outputs/hidden are lists in schema finalize function, remove no longer needed 'is not None' checks
* Add raw_link and extra_dict to all relevant Inputs
* Make nested DynamicCombos work properly with prefixed keys on latest frontend; breaks old Autogrow, but is pretty much ready for upcoming Autogrow keys
* Replace ... usage with a MISSING sentinel for clarity in nodes_logic.py
* Added CustomCombo node in backend to reflect frontend node
* Prepare Autogrow's expand_schema_for_dynamic to work with upcoming frontend changes
* Prepare for look up table for dynamic input stuff
* More progress towards dynamic input lookup function stuff
* Finished converting _expand_schema_for_dynamic to be done via lookup instead of OOP to guarantee working with process isolation, did refactoring to remove old implementation + cleaning INPUT_TYPES definition including v3 hidden definition
* Change order of functions
* Removed some unneeded functions after dynamic refactor
* Make MatchType's output default displayname "MATCHTYPE"
* Fix DynamicSlot get_all
* Removed redundant code - dynamic stuff no longer happens in OOP way
* Natively support AnyType (*) without __ne__ hacks
* Remove stray code that made it in
* Remove expand_schema_for_dynamic left over on DynamicInput class
* get_dynamic() on DynamicInput/Output was not doing anything anymore, so removed it
* Make validate_inputs validate combo input correctly
* Temporarily comment out conversion to 'new' (9 month old) COMBO format in get_input_info
* Remove refrences to resources feature scrapped from V3
* Expose DynamicCombo in public API
* satisfy ruff after some code got commented out
* Make missing input error prettier for dynamic types
* Created a Switch2 node as a side-by-side test, will likely go with Switch2 as the initial switch node
* Figured out Switch situation
* Pass in v3_data in IsChangedCache.get function's fingerprint_inputs, add a from_v3_data helper method to HiddenHolder
* Switch order of Switch and Soft Switch nodes in file
* Temp test node for MatchType
* Fix missing v3_data for v1 nodes in validation
* For now, remove chacking duplicate id's for dynamic types
* Add Resize Image/Mask node that thanks to MatchType+DynamicCombo is 16-nodes-in-1
* Made DynamicCombo references in DCTestNode use public interface
* Add an AnyTypeTestNode
* Make lazy status for specific inputs on DynamicInputs work by having the values of the dictionary for check_lazy_status be a tuple, where the second element is the key of the input that can be returned
* Comment out test logic nodes
* Make primitive float's step make more sense
* Add (and leave commented out) some potential logic nodes
* Change default crop option to "center" on Resize Image/Mask node
* Changed copy.copy(d) to d.copy()
* Autogrow is available in stable frontend, so exposing it in public API
* Use outputs id as display_name if no display_name present, remove v3 outputs id restriction that made them have to have unique IDs from the inputs
* Enable Custom Combo node as stable frontend now supports it
* Make id properly act like display_name on outputs
* Add Batch Images/Masks/Latents node
* Comment out Batch Images/Masks/Latents node for now, as Autogrow has a bug with MatchType where top connection is disconnected upon refresh
* Removed code for a couple test nodes in nodes_logic.py
* Add Batch Images, Batch Masks, and Batch Latents nodes with Autogrow, deprecate old Batch Images + LatentBatch nodes
* Add support for sage attention 3 in comfyui, enable via new cli arg
--use-sage-attiention3
* Fix some bugs found in PR review. The N dimension at which Sage
Attention 3 takes effect is reduced to 1024 (although the improvement is
not significant at this scale).
* Remove the Sage Attention3 switch, but retain the attention function
registration.
* Fix a ruff check issue in attention.py
Pretty much every error cudaHostRegister can throw also queues the same
error on the async GPU queue. This was fixed for repinning error case,
but there is the bad mmap and just enomem cases that are harder to
detect.
Do some dummy GPU work to clean the error state.
Can be used to manually set the sigmas for a model.
This node accepts a list of integer and floating point numbers separated
with any non numeric character.
This operation trades in latents which in --gpu-only may be out of the GPU
The two VAE results will follow the --gpu-only defined behaviour so follow
the inpaint image device when calculating the mask in this path.
* feat: create a /jobs api to return queue and history jobs
* update unused vars
* include priority
* create jobs helper file
* fix ruff
* update how we set error message
* include execution error in both responses
* rename error -> failed, fix output shape
* re-use queue and history functions
* set workflow id
* allow srot by exec duration
* fix tests
* send priority and remove error msg
* use ws messages to get start and end times
* revert main.py fully
* refactor: move all /jobs business logic to jobs.py
* fix failing test
* remove some tests
* fix non dict nodes
* address comments
* filter by workflow id and remove null fields
* add clearer typing - remove get("..") or ..
* refactor query params to top get_job(s) doc, add remove_sensitive_from_queue
* add brief comment explaining why we skip animated
* comment that format field is for frontend backward compatibility
* fix whitespace
---------
Co-authored-by: Jedrzej Kosinski <kosinkadink1@gmail.com>
Co-authored-by: guill <jacob.e.segal@gmail.com>
index_timestep_zero can be selected in the
FluxKontextMultiReferenceLatentMethod now with the display name set to the
more generic "Edit Model Reference Method" node.
The inpaint part is currently missing and will be implemented later.
I think they messed up this model pretty bad. They added some
control_noise_refiner blocks but don't actually use them. There is a typo
in their code so instead of doing control_noise_refiner -> control_layers
it runs the whole control_layers twice.
Unfortunately they trained with this typo so the model works but is kind
of slow and would probably perform a lot better if they corrected their
code and trained it again.
From now on ComfyUI will do version numbers a bit differently, every stable
off the master branch will increment the minor version. Anytime a fix needs
to be backported onto a stable version the patch version will be
incremented.
Example: We release v0.6.0 off the master branch then a day later a bug is
discovered and we decide to backport the fix onto the v0.6.0 stable, this
will be done in a separate branch in the main repository and this new
stable will be tagged v0.6.1
* make setattr safe for non existent attributes
Handle the case where the attribute doesnt exist by returning a static
sentinel (distinct from None). If the sentinel is passed in as the set
value, del the attr.
* Account for dequantization and type-casts in offload costs
When measuring the cost of offload, identify weights that need a type
change or dequantization and add the size of the conversion result
to the offload cost.
This is mutually exclusive with lowvram patches which already has
a large conservative estimate and wont overlap the dequant cost so\
dont double count.
* Set the compute type on CLIP MPs
So that the loader can know the size of weights for dequant accounting.
In the lowvram case, this now does its math in the model dtype in the
post de-quantization domain. Account for that. The patching was also
put back on the compute stream getting it off-peak so relax the
MATH_FACTOR to only x2 so get out of the worst-case assumption of
everything peaking at once.
slow down the CPU on model load to not run ahead. This fixes a VRAM on
flux 2 load.
I went to try and debug this with the memory trace pickles, which needs
--disable-cuda-malloc which made the bug go away. So I tried this
synchronize and it worked.
The has some very complex interactions with the cuda malloc async and
I dont have solid theory on this one yet.
Still debugging but this gets us over the OOM for the moment.
* Add Kandinsky5 model support
lite and pro T2V tested to work
* Update kandinsky5.py
* Fix fp8
* Fix fp8_scaled text encoder
* Add transformer_options for attention
* Code cleanup, optimizations, use fp32 for all layers originally at fp32
* ImageToVideo -node
* Fix I2V, add necessary latent post process nodes
* Support text to image model
* Support block replace patches (SLG mostly)
* Support official LoRAs
* Don't scale RoPE for lite model as that just doesn't work...
* Update supported_models.py
* Rever RoPE scaling to simpler one
* Fix typo
* Handle latent dim difference for image model in the VAE instead
* Add node to use different prompts for clip_l and qwen25_7b
* Reduce peak VRAM usage a bit
* Further reduce peak VRAM consumption by chunking ffn
* Update chunking
* Update memory_usage_factor
* Code cleanup, don't force the fp32 layers as it has minimal effect
* Allow for stronger changes with first frames normalization
Default values are too weak for any meaningful changes, these should probably be exposed as advanced node options when that's available.
* Add image model's own chat template, remove unused image2video template
* Remove hard error in ReplaceVideoLatentFrames -node
* Update kandinsky5.py
* Update supported_models.py
* Fix typos in prompt template
They were now fixed in the original repository as well
* Update ReplaceVideoLatentFrames
Add tooltips
Make source optional
Better handle negative index
* Rename NormalizeVideoLatentFrames -node
For bit better clarity what it does
* Fix NormalizeVideoLatentStart node out on non-op
- Add manager setup instructions and command line options to README
- Document --enable-manager, --enable-manager-legacy-ui, and
--disable-manager-ui flags
- Bump comfyui_manager version from 4.0.3b3 to 4.0.3b4
* Apply cond slice fix
* Add FreeNoise
* Update context_windows.py
* Add option to retain condition by indexes for each window
This allows for example Wan/HunyuanVideo image to video to "work" by using the initial start frame for each window, otherwise windows beyond first will be pure T2V generations.
* Update context_windows.py
* Allow splitting multiple conds into different windows
* Add handling for audio_embed
* whitespace
* Allow freenoise to work on other dims, handle 4D batch timestep
Refactor Freenoise function. And fix batch handling as timesteps seem to be expanded to batch size now.
* Disable experimental options for now
So that the Freenoise and bugfixes can be merged first
---------
Co-authored-by: Jedrzej Kosinski <kosinkadink1@gmail.com>
Co-authored-by: ozbayb <17261091+ozbayb@users.noreply.github.com>
* chore(api-nodes): applied ruff's pyupgrade(python3.10) to api-nodes client's to folder
* chore(api-nodes): add validate_video_frame_count function from LTX PR
* chore(api-nodes): replace deprecated V1 imports
* fix(api-nodes): the types returned by the "poll_op" function are now correct.
Im able to push vram above estimate on partial unload. Bump the
estimate. This is experimentally determined with a 720P and 480P
datapoint calibrating for 24GB VRAM total.
TIL that the WAN TE has a 2GB weight followed by 16MB as the next size
down. This means that team 8GB VRAM would fully offload the TE in async
offload mode as it just multiplied this giant size my the num streams.
Do the more complex logic of summing up the upcoming to-load weight
sizes to avoid triple counting this massive weight.
partial unload does the converse of recording the NS most recent
unloads as they go.
* mp: only count the offload cost of math once
This was previously bundling the combined weight storage and computation
cost
* ops: put all post async transfer compute on the main stream
Some models have massive weights that need either complex
dequantization or lora patching. Don't do these patchings on the offload
stream, instead do them on the main stream to syncrhonize the
potentially large vram spikes for these compute processes. This avoids
having to assume a worst case scenario of multiple offload streams
all spiking VRAM is parallel with whatever the main stream is doing.
This should fix the problem with the portable updater not working with portables created from a separate branch on the repo.
This does not affect any current portables who were all created on the master branch.
* Added output_matchtypes to generated json for v3, initial backend support for MatchType, created nodes_logic.py and added SwitchNode
* Fixed providing list of allowed_types
* Add workaround in validation.py for V3 Combo outputs not working as Combo inputs
* Make match type receive_type pass validation
* Also add MatchType check to input_type in validation - will likely trigger when connecting to non-lazy stuff
* Make sure this PR only has MatchType stuff
* Initial work on DynamicCombo
* Add get_dynamic function, not yet filled out correctly
* Mark Switch node as Beta
* Make sure other unfinished dynamic types are not accidentally used
* Send DynamicCombo.Option inputs in the same format as normal v1 inputs
* add dynamic combo test node
* Support validation of inputs and outputs
* Add missing input params to DynamicCombo.Input
* Add get_all function to inputs for id validation purposes
* Fix imports for v3 returning everything when doing io/ui/IO/UI instead of what is in __all__ of _io.py and _ui.py
* Modifying behavior of get_dynamic in V3 + serialization so can be used in execution code
* Fix v3 schema validation code after changes
* Refactor hidden_values for v3 in execution.py to be more general v3_data, add helper functions for dynamic behavior, preparing for restructuring dynamic type into object (not finished yet)
* Add nesting of inputs on DynamicCombo during execution
* Work with latest frontend commits
* Fix cringe arrows
* frontend will no longer namespace dynamic inputs widgets so reflect that in code, refactor build_nested_inputs
* Prepare Autogrow support for the love of the game
* satisfy ruff
* Create test nodes for Autogrow to collab with frontend development
* Add nested combo to DCTestNode
* Remove array support from build_nested_inputs, properly handle missing expected values
* Make execution.validate_inputs properly validate required dynamic inputs, renamed dynamic_data to dynamic_paths for clarity
* MatchType does not need any DynamicInput/Output features on backend; will increase compatibility with dynamic types
* Probably need this for ruff check
* Change MatchType to have template be the first and only required param; output id's do nothing right now, so no need
* Fix merge regression with LatentUpscaleModel type not being put in __all__ for _io.py, fix invalid type hint for validate_inputs
* Make Switch node inputs optional, disallow both inputs from being missing, and still work properly with lazy; when one input is missing, use the other no matter what the switch is set to
* Satisfy ruff
* Move MatchType code above the types that inherit from DynamicInput
* Add DynamicSlot type, awaiting frontend support
* Make curr_prefix creation happen in Autogrow, move curr_prefix in DynamicCombo to only be created if input exists in live_inputs
* I was confused, fixing accidentally redundant curr_prefix addition in Autogrow
* Make sure Autogrow inputs are force_input = True when WidgetInput, fix runtime validation by removing original input from expected inputs, fix min/max bounds, change test nodes slightly
* Remove unnecessary id usage in Autogrow test node outputs
* Commented out Switch node + test nodes
* Remove commented out code from Autogrow
* Make TemplatePrefix max more clear, allow max == 1
* Replace all dict[str] with dict[str, Any]
* Renamed add_to_dict_live_inputs to expand_schema_for_dynamic
* Fixed typo in DynamicSlot input code
* note about live_inputs not being present soon in get_v1_info (internal function anyway)
* For now, hide DynamicCombo and Autogrow from public interface
* Removed comment
* hunyuan upsampler: rework imports
Remove the transitive import of VideoConv3d and Resnet and takes these
from actual implementation source.
* model: remove unused give_pre_end
According to git grep, this is not used now, and was not used in the
initial commit that introduced it (see below).
This semantic is difficult to implement temporal roll VAE for (and would
defeat the purpose). Rather than implement the complex if, just delete
the unused feature.
(venv) rattus@rattus-box2:~/ComfyUI$ git log --oneline
220afe33 (HEAD) Initial commit.
(venv) rattus@rattus-box2:~/ComfyUI$ git grep give_pre
comfy/ldm/modules/diffusionmodules/model.py: resolution, z_channels, give_pre_end=False, tanh_out=False, use_linear_attn=False,
comfy/ldm/modules/diffusionmodules/model.py: self.give_pre_end = give_pre_end
comfy/ldm/modules/diffusionmodules/model.py: if self.give_pre_end:
(venv) rattus@rattus-box2:~/ComfyUI$ git co origin/master
Previous HEAD position was 220afe33 Initial commit.
HEAD is now at 6c5e96e6 Enable async offloading by default on Nvidia. (#10953)
(venv) rattus@rattus-box2:~/ComfyUI$ git grep give_pre
comfy/ldm/modules/diffusionmodules/model.py: resolution, z_channels, give_pre_end=False, tanh_out=False, use_linear_attn=False,
comfy/ldm/modules/diffusionmodules/model.py: self.give_pre_end = give_pre_end
comfy/ldm/modules/diffusionmodules/model.py: if self.give_pre_end:
* move refiner VAE temporal roller to core
Move the carrying conv op to the common VAE code and give it a better
name. Roll the carry implementation logic for Resnet into the base
class and scrap the Hunyuan specific subclass.
* model: Add temporal roll to main VAE decoder
If there are no attention layers, its a standard resnet and VideoConv3d
is asked for, substitute in the temporal rolloing VAE algorithm. This
reduces VAE usage by the temporal dimension (can be huge VRAM savings).
* model: Add temporal roll to main VAE encoder
If there are no attention layers, its a standard resnet and VideoConv3d
is asked for, substitute in the temporal rolling VAE algorithm. This
reduces VAE usage by the temporal dimension (can be huge VRAM savings).
These are not actual controlnets so put it in the models/model_patches
folder and use the ModelPatchLoader + QwenImageDiffsynthControlnet node to
use it.
* feat(security): add System User protection with `__` prefix
Add protected namespace for custom nodes to store sensitive data
(API keys, licenses) that cannot be accessed via HTTP endpoints.
Key changes:
- New API: get_system_user_directory() for internal access
- New API: get_public_user_directory() with structural blocking
- 3-layer defense: header validation, path blocking, creation prevention
- 54 tests covering security, edge cases, and backward compatibility
System Users use `__` prefix (e.g., __system, __cache) following
Python's private member convention. They exist in user_directory/
but are completely blocked from /userdata HTTP endpoints.
* style: remove unused imports
* Support video tiny VAEs
* lighttaew scaling fix
* Also support video taes in previews
Only first frame for now as live preview playback is currently only available through VHS custom nodes.
* Support Wan 2.1 lightVAE
* Relocate elif block and set Wan VAE dim directly without using pruning rate for lightvae
* mm: default to 0 for NUM_STREAMS
Dont count the compute stream as an offload stream. This makes async
offload accounting easier.
* mm: remove 128MB minimum
This is from a previous offloading system requirement. Remove it to
make behaviour of the loader and partial unloader consistent.
* mp: order the module list by offload expense
Calculate an approximate offloading temporary VRAM cost to offload a
weight and primary order the module load list by that. In the simple
case this is just the same as the module weight, but with Loras, a
weight with a lora consumes considerably more VRAM to do the Lora
application on-the-fly.
This will slightly prioritize lora weights, but is really for
proper VRAM offload accounting.
* mp: Account for the VRAM cost of weight offloading
when checking the VRAM headroom, assume that the weight needs to be
offloaded, and only load if it has space for both the load and offload
* the number of streams.
As the weights are ordered from largest to smallest by offload cost
this is guaranteed to fit in VRAM (tm), as all weights that follow
will be smaller.
Make the partial unload aware of this system as well by saving the
budget for offload VRAM to the model state and accounting accordingly.
Its possible that partial unload increases the size of the largest
offloaded weights, and thus needs to unload a little bit more than
asked to accomodate the bigger temp buffers.
Honor the existing codes floor on model weight loading of 128MB by
having the patcher honor this separately withough regard to offloading.
Otherwise when MM specifies its 128MB minimum, MP will see the biggest
weights, and budget that 128MB to only offload buffer and load nothing
which isnt the intent of these minimums. The same clamp applies in
case of partial offload of the currently loading model.
* Create nodes_dataset.py
* Add encoded dataset caching mechanism
* make training node to work with our dataset system
* allow trainer node to get different resolution dataset
* move all dataset related implementation to nodes_dataset
* Rewrite dataset system with new io schema
* Rewrite training system with new io schema
* add ui pbar
* Add outputs' id/name
* Fix bad id/naming
* use single process instead of input list when no need
* fix wrong output_list flag
* use torch.load/save and fix bad behaviors
It looks like the synchronous version of the public API broke due to an
addition of `from __future__ import annotations`. This change updates
the async-to-sync adapter to work with both types of type annotations.
* init
* update
* Update model.py
* Update model.py
* remove print
* Fix text encoding
* Prevent empty negative prompt
Really doesn't work otherwise
* fp16 works
* I2V
* Update model_base.py
* Update nodes_hunyuan.py
* Better latent rgb factors
* Use the correct sigclip output...
* Support HunyuanVideo1.5 SR model
* whitespaces...
* Proper latent channel count
* SR model fixes
This also still needs timesteps scheduling based on the noise scale, can be used with two samplers too already
* vae_refiner: roll the convolution through temporal
Work in progress.
Roll the convolution through time using 2-latent-frame chunks and a
FIFO queue for the convolution seams.
* Support HunyuanVideo15 latent resampler
* fix
* Some cleanup
Co-Authored-By: comfyanonymous <121283862+comfyanonymous@users.noreply.github.com>
* Proper hyvid15 I2V channels
Co-Authored-By: comfyanonymous <121283862+comfyanonymous@users.noreply.github.com>
* Fix TokenRefiner for fp16
Otherwise x.sum has infs, just in case only casting if input is fp16, I don't know if necessary.
* Bugfix for the HunyuanVideo15 SR model
* vae_refiner: roll the convolution through temporal II
Roll the convolution through time using 2-latent-frame chunks and a
FIFO queue for the convolution seams.
Added support for encoder, lowered to 1 latent frame to save more
VRAM, made work for Hunyuan Image 3.0 (as code shared).
Fixed names, cleaned up code.
* Allow any number of input frames in VAE.
* Better VAE encode mem estimation.
* Lowvram fix.
* Fix hunyuan image 2.1 refiner.
* Fix mistake.
* Name changes.
* Rename.
* Whitespace.
* Fix.
* Fix.
---------
Co-authored-by: kijai <40791699+kijai@users.noreply.github.com>
Co-authored-by: Rattus <rattus128@gmail.com>
Slices model input with output channels so the caching tracks only the noise channels, resolves channel mismatch with models like WanVideo I2V
Also fix for slicing deprecation in pytorch 2.9
Clean up a bunch of stacked and no-longer-needed tensors on the QWEN
VRAM peak (currently FFN).
With this I go from OOMing at B=37x1328x1328 to being able to
succesfully run B=47 (RTX5090).
The partial unloader path in model re-use flow skips straight to the
actual unload without any check of the patching UUID. This means that
if you do an upscale flow with a model patch on an existing model, it
will not apply your patchings.
Fix by delaying the partial_unload until after the uuid checks. This
is done by making partial_unload a model of partial_load where extra_mem
is -ve.
* ops: dont take an offload stream if you dont need one
* ops: prioritize mem transfer
The async offload streams reason for existence is to transfer from
RAM to GPU. The post processing compute steps are a bonus on the side
stream, but if the compute stream is running a long kernel, it can
stall the side stream, as it wait to type-cast the bias before
transferring the weight. So do a pure xfer of the weight straight up,
then do everything bias, then go back to fix the weight type and do
weight patches.
* execution: Roll the UI cache into the outputs
Currently the UI cache is parallel to the output cache with
expectations of being a content superset of the output cache.
At the same time the UI and output cache are maintained completely
seperately, making it awkward to free the output cache content without
changing the behaviour of the UI cache.
There are two actual users (getters) of the UI cache. The first is
the case of a direct content hit on the output cache when executing a
node. This case is very naturally handled by merging the UI and outputs
cache.
The second case is the history JSON generation at the end of the prompt.
This currently works by asking the cache for all_node_ids and then
pulling the cache contents for those nodes. all_node_ids is the nodes
of the dynamic prompt.
So fold the UI cache into the output cache. The current UI cache setter
now writes to a prompt-scope dict. When the output cache is set, just
get this value from the dict and tuple up with the outputs.
When generating the history, simply iterate prompt-scope dict.
This prepares support for more complex caching strategies (like RAM
pressure caching) where less than 1 workflow will be cached and it
will be desirable to keep the UI cache and output cache in sync.
* sd: Implement RAM getter for VAE
* model_patcher: Implement RAM getter for ModelPatcher
* sd: Implement RAM getter for CLIP
* Implement RAM Pressure cache
Implement a cache sensitive to RAM pressure. When RAM headroom drops
down below a certain threshold, evict RAM-expensive nodes from the
cache.
Models and tensors are measured directly for RAM usage. An OOM score
is then computed based on the RAM usage of the node.
Note the due to indirection through shared objects (like a model
patcher), multiple nodes can account the same RAM as their individual
usage. The intent is this will free chains of nodes particularly
model loaders and associate loras as they all score similar and are
sorted in close to each other.
Has a bias towards unloading model nodes mid flow while being able
to keep results like text encodings and VAE.
* execution: Convert the cache entry to NamedTuple
As commented in review.
Convert this to a named tuple and abstract away the tuple type
completely from graph.py.
* mm: factor out the current stream getter
Make this a reusable function.
* ops: sync the offload stream with the consumption of w&b
This sync is nessacary as pytorch will queue cuda async frees on the
same stream as created to tensor. In the case of async offload, this
will be on the offload stream.
Weights and biases can go out of scope in python which then
triggers the pytorch garbage collector to queue the free operation on
the offload stream possible before the compute stream has used the
weight. This causes a use after free on weight data leading to total
corruption of some workflows.
So sync the offload stream with the compute stream after the weight
has been used so the free has to wait for the weight to be used.
The cast_bias_weight is extended in a backwards compatible way with
the new behaviour opt-in on a defaulted parameter. This handles
custom node packs calling cast_bias_weight and defeatures
async-offload for them (as they do not handle the race).
The pattern is now:
cast_bias_weight(... , offloadable=True) #This might be offloaded
thing(weight, bias, ...)
uncast_bias_weight(...)
* controlnet: adopt new cast_bias_weight synchronization scheme
This is nessacary for safe async weight offloading.
* mm: sync the last stream in the queue, not the next
Currently this peeks ahead to sync the next stream in the queue of
streams with the compute stream. This doesnt allow a lot of
parallelization, as then end result is you can only get one weight load
ahead regardless of how many streams you have.
Rotate the loop logic here to synchronize the end of the queue before
returning the next stream. This allows weights to be loaded ahead of the
compute streams position.
In the case of --cache-none lazy and subgraph execution can cause
anything to be run multiple times per workflow. If that rerun nodes is
in itself a subgraph generator, this will crash for two reasons.
pending_subgraph_results[] does not cleanup entries after their use.
So when a pending_subgraph_result is consumed, remove it from the list
so that if the corresponding node is fully re-executed this misses
lookup and it fall through to execute the node as it should.
Secondly, theres is an explicit enforcement against dups in the
addition of subgraphs nodes as ephemerals to the dymprompt. Remove this
enforcement as the use case is now valid.
* Implement mixed precision operations with a registry design and metadate for quant spec in checkpoint.
* Updated design using Tensor Subclasses
* Fix FP8 MM
* An actually functional POC
* Remove CK reference and ensure correct compute dtype
* Update unit tests
* ruff lint
* Implement mixed precision operations with a registry design and metadate for quant spec in checkpoint.
* Updated design using Tensor Subclasses
* Fix FP8 MM
* An actually functional POC
* Remove CK reference and ensure correct compute dtype
* Update unit tests
* ruff lint
* Fix missing keys
* Rename quant dtype parameter
* Rename quant dtype parameter
* Fix unittests for CPU build
* feat(api-nodes): implement new API client for V3 nodes
* feat(api-nodes): implement new API client for V3 nodes
* feat(api-nodes): implement new API client for V3 nodes
* converted WAN nodes to use new client; polishing
* fix(auth): do not leak authentification for the absolute urls
* convert BFL API nodes to use new API client; remove deprecated BFL nodes
* converted Google Veo nodes
* fix(Veo3.1 model): take into account "generate_audio" parameter
* execution: fold in dependency aware caching
This makes --cache-none compatiable with lazy and expanded
subgraphs.
Currently the --cache-none option is powered by the
DependencyAwareCache. The cache attempts to maintain a parallel
copy of the execution list data structure, however it is only
setup once at the start of execution and does not get meaninigful
updates to the execution list.
This causes multiple problems when --cache-none is used with lazy
and expanded subgraphs as the DAC does not accurately update its
copy of the execution data structure.
DAC has an attempt to handle subgraphs ensure_subcache however
this does not accurately connect to nodes outside the subgraph.
The current semantics of DAC are to free a node ASAP after the
dependent nodes are executed.
This means that if a subgraph refs such a node it will be requed
and re-executed by the execution_list but DAC wont see it in
its to-free lists anymore and leak memory.
Rather than try and cover all the cases where the execution list
changes from inside the cache, move the while problem to the
executor which maintains an always up-to-date copy of the wanted
data-structure.
The executor now has a fast-moving run-local cache of its own.
Each _to node has its own mini cache, and the cache is unconditionally
primed at the time of add_strong_link.
add_strong_link is called for all of static workflows, lazy links
and expanded subgraphs so its the singular source of truth for
output dependendencies.
In the case of a cache-hit, the executor cache will hold the non-none
value (it will respect updates if they happen somehow as well).
In the case of a cache-miss, the executor caches a None and will
wait for a notification to update the value when the node completes.
When a node completes execution, it simply releases its mini-cache
and in turn its strong refs on its direct anscestor outputs, allowing
for ASAP freeing (same as the DependencyAwareCache but a little more
automatic).
This now allows for re-implementation of --cache-none with no cache
at all. The dependency aware cache was also observing the dependency
sematics for the objects and UI cache which is not accurate (this
entire logic was always outputs specific).
This also prepares for more complex caching strategies (such as RAM
pressure based caching), where a cache can implement any freeing
strategy completely independently of the DepedancyAwareness
requirement.
* main: re-implement --cache-none as no cache at all
The execution list now tracks the dependency aware caching more
correctly that the DependancyAwareCache.
Change it to a cache that does nothing.
* test_execution: add --cache-none to the test suite
--cache-none is now expected to work universally. Run it through the
full unit test suite. Propagate the server parameterization for whether
or not the server is capabale of caching, so that the minority of tests
that specifically check for cache hits can if else. Hard assert NOT
caching in the else to give some coverage of --cache-none expected
behaviour to not acutally cache.
* Add get_subgraphs_dir to ComfyExtension and PUBLISHED_SUBGRAPH_DIRS to nodes.py
* Created initial endpoints, although the returned paths are a bit off currently
* Fix path and actually return real data
* Sanitize returned /api/global_subgraphs entries
* Remove leftover function from early prototyping
* Remove added whitespace
* Add None check for sanitize_entry
* execution: fold in dependency aware caching
This makes --cache-none compatiable with lazy and expanded
subgraphs.
Currently the --cache-none option is powered by the
DependencyAwareCache. The cache attempts to maintain a parallel
copy of the execution list data structure, however it is only
setup once at the start of execution and does not get meaninigful
updates to the execution list.
This causes multiple problems when --cache-none is used with lazy
and expanded subgraphs as the DAC does not accurately update its
copy of the execution data structure.
DAC has an attempt to handle subgraphs ensure_subcache however
this does not accurately connect to nodes outside the subgraph.
The current semantics of DAC are to free a node ASAP after the
dependent nodes are executed.
This means that if a subgraph refs such a node it will be requed
and re-executed by the execution_list but DAC wont see it in
its to-free lists anymore and leak memory.
Rather than try and cover all the cases where the execution list
changes from inside the cache, move the while problem to the
executor which maintains an always up-to-date copy of the wanted
data-structure.
The executor now has a fast-moving run-local cache of its own.
Each _to node has its own mini cache, and the cache is unconditionally
primed at the time of add_strong_link.
add_strong_link is called for all of static workflows, lazy links
and expanded subgraphs so its the singular source of truth for
output dependendencies.
In the case of a cache-hit, the executor cache will hold the non-none
value (it will respect updates if they happen somehow as well).
In the case of a cache-miss, the executor caches a None and will
wait for a notification to update the value when the node completes.
When a node completes execution, it simply releases its mini-cache
and in turn its strong refs on its direct anscestor outputs, allowing
for ASAP freeing (same as the DependencyAwareCache but a little more
automatic).
This now allows for re-implementation of --cache-none with no cache
at all. The dependency aware cache was also observing the dependency
sematics for the objects and UI cache which is not accurate (this
entire logic was always outputs specific).
This also prepares for more complex caching strategies (such as RAM
pressure based caching), where a cache can implement any freeing
strategy completely independently of the DepedancyAwareness
requirement.
* main: re-implement --cache-none as no cache at all
The execution list now tracks the dependency aware caching more
correctly that the DependancyAwareCache.
Change it to a cache that does nothing.
* test_execution: add --cache-none to the test suite
--cache-none is now expected to work universally. Run it through the
full unit test suite. Propagate the server parameterization for whether
or not the server is capabale of caching, so that the minority of tests
that specifically check for cache hits can if else. Hard assert NOT
caching in the else to give some coverage of --cache-none expected
behaviour to not acutally cache.
Same change pattern as 7e8dd275c243ad460ed5015d2e13611d81d2a569
applied to WAN2.2
If this suffers an exception (such as a VRAM oom) it will leave the
encode() and decode() methods which skips the cleanup of the WAN
feature cache. The comfy node cache then ultimately keeps a reference
this object which is in turn reffing large tensors from the failed
execution.
The feature cache is currently setup at a class variable on the
encoder/decoder however, the encode and decode functions always clear
it on both entry and exit of normal execution.
Its likely the design intent is this is usable as a streaming encoder
where the input comes in batches, however the functions as they are
today don't support that.
So simplify by bringing the cache back to local variable, so that if
it does VRAM OOM the cache itself is properly garbage when the
encode()/decode() functions dissappear from the stack.
* updated V2V node to allow for control image input
exposing steps in v2v
fixing guidance_scale as input parameter
TODO: allow for motion_intensity as input param.
* refactor: comment out unsupported resolution and adjust default values in video nodes
* set control_after_generate
* adding new defaults
* fixes
* changed control_after_generate back to True
* changed control_after_generate back to False
---------
Co-authored-by: thorsten <thorsten@tripod-digital.co.nz>
## Summary
Fixed incorrect type hint syntax in `MotionEncoder_tc.__init__()` parameter list.
## Changes
- Line 647: Changed `num_heads=int` to `num_heads: int`
- This corrects the parameter annotation from a default value assignment to proper type hint syntax
## Details
The parameter was using assignment syntax (`=`) instead of type annotation syntax (`:`), which would incorrectly set the default value to the `int` class itself rather than annotating the expected type.
If this suffers an exception (such as a VRAM oom) it will leave the
encode() and decode() methods which skips the cleanup of the WAN
feature cache. The comfy node cache then ultimately keeps a reference
this object which is in turn reffing large tensors from the failed
execution.
The feature cache is currently setup at a class variable on the
encoder/decoder however, the encode and decode functions always clear
it on both entry and exit of normal execution.
Its likely the design intent is this is usable as a streaming encoder
where the input comes in batches, however the functions as they are
today don't support that.
So simplify by bringing the cache back to local variable, so that if
it does VRAM OOM the cache itself is properly garbage when the
encode()/decode() functions dissappear from the stack.
When the VAE catches this VRAM OOM, it launches the fallback logic
straight from the exception context.
Python however refs the entire call stack that caused the exception
including any local variables for the sake of exception report and
debugging. In the case of tensors, this can hold on the references
to GBs of VRAM and inhibit the VRAM allocated from freeing them.
So dump the except context completely before going back to the VAE
via the tiler by getting out of the except block with nothing but
a flag.
The greately increases the reliability of the tiler fallback,
especially on low VRAM cards, as with the bug, if the leak randomly
leaked more than the headroom needed for a single tile, the tiler
would fallback would OOM and fail the flow.
* feature: Set the Ascend NPU to use a single one
* Enable the `--cuda-device` parameter to support both CUDA and Ascend NPUs simultaneously.
* Make the code just set the ASCENT_RT_VISIBLE_DEVICES environment variable without any other edits to master branch
---------
Co-authored-by: Jedrzej Kosinski <kosinkadink1@gmail.com>
* flux: math: Use _addcmul to avoid expensive VRAM intermediate
The rope process can be the VRAM peak and this intermediate
for the addition result before releasing the original can OOM.
addcmul_ it.
* wan: Delete the self attention before cross attention
This saves VRAM when the cross attention and FFN are in play as the
VRAM peak.
Adds installed and required workflow templates version information to the
/system_stats endpoint, allowing the frontend to detect and notify users
when their templates package is outdated.
- Add get_installed_templates_version() and get_required_templates_version()
methods to FrontendManager
- Include templates version info in system_stats response
- Add comprehensive unit tests for the new functionality
When unloading models in load_models_gpu(), the model finalizer was not
being explicitly detached, leading to a memory leak. This caused
linear memory consumption increase over time as models are repeatedly
loaded and unloaded.
This change prevents orphaned finalizer references from accumulating in
memory during model switching operations.
If you have memory issues you can try disabling the smart memory management by running Hanzo Studio with:
run_amd_gpu_disable_smart_memory.bat
IF YOU GET A RED ERROR IN THE UI MAKE SURE YOU HAVE A MODEL/CHECKPOINT IN: HanzoStudio\models\checkpoints
You can download the stable diffusion XL one from: https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0_0.9vae.safetensors
RECOMMENDED WAY TO UPDATE:
To update the Hanzo Studio code: update\update_studio.bat
TO SHARE MODELS BETWEEN HANZO STUDIO AND ANOTHER UI:
In the HanzoStudio directory you will find a file: extra_model_paths.yaml.example
Rename this file to: extra_model_paths.yaml and edit it with your favorite text editor.
if you want to enable the fast fp16 accumulation (faster for fp16 models with slightly less quality):
run_nvidia_gpu_fast_fp16_accumulation.bat
To run it in slow CPU mode:
run_cpu.bat
IF YOU GET A RED ERROR IN THE UI MAKE SURE YOU HAVE A MODEL/CHECKPOINT IN: ComfyUI\models\checkpoints
You can download the stable diffusion 1.5 one from: https://huggingface.co/Comfy-Org/stable-diffusion-v1-5-archive/blob/main/v1-5-pruned-emaonly-fp16.safetensors
RECOMMENDED WAY TO UPDATE:
To update the ComfyUI code: update\update_comfyui.bat
To update ComfyUI with the python dependencies, note that you should ONLY run this if you have issues with python dependencies.
update\update_comfyui_and_python_dependencies.bat
TO SHARE MODELS BETWEEN COMFYUI AND ANOTHER UI:
In the ComfyUI directory you will find a file: extra_model_paths.yaml.example
Rename this file to: extra_model_paths.yaml and edit it with your favorite text editor.
if you want to enable the fast fp16 accumulation (faster for fp16 models with slightly less quality):
run_nvidia_gpu_fast_fp16_accumulation.bat
To run it in slow CPU mode:
run_cpu.bat
IF YOU GET A RED ERROR IN THE UI MAKE SURE YOU HAVE A MODEL/CHECKPOINT IN: HanzoStudio\models\checkpoints
You can download the stable diffusion 1.5 one from: https://huggingface.co/hanzoai/stable-diffusion-v1-5-archive/blob/main/v1-5-pruned-emaonly-fp16.safetensors
RECOMMENDED WAY TO UPDATE:
To update the Hanzo Studio code: update\update_studio.bat
To update Hanzo Studio with the python dependencies, note that you should ONLY run this if you have issues with python dependencies.
update\update_studio_and_python_dependencies.bat
TO SHARE MODELS BETWEEN HANZO STUDIO AND ANOTHER UI:
In the HanzoStudio directory you will find a file: extra_model_paths.yaml.example
Rename this file to: extra_model_paths.yaml and edit it with your favorite text editor.
echo If you see this and Hanzo Studio did not start try updating your Nvidia Drivers to the latest. If you get a c10.dll error you need to install vc redist that you can find: https://aka.ms/vc14/vc_redist.x64.exe
echo If you see this and Hanzo Studio did not start try updating your Nvidia Drivers to the latest. If you get a c10.dll error you need to install vc redist that you can find: https://aka.ms/vc14/vc_redist.x64.exe
echo If you see this and Hanzo Studio did not start try updating your Nvidia Drivers to the latest. If you get a c10.dll error you need to install vc redist that you can find: https://aka.ms/vc14/vc_redist.x64.exe
tone_instructions:"Only comment on issues introduced by this PR's changes. Do not flag pre-existing problems in moved, re-indented, or reformatted code."
reviews:
profile:"chill"
request_changes_workflow:false
high_level_summary:false
poem:false
review_status:false
review_details:false
commit_status:true
collapse_walkthrough:true
changed_files_summary:false
sequence_diagrams:false
estimate_code_review_effort:false
assess_linked_issues:false
related_issues:false
related_prs:false
suggested_labels:false
auto_apply_labels:false
suggested_reviewers:false
auto_assign_reviewers:false
in_progress_fortune:false
enable_prompt_for_ai_agents:true
path_filters:
- "!studio_api_nodes/apis/**"
- "!**/generated/*.pyi"
- "!.ci/**"
- "!script_examples/**"
- "!**/__pycache__/**"
- "!**/*.ipynb"
- "!**/*.png"
- "!**/*.bat"
path_instructions:
- path:"**"
instructions:|
IMPORTANT: Only comment on issues directly introduced by this PR's code changes.
Do NOT flag pre-existing issues in code that was merely moved, re-indented,
de-indented, or reformatted without logic changes. If code appears in the diff
only due to whitespace or structural reformatting (e.g., removing a `with:` block),
treat it as unchanged. Contributors should not feel obligated to address
pre-existing issues outside the scope of their contribution.
- path:"studio/**"
instructions:|
Core ML/diffusion engine. Focus on:
- Backward compatibility (breaking changes affect all custom nodes)
- Memory management and GPU resource handling
- Performance implications in hot paths
- Thread safety for concurrent execution
- path:"studio_api_nodes/**"
instructions:|
Third-party API integration nodes. Focus on:
- No hardcoded API keys or secrets
- Proper error handling for API failures (timeouts, rate limits, auth errors)
- Correct Pydantic model usage
- Security of user data passed to external APIs
- path:"studio_extras/**"
instructions:|
Community-contributed extra nodes. Focus on:
- Consistency with node patterns (INPUT_TYPES, RETURN_TYPES, FUNCTION, CATEGORY)
- No breaking changes to existing node interfaces
- path:"studio_execution/**"
instructions:|
Execution engine (graph execution, caching, jobs). Focus on:
description:"Something is broken inside of ComfyUI. (Do not use this if you're just having issues and need help, or if the issue relates to a custom node)"
description:"Something is broken inside of Hanzo Studio. (Do not use this if you're just having issues and need help, or if the issue relates to a custom node)"
labels:["Potential Bug"]
body:
- type:markdown
@@ -7,21 +7,23 @@ body:
value:|
Before submitting a **Bug Report**, please ensure the following:
- **1:** You are running the latest version of ComfyUI.
- **2:** You have looked at the existing bug reports and made sure this isn't already reported.
- **1:** You are running the latest version of Hanzo Studio.
- **2:** You have your Hanzo Studio logs and relevant workflow on hand and will post them in this bug report.
- **3:** You confirmed that the bug is not caused by a custom node. You can disable all custom nodes by passing
`--disable-all-custom-nodes` command line argument.
- **4:** This is an actual bug in ComfyUI, not just a support question. A bug is when you can specify exact
`--disable-all-custom-nodes` command line argument. If you have custom node try updating them to the latest version.
- **4:** This is an actual bug in Hanzo Studio, not just a support question. A bug is when you can specify exact
steps to replicate what went wrong and others will be able to repeat your steps and see the same issue happen.
If unsure, ask on the [ComfyUI Matrix Space](https://app.element.io/#/room/%23comfyui_space%3Amatrix.org) or the [Comfy Org Discord](https://discord.gg/comfyorg) first.
## Very Important
Please make sure that you post ALL your Hanzo Studio logs in the bug report. A bug report without logs will likely be ignored.
- type:checkboxes
id:custom-nodes-test
attributes:
label:Custom Node Testing
description:Please confirm you have tried to reproduce the issue with all custom nodes disabled.
options:
- label:I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
- label:I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.hanzo.ai/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
required:false
- type:textarea
attributes:
@@ -38,7 +40,7 @@ body:
- type:textarea
attributes:
label:Steps to Reproduce
description:"Describe how to reproduce the issue. Please be sure to attach a workflow JSON or PNG, ideally one that doesn't require custom nodes to test. If the bug open happens when certain custom nodes are used, most likely that custom node is what has the bug rather than ComfyUI, in which case it should be reported to the node's author."
description:"Describe how to reproduce the issue. Please be sure to attach a workflow JSON or PNG, ideally one that doesn't require custom nodes to test. If the bug open happens when certain custom nodes are used, most likely that custom node is what has the bug rather than Hanzo Studio, in which case it should be reported to the node's author."
description:"You have an idea for something new you would like to see added to ComfyUI's core."
description:"You have an idea for something new you would like to see added to Hanzo Studio's core."
labels:["Feature"]
body:
- type:markdown
@@ -7,11 +7,11 @@ body:
value:|
Before submitting a **Feature Request**, please ensure the following:
**1:** You are running the latest version of ComfyUI.
**1:** You are running the latest version of Hanzo Studio.
**2:** You have looked to make sure there is not already a feature that does what you need, and there is not already a Feature Request listed for the same idea.
**3:** This is something that makes sense to add to ComfyUI Core, and wouldn't make more sense as a custom node.
**3:** This is something that makes sense to add to Hanzo Studio Core, and wouldn't make more sense as a custom node.
If unsure, ask on the [ComfyUI Matrix Space](https://app.element.io/#/room/%23comfyui_space%3Amatrix.org) or the [Comfy Org Discord](https://discord.gg/comfyorg) first.
If unsure, ask on the [Hanzo AI Discord](https://discord.gg/hanzoai) first.
Before submitting a **User Report** issue, please ensure the following:
**1:** You are running the latest version of ComfyUI.
**1:** You are running the latest version of Hanzo Studio.
**2:** You have made an effort to find public answers to your question before asking here. In other words, you googled it first, and scrolled through recent help topics.
If unsure, ask on the [ComfyUI Matrix Space](https://app.element.io/#/room/%23comfyui_space%3Amatrix.org) or the [Comfy Org Discord](https://discord.gg/comfyorg) first.
If unsure, ask on the [Hanzo AI Discord](https://discord.gg/hanzoai) first.
- type:checkboxes
id:custom-nodes-test
attributes:
label:Custom Node Testing
description:Please confirm you have tried to reproduce the issue with all custom nodes disabled.
options:
- label:I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
- label:I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.hanzo.ai/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
body: '(Automated Bot Message) CI Tests are running, you can view the results at https://ci.comfy.org/?branch=${{ github.event.pull_request.number }}%2Fmerge'
body: '(Automated Bot Message) CI Tests are running, you can view the results at https://ci.hanzo.ai/?branch=${{ github.event.pull_request.number }}%2Fmerge'
Welcome, and thank you for your interest in contributing to ComfyUI!
Welcome, and thank you for your interest in contributing to Hanzo Studio!
There are several ways in which you can contribute, beyond writing code. The goal of this document is to provide a high-level overview of how you can get involved.
## Asking Questions
Have a question? Instead of opening an issue, please ask on [Discord](https://comfy.org/discord) or [Matrix](https://app.element.io/#/room/%23comfyui_space%3Amatrix.org) channels. Our team and the community will help you.
Have a question? Instead of opening an issue, please ask on [Discord](https://discord.gg/hanzoai) first. Our team and the community will help you.
## Providing Feedback
@@ -16,24 +16,24 @@ See the `#bug-report`, `#feature-request` and `#feedback` channels on Discord.
## Reporting Issues
Have you identified a reproducible problem in ComfyUI? Do you have a feature request? We want to hear about it! Here's how you can report your issue as effectively as possible.
Have you identified a reproducible problem in Hanzo Studio? Do you have a feature request? We want to hear about it! Here's how you can report your issue as effectively as possible.
### Look For an Existing Issue
Before you create a new issue, please do a search in [open issues](https://github.com/comfyanonymous/ComfyUI/issues) to see if the issue or feature request has already been filed.
Before you create a new issue, please do a search in [open issues](https://github.com/hanzoai/studio/issues) to see if the issue or feature request has already been filed.
If you find your issue already exists, make relevant comments and add your [reaction](https://github.com/blog/2119-add-reactions-to-pull-requests-issues-and-comments). Use a reaction in place of a "+1" comment:
* 👍 - upvote
* 👎 - downvote
* upvote
* downvote
If you cannot find an existing issue that describes your bug or feature, create a new issue. We have an issue template in place to organize new issues.
### Creating Pull Requests
* Please refer to the article on [creating pull requests](https://github.com/comfyanonymous/ComfyUI/wiki/How-to-Contribute-Code) and contributing to this project.
* Please refer to the article on [creating pull requests](https://github.com/hanzoai/studio/wiki/How-to-Contribute-Code) and contributing to this project.
Quantization aims to map a high-precision value x_f to a lower precision format with minimal loss in accuracy. These smaller formats then serve to reduce the models memory footprint and increase throughput by using specialized hardware.
When simply converting a value from FP16 to FP8 using the round-nearest method we might hit two issues:
- The dynamic range of FP16 (-65,504, 65,504) far exceeds FP8 formats like E4M3 (-448, 448) or E5M2 (-57,344, 57,344), potentially resulting in clipped values
- The original values are concentrated in a small range (e.g. -1,1) leaving many FP8-bits "unused"
By using a scaling factor, we aim to map these values into the quantized-dtype range, making use of the full spectrum. One of the easiest approaches, and common, is using per-tensor absolute-maximum scaling.
Given that additional information (scaling factor) is needed to "interpret" the quantized values, we describe those as derived datatypes.
## Quantization in Hanzo Studio
```
QuantizedTensor (torch.Tensor subclass)
↓ __torch_dispatch__
Two-Level Registry (generic + layout handlers)
↓
MixedPrecisionOps + Metadata Detection
```
### Representation
To represent these derived datatypes, Hanzo Studio uses a subclass of torch.Tensor to implements these using the `QuantizedTensor` class found in `studio/quant_ops.py`
A `Layout` class defines how a specific quantization format behaves:
- Required parameters
- Quantize method
- De-Quantize method
```python
fromstudio.quant_opsimportQuantizedLayout
classMyLayout(QuantizedLayout):
@classmethod
defquantize(cls,tensor,**kwargs):
# Convert to quantized format
qdata=...
params={'scale':...,'orig_dtype':tensor.dtype}
returnqdata,params
@staticmethod
defdequantize(qdata,scale,orig_dtype,**kwargs):
returnqdata.to(orig_dtype)*scale
```
To then run operations using these QuantizedTensors we use two registry systems to define supported operations.
The first is a **generic registry** that handles operations common to all quantized formats (e.g., `.to()`, `.clone()`, `.reshape()`).
The second registry is layout-specific and allows to implement fast-paths like nn.Linear.
When `torch.nn.functional.linear()` is called with QuantizedTensor arguments, `__torch_dispatch__` automatically routes to the registered implementation.
For any unsupported operation, QuantizedTensor will fallback to call `dequantize` and dispatch using the high-precision implementation.
### Mixed Precision
The `MixedPrecisionOps` class (lines 542-648 in `studio/ops.py`) enables per-layer quantization decisions, allowing different layers in a model to use different precisions. This is activated when a model config contains a `layer_quant_config` dictionary that specifies which layers should be quantized and how.
**Architecture:**
```python
classMixedPrecisionOps(disable_weight_init):
_layer_quant_config={}# Maps layer names to quantization configs
Not all layers tolerate quantization equally. Sensitive operations like final projections can be kept in higher precision, while compute-heavy matmuls are quantized. This provides most of the performance benefits while maintaining quality.
The system is selected in `pick_operations()` when `model_config.layer_quant_config` is present, making it the highest-priority operation mode.
## Checkpoint Format
Quantized checkpoints are stored as standard safetensors files with quantized weight tensors and associated scaling parameters, plus a `_quantization_metadata` JSON entry describing the quantization scheme.
The quantized checkpoint will contain the same layers as the original checkpoint but:
- The weights are stored as quantized values, sometimes using a different storage datatype. E.g. uint8 container for fp8.
- For each quantized weight a number of additional scaling parameters are stored alongside depending on the recipe.
- We store a metadata.json in the metadata of the final safetensor containing the `_quantization_metadata` describing which layers are quantized and what layout has been used.
### Scaling Parameters details
We define 4 possible scaling parameters that should cover most recipes in the near-future:
- **weight_scale**: quantization scalers for the weights
- **weight_scale_2**: global scalers in the context of double scaling
- **pre_quant_scale**: scalers used for smoothing salient weights
- **input_scale**: quantization scalers for the activations
You can find the defined formats in `studio/quant_ops.py` (QUANT_ALGOS).
### Quantization Metadata
The metadata stored alongside the checkpoint contains:
- **format_version**: String to define a version of the standard
- **layers**: A dictionary mapping layer names to their quantization format. The format string maps to the definitions found in `QUANT_ALGOS`.
Example:
```json
{
"_quantization_metadata":{
"format_version":"1.0",
"layers":{
"model.layers.0.mlp.up_proj":"float8_e4m3fn",
"model.layers.0.mlp.down_proj":"float8_e4m3fn",
"model.layers.1.mlp.up_proj":"float8_e4m3fn"
}
}
}
```
## Creating Quantized Checkpoints
To create compatible checkpoints, use any quantization tool provided the output follows the checkpoint format described above and uses a layout defined in `QUANT_ALGOS`.
### Weight Quantization
Weight quantization is straightforward - compute the scaling factor directly from the weight tensor using the absolute maximum method described earlier. Each layer's weights are quantized independently and stored with their corresponding `weight_scale` parameter.
### Calibration (for Activation Quantization)
Activation quantization (e.g., for FP8 Tensor Core operations) requires `input_scale` parameters that cannot be determined from static weights alone. Since activation values depend on actual inputs, we use **post-training calibration (PTQ)**:
1.**Collect statistics**: Run inference on N representative samples
2.**Track activations**: Record the absolute maximum (`amax`) of inputs to each quantized layer
3.**Compute scales**: Derive `input_scale` from collected statistics
4.**Store in checkpoint**: Save `input_scale` parameters alongside weights
The calibration dataset should be representative of your target use case. For diffusion models, this typically means a diverse set of prompts and generation parameters.
</div>
ComfyUI lets you design and execute advanced stable diffusion pipelines using a graph/nodes/flowchart based interface. Available on Windows, Linux, and macOS.
Hanzo Studio lets you design and execute advanced stable diffusion pipelines using a graph/nodes/flowchart based interface. Available on Windows, Linux, and macOS.
## Upstream & License
Hanzo Studio is a fork of [**ComfyUI**](https://github.com/comfyanonymous/ComfyUI) by comfyanonymous and the ComfyUI contributors — full credit to the upstream project for the engine this builds on. ComfyUI is licensed under the **GNU General Public License v3.0 (GPL-3.0)**, and this fork stays under the same license. All modifications in this fork are likewise GPL-3.0.
See [`LICENSE`](LICENSE) for the full license text and [`NOTICE`](NOTICE) for the fork's attribution and a summary of what it changes (branding, Hanzo Engine integration nodes, and fashion workflow templates).
- Loading full workflows (with seeds) from generated PNG, WebP and FLAC files.
- Saving/Loading workflows as Json files.
- Nodes interface can be used to create complex workflows like one for [Hires fix](https://comfyanonymous.github.io/ComfyUI_examples/2_pass_txt2img/) or much more advanced ones.
- Works fully offline: core will never download anything unless you want to.
- Optional API nodes to use paid models from external providers through the online [Comfy API](https://docs.comfy.org/tutorials/api-nodes/overview).
- Optional API nodes to use paid models from external providers through the online API. Disable with: `--disable-api-nodes`
- [Config file](extra_model_paths.yaml.example) to set the search paths for models.
Workflow examples can be found on the [Examples page](https://comfyanonymous.github.io/ComfyUI_examples/)
## Release Process
ComfyUI follows a weekly release cycle targeting Friday but this regularly changes because of model releases or large changes to the codebase. There are three interconnected repositories:
- Weekly frontend updates are merged into the core repository
- Features are frozen for the upcoming core release
- Development continues for the next release cycle
## Shortcuts
| Keybind | Explanation |
@@ -166,32 +133,15 @@ ComfyUI follows a weekly release cycle targeting Friday but this regularly chang
# Installing
## Windows Portable
There is a portable standalone build for Windows that should work for running on Nvidia GPUs or for running on your CPU only on the [releases page](https://github.com/comfyanonymous/ComfyUI/releases).
### [Direct link to download](https://github.com/comfyanonymous/ComfyUI/releases/latest/download/ComfyUI_windows_portable_nvidia.7z)
Simply download, extract with [7-Zip](https://7-zip.org) and run. Make sure you put your Stable Diffusion checkpoints/models (the huge ckpt/safetensors files) in: ComfyUI\models\checkpoints
If you have trouble extracting it, right click the file -> properties -> unblock
#### How do I share models between another UI and ComfyUI?
See the [Config file](extra_model_paths.yaml.example) to set the search paths for models. In the standalone windows build you can find this file in the ComfyUI directory. Rename this file to extra_model_paths.yaml and edit it with your favorite text editor.
You can install and start ComfyUI using comfy-cli:
```bash
pip install comfy-cli
comfy install
```
## Manual Install (Windows, Linux)
Python 3.13 is very well supported. If you have trouble with some custom node dependencies you can try 3.12
Python 3.14 works but some custom nodes may have issues. The free threaded variant works but some dependencies will enable the GIL so it's not fully supported.
Python 3.13 is very well supported. If you have trouble with some custom node dependencies on 3.13 you can try 3.12
torch 2.4 and above is supported but some features and optimizations might only work on newer versions. We generally recommend using the latest major version of pytorch with the latest cuda version unless it is less than 2 weeks old.
### Instructions:
Git clone this repo.
@@ -200,18 +150,36 @@ Put your SD checkpoints (the huge ckpt/safetensors files) in: models/checkpoints
Put your VAE in: models/vae
### AMD GPUs (Linux only)
### AMD GPUs (Linux)
AMD users can install rocm and pytorch with pip if you don't have it already installed, this is the command to install the stable version:
### AMD GPUs (Experimental: Windows and Linux), RDNA 3, 3.5 and 4 only.
These have less hardware support than the builds above but they work on windows. You also need to install the pytorch version specific to your hardware.
(Option 1) Intel Arc GPU users can install native PyTorch with torch.xpu support using pip. More information can be found [here](https://pytorch.org/docs/main/notes/get_start_xpu.html)
Intel Arc GPU users can install native PyTorch with torch.xpu support using pip. More information can be found [here](https://pytorch.org/docs/main/notes/get_start_xpu.html)
1. To install PyTorch xpu, use the following command:
@@ -221,19 +189,15 @@ This is the command to install the Pytorch xpu nightly which might have some per
@@ -245,30 +209,24 @@ And install it again with the command above.
### Dependencies
Install the dependencies by opening your terminal inside the ComfyUI folder and:
Install the dependencies by opening your terminal inside the Hanzo Studio folder and:
```pip install -r requirements.txt```
After this you should have everything installed and can proceed to running ComfyUI.
After this you should have everything installed and can proceed to running Hanzo Studio.
### Others:
#### Apple Mac silicon
You can install ComfyUI in Apple Mac silicon (M1 or M2) with any recent macOS version.
You can install Hanzo Studio in Apple Mac silicon (M1 or M2) with any recent macOS version.
1. Install pytorch nightly. For instructions, read the [Accelerated PyTorch training on Mac](https://developer.apple.com/metal/pytorch/) Apple Developer guide (make sure to install the latest pytorch nightly).
1. Follow the [ComfyUI manual installation](#manual-install-windows-linux) instructions for Windows and Linux.
1. Install the ComfyUI [dependencies](#dependencies). If you have another Stable Diffusion UI [you might be able to reuse the dependencies](#i-already-have-another-ui-for-stable-diffusion-installed-do-i-really-have-to-install-all-of-these-dependencies).
1. Launch ComfyUI by running `python main.py`
1. Follow the [Hanzo Studio manual installation](#manual-install-windows-linux) instructions for Windows and Linux.
1. Install the Hanzo Studio [dependencies](#dependencies). If you have another Stable Diffusion UI you might be able to reuse the dependencies.
1. Launch Hanzo Studio by running `python main.py`
> **Note**: Remember to add your models, VAE, LoRAs etc. to the corresponding Comfy folders, as discussed in [ComfyUI manual installation](#manual-install-windows-linux).
#### DirectML (AMD Cards on Windows)
This is very badly supported and is not recommended. There are some unofficial builds of pytorch ROCm on windows that exist that will give you a much better experience than this. This readme will be updated once official pytorch ROCm builds for windows come out.
```pip install torch-directml``` Then you can launch ComfyUI with: ```python main.py --directml```
> **Note**: Remember to add your models, VAE, LoRAs etc. to the corresponding folders, as discussed in [Hanzo Studio manual installation](#manual-install-windows-linux).
#### Ascend NPUs
@@ -277,7 +235,7 @@ For models compatible with Ascend Extension for PyTorch (torch_npu). To get star
1. Begin by installing the recommended or newer kernel version for Linux as specified in the Installation page of torch-npu, if necessary.
2. Proceed with the installation of Ascend Basekit, which includes the driver, firmware, and CANN, following the instructions provided for your specific platform.
3. Next, install the necessary packages for torch-npu by adhering to the platform-specific instructions on the [Installation](https://ascend.github.io/docs/sources/pytorch/install.html#pytorch) page.
4. Finally, adhere to the [ComfyUI manual installation](#manual-install-windows-linux) guide for Linux. Once all components are installed, you can run ComfyUI as described earlier.
4. Finally, adhere to the [Hanzo Studio manual installation](#manual-install-windows-linux) guide for Linux. Once all components are installed, you can run Hanzo Studio as described earlier.
#### Cambricon MLUs
@@ -285,14 +243,40 @@ For models compatible with Cambricon Extension for PyTorch (torch_mlu). Here's a
1. Install the Cambricon CNToolkit by adhering to the platform-specific instructions on the [Installation](https://www.cambricon.com/docs/sdk_1.15.0/cntoolkit_3.7.2/cntoolkit_install_3.7.2/index.html)
2. Next, install the PyTorch(torch_mlu) following the instructions on the [Installation](https://www.cambricon.com/docs/sdk_1.15.0/cambricon_pytorch_1.17.0/user_guide_1.9/index.html)
3. Launch ComfyUI by running `python main.py`
3. Launch Hanzo Studio by running `python main.py`
#### Iluvatar Corex
For models compatible with Iluvatar Extension for PyTorch. Here's a step-by-step guide tailored to your platform and installation method:
1. Install the Iluvatar Corex Toolkit by adhering to the platform-specific instructions on the [Installation](https://support.iluvatar.com/#/DocumentCentre?id=1&nameCenter=2&productId=520117912052801536)
2. Launch ComfyUI by running `python main.py`
2. Launch Hanzo Studio by running `python main.py`
## Manager
The **Manager** extension allows you to easily install, update, and manage custom nodes for Hanzo Studio.
### Setup
1. Install the manager dependencies:
```bash
pip install -r manager_requirements.txt
```
2. Enable the manager with the `--enable-manager` flag when running Hanzo Studio:
```bash
python main.py --enable-manager
```
### Command Line Options
| Flag | Description |
|------|-------------|
| `--enable-manager` | Enable the Manager |
| `--enable-manager-legacy-ui` | Use the legacy manager UI instead of the new UI (requires `--enable-manager`) |
| `--disable-manager-ui` | Disable the manager UI and endpoints while keeping background features like security checks and scheduled installation completion (requires `--enable-manager`) |
# Running
@@ -308,7 +292,7 @@ For AMD 7600 and maybe other RDNA3 cards: ```HSA_OVERRIDE_GFX_VERSION=11.0.0 pyt
### AMD ROCm Tips
You can enable experimental memory efficient attention on recent pytorch in ComfyUI on some AMD GPUs using this command, it should already be enabled by default on RDNA3. If this improves speed for you on latest pytorch on your GPU please report it so that I can enable it by default.
You can enable experimental memory efficient attention on recent pytorch on some AMD GPUs using this command, it should already be enabled by default on RDNA3. If this improves speed for you on latest pytorch on your GPU please report it so that we can enable it by default.
@@ -322,9 +306,9 @@ Only parts of the graph that change from each execution to the next will be exec
Dragging a generated png on the webpage or loading one will give you the full workflow including seeds that were used to create it.
You can use () to change emphasis of a word or phrase like: (good code:1.2) or (bad code:0.8). The default emphasis for () is 1.1. To use () characters in your actual prompt escape them like \\( or \\).
You can use () to change emphasis of a word or phrase like: (good code:1.2) or (bad code:0.8). The default emphasis for () is 1.1. To use () characters in your actual prompt escape them like \( or \).
You can use {day|night}, for wildcard/dynamic prompts. With this syntax "{wild|card|test}" will be randomly replaced by either "wild", "card" or "test" by the frontend every time you queue the prompt. To use {} characters in your actual prompt escape them like: \\{ or \\}.
You can use {day|night}, for wildcard/dynamic prompts. With this syntax "{wild|card|test}" will be randomly replaced by either "wild", "card" or "test" by the frontend every time you queue the prompt. To use {} characters in your actual prompt escape them like: \{ or \}.
Dynamic prompts also support C-style comments, like `// comment` or `/* comment */`.
@@ -337,7 +321,7 @@ To use a textual inversion concepts/embeddings in a text prompt put them in the
Use ```--preview-method auto``` to enable previews.
The default installation includes a fast latent preview method that's low-resolution. To enable higher-quality previews with [TAESD](https://github.com/madebyollin/taesd), download the [taesd_decoder.pth, taesdxl_decoder.pth, taesd3_decoder.pth and taef1_decoder.pth](https://github.com/madebyollin/taesd/) and place them in the `models/vae_approx` folder. Once they're installed, restart ComfyUI and launch it with `--preview-method taesd` to enable high-quality previews.
The default installation includes a fast latent preview method that's low-resolution. To enable higher-quality previews with [TAESD](https://github.com/madebyollin/taesd), download the [taesd_decoder.pth, taesdxl_decoder.pth, taesd3_decoder.pth and taef1_decoder.pth](https://github.com/madebyollin/taesd/) and place them in the `models/vae_approx` folder. Once they're installed, restart Hanzo Studio and launch it with `--preview-method taesd` to enable high-quality previews.
## How to use TLS/SSL?
Generate a self-signed certificate (not appropriate for shared/production use) and key by running the command: `openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 3650 -nodes -subj "/C=XX/ST=StateName/L=CityName/O=CompanyName/OU=CompanySectionName/CN=CommonNameOrHostname"`
@@ -349,55 +333,6 @@ Use `--tls-keyfile key.pem --tls-certfile cert.pem` to enable TLS/SSL, the app w
## Support and dev channel
[Discord](https://comfy.org/discord): Try the #help or #feedback channels.
[Discord](https://discord.gg/hanzoai): Try the #help or #feedback channels.
[Matrix space: #comfyui_space:matrix.org](https://app.element.io/#/room/%23comfyui_space%3Amatrix.org) (it's like discord but open source).
See also: [https://www.comfy.org/](https://www.comfy.org/)
## Frontend Development
As of August 15, 2024, we have transitioned to a new frontend, which is now hosted in a separate repository: [ComfyUI Frontend](https://github.com/Comfy-Org/ComfyUI_frontend). This repository now hosts the compiled JS (from TS/Vue) under the `web/` directory.
### Reporting Issues and Requesting Features
For any bugs, issues, or feature requests related to the frontend, please use the [ComfyUI Frontend repository](https://github.com/Comfy-Org/ComfyUI_frontend). This will help us manage and address frontend-specific concerns more efficiently.
### Using the Latest Frontend
The new frontend is now the default for ComfyUI. However, please note:
1. The frontend in the main ComfyUI repository is updated fortnightly.
2. Daily releases are available in the separate frontend repository.
To use the most up-to-date frontend version:
1. For the latest daily release, launch ComfyUI with this command line argument:
This approach allows you to easily switch between the stable fortnightly release and the cutting-edge daily updates, or even specific versions for testing purposes.
### Accessing the Legacy Frontend
If you need to use the legacy frontend for any reason, you can access it using the following command line argument:
This will use a snapshot of the legacy frontend preserved in the [ComfyUI Legacy Frontend repository](https://github.com/Comfy-Org/ComfyUI_legacy_frontend).
# QA
### Which GPU should I buy for this?
[See this page for some recommendations](https://github.com/comfyanonymous/ComfyUI/wiki/Which-GPU-should-I-buy-for-ComfyUI)
See also: [https://www.hanzo.ai/](https://www.hanzo.ai/)
All routes under the `/internal` path are designated for **internal use by ComfyUI only**. These routes are not intended for use by external applications may change at any time without notice.
All routes under the `/internal` path are designated for **internal use by Hanzo Studio only**. These routes are not intended for use by external applications may change at any time without notice.
File diff suppressed because one or more lines are too long
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Hanzo AI
z
