Compare commits

...
117 Commits
Author SHA1 Message Date
hanzo-dev 55075bcbbe release: v2.4.25 — layout: snap-to-grid, no corner chevrons, finer grid, text-size control 2026-07-18 23:30:53 -07:00
hanzo-dev ec595bc567 feat(world): layout polish — remove resize glyphs, finer grid, text-size control (held)
Owner feedback on the free layout, batched with the snap-to-grid fix:

- No visible corner/edge resize glyphs (#2): the dots + corner brackets are hidden
  (display:none on the handle ::after); the handles stay invisible hit areas so
  dragging from any edge/corner still resizes — the resize cursor is the affordance.
- Finer grid (#3): MIN_CELL_SIZE 120->80 + dock "Widget size" slider min 120->80,
  step 20->10, so the grid — and the snap, which reads cellSize — goes finer.
- Text-size / UI-scale control (#4, accessibility): a dock "Text size" slider drives
  a --ui-scale var that zooms the readable panel CONTENT only (never the panels-grid,
  so drag/snap coordinates are untouched); persisted in localStorage (utils/ui-scale.ts),
  applied before first paint. Per-user SQLite dashboard-sync picks up the key.
- e2e: glyphs-hidden-but-still-resizes, finer-80px-floor, text-size-scales+persists.
  Full layout suite 21/21 green; build clean.

Not released — batched with the snap fix; hanzo assigns version/sequence.
2026-07-18 23:29:35 -07:00
hanzo-dev f86fb162f8 fix(world): free-mode drags + resizes snap to a logical grid (held)
User feedback on the free layout: "the snap is not logical." Free mode placed
panels at arbitrary px, which felt loose to drag. Now free-mode drag snaps {x,y}
to grid-line origins and resize snaps {w,h} to whole-cell multiples, so panels
land on consistent, aligned tracks — while keeping the independence that made free
mode good (resize/move one panel, siblings never move; opposite edge stays pinned).

- ONE logical grid, DRY: grid-config.freeSnap() = { cell: getCellSize() (the
  slider-driven --panel-col-min), row: --panel-row (16), gap }. Free and grid share
  one grid definition — no second magic number.
- panel-drag: snapLine (position -> nearest grid line) + snapCells (size -> whole
  cells, >= floor); threaded through moveFree + applyFreeResize. Hold Alt for fine,
  un-snapped placement. Grid mode untouched.
- e2e: two new proofs (drag -> two panels align to the same track; resize -> whole-
  cell width + opposite edge pinned); the arbitrary-px tests now hold Alt. 18/18 green.

Not released — version/sequence assigned by hanzo.
2026-07-18 23:29:35 -07:00
hanzo-dev 3bf9cc994c fix(world): Enso benchmark shows only our measured results — confident, sells
Dropped the external "reported" comparison column + the self-deprecating footnote
from the public benchmark. The panel now shows ONLY Enso's own live-measured accuracy
and per-system cost, with a confident, true caveat: every number is measured against
real APIs through the Hanzo gateway, Enso reaches frontier-competitive accuracy at a
fraction of the blended cost by routing each task to the right model, and it reports
what it spends. No external names, no incoherent framing.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 23:06:54 -07:00
hanzo-dev 3d1f91ecd8 release: v2.4.24 — Enso benchmark branding
Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 22:52:08 -07:00
hanzo-dev da762aee5b fix(world): brand benchmarks as Enso — scrub the old codename
The Enso benchmark panel + summary data carried the pre-release codename in labels,
JSON keys, Go structs, and the TS consumer. Renamed consistently to Enso / Enso-Ultra
across the data, the /v1/world backend handler, and the frontend so the public
benchmark shows only the Enso brand. No numbers changed; identifiers stay in sync
(measured Enso vs Enso's originally-reported figures).

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 22:51:28 -07:00
hanzo-dev 032c33d55a release: v2.4.23 — geo-aware model-improvement consent + honest AI Compute gate
Sweeps the unreleased range: geo-aware consent toggle in the account menu (reflects
the backend's effective default — opt-out non-EU, opt-in EU/UK/EEA) with honest copy;
and the AI Compute panel showing a plain "Sign in" gate instead of a scary
"unreachable" when signed out. (Signup disclosure + privacy/terms shipped separately
in hanzo.id + hanzo.ai.)

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 21:09:52 -07:00
hanzo-dev 607edfdbcd feat(world/consent): honest geo-aware microcopy for the model-improvement toggle
The "Help improve Hanzo AI" account toggle already reads the effective consent
from ai (getTrainingContribution) and paints it once loaded. Make the microcopy
and comments honest for the GEO-AWARE model — opt-in/default-OFF in the EU, UK &
EEA; opt-out/default-ON, disclosed at signup, elsewhere — instead of claiming a
universal "off by default". No logic change: the toggle stays a verbatim mirror
of ai's answer, so a non-EU org's ON effective default now reads as an accurate
label rather than a contradicted one, and the service/loader comments no longer
claim "never opted in => OFF".

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 21:05:55 -07:00
hanzo-dev 2c0445a308 release: v2.4.22 — per-org Analytics + Insights cards (world cloud view) 2026-07-18 20:49:06 -07:00
zandGitHub b6c423e358 Merge pull request #18 from hanzoai/feat/org-analytics-insights-cards
feat(world/cloud): per-org Analytics + Insights cards
2026-07-18 20:48:14 -07:00
hanzo-dev 07607bf57a feat(world/cloud): per-org Analytics + Insights cards
Two org-scoped event-platform cards for the Cloud view, sourced from the
live native analytics warehouse on api.hanzo.ai (verified live):

- Analytics (org-analytics): /v1/analytics/overview + timeseries + top —
  per-org requests/visitors/tokens/spend, requests trend, top models.
- Insights (org-insights): /v1/insights/events — active users, top events,
  activity trend derived client-side from the real event stream.

Org pinned server-side from the validated bearer owner claim (principal.Org),
reusing org-scope.scopedHeaders() like getMyBilling — no new fetch mechanism.
Reuses the existing Panel + cloud-format primitives + cloud-* CSS. Honest
loading/signed-out/empty/error states; never fabricated numbers.
2026-07-18 20:20:15 -07:00
hanzo-dev 50a7448678 fix(world): AI Compute shows honest "sign in" gate, not a scary "unreachable"
Signed-out, the live compute pulse is account-scoped, but the panel rendered
"Compute telemetry unavailable — compute plane unreachable" which reads as a backend
outage. Match the Fleet / My Usage pattern: signed-out shows a plain "Sign in to see
the live inference plane…"; the unreachable error copy is reserved for a genuine
failure while signed in.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 20:13:20 -07:00
hanzo-dev 870ec49f3d release: v2.4.21 — mobile search icon + consent toggle (unique tag; 2.4.20 was claimed concurrently) 2026-07-18 17:45:22 -07:00
b7a90b2774 fix(world/header): give the mobile search button a magnifier icon (#16)
At <=680px both the Ctrl/Cmd-K hint (already hidden <=900px) and the
'Search' label were hidden, leaving the search button an empty bordered
box. Add an always-present magnifier icon that reveals once the K hint
collapses, so the button is never empty at any width. (2.4.20)

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-18 17:42:55 -07:00
hanzo-dev 493fb3d4e7 feat(world/consent): opt-in "Help improve Hanzo AI" toggle (default OFF) (2.4.20)
Add a model-improvement consent toggle to the account menu, wired to ai's
OrgSettings.TrainingContribution (the source of truth the automated judge + trainer
read). Honest, opt-in, off by default, no dark patterns.

- src/components/AccountMenu.ts: a "Data & privacy" switch in the signed-in dropdown.
  Painted OFF until the async read lands (never a false "on"); optimistic flip that
  reverts if the server refuses. Exact microcopy: rating keeps only a numeric score,
  never prompts/outputs, runs confidential/anonymous, off anytime.
- src/services/training-contribution.ts: get/set via the app's org-scoped authed
  fetch (orgHeaders). Signed-out / error / unset all read false — privacy-safe.
- internal/world/handlers_training.go: same-origin proxy /v1/world/training-contribution
  → ai get-/update-training-contribution, forwarding the caller's bearer + X-Org-Id.
  World holds NO copy; ai self-scopes to the caller's own org and is the sole authority.

Tests: proxy gates signed-out (401, no upstream call), forwards bearer + org, and
unwraps the casibase envelope to a bare {enabled} on both GET and POST.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 17:41:23 -07:00
hanzo-dev 09954907cf fix(world): default to free layout — panels move/resize independently, no reflow (2.4.19)
The dashboard defaulted to grid snap, so dragging or resizing one panel
reflowed all the others and resize jumped between coarse column/row spans.
CSS grid can't move a panel without disturbing siblings, so make FREE the
default layout: every panel owns its own {x,y,w,h}. The current grid
arrangement is frozen as the starting geometry (grid-config.applyDefaultLayout,
run once after panels+map lay out), so the flip is invisible; Grid stays a
dropdown choice and grid-config records an explicit pick so the default never
overrides it.

- Independent positioning (#1): free mode — moving/resizing one panel leaves
  every sibling exactly where it was (proven: siblings drift 0px).
- Granular resize (#2): pixel-exact w/h from any edge/corner, not span jumps.
- Narrower panels (#3): free floors 160->96 (map 240->140); grid track floor
  140->120. A panel drags to 96px, well under the old min.
- Viewport-flexible (#4): map + live-news resize freely, small to large.

Hidden panels no longer persist 0x0 rects; a panel shown while free seeds a
tidy default slot. Tests: +2 harness proofs (no-sibling-reflow, sub-160 width);
repaired two stale span-2 assertions to the fine-grid data-span reality.
2026-07-18 17:26:25 -07:00
hanzo-dev 08df02b535 release: v2.4.18 — dotted-land cybermap globe + fullscreen-video idle fix
Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 17:05:35 -07:00
hanzo-dev 138d96a1b8 fix(world/live-news): never idle-kill a fullscreen video (exited fullscreen + stopped)
Passively listening to a live stream in fullscreen fires no mousedown/keydown/scroll
(fullscreen video captures input), so the 5-min input-idle timer fired and pauseForIdle
→ destroyPlayer tore down the iframe — which force-exits fullscreen AND stops playback
(the reported "listening fullscreen and it exited + stopped"). Fix: treat fullscreen as
the strongest "actively watching" signal — never idle-kill when our player is the
fullscreen element — and count fullscreenchange as activity so entering fullscreen
resets the idle timer.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 17:05:34 -07:00
hanzo-dev cf2092380b feat(world): dotted-land "cybermap" globe — restore the dot map
Restores the dotted-land basemap (land shown as a lattice of glowing dots over a
black-ocean sphere — the abstract cybermap surface): new land-dots.ts (getLandDots()
samples /data/countries.geojson into a cached lat/long lattice, land points only),
GlobeNative renders it as a ScatterplotLayer on the 3D globe, variant.ts defaults
cloud + AI to the 'dot' style, and App opens cloud/AI on the 3D globe (the live-traffic
centerpiece). Recovered from a stashed WIP; DeckGLMap kept at current main (preserves
the shipped layer panel + control dropdowns), so this ports cleanly with no regression.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 17:00:43 -07:00
hanzo-dev d8175e0a37 feat(cloud): default the Cloud view to the immersive globe — bring back the cool 3D dot map
The flagship Cloud view was defaulting to the grid dashboard, demoting the
full-viewport 3D dot map to a small corner tile. Restore it: when the visitor
hasn't explicitly picked a layout, the Cloud variant opens on the immersive
globe (map fills the viewport, panels float over it). An explicit grid choice
and the ?layout= URL param still win; other variants unchanged.
2026-07-18 16:58:16 -07:00
hanzo-dev 97fed1fee2 release: v2.4.16 — analyst grounded in live cloud metrics (answers spend/fleet/enso)
Ships 3d892d96: the AI analyst now folds a live Hanzo-cloud snapshot into its context
(traffic, platform 24h, Enso router cost-saved/reward/engine-share, flywheel ledger,
billing), so it can answer questions about the dashboard's real numbers — on top of
its existing 26-command control surface.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 16:27:49 -07:00
hanzo-dev 3d892d9633 feat(world/analyst): ground the AI in live cloud metrics so it can answer anything
The analyst's context fed panel names + visibility but NOT the live values, so it
couldn't answer "what's my spend / how many DO nodes / is Enso saving money". On the
cloud board it now also folds a real-data snapshot — live traffic (rps/req), platform
24h (requests/tokens/nodes/GPUs/regions/uptime), the Enso router (cost saved, reward
rate, engine share, models routed), the flywheel ledger, and the caller's billing
(spend/balance/events). Best-effort + parallel with the news feeds; real data only,
mirroring the panels. Pairs with the 26-command control surface — the analyst can now
both control the dashboard AND answer questions about what it shows.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 16:27:04 -07:00
hanzo-dev 8c0e928789 release: world 2.4.15 — bake VITE_MAPBOX_TOKEN so Satellite/Terrain load
2.4.14 tagged before the build_secrets wiring landed, so it builds without the
Mapbox token. 2.4.15 = same tree + the KMS-sourced VITE_MAPBOX_TOKEN build-arg
(via hanzoai/ci v1.0.5), so satellite-streets/outdoors render in 2D and 3D
instead of the keyless dark fallback.
2026-07-18 12:34:27 -07:00
hanzo-dev 0b52de0236 release: world 2.4.14 — Satellite/Terrain via KMS Mapbox token
Root cause of the dark Sat/Terrain: no VITE_MAPBOX_TOKEN was baked at build, so
the token-gated Mapbox satellite-streets-v12 / outdoors-v12 styles silently fell
back to the keyless CartoDB dark basemap in both 2D and 3D. The publishable pk
token now lives in KMS (hanzo/deploy/VITE_MAPBOX_TOKEN, env=prod) and is sourced
by hanzoai/ci (v1.0.5) as a build_secret --build-arg into the Vite SPA.

Rides the pending main fixes (analyst reasoning recovery, AI Compute rate,
map-control dropdowns).
2026-07-18 12:33:11 -07:00
hanzo-dev 3fbbbbebf5 release: v2.4.14 — dropdown map controls, 2×2 panels, AI Compute fix, analyst recovery
Sweeps the unreleased range: map controls collapsed to 3 dropdowns + symmetric 2×2
tiles + dropped regions tile (3fc42b1f); AI Compute rate un-flattened (interval enum
bug) + honest sub-cent usage rows (4c41abbb); analyst chat recovers reasoning-model
answers instead of "empty response" (a17d63c1).

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 12:31:13 -07:00
hanzo-dev a17d63c10d fix(world/analyst): recover reasoning-model answers so chat stops returning "empty response"
The analyst rendered "The analyst couldn't answer — empty response" whenever the
served model streamed its answer on the reasoning channel with an EMPTY content
channel — gpt-oss / harmony, deepseek-r1 and kin. The ONE completion path
(chatMessagesModel + chatMessagesModelStream) read only the content channel, so a
reasoning model's output was dropped and surfaced as a blank.

- Read the reasoning channel (message.reasoning_content / reasoning, and the SSE
  delta.reasoning_content / reasoning) as a fallback when content is empty, and
  forward reasoning deltas live so the replyExtractor shows them on the thinking line.
  Content is always preferred; non-reasoning models (best/zen5) are unaffected.
- Replace the opaque "empty response" with an honest, model-named error
  ("the <model> model returned an empty answer"): never misreads a normal
  completion's response id as an error code, never trips the 402 top-up CTA.

Tests: reasoning-channel recovery (buffered + SSE) and the honest empty-answer error.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 12:30:15 -07:00
hanzo-dev 4c41abbb29 fix(world): AI Compute rate no longer flat (interval enum) + honest sub-cent usage rows
AI Compute (and cloud-pulse) rps/tps were dead flat: the usage series reports its
bucket width as a WORD enum ("hour"/"day"), but intervalSeconds parsed it with
time.ParseDuration — which fails on "hour" (needs "1h") — so usageRate always fell
back to total24h/86400, a constant. Resolve the enum (minute/hour/day/week) before
ParseDuration; the rate now reflects the freshest hour and moves. Regression tests
for intervalSeconds + usageRate.

My Usage & Bill: a real sub-cent inference charge (< $0.005) rounds to 0 integer
cents and rendered a bare "$0.00" that reads as "not billed". Show "< $0.01" for a
real row that rounded to zero — honest about a tiny real charge (the ledger only
carries integer cents on the wire).

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 12:17:03 -07:00
hanzo-dev 3fc42b1f1f fix(world): collapse map controls to dropdowns; symmetric 2×2 tiles; drop regions
Map controls:
- Projection (2D/3D), basemap (Dark/Sat/Terrain) and time-range each collapse into ONE
  compact dropdown (trigger shows the active value + caret; options in a popover). The
  top dock is now a single tidy row of three instead of two wrapping rows. Global
  .deckgl-dd treatment (one way, every map host). e2e specs open the dropdown before
  clicking an option via a shared clickMapControl helper.

Panel symmetry (real data, never padded):
- Live Traffic: added a real 4th tile (distinct countries with traffic) → 2×2.
- Model Improvement: split scored-events into its own tile → 2×2 (reward rate,
  requests scored, active days, retrains).
- Cloud Overview / Global Platform: dropped the 'regions' tile (fleet carries region
  detail) per CTO.
- New .cloud-stat-grid-2 (exactly two columns) for the symmetric 2×2 layout.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 12:13:53 -07:00
hanzo-devandz 5a65e739a9 fix(world): globe overlays hug the sphere — real occlusion, terrain seams, live dots (2.4.13)
The Hanzo-Cloud globe 'spazzed': data layers floated above the sphere, the
far hemisphere showed through, count badges hovered over the planet, terrain
striped at tile seams, and the live request-geo dots froze on the 3D view.

- GlobeNative seats every data layer against the depth-writing ocean sphere
  (depth-TEST, no write) so points/icons/text/paths/arcs occlude on the GPU.
- occludeFarSide now faces the globe's OWN live camera (mapbox is parked/frozen
  behind the native GlobeView) and culls TextLayer badges too, so a back-side
  count badge disappears instead of floating over the limb. Camera is fed each
  rebuild + on drag >2°; escape path (?globe=mapbox) keeps the live mapbox center.
- Draped ESRI terrain/satellite tiles stop writing depth (coplanar tiles were
  z-fighting at seams → striping); the ocean sphere owns the depth buffer.
- Cloud /v1/world/cloud/* poll gates on tab visibility, not renderPaused, so the
  realtime dots keep updating on the native 3D globe (the default view).
- Request-origin → serving-region arcs ON by default in Cloud (real feed, empty
  when no traffic — never demo).
- e2e/globe-render.spec.ts asserts occlusion (dot + badge), 2D/3D parity,
  terrain single-context, and live-dot updates.
2026-07-18 11:57:14 -07:00
hanzo-dev d4820277af release: v2.4.12 — map controls top-row + Layers, org logo, fleet RAM, globe cables
Ships the unreleased 2.4.11..HEAD range (all UI): globe cables hug the sphere, map
controls in one top row + top-left Layers button (footer/scrollbar removed), one Fleet
panel with system RAM, BYO once, and the org logo in the account switcher.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 11:53:11 -07:00
hanzo-dev d40c8560bb fix(world): render the org logo in the account switcher (was dropped)
The account chip showed the USER avatar next to the org name and never the org's
own image — IAM's get-organizations returns each org's Casdoor `logo`, but listOrgs
parsed only name + displayName and dropped it, so the hanzo org logo never rendered.

Thread `logo` end-to-end (OrgInfo → OrgOption → scope) and render an ORG avatar in
the switcher trigger: the active org's logo when set, else org initials (rounded-
square so it reads as an org, not a person). The dropdown's user row keeps the user
avatar. Falls back to the user avatar only when there's no org scope (signed out).

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 11:34:46 -07:00
hanzo-dev 7d737f2a53 fix(world): map controls to one top row + top-left Layers; BYO once; hide page scrollbar
Map controls (CTO revision of the bottom dock):
- 2D/3D toggle + Dark/Sat/Terrain switcher + time-range pills now sit in ONE row
  across the map's TOP, side by side (was a bottom-border dock). Zoom stays top-right,
  legend bottom.
- New top-left "Layers" button on the map toggles the floating layer panel (opens
  just below it). Removed the redundant Layers button from the page footer dock —
  layers now live on the map itself.
- Hid the .main-content right-edge scrollbar (wheel/trackpad scroll unchanged) — it
  read as a stray "sidebar scroller" over the map. Same treatment header/switchers use.

Fleet:
- BYO nodes rendered TWICE — once as a "byo" provider group and again as the BYO GPU
  workers list (same dbc/evo/spark). Managed providers (DO/GCP/AWS) now render as
  provider groups; BYO renders once, as the richer workers list.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 10:50:39 -07:00
hanzo-dev 35e07a65ae feat(world/fleet): render system memory in the machine spec line
The fleet machine rows showed GPUs · VRAM · vCPU · OS but never system RAM —
there was no memory field on the path at all. Thread `mem` end-to-end on the
world side: decode it from /v1/machines, carry it on fleetMachineRow, expose it
on FleetMachineRow, and render it in machineSpec (GPUs · VRAM · vCPU · RAM · OS)
only when present. The cloud service now populates `mem` (DO size-slug + upstream
memSize); world shows whatever is real and omits it otherwise — never fabricated.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 09:28:14 -07:00
hanzo-dev 8d3d20eb79 fix(world): globe cables hug sphere · docked map controls · one Fleet panel
Globe (3D):
- Cables + pipelines (PathLayer) drew straight chords across the sphere — deck.gl
  has no greatCircle for paths (only ArcLayer). Pre-densify each static path along
  its great circle into <=4-deg hops so the many short segments hug the surface.
  Flat map keeps the raw sparse vertices.

Map controls:
- The 2D/3D toggle, basemap switcher and time-range pills were scattered across the
  wide empty map header (top-center, mid-left, top-right). Dock them into ONE tidy
  strip along the map's bottom border via the existing controlsHost mechanism (App
  passed none). The dock mounts on the map container — a SIBLING of the deck wrapper
  — because 3D parks that wrapper (visibility:hidden) and renders via GlobeNative; a
  dock inside it would vanish on the globe. Reuses the .world-dock inlining CSS.

Fleet:
- "Fleet & GPUs" (FleetPanel, all users) and "Fleet & clusters" (CloudFleetPanel,
  admin) were duplicate platform-fleet renders. Collapse to ONE "Fleet & GPUs":
  widen FleetPanel, port the utilNote line, derive the provider-scope label from the
  providers actually reporting (not a hardcoded four), delete CloudFleetPanel.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-18 09:13:40 -07:00
hanzo-dev 0cda941c4c fix(world): builder needs git + gh_token for private indirect dep hanzoai/csqlite
golang:1.26-alpine ships no git; go resolves the private indirect
github.com/hanzoai/csqlite 'direct', so 'go mod download' died with
git-not-found. Add git, set GOPRIVATE, mount the gh_token secret ci already
passes. Unbreaks world main builds (broken since the Go-base-1.26 drop-in).
2026-07-17 23:29:05 -07:00
hanzo-dev 385a3a30bd world 2.4.11 — release (verify semver CR pin via hanzoai/ci@v1) 2026-07-17 23:18:40 -07:00
hanzo-dev 16d0f000ea fix(world): Docker Go base 1.26 for hanzoai/sqlite drop-in
go.mod now requires go >= 1.26.4 (github.com/hanzoai/sqlite). Bump the
builder base golang:1.25-alpine -> golang:1.26-alpine so the prod image
keeps building. The binary stays CGO-free (CGO_ENABLED=0), so hanzoai/sqlite
selects its pure-Go modernc backend — FTS5 built in, works. CGO test path
(no FTS5 without -tags sqlite_fts5 + libsqlcipher) is not what world ships.
2026-07-17 22:03:54 -07:00
hanzo-dev f52b5b2366 Merge feat/sqlite-hanzo-driver 2026-07-17 21:59:53 -07:00
hanzo-dev 3dd4823e3e fix(world): wide panels overflowed the fine grid (AI Compute rendered under things) + DRY o11y usage
.panel-wide kept grid-row: span 2 (the old 200px-grid assumption) after the move
to 16px rows — so all 7 wide panels (ai-compute, cloud-overview, live-news,
traffic-globe, enso-flywheel, model-improvement, hanzo-status) reserved only ~36px
of track while being 350px tall, overflowing their cell and painting over the
panels below. Fixed to span 18 (≈356px) with the shared 48px min-height floor.

Also: extract fetchLLMUsage (DRY) shared by cloud-pulse + ai-pulse, and fall back
to LLM observability in ai-pulse so AI Compute shows real usage (not zeros) for an
authorized operator.
2026-07-17 21:52:34 -07:00
hanzo-dev c8aae3366c feat(world): real per-model usage + volume from LLM observability (admin)
Model Usage showed 'warming up' / nothing and usage totals were 0 for a signed-in
admin whose super-admin usage ledger (?org=all) is denied. Fixed: the admin path
now folds REAL platform usage from LLM observability (/v1/admin/o11y) with the
caller's own bearer — measured requests, tokens, and top models by REAL name (not
the opaque router arms) — clearing volumeModeled. Falls back honestly to empty if
the caller isn't authorized for platform observability upstream.
2026-07-17 21:45:26 -07:00
hanzo-dev 997630fcb5 feat(world): admin-only Enso benchmark suite (head-to-head vs SOTA + Fugu)
Private, competitive data (names competitor models + Fugu), so it is gated
server-side, never bundled into the public SPA:

- backend: GET /v1/world/enso-benchmarks reuses the exact requireAdmin gate
  (401 no-token / 403 non-admin, IAM-introspection). Reshapes the SAME embedded
  enso-bench snapshot the flywheel reads (one embed, two views) into the measured
  head-to-head + verify-then-select ablation + agentic SWE-Bench Pro pilot +
  Fugu-reported reference + server-authored honest caveats. No upstream forward.
- snapshot: refresh ensodata/summary.json to the full measured set (LCB+GPQA+HLE,
  6 arms), ultra_ablation, and the committed agentic result folded in.
- frontend: EnsoBenchmarkPanel (admin-only; adminOnlyState when not admin) +
  services/enso-benchmarks.ts (bearer, 403->null). Registered in the admin-only
  Cloud panels. Theme-safe tables, scroll-in-box, no body h-scroll.
- privacy: no competitor names, arm ids, numbers, or our routing methodology
  names are baked into the public bundle — all arrive only via the gated endpoint.

Honest framing: enso MATCHES the best single SOTA arm at a fraction of the cost
(LCB 91.4% @ $5.24 vs opus $39.43; GPQA ties top); does NOT beat every SOTA;
HLE is preflight-only; we are below Fugu's REPORTED numbers. Decisive wins are
cost-efficiency, the ablation (better AND cheaper), and agentic step-routing.

Tests: gate matrix (anon 401 / non-admin 403 / admin 200 + no-leak) + real-number
payload assertions green; existing enso-training + red admin-gate suites green.
2026-07-17 21:41:55 -07:00
hanzo-dev 5d1c6fff5d fix(world): smooth fine 16px panel resize — no coarse jump, no min-height trap
Vertical resize snapped in coarse 200px tiers (120/200/400/600/800) and a hard
200px min-height blocked shrinking — the 'jump too much / can't fit content'
complaint. Now the grid is a fine 16px row unit (@hanzo/gui space-4): per-panel
resize snaps in ~20px steps, uncapped, and min-height drops to 48px so 200px is
just the DEFAULT for a new panel, not a floor. Tier presets (size menu + map) are
redefined on the same grid so they still work; the map resizes smoothly too.
Horizontal already fills width in 160px column steps.
2026-07-17 21:08:08 -07:00
hanzo-dev 84995c9ac1 fix(world): Model Usage never shows opaque arm-N — real model names or 'warming up'
The public router-stats by_model is relabeled to opaque arm-N ids on platform
scope (privacy), so it is NOT real model names. cloud-pulse wrongly surfaced it
as the model mix, so Model Usage showed 'arm-8, arm-7, arm-6'. Now the public
fallback carries real request rate + throughput only; the model mix comes solely
from the measured usage ledger (admin path) — otherwise Models stays empty and
the panel honestly shows 'warming up'.
2026-07-17 19:40:28 -07:00
hanzo-dev 4a269d65d4 feat: migrate SQLite driver to hanzoai/sqlite drop-in 2026-07-17 12:25:19 -07:00
hanzo-dev 768a0c6bf8 fix(world): cloud-pulse shows ALL real regions, not just the 8-city catalog
The region breakdown dropped any machine whose region wasn't one of the 8
catalog cities (real DO regions like tor1, or GCP/AWS regions), so a fleet in
uncatalogued regions rendered regions:0. Now every real region is counted —
catalog coords/name when known, else the raw region string as its own region.
2026-07-17 12:21:56 -07:00
hanzo-dev 06417f4243 feat(world): fleet drill-down — real DO nodes by region + BYO GPU specs; drop last demo fallbacks
Fleet & GPUs now shows every real machine grouped by provider/region (with GPU
model, VRAM, vCPU, OS) plus BYO GPU workers with specs (Spark/Evo/dbc: hostname,
GPU, VRAM, capabilities) — one shared renderer (cloud-fleet-view) used by both
FleetPanel (flagship, admin) and CloudFleetPanel (deep). Backend adds vCPU from
visor MachineView.

Removes the last fabricated data: byo-gpu and traffic map layers are now
real-or-empty (no demoGPUs / demoTraffic), so a signed-out visitor sees honest
empty globe layers, never fake clusters/arcs.
2026-07-17 12:18:59 -07:00
hanzo-dev 7e41959b01 feat(world): real user + signup metrics from IAM global-users
Wires the one signups/growth gap that has a real source: IAM /v1/iam/global-users
(all users, global-admin only), read with the signed-in admin's OWN bearer and
folded into AGGREGATES ONLY (never PII): total users, signups 24h/7d, active-now
(isOnline), and a 14-day daily-signup series (from each user's createdTime).
Attached to the cloud-pulse admin path only (p.users, omitempty — the public
teaser never carries it). CloudOverviewPanel renders users / +N signups·24h /
active-now tiles + a new-users sparkline, live at the 30s poll.

Remaining gaps stay honestly empty — they have NO upstream source (verified against
the visor/o11y OpenAPI specs): GPU util/mem/temp (visor GpuView: "Live telemetry is
omitted"), disk (no field), historical retention cohorts (no login-events store).
Those need real instrumentation upstream, never faked.

go build/vet/test green (TestCloudPulseAdminBearer extended); tsc + vite clean.
App.ts untouched.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-17 11:56:57 -07:00
hanzo-dev c7d3d11579 feat(world): real fleet + GPU globe + AI compute — whole Cloud tab admin-aware
Completes the "make it all real" pass on the cloud flagship:
- FleetPanel shows the REAL platform fleet (visor /v1/world/cloud/fleet) for an
  admin — machines + GPUs across DO/GCP/AWS/BYO, rolled up by region, 30s refresh.
  Removed the demo/sample-region fallback — never fabricates regions now.
- Globe BYO-GPU layer real for admins (handleCloudBYOGPU, caller bearer, no-store)
  — real GPU clusters instead of the 8 demoGPUs dots. Anonymous keeps the
  clearly-flagged demo sample (no public GPU inventory source).
- AiComputePanel: ai-pulse admin-aware; EventSource can't send a bearer, so the
  panel uses an authenticated poll when signed in — z@hanzo.ai gets the full
  measured compute pulse (tokens/s, req/s, spend, top models, fleet), own bearer.

Every admin/authed response is Vary: Authorization + private,no-store. Honest gaps
stay labeled (GPU util/mem/temp, disk, signups/cohorts need real collection
upstream). go build/vet/test green (TestBYOGPUAdminReal, TestAIPulseAdminPoll);
tsc + vite clean. App.ts untouched.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-17 10:59:07 -07:00
hanzo-dev 99bd0d71ac feat(world): admin-aware cloud-pulse — signed-in admin sees full real all-org data
cloud-pulse never looked at who was logged in (server service-token only), and the
real admin panels were gated to orgs {admin, built-in} — so z@hanzo.ai (org hanzo)
got 403 and saw only the public aggregate. Now:

- cloud-pulse is admin-aware (handlers_cloud.go): a signed-in admin's request is
  served the FULL measured all-org aggregate fetched with THEIR OWN bearer
  (get-cloud-usages?org=all + visor), Cache-Control: private,no-store + Vary:
  Authorization so it never mixes with the public edge cache. No server-side
  super-admin token needed — the admin's own login drives it. The ai gateway
  independently re-checks ?org=all, so a non-global-admin degrades to real public
  data, never fake (gateway is authoritative).
- Operator org recognized as admin (cloud_identity.go): {admin, built-in} +
  operator org (default hanzo, override WORLD_ADMIN_ORGS) — unlocks the existing
  real admin panels (services/llm/fleet/analytics) for z@hanzo.ai. Anonymous
  short-circuits, so the public path stays cheap.
- Frontend forwards the bearer (cloud-pulse.ts) and the client admin gate
  (iam.ts isAdmin/cachedIsAdmin) matches the server, so App.ts's existing
  isAdmin() reveal works — App.ts untouched.

go build/vet/test green (added TestCloudPulseAdminBearer); tsc + vite clean.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-17 10:24:34 -07:00
hanzo-dev 3c95187c8e fix(world): kill fabricated cloud metrics — real sources or honest-empty
The flagship cloud dashboard showed modeled numbers under a "live" badge:
demoPulse() sine-wave rps/24h/tokens + sparklines, demoModels() mix, demoRegions()
per-city rates, and a hardcoded 99.98% uptime. Replaced with an honest source
ladder (producePulse in handlers_cloud.go):
  - RequestsPerSec  <- /v1/traffic/globe totals.rps_1m (the real ~0.62)
  - Requests24h + sparkline <- /v1/router/stats?scope=platform
  - Models mix <- ledger byModel (super-admin) else router-stats by_model
  - UptimePct <- status.hanzo.ai Gatus up/total (0 -> tile dropped, never a const)
  - Regions <- real visor fleet
Tokens24h/TokenSeries have no non-ledger source -> honest 0/empty, volumeModeled
stays true (UI renders "—"). demo:true now means only "nothing measured yet".

Frontend (CloudOverview/LiveActivity/ModelUsage) keys the live badge on
!demo && !volumeModeled — no modeled number wears a live badge; LiveActivity
regions show real visor node counts. TrafficGlobe filters garbage country codes
to ISO-3166. Dead admin-panel gate fixed (mountAdminCloudPanels !== 'saas' ->
'cloud', so Cloud Fleet + LLM panels mount for admins).

Full all-org volume + per-model tokens need a super-admin HANZO_CLOUD_PULSE_TOKEN
in KMS; until then honestly blank. go build/vet/test green; tsc + vite clean.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-17 01:27:05 -07:00
hanzo-dev 32da053f28 feat(world): tab order Cloud·AI·Crypto·Finance·Tech·World, Title-case labels, SVG icons
- Header tabs reordered to Cloud, AI, Crypto, Finance, Tech, World (World last).
- Tab + status labels Title-case, not ALL CAPS (World/Crypto/Finance/Tech/Live;
  AI kept as the acronym) across en + nl.
- Export (⬇) and historical-playback ( ⏮ ◀ ▶ ⏭) emoji replaced with clean
  16px stroke SVGs matching the header's search/theme icons.

tsc clean.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-17 00:49:28 -07:00
06ff53615b feat(cloud): Cloud flagship view — H-merge + CLOUD tab, in-map controls, flicker fix, flywheel panels (#12)
* feat(cloud): Cloud-view polish — compact H + CLOUD tab, in-map controls, flywheel history, accurate legend

- Header: merge the redundant H-icon + "Hanzo" wordmark into one compact (16px)
  canonical H mark (brand/home toggle); rename the flagship tab "Hanzo" → "Cloud"
  (switcher: [Cloud | World | AI | Crypto | Finance | Tech]). Canonical variant id
  renamed hanzo → cloud, with hanzo + saas kept as runtime aliases (links + stored
  prefs keep working).
- Controls: dock the 2D/3D toggle, basemap switcher and time-range pills IN the map
  (controlsHost = map) alongside the zoom, layer toggles and legend that already
  overlay it, instead of floating them in the bottom toolbar; time pills move to the
  bottom-left so they clear the top-left layer toggles; the empty dock slot collapses.
- Flywheel history: a same-origin GET /v1/world/cloud/router-history proxy (→ ai's
  public /v1/router/history?scope=platform; honest empty, never demo) + a Model
  Improvement panel — reward-rate line chart with retrain-version markers, a ticking
  cumulative cost-saved hero, and an adoption sparkline. Charts start empty and GROW
  with real data; nothing is seeded.
- Legend: add a Cloud data-class legend (request origin·volume, validator node, GPU
  fleet, cloud region, datacenter) so it matches what the globe plots rather than the
  geopolitical default.

tsc + vite build clean; world Go build + gofmt clean.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

* fix(cloud): kill the 3D-globe flicker — settle the globe, guard the cables layer

Root cause (per a deep read-only diagnosis): the 3D globe is a custom GlobeNative
deck.gl renderer whose idle auto-rotate (~2°/s) keeps thin coplanar on-sphere vector
features (borders/coastlines/markers) perpetually sub-pixel-moving, re-rasterized
every frame — steady-state limb shimmer on real GPUs. Fixes:

- GlobeNative created with autoRotate:false — the globe SETTLES, so the deck render
  loop idles (verified: camera longitude does not drift and renders drop from ~40/s
  to a couple of data-sync ticks). Drag-to-rotate still works; re-enable the idle spin
  only with a coplanar depth/polygonOffset fix so it no longer z-fights the sphere.
- Guard createCablesLayer: filter UNDERSEA_CABLES to paths with >=2 finite [lon,lat]
  vertices, so one malformed cable can no longer make the PathLayer assert at init and
  silently drop the whole layer (the "[GlobeNative] cables-layer assertion").
- Cables OFF in the Cloud view (not a Hanzo Cloud data class; absent from the Cloud
  legend) so the globe stays to the plotted classes and the legend matches exactly.

Playwright: a new cloud-mode test asserts the 3D globe settles (lonDelta<0.001,
renders<12), the legend shows the Cloud classes (not the geopolitical default), and no
cables assertion fires. tsc + the full cloud-mode suite green.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

* feat(cloud): CLOUD variant rename, compact H, in-map controls, flywheel panel + proxy

Completes the Cloud-view content (the prior checkpoint captured only the file renames):

- variant.ts: canonical variant renamed hanzo → cloud, with hanzo + saas as runtime
  aliases; host-default cloud on world.hanzo.ai. isHanzoBrandHost gates the brand.
- App.ts: header merges the redundant H-icon + wordmark into one compact 16px
  canonical H (brand/home toggle); first tab renamed Hanzo → ☁️ Cloud; createPanels
  wires the cloud panel set incl. the Model Improvement panel; controlsHost = the map
  so the 2D/3D toggle + basemap + time pills dock IN the map cluster.
- ModelImprovementPanel + router-history service + /v1/world/cloud/router-history
  proxy: the flywheel-over-time story (reward-rate line + retrain markers, ticking
  cost-saved hero, adoption sparkline) — honest empty until the ledger fills.
- variants/cloud.ts (mirror) + saas.ts alias; flywheel + in-map-controls CSS.

tsc + vite build clean.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

* feat(cloud): regional activity — traffic-origin tooltip shows task mix + volume

Hover a traffic origin on the globe (or flat map) → its request volume + the router
task mix (code/reasoning/chat/vision — what customers there are DOING), falling back
to the coarse service class until the per-geo byTask recording lands. Uses the new
TrafficGlobePoint.byTask from /v1/traffic/globe. Works in both 2D and 3D (GlobeNative
delegates tooltips to this handler).

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

---------

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-16 20:13:22 -07:00
3c50a167c7 feat(hanzo): Hanzo mode — H-logo switcher + live-traffic globe (flagship default) (#11)
* feat(hanzo): Hanzo mode — H-logo switcher + live-traffic globe (flagship default)

world.hanzo.ai now leads with the first-class `hanzo` variant. The H logo toggles
Hanzo mode, revealing the [Hanzo | World | AI | Crypto | Finance | Tech] switcher
(Hanzo first + default on the world.hanzo.ai host). The Hanzo entry + toggle appear
ONLY on hanzo brand hosts (white-label rule); off-brand/OSS hosts keep the plain
home-link logo + an always-on switcher. The former `saas` variant is folded in and
kept as a runtime alias (saas → hanzo, normalized in variant.ts).

Globe: a new native deck.gl `traffic` layer plots WHERE requests hit api.hanzo.ai
from — pulsing points sized/heated by count at country/US-state centroids, breathing
on the shared cloud-pulse clock. Fed by a same-origin proxy /v1/world/cloud/traffic-
globe → the ai backend's public /v1/traffic/globe (aggregates only, no IPs). The
existing traffic-arc layer now draws from the SAME native source (one source of
truth). Both degrade to an HONEST empty state — never demo — until the ai release
lands.

Side panels (the globe's companions): Live Traffic (native throughput totals + top
origins), Enso Live Training + Enso Flywheel (router stats), AI Compute, Model Usage
(model mix), Fleet, Cloud Overview, Hanzo Status (uptime), My Usage.

Tests: Go proxy passthrough + honest-empty state; tsc + vite build; a Playwright spec
driving the H toggle → switcher reveal (6 tabs, Hanzo first) + the globe layer toggle,
plus the TrafficGlobePanel live + empty states.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

* refactor(hanzo): lead the globe with the honest native points layer, arcs off by default

The trafficArcs feed carries a demo fallback (demoTraffic), so enabling it by
default on the flagship could paint fabricated flows when native data is absent. The
new native `traffic` points layer is ALWAYS honest (empty until real traffic lands,
never demo), so the hanzo default now leads with it and ships trafficArcs OFF — still
one toggle away for the animated flow lines (which are native-sourced once live).

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

---------

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-16 15:50:14 -07:00
hanzo-dev b62ad51e7c feat(panels): AI Compute + Enso Flywheel on the DEFAULT world
The platform's own live compute/usage pulse and router-training flywheel
were gated to ?variant=ai — invisible on plain world.hanzo.ai. They are
front-page telemetry: registered in FULL_PANELS and instantiated for the
default (full) variant alongside ai.

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-16 14:36:55 -07:00
cbaae9e800 feat(analyst): content-free AI reward signals via @hanzo/ai SDK (#10)
Wire thumbs up/down + regenerate reward signals into the analyst chat,
keyed on the gateway response id, through a same-origin /v1/feedback BFF
proxy and the just-published @hanzo/ai@0.2.1 sendFeedback.

Backend:
- Thread the gateway response id out of every completion: chatResponse.ID
  (non-stream) + the per-chunk SSE frame id (first non-empty wins);
  runAnalystLoop returns it; the analyst done/JSON payloads carry "id".
- New same-origin BFF /v1/feedback (bare, matching the gateway path so the
  SDK's baseUrl:'' reaches it). Forwards the caller's IAM bearer +
  org/project selectors to a.base+"/feedback". Content-free is enforced
  server-side: the body is WHITELISTED to exactly {request_id, signal,
  rating?}; rating rides only with signal=="rating" and only for 1..3; a
  dismiss or any non-rating signal never forwards a rating. No bearer,
  bad signal, empty id, or malformed JSON => quiet 204 no-op (never 401).

Frontend:
- analyst-transport: surface id on AnalystResponse (JSON + SSE done).
- AnalystChat: store id + originating prompt on the assistant message; add
  thumbs-up/down (mutually-exclusive pressed state) + regenerate to the
  meta row, matching the copy button. emitFeedback helper calls the SDK
  same-origin with a lazy IAM bearer; opt-out via VITE_HANZO_FEEDBACK.
- icons: thumbs-up, thumbs-down, refresh-cw (lucide).

Content-free is a HARD invariant in BOTH layers: the SDK FeedbackInput type
union and the Go BFF whitelist. No prompt/response/text ever transits.

Tests: handlers_feedback_test.go (whitelist, rating rules, no-op paths,
preflight) + analyst id propagation. go test ./internal/world/... green;
tsc --noEmit + vite build green.

Rollout: merge to main ships via the world operator Service CR
(universe: infra/k8s/operator/crs/world.yaml) through hanzoai/ci.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-16 11:26:24 -07:00
zandGitHub 39288fe709 Merge pull request #9 from hanzoai/fix/world-router-stats-unwrap
fix(world 2.4.8): unwrap ai envelope in router-stats proxy
2026-07-16 09:17:16 -07:00
hanzo-dev f5a426c923 fix(world 2.4.8): unwrap ai casibase envelope in router-stats proxy
/v1/world/cloud/router-stats raw-passed ai's {status,data:{…}} envelope, but the
Enso Live Training panel (and this proxy's own unavailable fallback) expect the
BARE RouterStats — so live data would never render (the panel sat on
"connecting…" because stats.window was undefined). Unwrap `data` on success;
a missing/null data or non-ok status degrades to the honest unavailable payload
(never 5xx). Test now feeds the enveloped upstream and asserts the unwrap.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-16 09:16:46 -07:00
zandGitHub 0bdf531004 Merge pull request #8 from hanzoai/feat/enso-live-training
world 2.4.4: Enso Live Training panel (saas variant)
2026-07-16 09:08:35 -07:00
hanzo-dev 59c9eb52eb world 2.4.7: Enso Live Training panel (saas variant)
An "Enso Live Training" panel for the Hanzo Cloud (saas) variant of
world.hanzo.ai — live routing throughput, learned-engine agreement, per-arm
routing share (opaque arm-N), cumulative cost-saved, and the last-retrain gate
verdict — fed by the PUBLIC /v1/router/stats?scope=platform aggregate.

Data via the same-origin Go proxy /v1/world/cloud/router-stats (the established
/v1/world/cloud/* convention): hard-pins scope=platform (client can't request a
vendor-labeled scope), clamps hours to [1,168], degrades to 200 {unavailable}
on an unreachable upstream (never 5xx, never fabricated). API base configurable
via HANZO_API_BASE (default https://api.hanzo.ai).

Enso stays private: the public scope already relabels arms to opaque arm-N; the
panel labels them "Enso arm N" and an e2e test asserts no vendor token ever
reaches the DOM.

Reconciled with the parallel Enso Flywheel panel that landed on main (ai
variant, /v1/world/enso-training): distinct variant + audience (this is the
Cloud self-metrics economics view; the flywheel is the public eval-loop view).
The colliding services/enso-training.ts was split — theirs kept, mine moved to
services/router-stats.ts — so both panels coexist with no shared symbol. VERSION
bumped past the flywheel's 2.4.6 to 2.4.7.

Claude-Session: https://claude.ai/code/session_015Z1iLf7QBrq1LhignJrzDw
2026-07-16 09:06:46 -07:00
hanzo-dev dee04c5c89 feat(world/ai): Enso Flywheel panel + /v1/world/enso-training
The router self-improvement loop made visible for the AI variant.
/v1/world/enso-training folds three honest sources: the routing-decision ledger
tail (export-routing-ledger ?since=, super-admin JSONL) → decision count,
engine-vs-heuristic mix, a routing-confidence histogram, task/model distributions;
the reward tail (export-routing-rewards) → rewarded count + mean reward; and the
latest enso-bench eval scores. Eval scores come from a committed snapshot of
enso-bench results/summary.json (go:embed), overridable live via ENSO_BENCH_URL —
so the panel is useful even signed-out; state (live/partial/demo) says which live
sources resolved. The payload is event-typed (Events[]) so retrain/deploy
milestones slot into the same timeline.

Frontend: enso-training.ts + EnsoFlywheelPanel registered for the ai variant,
reusing the existing cloud stat-tile / model-row / share-bar idioms.

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-16 00:33:23 -07:00
hanzo-dev 7cd8035964 feat(world/ai): AI Compute panel + /v1/world/ai-pulse SSE
Live Hanzo inference-plane telemetry for the AI variant. /v1/world/ai-pulse
reuses the analyst SSE plumbing + the shared cloud-usage seam: it polls the
platform usage ledger (get-cloud-usages allOrgs → tokens/s, req/s, spend, top
models) + visor (gpu/machine/region counts) on an interval and pushes typed
usage/fleet/status frames to EventSource clients, coalesced through cache.go so N
streams cause ~one upstream sweep per 10s. A plain GET returns one JSON snapshot —
the panel's poll fallback. No token ⇒ honest state:"unavailable".

Frontend: ai-pulse.ts (SSE stream + poll fallback, cloned from cloud-pulse.ts),
AiComputePanel registered for the ai variant (panels.ts AI_PANELS + variants/ai.ts
+ App.ts). Reuses the existing cloud-panel stat-tile / model-row idioms.

Shared usage helpers (usageRate/topModelsFromUsage) decomplected into
cloud_service.go so cloud-pulse and ai-pulse derive rate/top-models one way.

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-16 00:27:00 -07:00
hanzo-dev 4ef1b36e45 feat(world/cloud): cloud-pulse folds real platform volume (get-cloud-usages allOrgs)
/v1/world/cloud-pulse now reads MEASURED 24h request/token volume + top models
from the ClickHouse-backed usage ledger (get-cloud-usages ?org=all, super-admin)
on top of the live model/node/GPU/region counts, dropping demo:true AND
volumeModeled:true. Without a token — or with a non-admin token that can't reach
the allOrgs ledger — it stays honestly demo/modeled; platform numbers are never
faked silently, and the service bearer never reaches the browser.

One credential, one accessor: serviceToken()/serviceAuth()/apiHost() are now the
single service-plane seam (cloud-pulse + cloud-map share it) reading the existing
KMS-injected HANZO_CLOUD_PULSE_TOKEN. New cloud_service.go holds the shared
get-cloud-usages decode + fetcher, reused by the forthcoming ai-pulse.

Claude-Session: https://claude.ai/code/session_018PmFAHZvbBSTsuWyebwMra
2026-07-16 00:18:49 -07:00
hanzo-dev 7bbfe0f5a7 world 2.4.3: analyst top-up CTA on out-of-credits + 'Ask anything. Update your world.' placeholder
The analyst returned an opaque 'empty response' when a signed-in user's AI
credits were exhausted. The ONE AI gateway (hanzo/ai routers/filter_balance.go)
signals this as HTTP 402 {code: insufficient_balance}; the gateway can also
answer 2xx with an error envelope carrying no choices. World now detects EXACTLY
that one contract (no keyword heuristics, no invented codes — world stays a thin
client of api.hanzo.ai) and renders a wallet CTA: 'You're out of AI credits →
Add credits (console.hanzo.ai/billing) / View usage' instead of a dead bubble.
Adds failure logging on the analyst path (was silent). Placeholder set to the
canonical 'Ask anything. Update your world.'
2026-07-15 12:40:48 -07:00
hanzo-dev 426f918286 fix(world/analyst): shrink chat placeholder so it stops wrapping to two lines
The ~52-char placeholder wrapped at 14px inside the ~360px composer,
shifting the box when it did. The placeholder now renders at 12.5px; the
unitless 1.5 line-height stays pegged to the input's 14px row, so a
smaller placeholder never moves the row vertically — typed text stays
14px, the composer stays one line.
2026-07-15 01:57:32 -07:00
hanzo-dev 322248b98c chore(world): 2.4.2 — streaming analyst + model menu
Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 21:54:39 -07:00
hanzo-dev 27e52c0ea2 feat(world/ai): the analyst answers stream live (SSE)
/v1/world/analyst with Accept: text/event-stream now emits round /
think / delta / tool / done events while the agentic loop runs; the
done event carries exactly the JSON path's payload, so deltas are
cosmetic and the final render is unchanged. Server: chatMessagesModelStream
reads upstream OpenAI SSE; replyExtractor incrementally decodes the
strict-JSON envelope's "reply" value (chunk-safe escapes, byte-at-a-time
tested) and routes a reasoning model's pre-envelope prose to think
events. Tool rounds stream silently; tools surface as live chips.
Client: streaming bubble with breathing caret, dim italic thinking that
real answer text supersedes; non-SSE responses (old backend, sign-in
gate, agent path) fall back to the JSON contract in the same call.
Proven live: 61 think + 41 delta frames then done, gpt-oss-120b.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 21:49:28 -07:00
hanzo-dev 5b3f257a78 feat(world/ai): grouped model menu with family marks — no Zen ring on gpt-oss
The composer pill (fixed Zen logo + native <select>) becomes a proper
listbox popover: rows carry per-family marks (Zen ring ONLY on zen*,
sparkles for the Best router, G/L/C monograms for gpt/llama/claude,
bot for agents), group headers, active check. Assistant avatars and the
thinking row now wear the SERVING model's mark. One mapping in
src/utils/model-marks.ts, mirrored server-side by modelGroup() so the
roster groups honestly (Auto/Zen/GPT/Llama/Claude). Menu is
body-mounted + fixed so panel overflow never clips it. npm run dev
gains a /v1 proxy (prod by default, VITE_DEV_API_PROXY override).

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 17:55:17 -07:00
hanzo-dev 9ff7367f47 Merge feat/china-macro: China macro snapshot + release calendar
Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 17:08:00 -07:00
hanzo-dev b2c86b60f5 feat(world/econ): China macro snapshot + release calendar
/v1/world/china-macro — OECD CPI+CLI (2 sequential, non-retryable), BIS
policy rate (kv-seeded; not_configured until a feed exists), FRED
DEXCHUS, HKMA CNY context, NBS release-calendar scrape + PBoC LPR
candidates (2026 CN business calendar; 2027 entry required before Jan).
launchReady gates on all four required categories;
contentObservationDate anchors to the OLDEST required observation.
EconomicPanel CN tab renders staleness-honest tiles + upcoming releases;
keys mirrored across all 13 locales. Ported from upstream worldmonitor.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 17:07:52 -07:00
hanzo-dev ca6ab46c0a fix(world/tech): CISA feed rides our same-origin /v1 rss-proxy
The one feed still pinned to upstream's rss.worldmonitor.app/api host;
cisa.gov is already on the server allowlist.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 17:00:42 -07:00
hanzo-dev 9e56b6768f chore(world): 2.4.1 changelog; PWA guardrail asserts real precache intent
The stale guard demanded an html-free precache glob, contradicting the
committed vite.config (offline.html precached, index/settings excluded
via globIgnores). Assert the actual invariant instead.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 16:57:31 -07:00
hanzo-dev df125ac864 feat(world/natural): Western Pacific cyclone attribution
Cross-agency TC canonicalization ported from upstream worldmonitor:
GDACS WP-box observations + HKO warning signals (new same-origin
/v1/world/hko-warnings passthrough; browser CORS) dedup by alias
(<=750km/<=18h) or unnamed proximity (<=90km/<=3h), preserving each
agency's own wind + averaging period. Popup renders per-agency wind
rows and canonical-match confidence. JMA/JTWC upstream stubs skipped
(preflight-blocked upstream, fetch nothing).

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 16:57:31 -07:00
hanzo-dev 8f024d9227 perf(world/go): round sparkline closes to 7 significant digits
Yahoo closes come back float32-widened (17.209999084472656); Go re-emits
the noise verbatim, ~40-50% of sparkline payload bytes. One sparkline()
emit path (né tailN) rounds via strconv 'g'/7; scalars stay exact.
Ported from upstream worldmonitor seed-utils.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 16:57:18 -07:00
hanzo-dev f4d5827263 perf(world/ui): paint news before ML, bound analyst snapshot fetches
News panels gated their first DOM write on enrichWithVelocityML — on a
cold browser that means a 65MB sentiment-model download before any
headline appears (15-25s after the feeds answered in <1s). Paint with
keyword velocity immediately, upgrade sentiment in place, guarded by
renderRequestId. Analyst context fetches now cap at 2.5s so a cold
endpoint can't delay the chat send.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 16:57:18 -07:00
hanzo-dev 8511a46d38 perf(world/go): SWR cache with single-flight + GDELT warm keys
cachedJSON/passthrough now serve stale instantly and refresh in the
background (one coalesced flight per key; cold misses single-flighted).
gdelt-doc/gdelt-geo stalled every request ~10s on each 5-minute TTL
lapse; the analyst grounding + protests-layer keys are additionally
kept warm every ~4min, GDELT-paced.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 16:57:18 -07:00
hanzo-dev 014080d3e8 fix(world/ai): 'best' default model, honest upstream errors, roster lead
zen5 went dark when the inference plane's claim catalog shifted — an
unclaimed id falls through to a proxy that rejects every credential, so
every AI endpoint died with an opaque 402/401. 'best' is the gateway's
own routing alias (owned_by hanzo) and always resolves; non-2xx bodies
now surface their code (insufficient_balance, spend_cap_exceeded, …) in
the chat instead of a bare status.

Claude-Session: https://claude.ai/code/session_01SpMZ69ur3tjAXCiwaa7Wv2
2026-07-14 16:57:02 -07:00
hanzo-dev bd5aec5e12 fix(world/globe): arcs hug the sphere (greatCircle) + compact model selector
The 3D globe launched arcs far off the silhouette: deck.gl ArcLayer arches each
arc by ~1x the straight-line CHORD, so long cross-continent arcs (displacement
flows, traffic pulses) ballooned into space — the 'off-globe' glitch in the
screenshots. On the globe (mode==='3d') set greatCircle:true + getHeight:0 so
arcs follow the great circle flat on the surface; the flat map keeps its arch.

Also: the analyst model selector was a wide bar (max-width 160px showing full
model ids) — capped to 104px + ellipsis so it reads as a compact chip.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-13 08:20:16 -07:00
hanzo-dev 7f58503cf6 Merge feat/analyst-full-control: layout/monitor/queue/language commands 2026-07-12 11:31:56 -07:00
hanzo-dev baac10392f feat(world/ai): full app control from chat — layout mode, monitors, queue, language
The analyst could show/hide/move panels and drive the map, but NOT change the
layout mode, add a topic to watch, or drive the Watch Queue — so 'reconfigure
the layout and all widgets from the chat' was not actually true.

New commands (each drives the SAME public API the UI does; no second path):
- set_layout_mode: grid / free / immersive
- set_immersive_background: map | video
- set_language
- add_monitor / remove_monitor — 'watch for X'; goes through the one monitor
  path, so an AI-added monitor persists + matches server-side like a typed one
- queue_next / queue_prev — drive the Watch Queue

AppState snapshot now exposes layoutMode, immersiveBg, language, the user's
monitors, and queue depth — the model cannot drive what it cannot see.

Fixes a real WatchQueue restore bug the e2e caught: a persisted blob whose
currentId pointed at a non-'watching' item silently lost watch-tracking
(next() only finishes a 'watching' item). load() now restores the invariant.

e2e/analyst-full-control.spec.ts: AI really flips to immersive, really adds a
monitor that lands in storage, really advances the queue — 3/3. Full suite 56.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-12 11:31:56 -07:00
hanzo-dev 881c464e08 Merge feat/go-native-monitors: server-side monitor list + lake-wide matching 2026-07-12 09:23:02 -07:00
hanzo-dev b03f317986 feat(world/go): Go-native monitors — server-side list + lake-wide matching
Monitors lived entirely in the browser: keywords in localStorage, matching done
client-side over whatever headlines that tab had loaded. So a monitor only ever
saw one client's slice of the corpus, and it ceased to exist when you closed the
tab or switched device.

- /v1/world/monitors (GET/PUT): the list is persisted PER IDENTITY, reusing the
  existing per-identity store (namespaced 'monitors') — no new mechanism, no new
  table. Bounded + normalised at the boundary (sanitizeMonitors).
- /v1/world/monitors/matches: matching runs against the LAKE — every item the
  backend ingested — via the existing FTS search. FTS tokenising can over-match,
  so each hit is re-checked against the same word-boundary rule the browser used
  ('ai' must never trip on 'train').
- store: Lake.Flush() — drains the write-behind buffer synchronously. Run() is
  still the steady-state consumer; Flush is for callers that need the write to be
  VISIBLE (deterministic tests, and a shutdown that wants its buffer on disk).
- services/monitors.ts: the ONE path to where monitors live. Signed in → server
  (and localStorage as the offline mirror); signed out → localStorage, unchanged.
  Local monitors are ADOPTED on first sign-in rather than silently dropped.
- MonitorPanel.renderServerMatches renders lake matches with the same markup (and
  the data-ctx-* attrs, so Summarize-with-AI works on them too).

Tests run against the real SQLite lake (temp WORLD_DATA_DIR), not a mock:
per-identity round-trip + isolation (user-2 cannot see user-1's monitors),
matching against the lake, and the word-boundary guarantee. 4/4 pass.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-12 09:23:02 -07:00
hanzo-dev 0ad40998e1 Merge feat/go-native-enrichment: server-side threat + geo enrichment 2026-07-12 09:11:19 -07:00
hanzo-dev a452df5282 feat(world/go): move news enrichment into the Go backend (proven parity)
The browser was classifying threat level and inferring geo for every headline,
on every client, on every render — the data and the logic both lived in the
frontend. The Go backend is the product; the browser is a view.

- enrichdata/{threat-keywords,geo-hubs}.json: the tables are DATA now, one
  source of truth (extracted by EVALUATING the TS, not transcribed).
- internal/world/enrich.go: Go classifier + geo inference. Two parity traps
  handled deliberately — JS object key order IS match priority (tables stay
  ordered slices, not maps), and JS sort is stable (sort.SliceStable).
- feeds-batch now returns threat + resolved geo (hub, lat/lon, name) per item;
  parseFeedItems stays a pure XML parser, enrichFeedItems does the meaning.
- rss.ts consumes the server's enrichment; the browser no longer geo-locates
  anything (the import is gone). Single feeds now batch too, so the per-feed
  browser path survives only for feeds that aren't proxy-shaped.

PROOF, not a claim: enrich_test.go replays a 494-case golden corpus generated
from the real TypeScript (every keyword × both variants, every geo-hub keyword,
every exclusion, plus adversarial word-boundary cases) — 494×2 classifications
and all geo inferences are byte-identical to the code being replaced.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-12 09:11:19 -07:00
hanzo-dev 959138ac28 Merge feat/ui-polish: summarize context action, chrome cleanup, auth-aware CTA 2026-07-11 23:57:29 -07:00
hanzo-dev 329a741409 test(world): refresh e2e screenshots from the ui-polish run 2026-07-11 23:57:29 -07:00
hanzo-dev a8485d11af feat(world/ui): Summarize-with-AI on right-click; drop header rule, drag pill; hide Try-Hanzo when signed in
- Summarize with AI: any item already annotated data-ctx-headline (news, custom
  feeds) gets it as the FIRST context-menu entry. It routes into the ONE analyst
  dock via a capability port (registerSummarizePort), mirroring the existing
  map port — the menu never reaches into the AI. New AnalystChat.ask() /
  AiAnalystDock.askInDock() drive the same send path as typing; no second AI.
- Header: drop the border-bottom. The header shares the app background, so the
  rule just cut the page in half.
- Map: drop the visible drag pill (::after). The grip strip is still the drag
  target (cursor:grab) — the map stays chromeless, drag is unchanged.
- Try Hanzo is an ACQUISITION CTA: it now disappears once identity resolves as
  signed-in, driven by one new 'hanzo:auth' event emitted from the single place
  auth resolves (AccountMenu.refresh).
  FIXES A REAL BUG the e2e caught: .try-hanzo sets display:inline-flex, which
  beats [hidden]'s UA rule — so hiding it would have silently done nothing.

e2e/ui-polish.spec.ts covers all four against real DOM/CSS: 4/4 pass.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-11 23:51:55 -07:00
hanzo-dev abfbd61d1d feat(world/queue): keyless YouTube search — no API key required
queue_video was dead in prod: no YOUTUBE_API_KEY is configured, so Data-API
search returned empty. Scrape the results page instead — the same keyless
pattern handleYouTubeLive already uses (browser UA + consent cookies, since
datacenter IPs get an interstitial). The Data API stays as the fast path when
a key exists; the scraper degrades to an honest empty list, never a fake hit.

Verified live: 'Milken Institute Jensen Huang' returns the real talks
(HT8-KPAjpiA = Global Conference 2025) as the top results.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-11 23:23:25 -07:00
hanzo-dev cb67e2bbec chore(world): drop the accidentally-committed cmd/world binary; gitignore it 2026-07-11 23:22:00 -07:00
hanzo-dev 44ad7190cc Merge feat/watch-queue: persistent Watch Queue + AI queue_video + youtube search 2026-07-11 23:21:51 -07:00
hanzo-dev bc11f1111e feat(world/queue): persistent Watch Queue — AI-queued video, skip/back, tracked watched
Content the AI surfaced was ephemeral: a video it found (e.g. the Milken
Institute Jensen Huang talk) was lost on reload. Now everything consumable
lands in ONE persistent queue.

- WatchQueuePanel: stage (video/image/story) + list; prev / finish-and-next /
  fullscreen; click a row to play, x to remove. Owns no state — a pure view
  over watchQueue, so panel and immersive views can never disagree.
- queue_video app-command: the analyst resolves free text ('milken jensen
  2025') to a real video and queues it — reachable over MCP like every other
  command.
- /v1/world/youtube/search: Data-API search, cached, honest empty result when
  no key is configured (never fakes hits).
- Queue persists to localStorage: survives reload, tracks watched/finished.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-11 23:21:51 -07:00
hanzo-dev c75b442523 feat(world/queue): WatchQueue core — one source of truth for consumable media
Unifies non-live news clips, AI-surfaced media, and story cards into ONE
ordered queue with tracked per-item state (queued/watching/watched), so
content can be consumed reliably instead of flashing by. Pure logic, no DOM;
persisted to localStorage so the queue + watched history survive reload.
Idempotent enqueue (dedup by id) + bounded size (prune oldest watched).

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-11 11:10:22 -07:00
hanzo-dev 6bf39c2ebd feat(world/video): fullscreen/enlarge button on the live-news player
Adds a fullscreen control next to mute/play that promotes the video player
container to true fullscreen (and toggles back). CSS makes the iframe fill
the fullscreen root edge-to-edge. Answers 'make the video bigger / go full
screen so I can enjoy streaming it'.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-11 11:02:28 -07:00
hanzo-dev 2351a1e5bb fix(world/video): stop idle timer from killing a video you're watching
Both live-video panels ran a 5-min input-idle timer (mousedown/keydown/
scroll/touchstart on document). Passively watching a video fires none of
those, so after ~5 min the timer destroyed the player/iframes mid-watch —
exactly the 'video disappears after a few minutes' report. LiveNewsPanel
had no non-visibility resume path, so it stayed dead until a tab switch.

Fix: when the idle timer fires while the video is actively playing and the
tab is visible, reschedule instead of tearing down. The visibilitychange
handler (tab hidden) and the webcam IntersectionObserver (panel off-screen)
still cover the genuine 'user left' cases, so abandoned tabs still pause.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-11 10:58:02 -07:00
hanzo-dev 6461b9f7eb Merge perf/world-gzip-cache: gzip + immutable-cache static SPA 2026-07-11 10:50:06 -07:00
hanzo-dev 9f6ae73148 perf(world): gzip + immutable-cache the static SPA
The cutover from the hanzoai/static image to the cmd/world Go binary dropped
response compression and asset caching, so every cold load pulled the Vite
bundle raw: main.js 1.5MB, map.js 2.7MB — ~4.5MB of uncompressed JS.

- gzipStatic wraps ONLY the SPA handler (never /v1/world/*, which streams):
  gzip for compressible types, Range + non-gzip clients pass through,
  already-compressed media skipped. main.js -73%, map.js -72% on the wire.
- setCacheHeaders: content-hashed assets/ → immutable 1y; every unhashed file
  (favicons, manifest, service worker) stays no-cache so SW updates aren't
  stuck behind a stale cache.
- static_test.go: proves encoding, ratio, round-trip decode, cache policy.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-11 10:50:00 -07:00
hanzo-dev 43bfcd9463 feat(world/layout): resize free-mode panels from all four corners
Free mode gains NW/NE/SW corner grips alongside the existing SE one, so a
panel can be resized from any corner like a real window manager. The resize
is anchor-aware: dragging a corner pins the opposite corner — w/n edges shift
the panel's left/top in lock-step with the size, e/s grow from a fixed
top-left as before. One primitive (applyFreeResize over a {n,s,e,w} edge set)
drives every edge and corner handle; attachPanelCornerResize takes a `corner`
and only the SE corner also drives grid-mode span/column snapping (top/left
resize has no meaning in grid flow, so those grips are free-mode only, hidden
in grid via CSS). Header controls are raised above the grips so the close/info
buttons stay clickable where a top corner overlaps them.

Verified: e2e all-four-corners test asserts each corner resizes and pins its
opposite corner (14/14 panel-drag specs green); tsc clean.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 21:46:29 -07:00
hanzo-dev 71c53613e4 test(world): keep RED adversarial admin-gate matrix (17 cases, hermetic) — CI coverage for requireAdmin fail-closed 2026-07-10 14:32:22 -07:00
hanzo-dev d5cccfa026 chore(world): gitignore the embedded SQLite lake (world.db + WAL/SHM)
The datastore lake/settings DB (store.dbFile = "world.db", opened in WAL
mode) lands in the repo's data/ dir when WORLD_DATA_DIR points there in
local dev. Ignore world.db and its -wal/-shm sidecars alongside the
existing world-model runtime snapshots so they are never committed.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 14:32:22 -07:00
hanzo-dev fb5b277927 refactor(world/src): delete 3 orphaned data-fetcher services
conflict-impact, github-trending, hackernews (244 LOC): none imported
anywhere, none re-exported via services/index.ts, and every exported
symbol (correlateConflictImpact, fetchGitHubTrending, fetchHackerNews, …)
has zero references repo-wide. The github/HN panels now fetch through the
/v1/world/* URL registry in config/variants/base.ts, which superseded
these client-side fetchers. tsc + vite build green; bundle unaffected.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 14:32:22 -07:00
hanzo-dev c90e39c018 chore(world): drop unused deps @vercel/analytics, youtubei.js
Neither is imported anywhere in src/, src-tauri/, or api/ (0 refs
repo-wide). Removing them drops 4 packages from the lockfile (the two
plus their now-orphaned transitives @bufbuild/protobuf and meriyah) and
~26 MB of install footprint (youtubei.js alone is 21 MB).

Proven inert: with all four physically removed from node_modules,
`tsc --noEmit` and `vite build` pass and the JS bundle is byte-identical
(6,139,678 B) — they were never bundled, so bundle-size delta is 0; the
win is pure supply-chain / install-surface reduction.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 14:32:22 -07:00
hanzo-dev a5d6da77ae refactor(world): one IAM-userinfo introspection path
introspectOwner and introspectIdentity were the same memoized userinfo
call under two cache-key namespaces ("cloud-owner:" vs "identity:"), so a
token used for both the admin gate and settings sync was fetched and
cached twice. Delete introspectOwner; route requireAdmin through
introspectIdentity and read the owner claim off it. One function, one
cache entry per token, one source of identity truth.

Fail-closed semantics unchanged: no token → 401, introspection error or
non-admin owner → 403. Admin-gate and settings tests green.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 14:32:22 -07:00
hanzo-dev 217f13a582 refactor(world): delete dead code (deadcode-verified, tests included)
Five funcs unreachable even counting test callers, per
golang.org/x/tools/cmd/deadcode -test:

- strings.go fieldsCollapse   — 0 refs
- mcp.go     toolError         — 0 refs
- model.Engine.History()       — dead accessor; the /history route reads
                                 e.history directly (comment was stale)
- model.Engine.Context()       — superseded grounding path; AI handlers
                                 ground via CountryContext + HistoryDigest
                                 (comment "called by the AI handlers" was false)
- model.toneNote()             — only called by the removed Context()

ftoa/signed/round2 verified still live (history.go, model.go). Build,
vet, and internal/world tests green.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 14:32:22 -07:00
hanzo-dev 73bd9d78f8 feat(world): "Try Hanzo" product-switcher dropdown in header
Replace the signed-out header CTA with the hanzo.ai "Try" pattern: a white
.hz-cta pill "Try Hanzo ▾" that drops a monochrome menu of Hanzo products
(World/Chat/Dev/App/Cloud/Desktop), with world.hanzo.ai highlighted as the
current product. The former "Try Hanzo World" pill was the AccountMenu login
CTA — relabel it to a secondary "Sign in" so the header has one primary action
(hanzo.ai layout: [Try Hanzo ▾] [Sign in]).

- New self-contained stylesheet src/styles/try-hanzo.css (not main.css).
- Menu portaled to <body> + position:fixed so it never widens the page and is
  never clipped by the header's mobile overflow-y:hidden tab scroller.
- Closes on click-away and Escape; icon-only trigger < 680px. No h-overflow at
  1440/390; theme-aware (white pill dark / black pill light).
- All product URLs verified live (200): world.hanzo.ai, hanzo.chat,
  hanzo.ai/code, hanzo.app, console.hanzo.ai, hanzo.ai/desktop.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 14:28:33 -07:00
hanzo-dev 5dc7232cda feat(world/analyst): redesign the analyst copilot — markdown, real Zen mark, window modes
The dock chat is rebuilt to a native-@hanzo/ai, hanzo.chat-grade copilot in a
self-contained `hzc-` namespace + its own stylesheet (analyst-chat.css), so it
never touches main.css.

P0 fixes
- Markdown now renders: a tiny XSS-safe renderer (utils/markdown.ts) does
  bold/italic/lists/headings/code/links/tables — no raw HTML passthrough, every
  text run escaped once, every href run through sanitizeUrl.
- HTML entities decoded at the source: the backend HTML-escapes replies, so an
  apostrophe arrived as the literal `&#39;`. decodeEntities() runs first, then a
  single re-escape on output — the browser shows `'`, never `&#39;` and never the
  double-escaped `&amp;#39;`.

Brand + look
- Real Zen mark everywhere (the notched ring from zen/logo/dist/logo-mono.svg):
  icon('zen') is now the scaled logo and a new zenLogo() drives the FAB, the
  header, the assistant avatar, and the pulsing "thinking" state — no more
  generic enso/sparkle.
- vercel-black monochrome, Geist, sentence case: clean user/assistant bubbles,
  a card composer with an inline model selector, the model tag on each reply,
  tidy action-log + tool-trace styling.

Window modes (expanded scope)
- Closed = a small Zen-mark FAB bottom-right. Open = dock / sidebar / split /
  fullscreen, toggled from the header. Sidebar is the default and reflows #app
  via `body.hzc-shift` + the `--hzc-width` var (the auto-fit grid reflows
  cleanly); left-edge drag resizes; mode/width/open-state persist so a
  variant-switch reload reopens the same rail.

App control + data
- New bulk commands hide_all + show_only in the command registry (composed from
  existing AppHost methods, so they auto-propagate to the backend via
  commandManifest()); "hide all panels / show news / flip to finance" now
  visibly re-aligns the dashboard with an action-log receipt.
- MCP data-tool traces render inline as compact tables, not raw JSON.

Transport is still request/response (no SSE server yet), so slow zen5 shows a
live pulsing-Zen "thinking" state rather than a dead spinner; the SSE seam stays
in analyst-transport.ts for when a streaming cap lands.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 14:17:43 -07:00
hanzo-dev 9119ebbf3c fix(world/iam): sign-out clears local session first, bounds the IdP call (2.5s) so it never hangs; RP-logout with id_token_hint ends SSO
Claude-Session: https://claude.ai/code/session_01CDooqWJiB7yNNaSjGQjdL7
2026-07-10 14:05:47 -07:00
hanzo-dev 966882d9b7 fix(world): uncapped panel height resize (no max) + silence CartoDB CJK glyph 403 spam
1. PANEL HEIGHT CAP (CTO bug): grid-mode height snapped to the span ladder and
   hard-capped at span-4 (~800px); dragging the bottom edge further did nothing.
   Now height snaps to the fine tier ladder [120,200,400,600,800] up to its top,
   then grows CONTINUOUSLY with NO upper cap — each extra ~200px adds a whole row
   (spanForHeight, shared by the bottom-edge + corner handles). Spans beyond the
   CSS ladder (>4) are driven inline (grid-row + min-height; dataset.span is the
   source of truth via currentSpan), so a panel can be made arbitrarily tall.
   Free mode was already pixel-unbounded. MIN (120px) kept, no MAX. resetHeight
   clears the inline geometry too.
   Proof: grid 200→1628px, free 200→1700px — both persist across reload.

2. CJK glyph 403 spam: CartoDB's CJK glyph fonts (HanWang / NanumBarun / Noto
   Sans CJK…) 403 on CORS whenever a CJK label is in view, flooding the console.
   A map transformRequest repoints just those glyph fetches at the CORS-open Latin
   font (Montserrat), so CJK labels fall back cleanly with zero failed requests.
   Latin fonts (incl. "Noto Sans Regular") are untouched.
   Proof (panned to East Asia): 0 font 403s; Montserrat fallback in use.

Gate: typecheck + build clean; e2e panel-drag 13/13; responsive 17/17.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 11:49:01 -07:00
hanzo-dev 56c8f3770a feat(world/menu): workspace context menu on empty background — always a way back
Right-clicking the empty workspace (grid / content background, no panel or map
under the cursor) now opens a workspace menu: Grid / Free / Immersive layout, Add
widget, Reset layout. So even if the user hides every panel and the map, a
right-click always offers a way to reset and restore. Native menu still shows in
text inputs and over the chrome (header, dock, modals); panel/map right-clicks keep
their existing specific menus.

Proof: synthetic contextmenu on .main-content → menu with
[Grid layout, Free layout, Immersive layout, Add widget, Reset layout].

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 11:20:58 -07:00
hanzo-dev 897a9e1280 feat(world/layout): window-manager snap-zone tiling — drag to halves/quadrants with live preview
Free-mode drag now tiles like macOS / FancyZones: dragging a panel near an EDGE
previews a HALF, near a CORNER previews a QUADRANT, the centre stays free
placement. A monochrome translucent target rectangle fades in during the drag and
glides between zones as the cursor moves; on drop the panel resizes + positions to
the zone (with a clean gutter). Escape cancels (restores the pre-drag position);
reduced-motion drops the glide. One mechanism: the zone's viewport rect converts to
the panel's grid-relative free geometry on commit — no new coordinate system.

Zones are computed against the visible workspace (grid ∩ scroll viewport), so
tiling always fills the on-screen area regardless of scroll. This is the "toss
panels around like a window manager" experience over the map/globe.

Proof: drag → left-half preview 720×810 → drop → panel = left half (708×798);
top-right corner → quadrant 720×405 → panel = top-right quarter. Screenshots of the
mid-drag highlight captured.

Follow-ups (noted, not in this commit): center/left/right THIRDS, and split-view
when dragging over another panel's half.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 11:17:10 -07:00
hanzo-dev c953754197 fix(world/map): never black out without a Mapbox token — keyless deck raster basemap
mapbox-gl v3 refuses to paint ANY basemap without a token (the canvas goes fully
black even though the third-party CartoDB tiles fetch 200), which the CTO hit on
localhost and any deploy missing VITE_MAPBOX_TOKEN. Fix: when no token is present,
drape a keyless CartoDB raster basemap rendered by DECK (token-independent, same as
the data dots and the native ESRI globe) at the bottom of the overlaid deck stack —
theme-aware (dark_all / light_all), works in both the flat MapView and the globe
_GlobeView. Data layers draw on top. The token path is untouched: with a token the
mapbox vector basemap stands and no deck basemap is added.

Proof (VITE_MAPBOX_TOKEN unset): 2D = full CartoDB dark basemap w/ coastlines +
labels (60% non-black, was 0), 3D = basemap draped on the globe. Never a black void.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 11:17:10 -07:00
hanzo-dev 490acbb1c5 fix(world/panels): header cleanup — count clear of ✕ + hidden at 0, retire LIVE chip, label the summarize button
- Item count moves inline after the title (left), so it never collides with the
  hover-✕ (top-right); hidden entirely until count>0 (a loading "0" read as broken).
- Retire the "LIVE" text chip: it's all live by default and the static label
  misled while a feed was loading/empty. Live state now shows only a subtle pulsing
  dot (reduced-motion: static); cached/unavailable keep their informative labels.
- Summarize: the cryptic  becomes a labelled " Summarize" button (monochrome,
  sentence case, aria-label), reserved clear of the ✕. Funded-key path unchanged.

Playwright: 0 LIVE text chips, 0 zero/empty counts shown, summarize reads
" Summarize".

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 11:17:10 -07:00
hanzo-dev da5be4b051 feat(world/layout): one Grid/Free/Immersive mode dropdown; decomplect immersive (map bg) from snap; news-tag WCAG fix
- Dock: replace the Immersive button + Grid/Free segmented control with a single
  Layout dropdown (Grid / Free / Immersive). One control, one way.
- Decomplect background from snap: immersive is a BACKGROUND mode (map fills the
  viewport, panels float on top); grid/free is the SNAP behaviour. Immersive
  defaults to freestyle floating (free snap) over the FULL viewport — the old
  narrow right-docked column is gone (it collapsed once free-mode made the cards
  absolute). Map stays fixed full-viewport via !important over the inline free geom.
- Background Map/Video + collapse controls show only while immersive is active.
- NewsPanel category tags: drop inline color (threat colour as text failed WCAG);
  text now var(--text), threat colour carried by border + translucent bg only.

Verified: dropdown drives grid→free→immersive→grid (map fixed full-viewport in
immersive, cards float visibly over it); 17/17 P0 checks still green.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 11:17:10 -07:00
hanzo-dev 678bd10d0d fix(world/layout): kill map/panel overlap, fill full width, retire mobile modal, map edge-to-edge + maplogo flag
Layered on the integrated layout engine (194e6657). The grid's auto-rows
collapsed to their 120px floor, so every span-2 panel (min-height 400) — the
map included — overflowed ~150px onto the row below and OVERLAPPED it
(43 overlaps across 41 panels). Root fix: definite 200px row unit
(grid-auto-rows: var(--panel-row,200px)) so span-N reserves exactly its height;
content that exceeds a panel scrolls inside it, never onto neighbours.

- Full width: auto-fill → auto-fit so real columns always stretch to consume
  100% width, no dead right-hand gutter at any size (verified 9 widths).
- Map edge-to-edge: the 14px drag-grip is now an absolute overlay, not in flow,
  so the canvas fills the panel top (black top strip 15px → 1px).
- ?maplogo=0 (+ localStorage) hides the basemap wordmark; a compact "ⓘ"
  AttributionControl is always mounted so basemap ToS still holds.
- Mobile-view warning modal retired (layout is responsive to 390px).
- span-0 stays a genuine 120px short tier (top-aligned in its 200px track).

Proof: 17/17 playwright checks (full/crypto/saas × 1920–390): hOverflow=0,
0 overlaps, gutter=grid-padding only, no modal. Live-news video grows to full
width (353→1430px, 16:9, no cap). Dark/Sat/Terrain inline with 2D/3D in the dock.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 11:17:10 -07:00
hanzo-dev a338fc440b feat(world): panel layout engine integrated onto origin/main + free live-news video
Rebuilds the free-form + snap-to-grid layout engine on top of the advanced
origin/main (footer dock + responsive work) and unifies the two grid-cell
mechanisms into one:

- grid-config drives the dock's `--panel-col-min` (default 160px, range 140–360,
  matching the base .panels-grid rule + the dock slider) — the duplicate
  `--grid-cell` override is gone. One variable, one way. Default 160 = the grid
  is byte-identical until the slider moves it, so no fixed-span panel shrinks.
- window.worldGrid exposes the engine to the footer dock's Grid⇄Free toggle +
  cell-size slider (App.gridApi delegates to it).
- main.css layout-engine section is append-only after the dock CSS; only NEW
  selectors (corner grip, snap overlay, free-mode absolute layout, mobile guard).

Live News (video) now resizes freely with the video filling at any size (CTO
ask): the panel carries id="live-news" so the intended `#live-news .panel-content`
fill CSS (padding:0 + flex-column) finally applies — the 16:9 `.live-news-player`
fills the container edge-to-edge (was 16px short under generic padding). With the
new right-edge + corner grips and the uncapped column snap (span up to full
width), Live News drags to 2-3 cols or full-width-top in grid mode and to any
pixel size in free mode; the width-driven video fills at every size.

Tests (13, all green; typecheck + build pass): 4 original drag/resize + 7
layout-engine (grid snap, corner resize, overlay, cell re-snap, free drag+resize
persist across reload, map participates, toggle) + 2 real-app live-news (grid
default→3col→full-width fill setup + resize, free arbitrary-size). Screenshot:
e2e/layout-shots/live-news-fullwidth.png (full-width-top, large video area).

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 11:17:10 -07:00
hanzo-dev 6fdfc0d3a6 feat(world): instant news via warm feed cache + queryable ingested-data lake
Make news/feeds INSTANT and give world one place to query everything.

Feed hot cache (two-tier, ONE way to read a feed body):
- L1 per-pod in-mem mirror (sub-ms hot reads) + L2 hanzo-kv (shared across
  pods, survives restart). A body warmed by any pod is instantly served by
  every pod; a restarted pod reads the still-warm shared cache instead of
  cold-starting. Graceful degrade to L1-only when hanzo-kv is unreachable.
- Background warmer: on boot + every ~5m (jittered) fetches every warm feed,
  write-throughs the body, and folds items into the lake. rss-proxy and
  feeds-batch serve from the warm cache and never block on upstream while any
  cached copy exists (stale-while-revalidate; revalidation runs in background).
- Demand-driven warm set (fleet-wide via hanzo-kv) + a curated seed of the
  crypto / financial-regulation / crypto-news feeds so a brand-new pod warms
  THOSE first. Cross-pod fetch dedupe via shared copy freshness.

Ingested-data lake (embedded SQLite, modernc.org/sqlite — pure Go, FTS5):
- One normalized `items` table (id, kind, source, ts, title, text, tickers,
  country, geo, payload) fed by feed news AND world-model observations, so
  everything is searchable/countable together.
- GET /v1/world/search (FTS5 bm25 ranking; filters kind/since/country/ticker)
  and GET /v1/world/analytics (totals, by kind/source, top tickers). Rolling
  retention window with an hourly prune job. Write-behind ingest (never on the
  request path).

Per-identity settings (same SQLite DB, own table):
- GET/PUT /v1/world/settings, bearer-gated, keyed by (org, user_sub, project)
  from IAM userinfo, so a signed-in dashboard syncs across devices. Anonymous
  → 401 (client keeps localStorage).

Everything degrades cleanly (never-5xx): no hanzo-kv → per-pod cache; no SQLite
→ empty search/analytics + settings say not-stored. Binary stays CGO-free
(CGO_ENABLED=0). go directive + Dockerfile bumped to 1.25 for modernc.

Latency (local, warm): feeds-batch warm 0.5-1.9ms vs 154ms cold; search 0.8-1.5ms.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 10:51:22 -07:00
hanzo-dev 1043182671 fix(world/analyst): enforce paid usage — chat needs a user bearer, never the funded key
The analyst chat is paid usage: metered to the signed-in user's org via their
own IAM bearer. handleAnalyst used s.ai.bearerFor(r), which falls back to the
funded service key (HANZO_AI_KEY) when no user token is present — so an anonymous
chat silently ran on the shared key (proven live: an unauthenticated POST to
/v1/world/analyst returns a real 200 answer). That made the backend "Sign in to
chat" gate dead code and meant paid usage was never actually enforced or metered.

Use userBearer(r) for the chat path (user-only). The funded key now backs ONLY the
anonymous auto-insight endpoints (summarize/classify/country-intel), exactly the
intended pro model. The sign-in gate is now live server-side, and the frontend
gate gains defense-in-depth. Roster listing (handleModels) stays on bearerFor —
it is read-only, not metered inference.

Test: TestAnalystChatRequiresUserBearer pins the contract (funded key present,
anonymous chat still refused with the sign-in skip). TestAnalystDataToolLoop now
drives the chat as a signed-in caller (Authorization header), the real paid path.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 10:25:14 -07:00
hanzo-dev 68a28f2eda feat(world/map): dark/satellite/terrain basemap styles on the native globe
The native deck GlobeView now honors the 2D basemap switcher (dark | satellite |
terrain), reading the SAME localStorage key (hanzo-world-basemap-style) and
reacting live to a basemap-style-changed event DeckGLMap fires — no reload.

- dark: the existing monochrome vercel-black sphere.
- satellite: keyless ESRI World Imagery draped via a deck TileLayer/BitmapLayer,
  back-faces culled so the far hemisphere never bleeds through; the near-black
  ocean sphere underneath supplies depth + darkens the imagery so data dots stay
  legible. Thin light country borders overlaid for intel orientation.
- terrain: keyless ESRI World Physical Map (natural relief).

Data layers (dots/arcs/borders) render ON TOP and keep picking; ocean sphere is
tinted dark slate-blue on bright styles so the Web-Mercator polar gap blends.
Since native uses keyless ESRI, the switcher's sat/terrain buttons no longer need
a Mapbox token when native is the 3D renderer (default), and setBasemapStyle
notifies the globe before the tokenless mapbox setStyle so a missing token can't
block the switch. Adds a style-switch e2e (single-context preserved, data still
picks) + harness hooks.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 10:20:44 -07:00
hanzo-dev a8af187b99 fix(world): split header/tooling into a bottom dock; kill 1366 overflow; layers collapsed; globe fit
Live 1366×768 had 167px of horizontal page overflow (documentElement.scrollWidth
1533 vs innerWidth 1366): the header's action row (Copy Link / theme / Immersive /
PANELS / SOURCES + variant tabs) could not shrink and pushed the account CTA off the
right edge. The layers panel also rendered OPEN by default, floating over the map and
the panels beneath it.

CTO-approved restructure: the header is now IDENTITY + NAV only (Hanzo mark, variant
tabs, LIVE, search, theme, account). All operational controls move to a new slim,
monochrome, collapsible BOTTOM TOOLBAR DOCK:

- Map's 2D/3D toggle, basemap switcher (Dark/Sat/Terrain) and time-range pills mount
  into the dock via a new optional DeckGLMap `controlsHost` (defaults to the map
  container, so the e2e harnesses that drive the overlay controls are unchanged).
- Region select, Layers toggle, Immersive controls, Panels, Sources, Copy link and
  Fullscreen relocate into the dock (same ids → existing handlers still bind).
- New controls: layout-mode (Grid ⇄ Free) + widget-size slider, wired through a
  feature-checked adapter that prefers window.worldGrid (world-layoutengine) and
  otherwise applies a self-contained fallback (CSS column-floor var + body data attr);
  and "+ Add widget", a searchable palette over the panel registry that shows a panel
  via the one setPanelEnabled() path — works in both grid and immersive layouts.

Fixes:
1. Horizontal overflow — scrollWidth === innerWidth at 1920/1440/1366/1024/768/390.
   Header groups are min-width:0 shrinkable; variant tabs collapse to icons then scroll
   INSIDE their own strip; the dock scrolls horizontally so the PAGE never scrolls.
2. Layers panel — hidden by default (`.open` reveals it via the dock's Layers button),
   anchored top-left WITHIN the map box, height-capped with internal scroll, draggable.
3. Basemap switcher — verified switching Dark/Sat/Terrain in 2D (tiles load, deck data
   layers survive setStyle, choice persists in localStorage). The token IS in the
   runtime bundle. In native-3D-globe mode the deck GlobeView renders its own sphere,
   so sat/terrain don't retexture it there — a pre-existing renderer limitation, noted.
4. Globe fit — the deck globe canvas fills the map container exactly (measured
   1356×384 == map box; no letterboxing); CSS forces the wrappers/canvas to 100%.

Gates: typecheck ✓, vite build ✓, e2e globe-overlay 1/1 ✓, panel-drag 4/4 ✓.

Claude-Session: https://claude.ai/code/session_013jh8aka8q8RvhhVQ1psMeW
2026-07-10 10:01:09 -07:00
226 changed files with 33802 additions and 1717 deletions
+10
View File
@@ -27,3 +27,13 @@ data/world-model.json
data/world-model.json.tmp
data/world-model-history.json.gz
data/world-model-history.json.gz.tmp
# embedded datastore lake + settings (SQLite, WAL mode) — regenerated on boot,
# lands here only when WORLD_DATA_DIR points at the repo (local dev)
world.db
world.db-wal
world.db-shm
data/world.db
data/world.db-wal
data/world.db-shm
/world
+75
View File
@@ -2,6 +2,81 @@
All notable changes to World Monitor are documented here.
## [2.4.19]
### Fixed
- **Moving/resizing a panel no longer shifts the others** (the "when I shift the 3D map it shifts all other components" report). The dashboard now DEFAULTS to the free layout: every panel owns its own {x,y,w,h}, so a drag or resize leaves every sibling exactly where it was — no more grid reflow. The switch is invisible: the current grid arrangement is frozen as the starting geometry (`grid-config.applyDefaultLayout`), and Grid stays one dropdown-click away for anyone who wants snap-to-grid back.
- **Panels resize granularly** — pixel-exact width AND height from any edge or corner, instead of jumping between coarse column/row spans.
- **Panels can be made much narrower/smaller** ("constrained on min width"): the free-mode floors drop to 96×40px (map 140px), and the grid widget-size slider now reaches a 120px column track (was 140).
### Changed
- A panel shown from the Panels menu / "+ Add widget" while in free mode seeds a tidy default slot below the placed panels (never a full-width block under its absolute siblings). Hidden panels no longer persist 0×0 geometry.
## [2.4.13]
### Fixed
- **Globe overlays now sit ON the sphere with correct occlusion** (the "map spazzes" report). On the native deck.gl GlobeView, data layers (shared with the 2D map) carried no depth parameters, so the far hemisphere showed through and count badges floated above the limb. GlobeNative now seats every data layer against the depth-writing ocean sphere (test, don't write — no inter-marker z-fight), and the far-side billboard cull (`occludeFarSide`) faces the globe's OWN live camera instead of the parked (frozen) mapbox center — extended to `TextLayer` so a back-side "36" count badge disappears instead of hovering over the planet.
- **Terrain striping on the globe**: the draped ESRI imagery tiles each wrote depth, so coplanar neighbours z-fought at the tile seams (horizontal stripes / partial render). Tiles now depth-TEST only; the ocean sphere owns the depth buffer.
- **Live request-geo dots kept polling on the 3D globe**: the `/v1/world/cloud/*` poll was gated on `renderPaused`, which the native GlobeView sets on activation — so the realtime dots froze the moment you entered the default 3D view. It now gates on tab visibility, so the dots stay live (and fade) on the globe.
### Changed
- **Request-origin → serving-region arcs on by default** in the Cloud view: the `/cloud/traffic` arcs derive from the same real native request-geo points as the dots (origin country → nearest Hanzo region) and degrade to empty — never demo.
## [2.4.6]
### Added
- **Enso Flywheel panel (AI variant)**: the router self-improvement loop made visible. New `/v1/world/enso-training` folds the routing-decision ledger tail + reward tail (`export-routing-ledger` / `export-routing-rewards`, super-admin) into ledger growth, engine-vs-heuristic mix, a routing-confidence histogram, and task/model distributions, alongside the latest enso-bench eval scores (an embedded `results/summary.json` snapshot, overridable live via `ENSO_BENCH_URL`). The response is event-typed so future retrain/deploy milestones slot into the same timeline. Eval scores render even signed-out; `state` (live/partial/demo) says which live sources resolved.
## [2.4.5]
### Added
- **AI Compute panel (AI variant)**: live Hanzo inference-plane telemetry over SSE — tokens/s, requests/s, 24h spend, top models by real spend, and the serving fleet (GPUs, machines online, models served). New `/v1/world/ai-pulse` pushes typed `usage`/`fleet`/`status` frames (EventSource) and answers a plain GET with one JSON snapshot as the poll fallback. Honest "connecting"/"unavailable" states — never a zero dressed up as live; the service bearer stays server-side.
## [2.4.4]
### Changed
- **Cloud Pulse is real when a service token is wired**: `/v1/world/cloud-pulse` now folds MEASURED platform-wide 24h request/token volume from the ClickHouse-backed usage ledger (`get-cloud-usages ?org=all`, super-admin) on top of the live model/node/GPU/region counts — dropping both `demo:true` and `volumeModeled:true`. Top models come from real ledger spend. Without a token, or with a non-admin token, it stays honestly demo/modeled — platform numbers are never faked silently. The service bearer stays server-side (never sent to the browser).
## [2.4.2]
### Added
- **Streaming analyst**: answers flow in live over SSE — reasoning shows as dim thinking text, tool calls appear as chips the moment they run, the reply types itself in; the final render and command dispatch are unchanged (done event = the old JSON contract)
- **Model menu**: the composer pill opens a grouped popover (Auto / Zen / GPT / Llama / Claude / Agents) with per-family marks and an active check
### Fixed
- **Model identity**: the Zen ring appears only on zen* models — gpt-oss/llama/claude get their own marks on the pill, menu rows, avatars, and the thinking row
- **Dev server**: `npm run dev` proxies /v1 to production by default (VITE_DEV_API_PROXY overrides)
## [2.4.1]
### Added
- **Western Pacific cyclones**: cross-agency tropical-cyclone attribution (GDACS + HKO warnings via new `/v1/world/hko-warnings` proxy) with per-agency wind observations, canonical dedup, and map popup detail rows
- **China macro snapshot**: `/v1/world/china-macro` — CPI/CLI (OECD), policy rate, USD/CNY (FRED), HKMA context, NBS release calendar + PBoC LPR dates, surfaced with staleness-honest indicator tiles
- **Model roster**: `Best (auto)` leads the analyst model picker — the gateway routing alias that always resolves
### Changed
- **AI default model**: `zen5``best`; a pinned family id goes dark when the inference plane's claim catalog shifts, the routing alias never does
- **Server cache is now stale-while-revalidate**: `cachedJSON`/`passthrough` serve stale instantly and refresh in the background (single-flight); GDELT/theater-posture no longer stall requests ~10s on TTL expiry
- **GDELT cache warmers**: hot keys (analyst grounding, protests layer) refreshed every ~4min so no user ever eats a cold miss
- **Sparkline payloads**: close arrays rounded to 7 significant digits (float32-widening noise stripped, ~40% smaller; scalars untouched)
### Fixed
- **News first paint**: panels no longer gate their first DOM write on a 65MB ML sentiment model download — headlines paint immediately, sentiment refines in place
- **Analyst grounding snapshot**: context fetches bounded at 2.5s so a cold endpoint can't hold the chat send hostage
- **AI errors are honest**: upstream error codes (`insufficient_balance`, `spend_cap_exceeded`, …) surface in the chat instead of a bare `status 402`
## [2.4.0] - 2026-02-19
### Added
+18 -3
View File
@@ -37,10 +37,25 @@ ENV VITE_MAPBOX_TOKEN=$VITE_MAPBOX_TOKEN
RUN npm run build
# ---- go stage: build the static server binary (CGO-free) -----------------
FROM golang:1.23-alpine AS gobuild
# go 1.26: go.mod requires >= 1.26.4 (github.com/hanzoai/sqlite drop-in). The
# binary stays CGO-free — with CGO_ENABLED=0, hanzoai/sqlite selects its pure-Go
# modernc backend (FTS5 built in), so no C toolchain is added and FTS5 works.
FROM golang:1.26-alpine AS gobuild
WORKDIR /src
# stdlib-only module: go.mod has no requires, so there is no go.sum to copy.
COPY go.mod ./
# git: go resolves the PRIVATE indirect dep github.com/hanzoai/csqlite 'direct'
# (not via the module proxy), which needs the git binary + an https credential.
# alpine ships neither, so add git and mount the gh_token BuildKit secret that
# hanzoai/ci passes (--secret id=gh_token); the mount is a no-op for public builds.
RUN apk add --no-cache git
ENV GOPRIVATE=github.com/hanzoai,github.com/luxfi,github.com/zooai
# Deps: hanzo-kv client (go-redis) + embedded SQLite (modernc). Download once for
# a cached layer before the source is copied.
COPY go.mod go.sum ./
RUN --mount=type=secret,id=gh_token \
if [ -s /run/secrets/gh_token ]; then \
git config --global url."https://x-access-token:$(cat /run/secrets/gh_token)@github.com/".insteadOf "https://github.com/"; \
fi; \
go mod download
COPY cmd ./cmd
COPY internal ./internal
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/world ./cmd/world
+1
View File
@@ -0,0 +1 @@
2.4.22
+115
View File
@@ -0,0 +1,115 @@
package main
import (
"compress/gzip"
"io"
"net/http"
"strings"
"sync"
)
// gzip on the fly for the static SPA. The hanzoai/static image this binary
// replaced compressed responses for us; serving the Vite bundle raw (main.js
// 1.5MB, map.js 2.7MB) is the whole reason a cold load felt slow. Restoring
// gzip here cuts the JS/CSS wire size ~75%. Paired with immutable cache headers
// on hashed assets (see setCacheHeaders), each client pays the transfer once.
//
// Scope: wraps ONLY the SPA handler, never /v1/world/* — those include SSE /
// streaming endpoints that must not be buffered through a gzip.Writer.
var gzipPool = sync.Pool{
New: func() any {
w, _ := gzip.NewWriterLevel(io.Discard, gzip.DefaultCompression)
return w
},
}
// gzipStatic compresses compressible responses when the client accepts gzip.
// Range requests pass through untouched (compressing a byte-range is invalid).
func gzipStatic(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Range") != "" || !acceptsGzip(r) {
next.ServeHTTP(w, r)
return
}
gw := &gzipResponseWriter{ResponseWriter: w}
defer gw.close()
next.ServeHTTP(gw, r)
})
}
func acceptsGzip(r *http.Request) bool {
for _, enc := range strings.Split(r.Header.Get("Accept-Encoding"), ",") {
if strings.EqualFold(strings.TrimSpace(strings.SplitN(enc, ";", 2)[0]), "gzip") {
return true
}
}
return false
}
// compressible reports whether a content type shrinks under gzip. Already-
// compressed media (png/jpg/woff2/…) is skipped so we never waste CPU or grow
// the payload.
func compressible(ct string) bool {
ct = strings.ToLower(ct)
switch {
case strings.HasPrefix(ct, "text/"),
strings.Contains(ct, "javascript"),
strings.Contains(ct, "json"),
strings.Contains(ct, "svg"),
strings.Contains(ct, "xml"),
strings.Contains(ct, "wasm"),
strings.Contains(ct, "manifest"):
return true
}
return false
}
type gzipResponseWriter struct {
http.ResponseWriter
gz *gzip.Writer
decided bool
compress bool
}
func (g *gzipResponseWriter) WriteHeader(status int) {
g.decide(status)
g.ResponseWriter.WriteHeader(status)
}
func (g *gzipResponseWriter) Write(b []byte) (int, error) {
if !g.decided {
g.decide(http.StatusOK) // FileServer may Write without an explicit WriteHeader
}
if g.compress {
return g.gz.Write(b)
}
return g.ResponseWriter.Write(b)
}
// decide inspects the headers the wrapped handler set (Content-Type is already
// populated by http.ServeContent at this point) and commits to compress-or-not
// exactly once.
func (g *gzipResponseWriter) decide(status int) {
if g.decided {
return
}
g.decided = true
h := g.Header()
if status == http.StatusOK && h.Get("Content-Encoding") == "" && compressible(h.Get("Content-Type")) {
g.compress = true
h.Del("Content-Length") // length changes after compression
h.Set("Content-Encoding", "gzip")
h.Add("Vary", "Accept-Encoding")
g.gz = gzipPool.Get().(*gzip.Writer)
g.gz.Reset(g.ResponseWriter)
}
}
func (g *gzipResponseWriter) close() {
if g.gz != nil {
_ = g.gz.Close()
gzipPool.Put(g.gz)
g.gz = nil
}
}
+18 -2
View File
@@ -46,12 +46,16 @@ func main() {
world.LoadKMSSecrets(rootCtx)
srv := world.NewServer()
srv.StartModel(rootCtx) // continuously-folded world-state engine
defer srv.Close() // release hanzo-kv + embedded datastore handles
srv.StartModel(rootCtx) // continuously-folded world-state engine
srv.StartDatastore(rootCtx) // shared feed warmer + lake write-behind/prune
mux := http.NewServeMux()
srv.Mount(mux) // /v1/world/* routes
// Static SPA + fallback handles everything not matched by an /api route.
mux.Handle("/", newSPAHandler(*root))
// gzipStatic wraps ONLY this handler — /v1/world/* keeps its streaming
// endpoints unbuffered.
mux.Handle("/", gzipStatic(newSPAHandler(*root)))
httpSrv := &http.Server{
Addr: *addr,
@@ -123,6 +127,7 @@ func (h *spaHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
info, err := os.Stat(full)
switch {
case err == nil && !info.IsDir():
setCacheHeaders(w, rel)
h.fileSrv.ServeHTTP(w, r) // real file
case errors.Is(err, fs.ErrNotExist) && !hasExt(rel):
h.serveIndex(w, r) // client-routed path → SPA shell
@@ -147,6 +152,17 @@ func (h *spaHandler) serveIndex(w http.ResponseWriter, r *http.Request) {
}
}
// setCacheHeaders makes Vite's content-hashed bundles cacheable forever while
// keeping every unhashed file (favicons, manifest, service worker) revalidated
// so SW updates and icon swaps are never stuck behind a stale cache.
func setCacheHeaders(w http.ResponseWriter, rel string) {
if strings.HasPrefix(rel, "assets/") {
w.Header().Set("Cache-Control", "public, max-age=31536000, immutable")
return
}
w.Header().Set("Cache-Control", "no-cache")
}
// hasExt reports whether the last path segment has a file extension, used to
// distinguish an asset miss (foo.js → 404) from a client route (/country/US →
// SPA shell).
+123
View File
@@ -0,0 +1,123 @@
package main
import (
"bytes"
"compress/gzip"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
)
// writeTree lays out a minimal Vite-style dist for the SPA handler tests.
func writeTree(t *testing.T) string {
t.Helper()
root := t.TempDir()
if err := os.MkdirAll(filepath.Join(root, "assets"), 0o755); err != nil {
t.Fatal(err)
}
// A hashed JS bundle large enough that gzip clearly wins.
js := strings.Repeat("export const answer = 42;\n", 4000)
files := map[string]string{
"index.html": "<!doctype html><title>world</title>",
"assets/main-abc123.js": js,
"assets/main-abc123.css": strings.Repeat(".panel{display:flex}\n", 2000),
"favicon.ico": "\x00\x00binary",
"manifest.webmanifest": `{"name":"world"}`,
}
for name, body := range files {
if err := os.WriteFile(filepath.Join(root, name), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
return root
}
func TestStaticGzipAndCache(t *testing.T) {
root := writeTree(t)
srv := httptest.NewServer(gzipStatic(newSPAHandler(root)))
defer srv.Close()
get := func(path, ae string) *http.Response {
req, _ := http.NewRequest(http.MethodGet, srv.URL+path, nil)
req.Header.Set("Accept-Encoding", ae)
resp, err := http.DefaultTransport.RoundTrip(req) // no transparent gunzip
if err != nil {
t.Fatalf("GET %s: %v", path, err)
}
return resp
}
rawLen := func(root, name string) int {
b, _ := os.ReadFile(filepath.Join(root, name))
return len(b)
}
t.Run("hashed js is gzipped, immutable, and smaller on the wire", func(t *testing.T) {
resp := get("/assets/main-abc123.js", "gzip")
defer resp.Body.Close()
if got := resp.Header.Get("Content-Encoding"); got != "gzip" {
t.Fatalf("Content-Encoding = %q, want gzip", got)
}
if got := resp.Header.Get("Cache-Control"); !strings.Contains(got, "immutable") {
t.Fatalf("Cache-Control = %q, want immutable", got)
}
if !strings.Contains(resp.Header.Get("Vary"), "Accept-Encoding") {
t.Fatalf("Vary = %q, want Accept-Encoding", resp.Header.Get("Vary"))
}
body, _ := io.ReadAll(resp.Body)
raw := rawLen(root, "assets/main-abc123.js")
if len(body) >= raw {
t.Fatalf("gzip wire size %d not smaller than raw %d", len(body), raw)
}
// And it must actually decode back to the original bytes.
zr, err := gzip.NewReader(bytes.NewReader(body))
if err != nil {
t.Fatalf("gzip.NewReader: %v", err)
}
dec, _ := io.ReadAll(zr)
if len(dec) != raw {
t.Fatalf("decoded %d bytes, want %d", len(dec), raw)
}
})
t.Run("no gzip when client does not accept it", func(t *testing.T) {
resp := get("/assets/main-abc123.js", "identity")
defer resp.Body.Close()
if resp.Header.Get("Content-Encoding") != "" {
t.Fatalf("unexpected Content-Encoding %q", resp.Header.Get("Content-Encoding"))
}
})
t.Run("unhashed files revalidate, never immutable", func(t *testing.T) {
for _, p := range []string{"/favicon.ico", "/manifest.webmanifest"} {
resp := get(p, "gzip")
resp.Body.Close()
if cc := resp.Header.Get("Cache-Control"); cc != "no-cache" {
t.Fatalf("%s Cache-Control = %q, want no-cache", p, cc)
}
}
})
t.Run("already-compressed media is not re-gzipped", func(t *testing.T) {
resp := get("/favicon.ico", "gzip")
defer resp.Body.Close()
if resp.Header.Get("Content-Encoding") == "gzip" {
t.Fatal("favicon.ico should not be gzipped")
}
})
t.Run("index shell served with no-cache and gzip", func(t *testing.T) {
resp := get("/", "gzip")
defer resp.Body.Close()
if cc := resp.Header.Get("Cache-Control"); cc != "no-cache" {
t.Fatalf("index Cache-Control = %q, want no-cache", cc)
}
if resp.Header.Get("Content-Encoding") != "gzip" {
t.Fatalf("index should be gzipped, got %q", resp.Header.Get("Content-Encoding"))
}
})
}
+147
View File
@@ -0,0 +1,147 @@
import { expect, test, type Page, type Route } from '@playwright/test';
/**
* Full app control from the chat widget.
*
* The analyst's control surface is the app-command registry (services/app-commands.ts)
* driven through the AppHost port. Until now it could show/hide/move panels and
* touch the map, but it could NOT change the layout mode, add a topic to monitor,
* or drive the Watch Queue — so "reconfigure the layout and all widgets from the
* chat" was not actually true.
*
* Each test stubs the analyst to return ONE command and asserts the app really
* changed — the command is dispatched through the same path a real model reply
* takes. No mocked host, no unit-test stand-in for the DOM.
*/
const SCREENS = 'e2e/screens';
async function stubAnalyst(page: Page, actions: unknown[], reply = 'Done.'): Promise<void> {
await page.route('**/v1/world/**', (route: Route) => {
const url = route.request().url();
if (url.includes('/v1/world/analyst')) {
return route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ reply, actions, model: 'zen5', tokens: 0, traces: [] }),
});
}
if (url.includes('/v1/world/models')) {
return route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ models: [{ id: 'zen5', name: 'Zen 5' }], default: 'zen5' }),
});
}
// Monitors: signed-in sync endpoints. Keep them empty + accepting.
if (url.includes('/v1/world/monitors')) {
return route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ monitors: [], matches: [], ok: true }),
});
}
return route.fulfill({ status: 200, contentType: 'application/json', body: '[]' });
});
}
async function signIn(page: Page): Promise<void> {
await page.addInitScript(() => {
localStorage.setItem('hanzo_iam_access_token', 'e2e-fake-token');
localStorage.setItem('hanzo_iam_expires_at', String(Date.now() + 3_600_000));
localStorage.setItem('hanzo_iam_owner', 'e2e-org');
});
}
async function appReady(page: Page): Promise<void> {
await page.goto('/');
await page.waitForSelector('.panel', { timeout: 30_000 });
}
/** Send any message; the stub decides which command comes back. */
// Two AnalystChats exist (the in-grid panel and the dock) — they share the SAME
// code path, so drive the dock's explicitly rather than matching both.
async function ask(page: Page, text: string): Promise<void> {
await page.click('.hzc-fab');
const dock = page.locator('.hzc-panel');
await expect(dock).toBeVisible();
const composer = dock.locator('.hzc-input');
await expect(composer).toBeVisible();
await composer.fill(text);
await dock.locator('.hzc-send').click();
}
test.describe('analyst full app control', () => {
test('set_layout_mode switches the app into immersive', async ({ page }) => {
await signIn(page);
await stubAnalyst(page, [{ type: 'set_layout_mode', mode: 'immersive' }]);
await appReady(page);
await expect(page.locator('body')).not.toHaveClass(/immersive/);
await ask(page, 'go immersive');
// The app really entered immersive (body class is the mode's source of truth)…
await expect(page.locator('body')).toHaveClass(/immersive/);
// …and the dock select agrees — the AI and the UI cannot disagree about mode.
await expect(page.locator('#dockModeSelect')).toHaveValue('immersive');
await page.screenshot({ path: `${SCREENS}/analyst-immersive.png` });
});
test('add_monitor makes the analyst able to watch a new topic', async ({ page }) => {
await signIn(page);
await stubAnalyst(page, [{ type: 'add_monitor', keywords: 'nvidia, gpu' }]);
await appReady(page);
await ask(page, 'watch for nvidia and gpu');
// The monitor list really gained the topic (the panel renders it).
const monitors = page.locator('#monitorsList');
await expect(monitors).toContainText(/nvidia/i);
await expect(monitors).toContainText(/gpu/i);
// And it persisted through the ONE monitor path (localStorage mirror).
const stored = await page.evaluate(() =>
JSON.parse(localStorage.getItem('worldmonitor-monitors') || '[]'),
);
expect(JSON.stringify(stored)).toContain('nvidia');
});
test('queue_next advances the Watch Queue', async ({ page }) => {
await signIn(page);
await stubAnalyst(page, [{ type: 'queue_next' }]);
await appReady(page);
// Seed two items straight into the queue's own store, then let the analyst advance it.
await page.evaluate(() => {
localStorage.setItem(
'hanzo-world-watch-queue',
JSON.stringify({
items: [
{ id: 'a', kind: 'video', title: 'First talk', source: 'X', ref: 'aaaaaaaaaaa', addedAt: 1, status: 'queued' },
{ id: 'b', kind: 'video', title: 'Second talk', source: 'X', ref: 'bbbbbbbbbbb', addedAt: 2, status: 'queued' },
],
currentId: 'a',
}),
);
});
await page.reload();
await page.waitForSelector('.panel', { timeout: 30_000 });
await ask(page, 'next video');
// 'a' is finished and 'b' is now current — tracked consumption, driven by the AI.
await expect
.poll(async () =>
page.evaluate(() => {
const q = JSON.parse(localStorage.getItem('hanzo-world-watch-queue') || '{}');
return q.currentId;
}),
)
.toBe('b');
const statusOfA = await page.evaluate(() => {
const q = JSON.parse(localStorage.getItem('hanzo-world-watch-queue') || '{}');
return (q.items || []).find((i: { id: string }) => i.id === 'a')?.status;
});
expect(statusOfA).toBe('watched');
});
});
+155
View File
@@ -0,0 +1,155 @@
import { expect, test } from '@playwright/test';
import { clickMapControl } from './helpers/map-controls';
// Cloud mode: on world.hanzo.ai (and local dev — a hanzo brand host) the H logo is a
// toggle that reveals the [Hanzo | World | AI | Crypto | Finance | Tech] switcher,
// and the flagship `hanzo` view ships the live-traffic globe layer. This spec drives
// the real header + map on the main app, plus the TrafficGlobePanel in isolation.
const TRAFFIC = {
updatedAt: '2026-07-16T12:00:00Z',
live: true,
window: { minutes: 60, since: '2026-07-16T11:00:00Z', until: '2026-07-16T12:00:00Z' },
points: [
{ country: 'US', region: 'CA', lat: 36.12, lon: -119.68, count: 42, byService: { chat: 40, media: 2 } },
{ country: 'GB', lat: 55.38, lon: -3.44, count: 12, byService: { models: 12 } },
{ country: 'DE', lat: 51.17, lon: 10.45, count: 7, byService: { embeddings: 7 } },
],
totals: { rps_1m: 0.7, rpm_60m: 0.35, top_countries: [{ country: 'US', count: 42 }, { country: 'GB', count: 12 }, { country: 'DE', count: 7 }] },
};
test.describe('Cloud mode', () => {
test('H logo reveals the switcher; cloud view ships the live-traffic globe layer', async ({ page }) => {
// Feed the globe layer real points so its poll resolves (avoids a 404 empty state).
await page.route('**/v1/world/cloud/traffic-globe', (route) =>
route.fulfill({ status: 200, contentType: 'application/json', body: JSON.stringify(TRAFFIC) }),
);
await page.goto('/?variant=cloud');
// The H logo renders as the Hanzo-mode toggle on this (hanzo brand) host.
const hLogo = page.locator('[data-hanzo-toggle]');
await expect(hLogo).toBeVisible();
// Flagship default: the switcher starts collapsed (revealed by the H, not shown
// by default), so the clean globe view leads.
const switcher = page.locator('.variant-switcher');
await expect(switcher).toBeHidden();
// Click the H → Cloud mode reveals the switcher.
await hLogo.click();
await expect(page.locator('.header.hanzo-mode')).toHaveCount(1);
await expect(switcher).toBeVisible();
await expect(hLogo).toHaveAttribute('aria-expanded', 'true');
// Exactly the [Cloud | World | AI | Crypto | Finance | Tech] tabs, Cloud first.
await expect(switcher.locator('.variant-option')).toHaveCount(6);
await expect(switcher.locator('.variant-option').first()).toHaveAttribute('data-variant', 'cloud');
await expect(page.locator('.variant-option[data-variant="cloud"].active')).toHaveCount(1);
// Click the H again → collapses.
await hLogo.click();
await expect(switcher).toBeHidden();
// The globe's native traffic layer is wired into the layer-toggle system and ON
// by default in the hanzo view — i.e. the globe layer is toggleable.
await expect(page.locator('.deckgl-map-wrapper')).toBeVisible({ timeout: 45000 });
const trafficToggle = page.locator('.layer-toggle[data-layer="traffic"]');
await expect(trafficToggle).toHaveCount(1);
await expect(trafficToggle.locator('input[type="checkbox"]')).toBeChecked();
});
test('TrafficGlobePanel renders live throughput + top origins, and an honest empty state', async ({ page }) => {
await page.goto('/tests/runtime-harness.html');
// Live payload → throughput tiles + ranked origin countries.
const live = await page.evaluate(async (payload) => {
const orig = window.fetch;
window.fetch = (async () => ({ ok: true, status: 200, json: async () => payload })) as typeof window.fetch;
const { TrafficGlobePanel } = await import('/src/components/TrafficGlobePanel.ts');
const panel = new TrafficGlobePanel();
const root = panel.getElement();
document.body.appendChild(root);
const deadline = Date.now() + 3000;
while (Date.now() < deadline && root.querySelectorAll('.traffic-row').length === 0) {
await new Promise((r) => setTimeout(r, 30));
}
const text = root.textContent ?? '';
const rows = root.querySelectorAll('.traffic-row').length;
panel.destroy(); root.remove(); window.fetch = orig;
return { text, rows };
}, TRAFFIC);
expect(live.rows).toBe(3);
expect(live.text).toContain('US');
expect(live.text).toContain('requests / sec');
expect(live.text.toLowerCase()).toContain('active regions');
// Empty payload → honest zero state, never fabricated numbers.
const empty = await page.evaluate(async () => {
const orig = window.fetch;
const EMPTY = { updatedAt: '', live: false, window: { minutes: 60, since: '', until: '' }, points: [], totals: { rps_1m: 0, rpm_60m: 0, top_countries: [] } };
window.fetch = (async () => ({ ok: true, status: 200, json: async () => EMPTY })) as typeof window.fetch;
const { TrafficGlobePanel } = await import('/src/components/TrafficGlobePanel.ts');
const panel = new TrafficGlobePanel();
const root = panel.getElement();
document.body.appendChild(root);
const deadline = Date.now() + 3000;
while (Date.now() < deadline && !(root.textContent ?? '').includes('No live traffic')) {
await new Promise((r) => setTimeout(r, 30));
}
const text = root.textContent ?? '';
panel.destroy(); root.remove(); window.fetch = orig;
return text;
});
expect(empty).toContain('No live traffic yet');
});
test('3D globe settles (no idle-spin flicker) + Cloud-accurate legend, no cables assertion', async ({ page }) => {
await page.route('**/v1/world/cloud/traffic-globe', (route) =>
route.fulfill({ status: 200, contentType: 'application/json', body: JSON.stringify(TRAFFIC) }),
);
const errors: string[] = [];
page.on('console', (m) => { if (m.type() === 'error') errors.push(m.text()); });
await page.goto('/?variant=cloud');
await expect(page.locator('.deckgl-map-wrapper')).toBeVisible({ timeout: 45000 });
// Legend must MATCH what the globe plots — the Cloud data classes, not the
// geopolitical default (which would mislabel a traffic dot as "high alert").
const legend = page.locator('.deckgl-legend');
await expect(legend).toBeVisible();
const legendText = (await legend.innerText()).toLowerCase();
for (const cls of ['request origin', 'validator node', 'gpu fleet', 'cloud region', 'datacenter']) {
expect(legendText).toContain(cls);
}
for (const geo of ['high alert', 'nuclear', 'elevated']) {
expect(legendText).not.toContain(geo);
}
// Switch to the 3D globe (GlobeNative is exposed on window in dev/e2e).
await clickMapControl(page, '.deckgl-projection-toggle .proj-btn[data-mode="3d"]');
await expect
.poll(() => page.evaluate(() => Boolean((window as unknown as { __globeNative?: unknown }).__globeNative)), { timeout: 20000 })
.toBe(true);
// Idle auto-rotate is OFF, so the globe SETTLES: the camera longitude does not
// drift and the deck render loop idles (a few data-sync renders, not ~40/s). A
// perpetually-spinning globe re-rasterizes thin vector lines every frame — that is
// the shimmer; a settled globe does not.
const settled = await page.evaluate(async () => {
const g = (window as unknown as {
__globeNative: { getCenter: () => { lon: number }; getDeck: () => { setProps: (p: unknown) => void } };
}).__globeNative;
const c0 = g.getCenter();
let renders = 0;
g.getDeck().setProps({ onAfterRender: () => { renders++; } });
await new Promise((r) => setTimeout(r, 1300));
return { lonDelta: Math.abs(g.getCenter().lon - c0.lon), renders };
});
expect(settled.lonDelta).toBeLessThan(0.001); // no idle auto-rotate drift
expect(settled.renders).toBeLessThan(12); // render loop idles (vs ~40/s spinning)
// The broken cables PathLayer must not assert (cables off in Cloud + path guarded).
expect(errors.filter((e) => /cables-layer|assertion failed/i.test(e))).toEqual([]);
});
});
+69
View File
@@ -0,0 +1,69 @@
import { expect, test } from '@playwright/test';
// Mounts the real EnsoTrainingPanel against a stubbed router-stats payload and
// asserts the rendered DOM: opaque "Enso arm N" labels (NEVER a vendor name),
// the cost-saved headline, the retrain gate verdict, and the honest "—" shadow
// state. Mirrors e2e/investments-panel.spec.ts (runtime-harness).
test.describe('Enso Live Training panel', () => {
test('renders opaque arms, cost-saved headline and retrain gate — no vendor names', async ({ page }) => {
await page.goto('/tests/runtime-harness.html');
const result = await page.evaluate(async () => {
const CANNED = {
scope: 'platform',
window: { since: '2026-07-15T00:00:00Z', until: '2026-07-16T00:00:00Z', events: 48210 },
cost: { saved_pct: 21.5, cumulative_saved_index: 1372, baseline_model: 'arm-1', priced_events: 41000 },
quality: { reward_rate: 0.31, rewarded_events: 1200, engine_share: 0.62, avg_confidence: 0.74, shadow_agreement: null },
by_task: { chat: { events: 30000, models: { 'arm-1': 18000, 'arm-2': 12000 } } },
by_model: { 'arm-1': 30000, 'arm-2': 14000, 'arm-3': 4210 },
throughput: { per_hour: Array.from({ length: 24 }, (_, i) => 1500 + i * 40), total_window: 48210 },
retrain: {
version: 'router-2026.07.16', trained_time: '2026-07-16T00:00:00Z', events: 48210,
gate_passed: true, published: true, gate_kind: 'holdout', gate_metric: 'reward',
gate_value: 0.312, gate_base: 0.298, note: 'shipped',
},
};
// Stub the same-origin proxy the service calls.
const origFetch = window.fetch;
window.fetch = (async () =>
({ ok: true, status: 200, json: async () => CANNED })) as typeof window.fetch;
const { EnsoTrainingPanel } = await import('/src/components/EnsoTrainingPanel.ts');
const panel = new EnsoTrainingPanel();
const root = panel.getElement();
document.body.appendChild(root);
// Wait for the async fetch → render to land.
const deadline = Date.now() + 3000;
while (Date.now() < deadline && root.querySelectorAll('.cloud-model-row').length === 0) {
await new Promise((r) => setTimeout(r, 30));
}
const armLabels = Array.from(root.querySelectorAll('.cloud-model-name')).map(
(n) => (n.textContent ?? '').trim(),
);
const text = root.textContent ?? '';
panel.destroy();
root.remove();
window.fetch = origFetch;
return { armLabels, text };
});
// Three opaque arms, labeled by tier — never a vendor name.
expect(result.armLabels.length).toBe(3);
expect(result.armLabels[0]).toContain('Enso arm 1');
expect(result.armLabels[1]).toContain('Enso arm 2');
expect(result.armLabels[2]).toContain('Enso arm 3');
for (const vendor of ['claude', 'anthropic', 'gpt', 'openai', 'deepseek', 'qwen', 'gemini', 'llama']) {
expect(result.text.toLowerCase()).not.toContain(vendor);
}
// Headline cost-saved + retrain gate verdict + honest empty shadow state.
expect(result.text).toContain('21.5%');
expect(result.text).toContain('passed');
expect(result.text).toContain('Shadow-vs-served agreement: —');
});
});
+22
View File
@@ -24,6 +24,8 @@ type GlobeHarness = {
setCamera: (lon: number, lat: number, zoom: number) => void;
stopSpin: () => void;
pickAtLonLat: (lon: number, lat: number, radius?: number) => PickResult;
setBasemapStyle: (style: 'dark' | 'satellite' | 'terrain') => void;
getBasemapStyle: () => string;
destroy: () => void;
};
@@ -111,4 +113,24 @@ test.describe('native deck.gl GlobeView', () => {
await page.evaluate(() => (window as HarnessWindow).__globeHarness!.nativeEnabled),
).toBe(false);
});
test('basemap style switches dark/satellite/terrain, stays single-context, data still picks', async ({
page,
}) => {
await ready(page, '?globe=native');
const setStyle = (s: 'dark' | 'satellite' | 'terrain') =>
page.evaluate((style) => (window as HarnessWindow).__globeHarness!.setBasemapStyle(style), s);
const getStyle = () =>
page.evaluate(() => (window as HarnessWindow).__globeHarness!.getBasemapStyle());
for (const style of ['satellite', 'terrain', 'dark'] as const) {
await setStyle(style);
expect(await getStyle()).toBe(style);
// Draping imagery must not spawn a second WebGL context...
expect(await page.evaluate(() => document.querySelectorAll('canvas').length)).toBe(1);
// ...and the data layers keep rendering ON TOP (still pickable on the sphere).
await expect.poll(async () => (await pickHotspot(page)).found, { timeout: 15000 }).toBe(true);
}
});
});
+4 -3
View File
@@ -1,4 +1,5 @@
import { expect, test } from '@playwright/test';
import { clickMapControl } from './helpers/map-controls';
// P0-1: deck.gl overlays must actually RENDER on the mapbox-gl v3 globe.
//
@@ -110,7 +111,7 @@ test.describe('P0-1 deck overlay renders on the globe', () => {
// → 3D globe. Deck's active viewport must be a GlobeViewport (proves deck is
// reprojecting onto the sphere) and the dot must still pick on the globe.
await page.locator('.deckgl-projection-toggle .proj-btn[data-mode="3d"]').click();
await clickMapControl(page, '.deckgl-projection-toggle .proj-btn[data-mode="3d"]');
await expect.poll(() => projection(page), { timeout: 30000 }).toBe('globe');
await page.evaluate(() => (window as HarnessWindow).__mapHarness!.stopIdleSpin());
await expect.poll(() => viewportType(page), { timeout: 20000 }).toMatch(/Globe/i);
@@ -128,10 +129,10 @@ test.describe('P0-1 deck overlay renders on the globe', () => {
await testInfo.attach('globe-3d-dots', { body: shot, contentType: 'image/png' });
// Round-trip 2D → 3D again; picks must survive each transition.
await page.locator('.deckgl-projection-toggle .proj-btn[data-mode="2d"]').click();
await clickMapControl(page, '.deckgl-projection-toggle .proj-btn[data-mode="2d"]');
await expect.poll(() => projection(page), { timeout: 20000 }).toBe('mercator');
await pollPickFound(page);
await page.locator('.deckgl-projection-toggle .proj-btn[data-mode="3d"]').click();
await clickMapControl(page, '.deckgl-projection-toggle .proj-btn[data-mode="3d"]');
await expect.poll(() => projection(page), { timeout: 20000 }).toBe('globe');
await page.evaluate(() => (window as HarnessWindow).__mapHarness!.stopIdleSpin());
await pollPickFound(page);
+207
View File
@@ -0,0 +1,207 @@
import { expect, test } from '@playwright/test';
import { clickMapControl } from './helpers/map-controls';
// Globe render-fix acceptance suite (world.hanzo.ai Hanzo-Cloud view).
//
// Guards the three "spazz" bugs the CTO reported and the 2D/3D parity + live-analytics
// contract:
// 1. Overlays must sit ON the globe surface with correct occlusion — a back-hemisphere
// dot/badge is HIDDEN, a front one is drawn (no floating layers above the sphere).
// 2. Terrain drapes on the globe in a single WebGL context (no second canvas, no
// striping regression — depth-write disabled on the coplanar imagery tiles).
// 3. Every cloud data layer mounts in BOTH 2D (mercator) and 3D (globe) — parity.
// Plus: the REAL live request-geo dots appear and update.
//
// Data is mocked at the same-origin /v1/world/cloud/* contracts so the run is
// deterministic and offline (production serves the real, live payloads).
type DeckLayer = { id: string; props: Record<string, unknown> };
type Rgba = [number, number, number, number];
const cloudMap = {
// Two request-origin points on OPPOSITE hemispheres so occlusion is testable:
// FRONT (lon 0) faces a camera centred at lon 0; BACK (lon 180) is behind the globe.
trafficGlobe: {
updatedAt: '2026-07-18T12:00:00Z',
live: true,
window: { minutes: 60, since: '2026-07-18T11:00:00Z', until: '2026-07-18T12:00:00Z' },
points: [
{ country: 'FR', lat: 20, lon: 0, count: 90, byService: { models: 90 } },
{ country: 'US', lat: 37.09, lon: -95.71, count: 42, byService: { models: 42 } },
{ country: 'JP', lat: 20, lon: 180, count: 30, byService: { models: 30 } },
],
totals: { rps_1m: 1.6, rpm_60m: 96, top_countries: [{ country: 'FR', count: 90 }, { country: 'US', count: 42 }, { country: 'JP', count: 30 }] },
},
traffic: {
updatedAt: '2026-07-18T12:00:00Z', demo: false,
arcs: [
{ fromLat: 20, fromLon: 0, toLat: 51.5, toLon: -0.12, weight: 1, label: 'FR → lon' },
{ fromLat: 37.09, fromLon: -95.71, toLat: 40.71, toLon: -74.0, weight: 0.6, label: 'US → nyc' },
],
},
chainNodes: {
updatedAt: '2026-07-18T12:00:00Z', positionsModeled: true,
networks: [{ id: 'lux', name: 'Lux Network', chainId: 96369, blockHeight: 1096461, peers: 3, live: true,
nodes: [{ lat: 40.71, lon: -74.0, city: 'New York', kind: 'validator' }, { lat: 37.77, lon: -122.42, city: 'San Francisco', kind: 'validator' }] }],
},
byoGpu: { updatedAt: '2026-07-18T12:00:00Z', demo: false, gpus: [] },
};
async function mockCloud(page: import('@playwright/test').Page, traffic = cloudMap.trafficGlobe): Promise<void> {
const json = (body: unknown) => ({ status: 200, contentType: 'application/json', body: JSON.stringify(body) });
await page.route('**/v1/world/cloud/traffic-globe', (r) => r.fulfill(json(traffic)));
await page.route('**/v1/world/cloud/traffic', (r) => r.fulfill(json(cloudMap.traffic)));
await page.route('**/v1/world/cloud/chain-nodes', (r) => r.fulfill(json(cloudMap.chainNodes)));
await page.route('**/v1/world/cloud/byo-gpu', (r) => r.fulfill(json(cloudMap.byoGpu)));
}
const layerIds = (page: import('@playwright/test').Page): Promise<string[]> =>
page.evaluate(() => {
const m = (window as unknown as { __deckMap?: { asGlobeSource: () => { buildLayers: () => DeckLayer[] } } }).__deckMap;
return (m?.asGlobeSource().buildLayers() ?? []).flat(Infinity).filter(Boolean).map((l: DeckLayer) => l.id);
});
const go3D = async (page: import('@playwright/test').Page): Promise<void> => {
await clickMapControl(page, '.deckgl-projection-toggle .proj-btn[data-mode="3d"]');
await expect
.poll(() => page.evaluate(() => Boolean((window as unknown as { __globeNative?: unknown }).__globeNative)), { timeout: 25000 })
.toBe(true);
await page.waitForTimeout(1200); // let the first data-sync pull + push layers
};
test.describe('Globe render fixes — occlusion, terrain, 2D/3D parity, live dots', () => {
test.describe.configure({ retries: 1 });
test.use({ reducedMotion: 'reduce' });
// The cloud data layers that must exist on the Hanzo-Cloud globe once feeds resolve.
const CLOUD_LAYERS = ['traffic', 'trafficArcs', 'chainNodes', 'datacenter-clusters-layer'];
test('3D: cloud layers mount, live dots render on the sphere, no WebGL errors', async ({ page }, testInfo) => {
const errors: string[] = [];
const ignorable = [/could not compile fragment shader/i, /image.*could not be decoded/i, /the layer 'background'/i, /status of 40[13]/i];
page.on('console', (m) => { if (m.type() === 'error') errors.push(m.text()); });
page.on('pageerror', (e) => errors.push(e.message));
await mockCloud(page);
await page.goto('/?variant=cloud');
await expect(page.locator('.deckgl-map-wrapper')).toBeVisible({ timeout: 45000 });
await go3D(page);
// Single WebGL context for the globe (deck GlobeView, not a 2nd overlay canvas).
expect(await page.evaluate(() => document.querySelectorAll('.globe-native-canvas').length)).toBe(1);
await expect.poll(() => page.evaluate(() => (window as unknown as { __globeNative: { getViewportType: () => string | null } }).__globeNative.getViewportType())).toMatch(/Globe/i);
// Every cloud data layer is mounted.
const ids = await layerIds(page);
for (const id of CLOUD_LAYERS) expect(ids).toContain(id);
// The live request-geo dots are present with the real (mocked) point count.
const trafficCount = await page.evaluate(() => {
const m = (window as unknown as { __deckMap: { asGlobeSource: () => { buildLayers: () => DeckLayer[] } } }).__deckMap;
const t = m.asGlobeSource().buildLayers().flat(Infinity).find((l: DeckLayer) => l?.id === 'traffic');
return (t?.props?.data as unknown[])?.length ?? 0;
});
expect(trafficCount).toBe(cloudMap.trafficGlobe.points.length);
const shot = await page.locator('.globe-native-wrapper').screenshot();
await testInfo.attach('3d-globe-cloud-layers', { body: shot, contentType: 'image/png' });
expect(errors.filter((e) => !ignorable.some((p) => p.test(e)))).toEqual([]);
});
test('occlusion: a back-hemisphere dot is culled to transparent, a front dot is opaque', async ({ page }) => {
await mockCloud(page);
await page.goto('/?variant=cloud');
await expect(page.locator('.deckgl-map-wrapper')).toBeVisible({ timeout: 45000 });
await go3D(page);
// Face the camera at lon 0 (the FRONT point). Read the REAL traffic layer's fill
// accessor for each point: front → opaque (alpha>0), back (lon 180) → alpha 0.
const alphas = await page.evaluate(() => {
const m = (window as unknown as { __deckMap: { setOcclusionCenter: (lng: number, lat: number) => void; asGlobeSource: () => { buildLayers: () => DeckLayer[] } } }).__deckMap;
m.setOcclusionCenter(0, 20);
const t = m.asGlobeSource().buildLayers().flat(Infinity).find((l: DeckLayer) => l?.id === 'traffic') as DeckLayer;
const data = t.props.data as Array<{ lon: number }>;
const getFill = t.props.getFillColor as (d: unknown) => Rgba;
const front = getFill(data.find((d) => d.lon === 0));
const back = getFill(data.find((d) => d.lon === 180));
return { frontAlpha: front[3], backAlpha: back[3] };
});
expect(alphas.frontAlpha).toBeGreaterThan(0);
expect(alphas.backAlpha).toBe(0);
});
test('occlusion: a back-hemisphere count badge is culled (no floating badge over the globe)', async ({ page }) => {
await mockCloud(page);
await page.goto('/?variant=cloud');
await expect(page.locator('.deckgl-map-wrapper')).toBeVisible({ timeout: 45000 });
await go3D(page);
// datacenter count badges are a TextLayer; a back-side badge's glyph AND pill must
// both go transparent. Probe both hemispheres against a fixed camera centre.
const badge = await page.evaluate(() => {
const m = (window as unknown as { __deckMap: { setOcclusionCenter: (lng: number, lat: number) => void; asGlobeSource: () => { buildLayers: () => DeckLayer[] } } }).__deckMap;
m.setOcclusionCenter(0, 20);
const b = m.asGlobeSource().buildLayers().flat(Infinity).find((l: DeckLayer) => l?.id === 'datacenter-clusters-badge') as DeckLayer | undefined;
if (!b) return { skip: true };
const getColor = b.props.getColor as (d: unknown) => Rgba;
const getBg = b.props.getBackgroundColor as (d: unknown) => Rgba;
const front = { lon: 0, lat: 20, count: 5 };
const back = { lon: 180, lat: 20, count: 5 };
return { skip: false, frontText: getColor(front)[3], backText: getColor(back)[3], backBg: getBg(back)[3] };
});
if (badge.skip) test.skip(true, 'no datacenter badge in this build');
expect(badge.frontText).toBeGreaterThan(0);
expect(badge.backText).toBe(0);
expect(badge.backBg).toBe(0);
});
test('2D/3D parity: every cloud layer that mounts in 3D also mounts in 2D', async ({ page }, testInfo) => {
await mockCloud(page);
await page.goto('/?variant=cloud');
await expect(page.locator('.deckgl-map-wrapper')).toBeVisible({ timeout: 45000 });
await page.waitForTimeout(1500); // 2D feeds settle
const ids2d = await layerIds(page);
for (const id of CLOUD_LAYERS) expect(ids2d, `2D missing ${id}`).toContain(id);
await testInfo.attach('2d-map-cloud-layers', { body: await page.locator('.deckgl-map-wrapper').screenshot(), contentType: 'image/png' });
await go3D(page);
const ids3d = await layerIds(page);
for (const id of CLOUD_LAYERS) expect(ids3d, `3D missing ${id}`).toContain(id);
await testInfo.attach('3d-globe-cloud-layers-parity', { body: await page.locator('.globe-native-wrapper').screenshot(), contentType: 'image/png' });
});
test('terrain drapes on the globe in a single context (no second canvas)', async ({ page }, testInfo) => {
await mockCloud(page);
await page.goto('/?variant=cloud');
await expect(page.locator('.deckgl-map-wrapper')).toBeVisible({ timeout: 45000 });
await go3D(page);
await page.evaluate(() => (window as unknown as { __globeNative: { setBasemapStyle: (s: string) => void } }).__globeNative.setBasemapStyle('terrain'));
await expect
.poll(() => page.evaluate(() => (window as unknown as { __globeNative: { getDeck: () => { props: { layers: DeckLayer[] } } } }).__globeNative.getDeck().props.layers.flat(Infinity).some((l: DeckLayer) => l?.id?.startsWith('globe-imagery-terrain'))), { timeout: 20000 })
.toBe(true);
// Still exactly one canvas — imagery drapes as deck tiles, not a 2nd WebGL context.
expect(await page.evaluate(() => document.querySelectorAll('.globe-native-wrapper canvas').length)).toBe(1);
await page.waitForTimeout(2500);
await testInfo.attach('3d-terrain-globe', { body: await page.locator('.globe-native-wrapper').screenshot(), contentType: 'image/png' });
});
test('live dots update when the feed changes', async ({ page }) => {
await mockCloud(page, { ...cloudMap.trafficGlobe, points: cloudMap.trafficGlobe.points.slice(0, 1) });
await page.goto('/?variant=cloud');
await expect(page.locator('.deckgl-map-wrapper')).toBeVisible({ timeout: 45000 });
await go3D(page);
const count = () => page.evaluate(() => {
const m = (window as unknown as { __deckMap: { asGlobeSource: () => { buildLayers: () => DeckLayer[] } } }).__deckMap;
const t = m.asGlobeSource().buildLayers().flat(Infinity).find((l: DeckLayer) => l?.id === 'traffic');
return (t?.props?.data as unknown[])?.length ?? 0;
});
await expect.poll(count, { timeout: 15000 }).toBe(1);
// Swap the feed to the full 3-point payload; the DeckGLMap poll picks it up.
await mockCloud(page, cloudMap.trafficGlobe);
await expect.poll(count, { timeout: 20000 }).toBe(cloudMap.trafficGlobe.points.length);
});
});
+3 -2
View File
@@ -1,4 +1,5 @@
import { expect, test } from '@playwright/test';
import { clickMapControl } from './helpers/map-controls';
type LayerSnapshot = { id: string; dataCount: number };
@@ -160,7 +161,7 @@ test.describe('3D globe', () => {
}
// Click the real 3D pill in the map header.
await page.locator('.deckgl-projection-toggle .proj-btn[data-mode="3d"]').click();
await clickMapControl(page, '.deckgl-projection-toggle .proj-btn[data-mode="3d"]');
await expect.poll(() => projectionType(page), { timeout: 30000 }).toBe('globe');
await expect(page.locator('.deckgl-projection-toggle .proj-btn[data-mode="3d"]')).toHaveClass(/active/);
@@ -186,7 +187,7 @@ test.describe('3D globe', () => {
}
// Flip back to the flat map.
await page.locator('.deckgl-projection-toggle .proj-btn[data-mode="2d"]').click();
await clickMapControl(page, '.deckgl-projection-toggle .proj-btn[data-mode="2d"]');
await expect.poll(() => projectionType(page), { timeout: 20000 }).toBe('mercator');
expect(pageErrors.filter((e) => !ignorable.some((p) => p.test(e)))).toEqual([]);
+18
View File
@@ -0,0 +1,18 @@
import type { Page, Locator } from '@playwright/test';
// The map's projection / basemap / time-range controls collapsed into dropdowns:
// each option button (.proj-btn / .style-btn / .time-btn) lives in a popover that a
// trigger opens. Tests that click an option must open its dropdown first. This helper
// opens the button's parent .deckgl-dd (if it's in one) and clicks the option — a
// no-op open when the button isn't inside a dropdown, so it's safe everywhere.
export async function clickMapControl(page: Page, buttonSelector: string): Promise<void> {
const btn: Locator = page.locator(buttonSelector).first();
const dd = btn.locator('xpath=ancestor::div[contains(concat(" ", normalize-space(@class), " "), " deckgl-dd ")][1]');
if (await dd.count()) {
const trigger = dd.locator('.dd-trigger').first();
if (!(await dd.evaluate((el) => el.classList.contains('open')).catch(() => false))) {
await trigger.click();
}
}
await btn.click();
}
+9 -5
View File
@@ -1,6 +1,7 @@
import { expect, test, type Page } from '@playwright/test';
import { mkdirSync } from 'node:fs';
import { join } from 'node:path';
import { clickMapControl } from './helpers/map-controls';
// Capture the CTO-facing proof screenshots for the layout/style batch. These drive
// the REAL app (not a harness) so they show the shipped chrome + layout. They are
@@ -33,7 +34,7 @@ test.describe('layout/style batch — deliverable screenshots', () => {
await boot(page);
// Globe on.
await page.locator('.deckgl-projection-toggle .proj-btn[data-mode="3d"]').click().catch(() => {});
await clickMapControl(page, '.deckgl-projection-toggle .proj-btn[data-mode="3d"]').catch(() => {});
await page.waitForTimeout(1200);
// Enter immersive.
@@ -59,19 +60,22 @@ test.describe('layout/style batch — deliverable screenshots', () => {
test('basemap style switcher (dark / satellite / terrain)', async ({ page }) => {
await page.setViewportSize({ width: 1280, height: 800 });
await boot(page);
// The basemap control collapsed into a dropdown: assert its trigger, then open it.
await expect(page.locator('.deckgl-dd[data-dd="basemap"] .dd-trigger')).toBeVisible();
await page.locator('.deckgl-dd[data-dd="basemap"] .dd-trigger').click();
await expect(page.locator('.deckgl-style-switcher')).toBeVisible();
await shot(page, 'style-switcher-dark.png');
// Satellite + terrain need a Mapbox token (VITE_MAPBOX_TOKEN); the buttons are
// disabled without one. When a token is configured, actually switch and let the
// relief render before capturing.
// relief render before capturing. clickMapControl reopens the dropdown each time
// (picking an option closes it).
const sat = page.locator('.deckgl-style-switcher .style-btn[data-style="satellite"]');
if (!(await sat.isDisabled())) {
await sat.click();
await clickMapControl(page, '.deckgl-style-switcher .style-btn[data-style="satellite"]');
await page.waitForTimeout(4000);
await shot(page, 'style-switcher-satellite.png');
const terrain = page.locator('.deckgl-style-switcher .style-btn[data-style="terrain"]');
await terrain.click();
await clickMapControl(page, '.deckgl-style-switcher .style-btn[data-style="terrain"]');
await page.waitForTimeout(4000);
await shot(page, 'style-switcher-terrain.png');
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

+576
View File
@@ -4,6 +4,9 @@ import { expect, test, type Page } from '@playwright/test';
// FLIP reflow, snapping resize) through the deterministic harness.
const HARNESS = '/tests/panel-drag-harness.html';
const LAYOUT_HARNESS = '/tests/layout-harness.html';
// Screenshots land in a repo-relative artifacts dir (Playwright creates it).
const SHOTS = 'e2e/layout-shots';
interface PanelDragHarness {
ready: boolean;
@@ -12,9 +15,31 @@ interface PanelDragHarness {
order: () => string[];
}
interface Rect {
left: number;
top: number;
width: number;
height: number;
}
interface LayoutHarness {
ready: boolean;
mode: () => 'grid' | 'free';
setMode: (m: 'grid' | 'free') => void;
toggle: () => 'grid' | 'free';
cell: () => number;
setCell: (px: number) => void;
rect: (id: string) => Rect | null;
colStep: () => { step: number; cols: number; padL: number };
order: () => string[];
overlayVisible: () => boolean;
gridColumnOf: (id: string) => string;
}
declare global {
interface Window {
__panelDragHarness?: PanelDragHarness;
__layoutHarness?: LayoutHarness;
}
}
@@ -103,3 +128,554 @@ test.describe('panel drag + resize', () => {
expect(span).toBe(2);
});
});
// ── Layout engine: grid ⇄ free, corner resize, cell-size, overlay ──────────
// Drives the real Panel grips + grid-config against the true main.css, at
// 1440x900 (see playwright.layout.config.ts).
const lh = async (page: Page): Promise<void> => {
await page.goto(LAYOUT_HARNESS);
await page.waitForFunction(() => window.__layoutHarness?.ready === true);
await page.waitForTimeout(60); // let the queued registerPanel microtasks settle
};
const rect = (page: Page, id: string): Promise<Rect> =>
page.evaluate((pid) => window.__layoutHarness!.rect(pid)!, id);
const headerBox = async (page: Page, id: string) =>
(await page.locator(`[data-panel="${id}"] .panel-header`).first().boundingBox())!;
test.describe('layout engine', () => {
// Gate viewport: 1440x900 (independent of the base config's default size).
test.use({ viewport: { width: 1440, height: 900 } });
test('grid mode: a dropped panel lands on a cell boundary', async ({ page }) => {
await lh(page);
expect(await page.evaluate(() => window.__layoutHarness!.mode())).toBe('grid');
const src = await headerBox(page, 'charlie');
const dst = (await page.locator('[data-panel="echo"]').boundingBox())!;
await page.mouse.move(src.x + 30, src.y + src.height / 2);
await page.mouse.down();
await page.mouse.move(src.x + 60, src.y + src.height / 2, { steps: 4 });
await page.mouse.move(dst.x + dst.width * 0.8, dst.y + dst.height / 2, { steps: 16 });
await page.mouse.up();
// Final resting position is snapped to a grid cell (multiple of the column step).
const { step, padL } = await page.evaluate(() => window.__layoutHarness!.colStep());
const r = await rect(page, 'charlie');
const k = Math.round((r.left - padL) / step);
expect(Math.abs(r.left - padL - k * step)).toBeLessThan(4);
await page.screenshot({ path: `${SHOTS}/grid-snap.png` });
});
test('grid mode: bottom-right corner resizes width + height (snapped)', async ({ page }) => {
await lh(page);
const corner = (await page
.locator('[data-panel="delta"] .panel-corner-resize-handle.se')
.boundingBox())!;
const { step } = await page.evaluate(() => window.__layoutHarness!.colStep());
await page.mouse.move(corner.x + corner.width / 2, corner.y + corner.height / 2);
await page.mouse.down();
// Pull out ~1.5 columns wide and ~250px tall.
await page.mouse.move(
corner.x + corner.width / 2 + step * 1.5,
corner.y + corner.height / 2 + 250,
{ steps: 16 },
);
await page.mouse.up();
// Height grew (fine 16px row grid → a data-span well above the ~100px min) and
// width snapped to multiple columns. Resize lands on the fine grid, not the
// coarse span-N tier classes, so the panel carries `resized` + a data-span.
await expect(page.locator('[data-panel="delta"]')).toHaveClass(/resized/);
const span = await page.evaluate(() =>
parseInt(document.querySelector<HTMLElement>('[data-panel="delta"]')!.dataset.span ?? '0', 10),
);
expect(span).toBeGreaterThan(5); // taller than its start via the fine row grid
const gc = await page.evaluate(() => window.__layoutHarness!.gridColumnOf('delta'));
expect(gc).toMatch(/span [2-9]/);
await page.screenshot({ path: `${SHOTS}/resized-from-corner.png` });
});
test('grid mode: overlay appears only while dragging', async ({ page }) => {
await lh(page);
expect(await page.evaluate(() => window.__layoutHarness!.overlayVisible())).toBe(false);
const src = await headerBox(page, 'bravo');
await page.mouse.move(src.x + 30, src.y + src.height / 2);
await page.mouse.down();
await page.mouse.move(src.x + 120, src.y + 40, { steps: 8 });
// Mid-drag: the faint track overlay is shown.
await expect
.poll(() => page.evaluate(() => window.__layoutHarness!.overlayVisible()))
.toBe(true);
await page.screenshot({ path: `${SHOTS}/grid-overlay.png` });
await page.mouse.up();
await expect
.poll(() => page.evaluate(() => window.__layoutHarness!.overlayVisible()))
.toBe(false);
});
test('grid mode: changing cell size re-snaps the grid', async ({ page }) => {
await lh(page);
const before = await page.evaluate(() => window.__layoutHarness!.colStep());
await page.evaluate(() => window.__layoutHarness!.setCell(240));
await page.waitForTimeout(50);
const after = await page.evaluate(() => window.__layoutHarness!.colStep());
// Wider cells ⇒ fewer, wider columns: the panels re-snap to a new track grid.
expect(after.step).toBeGreaterThan(before.step + 20);
expect(after.cols).toBeLessThanOrEqual(before.cols);
expect(await page.evaluate(() => window.__layoutHarness!.cell())).toBe(240);
});
test('free mode: pixel drag + corner resize persist across reload', async ({ page }) => {
await lh(page);
await page.evaluate(() => window.__layoutHarness!.setMode('free'));
await page.waitForTimeout(40);
expect(await page.evaluate(() => window.__layoutHarness!.mode())).toBe('free');
// Hold Alt to bypass grid snapping — this test asserts that exact arbitrary-pixel
// geometry survives a reload (snapping has its own tests below).
await page.keyboard.down('Alt');
// Drag alpha by an arbitrary (non-cell) pixel delta.
const start = await rect(page, 'alpha');
const hdr = await headerBox(page, 'alpha');
const DX = 223;
const DY = -137;
await page.mouse.move(hdr.x + 30, hdr.y + hdr.height / 2);
await page.mouse.down();
await page.mouse.move(hdr.x + 40, hdr.y + hdr.height / 2, { steps: 3 });
await page.mouse.move(hdr.x + 30 + DX, hdr.y + hdr.height / 2 + DY, { steps: 16 });
await page.mouse.up();
const moved = await rect(page, 'alpha');
expect(Math.abs(moved.left - (start.left + DX))).toBeLessThan(6);
expect(Math.abs(moved.top - (start.top + DY))).toBeLessThan(6);
// Arbitrary pixel position — not snapped to a cell.
await page.screenshot({ path: `${SHOTS}/free-form.png` });
// Resize from the corner to an arbitrary size.
const corner = (await page
.locator('[data-panel="alpha"] .panel-corner-resize-handle.se')
.boundingBox())!;
const WD = 118;
const HD = 94;
await page.mouse.move(corner.x + corner.width / 2, corner.y + corner.height / 2);
await page.mouse.down();
await page.mouse.move(
corner.x + corner.width / 2 + WD,
corner.y + corner.height / 2 + HD,
{ steps: 16 },
);
await page.mouse.up();
const resized = await rect(page, 'alpha');
expect(Math.abs(resized.width - (moved.width + WD))).toBeLessThan(6);
expect(Math.abs(resized.height - (moved.height + HD))).toBeLessThan(6);
await page.keyboard.up('Alt');
// Reload: the mode + exact geometry are restored.
await page.reload();
await page.waitForFunction(() => window.__layoutHarness?.ready === true);
await page.waitForTimeout(80);
expect(await page.evaluate(() => window.__layoutHarness!.mode())).toBe('free');
const restored = await rect(page, 'alpha');
expect(Math.abs(restored.left - resized.left)).toBeLessThan(3);
expect(Math.abs(restored.top - resized.top)).toBeLessThan(3);
expect(Math.abs(restored.width - resized.width)).toBeLessThan(3);
expect(Math.abs(restored.height - resized.height)).toBeLessThan(3);
});
test('free mode: all four corners resize and pin the opposite corner', async ({ page }) => {
await lh(page);
await page.evaluate(() => window.__layoutHarness!.setMode('free'));
await page.waitForTimeout(40);
expect(await page.evaluate(() => window.__layoutHarness!.mode())).toBe('free');
// Alt bypasses snapping so the shrink is exactly the drag delta; this test is
// about the anchor-pinning invariant, which holds with or without snapping.
await page.keyboard.down('Alt');
const D = 60; // drag each corner inward (toward the panel centre) — always in-bounds
// corner → inward drag sign + which opposite corner must stay pinned.
const cases = [
{ panel: 'alpha', corner: 'nw', sx: 1, sy: 1, fix: 'br' },
{ panel: 'bravo', corner: 'ne', sx: -1, sy: 1, fix: 'bl' },
{ panel: 'charlie', corner: 'sw', sx: 1, sy: -1, fix: 'tr' },
{ panel: 'delta', corner: 'se', sx: -1, sy: -1, fix: 'tl' },
] as const;
for (const c of cases) {
const b = await rect(page, c.panel);
const h = (await page
.locator(`[data-panel="${c.panel}"] .panel-corner-resize-handle.${c.corner}`)
.boundingBox())!;
const px = h.x + h.width / 2;
const py = h.y + h.height / 2;
await page.mouse.move(px, py);
await page.mouse.down();
await page.mouse.move(px + c.sx * D, py + c.sy * D, { steps: 12 });
await page.mouse.up();
const a = await rect(page, c.panel);
// The grabbed corner pulled inward on both axes → the panel shrank.
expect(a.width).toBeLessThan(b.width - 20);
expect(a.height).toBeLessThan(b.height - 20);
// The OPPOSITE corner never moved — proof the resize is anchor-aware
// (nw/ne/sw shift left/top to hold the far edge, not just grow w/h).
const bR = b.left + b.width;
const bB = b.top + b.height;
const aR = a.left + a.width;
const aB = a.top + a.height;
if (c.fix === 'br') {
expect(Math.abs(aR - bR)).toBeLessThan(4);
expect(Math.abs(aB - bB)).toBeLessThan(4);
} else if (c.fix === 'bl') {
expect(Math.abs(a.left - b.left)).toBeLessThan(4);
expect(Math.abs(aB - bB)).toBeLessThan(4);
} else if (c.fix === 'tr') {
expect(Math.abs(aR - bR)).toBeLessThan(4);
expect(Math.abs(a.top - b.top)).toBeLessThan(4);
} else {
expect(Math.abs(a.left - b.left)).toBeLessThan(4);
expect(Math.abs(a.top - b.top)).toBeLessThan(4);
}
}
await page.keyboard.up('Alt');
await page.screenshot({ path: `${SHOTS}/free-all-corners.png` });
});
test('free mode: moving one panel never shifts its siblings (no reflow)', async ({ page }) => {
// The owner's #1 complaint: dragging the map/one panel "shifts all other
// components". In free mode each panel is independent — this proves a big drag
// of one leaves every sibling byte-for-byte where it was.
await lh(page);
await page.evaluate(() => window.__layoutHarness!.setMode('free'));
await page.waitForTimeout(40);
const siblings = ['bravo', 'charlie', 'delta', 'echo'] as const;
const before: Record<string, Rect> = {};
for (const id of siblings) before[id] = await rect(page, id);
const hdr = await headerBox(page, 'alpha');
await page.mouse.move(hdr.x + 30, hdr.y + hdr.height / 2);
await page.mouse.down();
await page.mouse.move(hdr.x + 40, hdr.y + hdr.height / 2, { steps: 3 });
await page.mouse.move(hdr.x + 300, hdr.y + hdr.height / 2 + 200, { steps: 18 });
await page.mouse.up();
for (const id of siblings) {
const a = await rect(page, id);
const b = before[id]!;
expect(Math.abs(a.left - b.left)).toBeLessThan(2);
expect(Math.abs(a.top - b.top)).toBeLessThan(2);
expect(Math.abs(a.width - b.width)).toBeLessThan(2);
expect(Math.abs(a.height - b.height)).toBeLessThan(2);
}
});
test('free mode: a panel resizes narrower than the old 160px min width', async ({ page }) => {
// The owner's #3 complaint: panels are "constrained on min width". Free mode's
// floor is now ~96px, so a panel can be pulled well under the old 160px track.
await lh(page);
await page.evaluate(() => window.__layoutHarness!.setMode('free'));
await page.waitForTimeout(40);
const before = await rect(page, 'bravo');
const corner = (await page
.locator('[data-panel="bravo"] .panel-corner-resize-handle.se')
.boundingBox())!;
// Pull the SE corner far left → collapse width toward the low floor. Hold Alt for
// fine (un-snapped) sizing — snapping would otherwise land on a whole cell (≥160).
await page.keyboard.down('Alt');
await page.mouse.move(corner.x + corner.width / 2, corner.y + corner.height / 2);
await page.mouse.down();
await page.mouse.move(corner.x - before.width, corner.y + corner.height / 2, { steps: 18 });
await page.mouse.up();
await page.keyboard.up('Alt');
const after = await rect(page, 'bravo');
expect(after.width).toBeLessThan(150); // narrower than the old 160px floor
expect(after.width).toBeGreaterThanOrEqual(90); // …but not past the new ~96px floor
});
test('free mode: the map participates with a 240px floor', async ({ page }) => {
await lh(page);
await page.evaluate(() => window.__layoutHarness!.setMode('free'));
await page.waitForTimeout(40);
const pos = await page.evaluate(
() => getComputedStyle(document.querySelector('[data-panel="map"]')!).position,
);
expect(pos).toBe('absolute');
const r = await rect(page, 'map');
expect(r.width).toBeGreaterThanOrEqual(240);
expect(r.height).toBeGreaterThanOrEqual(240);
});
test('free mode: dragging snaps to logical grid lines (panels align to shared tracks)', async ({ page }) => {
// The owner's feedback: "the snap is not logical." Two panels dragged to targets
// less than half a cell apart must land on the SAME grid line — proof placement is
// quantised to the logical grid, not arbitrary px.
await lh(page);
await page.evaluate(() => window.__layoutHarness!.setMode('free'));
await page.waitForTimeout(40);
const cell = await page.evaluate(() => window.__layoutHarness!.cell());
const colPitch = cell + 4; // + gap
const rowPitch = 16 + 4; // ROW_UNIT + gap
const dragTo = async (id: string, x: number, y: number) => {
const hdr = await headerBox(page, id);
await page.mouse.move(hdr.x + 20, hdr.y + hdr.height / 2);
await page.mouse.down();
await page.mouse.move(hdr.x + 30, hdr.y + hdr.height / 2, { steps: 3 });
await page.mouse.move(x, y, { steps: 16 });
await page.mouse.up();
};
const gridBox = (await page.locator('#panelsGrid').boundingBox())!;
const tx = gridBox.x + colPitch * 2 + 20;
const ty = gridBox.y + rowPitch * 8 + 30;
await dragTo('alpha', tx, ty);
await dragTo('bravo', tx + 30, ty + 8); // < half a cell/row from alpha's target
const a = await rect(page, 'alpha');
const b = await rect(page, 'bravo');
// Both snapped to the SAME grid line instead of sitting 30/8px apart.
expect(Math.abs(a.left - b.left)).toBeLessThanOrEqual(1);
expect(Math.abs(a.top - b.top)).toBeLessThanOrEqual(1);
});
test('free mode: resizing snaps width to whole cells and pins the opposite edge', async ({ page }) => {
await lh(page);
await page.evaluate(() => window.__layoutHarness!.setMode('free'));
await page.waitForTimeout(40);
const cell = await page.evaluate(() => window.__layoutHarness!.cell());
const GAP = 4;
const before = await rect(page, 'alpha');
const corner = (await page
.locator('[data-panel="alpha"] .panel-corner-resize-handle.se')
.boundingBox())!;
// Pull the SE corner out by a non-cell amount; width must land on a whole-cell size.
await page.mouse.move(corner.x + corner.width / 2, corner.y + corner.height / 2);
await page.mouse.down();
await page.mouse.move(corner.x + corner.width / 2 + 210, corner.y + corner.height / 2, { steps: 16 });
await page.mouse.up();
const after = await rect(page, 'alpha');
// width == N*cell + (N-1)*gap for some integer N ≥ 1.
const n = Math.round((after.width + GAP) / (cell + GAP));
expect(n).toBeGreaterThanOrEqual(1);
expect(Math.abs(after.width - (n * cell + (n - 1) * GAP))).toBeLessThanOrEqual(2);
// The SE resize pins the top-left corner — it must not have moved.
expect(Math.abs(after.left - before.left)).toBeLessThanOrEqual(1);
expect(Math.abs(after.top - before.top)).toBeLessThanOrEqual(1);
});
test('resize handles show no visible glyphs (owner request) but still resize', async ({ page }) => {
await lh(page);
await page.evaluate(() => window.__layoutHarness!.setMode('free'));
await page.waitForTimeout(40);
// The corner/edge resize glyphs are hidden (display:none on the ::after marks) —
// no visible "< >" chevrons — while the handles stay live.
const displays = await page.evaluate(() => {
const d = (sel: string) => {
const el = document.querySelector(sel);
return el ? getComputedStyle(el, '::after').display : 'missing';
};
return {
corner: d('[data-panel="alpha"] .panel-corner-resize-handle.se'),
col: d('[data-panel="alpha"] .panel-col-resize-handle'),
row: d('[data-panel="alpha"] .panel-resize-handle'),
};
});
expect(displays.corner).toBe('none');
expect(displays.col).toBe('none');
expect(displays.row).toBe('none');
// Functionality intact: the (now glyph-less) SE corner still resizes.
const before = await rect(page, 'alpha');
const corner = (await page.locator('[data-panel="alpha"] .panel-corner-resize-handle.se').boundingBox())!;
await page.mouse.move(corner.x + corner.width / 2, corner.y + corner.height / 2);
await page.mouse.down();
await page.mouse.move(corner.x + 180, corner.y + 120, { steps: 12 });
await page.mouse.up();
expect((await rect(page, 'alpha')).width).toBeGreaterThan(before.width + 40);
});
test('cell size can go down to the finer 80px floor', async ({ page }) => {
await lh(page);
await page.evaluate(() => window.__layoutHarness!.setCell(80));
await page.waitForTimeout(30);
expect(await page.evaluate(() => window.__layoutHarness!.cell())).toBe(80);
// Clamped: a request below the floor snaps back up to 80, never lower.
await page.evaluate(() => window.__layoutHarness!.setCell(40));
await page.waitForTimeout(30);
expect(await page.evaluate(() => window.__layoutHarness!.cell())).toBe(80);
});
test('toggle flips grid ⇄ free and back', async ({ page }) => {
await lh(page);
expect(await page.evaluate(() => window.__layoutHarness!.toggle())).toBe('free');
expect(await page.evaluate(() => window.__layoutHarness!.mode())).toBe('free');
expect(await page.evaluate(() => window.__layoutHarness!.toggle())).toBe('grid');
// Back in grid mode the free inline geometry is stripped.
const pos = await page.evaluate(
() => document.querySelector<HTMLElement>('[data-panel="alpha"]')!.style.position,
);
expect(pos).toBe('');
});
});
// ── Live News (video) resizes freely; the video fills the panel at any size ──
// Real app: proves the CTO requirement — Live News can grow to 2-3 cols / full
// width (grid) or any pixel size (free), and the 16:9 video (`.live-news-player`,
// width-driven) scales to fill, never capped small. NOTE: in the offline e2e
// runtime the YouTube embed eventually errors and replaces `.live-news-player`
// with a message, so the video-fill invariant is asserted where it's reliably
// present (default size) and the resize is proved via the player's containing
// block (`.panel-content`, always present), which the player fills 1:1.
const rectW = (page: Page, sel: string): Promise<number> =>
page.evaluate((s) => {
const el = document.querySelector<HTMLElement>(s);
return el ? Math.round(el.getBoundingClientRect().width) : -1;
}, sel);
const LN = '[data-panel="live-news"]';
const LN_CONTENT = `${LN} .panel-content`;
const LN_PLAYER = `${LN} .live-news-player`;
test.describe('live news video resize (real app)', () => {
test.use({ viewport: { width: 1440, height: 900 } });
test('grid: default → 3 cols → full-width; the video fills at every width', async ({ page }) => {
await page.goto('/');
await page.waitForSelector(LN, { timeout: 45000 });
await page.waitForTimeout(500);
// The app defaults to free layout now; this test asserts GRID column-span
// semantics, so pin grid explicitly (setLayoutMode marks it an explicit choice
// so the deferred default-to-free won't re-flip it).
await page.evaluate(() => (window as unknown as { worldGrid?: { setLayoutMode(m: string): void } }).worldGrid?.setLayoutMode('grid'));
await page.waitForTimeout(80);
const grid = (await page.locator('#panelsGrid').boundingBox())!;
const ln = page.locator(LN);
const colHandle = ln.locator('.panel-col-resize-handle');
// The fill setup is active: the panel-content is a full-bleed flex column
// (padding 0) so the width-driven 16:9 `.live-news-player` fills it edge-to-edge
// at any width. (Keyed on #live-news — dead until the id fix in LiveNewsPanel.)
const setup = await page.evaluate((sel) => {
const c = document.querySelector<HTMLElement>(sel);
if (!c) return null;
const s = getComputedStyle(c);
return { display: s.display, dir: s.flexDirection, padLeft: s.paddingLeft };
}, LN_CONTENT);
expect(setup).toEqual({ display: 'flex', dir: 'column', padLeft: '0px' });
// The video, while present (offline embed errors after a beat), fills the
// container 1:1 at the default width.
const startContent = await rectW(page, LN_CONTENT);
const startVideo = await rectW(page, LN_PLAYER);
if (startVideo > 0) expect(Math.abs(startVideo - startContent)).toBeLessThan(4);
// Drag the right edge out to ~3 columns.
const h1 = (await colHandle.boundingBox())!;
await page.mouse.move(h1.x + h1.width / 2, h1.y + h1.height / 2);
await page.mouse.down();
await page.mouse.move(grid.x + grid.width * 0.42, h1.y + h1.height / 2, { steps: 14 });
await page.mouse.up();
await page.waitForTimeout(200);
const midContent = await rectW(page, LN_CONTENT);
expect(midContent).toBeGreaterThan(startContent + 40); // genuinely wider
// Drag the right edge all the way out → full width. No column cap.
const h2 = (await colHandle.boundingBox())!;
await page.mouse.move(h2.x + h2.width / 2, h2.y + h2.height / 2);
await page.mouse.down();
await page.mouse.move(grid.x + grid.width + 200, h2.y + h2.height / 2, { steps: 16 });
await page.mouse.up();
await page.waitForTimeout(200);
const fullContent = await rectW(page, LN_CONTENT);
expect(fullContent).toBeGreaterThan(midContent);
expect(fullContent).toBeGreaterThan(grid.width * 0.9); // ~full grid width
// The video (when present) fills the now-full-width container 1:1.
const fullVideo = await rectW(page, LN_PLAYER);
if (fullVideo > 0) expect(Math.abs(fullVideo - fullContent)).toBeLessThan(6);
// Make it tall too (drag the bottom edge down) so the 16:9 video is large.
const bottom = ln.locator('.panel-resize-handle');
const b = (await bottom.boundingBox())!;
await page.mouse.move(b.x + b.width / 2, b.y + b.height / 2);
await page.mouse.down();
await page.mouse.move(b.x + b.width / 2, b.y + 520, { steps: 16 });
await page.mouse.up();
await page.waitForTimeout(250);
await page.evaluate(() => document.querySelector('[data-panel="live-news"]')?.scrollIntoView({ block: 'start' }));
await page.waitForTimeout(150);
await page.screenshot({ path: 'e2e/layout-shots/live-news-fullwidth.png' });
// Container is now full-width AND tall → a large video area.
expect(await rectW(page, LN_CONTENT)).toBeGreaterThan(900);
});
test('free: Live News resizes to an arbitrary pixel size; video fills', async ({ page }) => {
await page.goto('/');
await page.waitForSelector(LN_PLAYER, { timeout: 45000 });
await page.waitForTimeout(600);
const startContent = await rectW(page, LN_CONTENT);
await page.evaluate(() => (window as unknown as { worldGrid?: { setLayoutMode(m: string): void } }).worldGrid?.setLayoutMode('free'));
await page.waitForTimeout(200);
const corner = page.locator(`${LN} .panel-corner-resize-handle.se`);
const c = (await corner.boundingBox())!;
await page.mouse.move(c.x + c.width / 2, c.y + c.height / 2);
await page.mouse.down();
await page.mouse.move(c.x + 240, c.y + 260, { steps: 18 });
await page.mouse.up();
await page.waitForTimeout(200);
const content = await rectW(page, LN_CONTENT);
expect(content).toBeGreaterThan(startContent + 120); // grew to an arbitrary pixel width
const video = await rectW(page, LN_PLAYER);
if (video > 0) expect(Math.abs(video - content)).toBeLessThan(6); // fills at that size
await page.evaluate(() => (window as unknown as { worldGrid?: { setLayoutMode(m: string): void } }).worldGrid?.setLayoutMode('grid'));
});
});
// Text size / UI scale (accessibility) — real app.
test.describe('text size control (real app)', () => {
test.use({ viewport: { width: 1440, height: 900 } });
test('the dock text-size slider scales panel content and persists across reload', async ({ page }) => {
await page.goto('/');
await page.waitForSelector('[data-panel="live-news"]', { timeout: 45000 });
// Drive the dock "Text size" slider to 1.4.
await page.evaluate(() => {
const el = document.getElementById('dockFontSize') as HTMLInputElement;
el.value = '1.4';
el.dispatchEvent(new Event('input', { bubbles: true }));
});
await expect
.poll(() => page.evaluate(() => getComputedStyle(document.documentElement).getPropertyValue('--ui-scale').trim()))
.toBe('1.4');
expect(await page.evaluate(() => localStorage.getItem('hanzo-world-ui-scale'))).toBe('1.4');
const zoom = await page.evaluate(() => {
const c = document.querySelector('.panel-content');
return c ? getComputedStyle(c).zoom : '';
});
expect(parseFloat(zoom)).toBeCloseTo(1.4, 1);
// Persists across reload (applyStoredUiScale runs at boot).
await page.reload();
await page.waitForSelector('[data-panel="live-news"]', { timeout: 45000 });
expect(await page.evaluate(() => getComputedStyle(document.documentElement).getPropertyValue('--ui-scale').trim())).toBe('1.4');
});
});
Binary file not shown.

After

Width:  |  Height:  |  Size: 146 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 39 KiB

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 95 KiB

After

Width:  |  Height:  |  Size: 165 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 134 KiB

After

Width:  |  Height:  |  Size: 199 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 55 KiB

After

Width:  |  Height:  |  Size: 151 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 99 KiB

After

Width:  |  Height:  |  Size: 176 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 638 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 416 KiB

+150
View File
@@ -0,0 +1,150 @@
import { expect, test, type Page, type Route } from '@playwright/test';
/**
* UI/UX polish e2e — the four changes shipped together, each asserted against
* the real DOM/CSS rather than by eye:
*
* 1. Right-click a headline → "Summarize with AI" is the FIRST item, and it
* opens the analyst dock with the story as the question.
* 2. The app header carries no underline (border-bottom).
* 3. The map's drag pill is gone, but the grip strip is still the drag target.
* 4. "Try Hanzo" is an acquisition CTA: visible signed-out, hidden once
* identity resolves as signed-in.
*/
const SCREENS = 'e2e/screens';
async function stubWorldApi(page: Page): Promise<void> {
await page.route('**/v1/world/**', (route: Route) => {
const url = route.request().url();
if (url.includes('/v1/world/analyst')) {
return route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
reply: 'Stubbed summary.',
actions: [],
model: 'zen5',
tokens: 0,
traces: [],
}),
});
}
if (url.includes('/v1/world/models')) {
return route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({ models: [{ id: 'zen5', name: 'Zen 5' }], default: 'zen5' }),
});
}
return route.fulfill({ status: 200, contentType: 'application/json', body: '[]' });
});
}
/** Sign in the way the analyst e2e does — the analyst only accepts a question
* when identity is present, so the summarize path needs a signed-in session. */
async function signIn(page: Page): Promise<void> {
await page.addInitScript(() => {
localStorage.setItem('hanzo_iam_access_token', 'e2e-fake-token');
localStorage.setItem('hanzo_iam_expires_at', String(Date.now() + 3_600_000));
localStorage.setItem('hanzo_iam_owner', 'e2e-org');
});
}
async function appReady(page: Page): Promise<void> {
await page.goto('/');
await page.waitForSelector('.panel', { timeout: 30_000 });
}
/** Inject a news-shaped item — the exact data-ctx-* convention NewsPanel emits. */
async function addNewsItem(page: Page): Promise<void> {
await page.evaluate(() => {
const grid = document.querySelector('#panelsGrid')!;
const panel = document.createElement('div');
panel.className = 'panel';
panel.dataset.panel = 'e2e-ui-panel';
panel.innerHTML =
'<div class="panel-content"><div class="item" id="e2e-news-item" ' +
'data-ctx-url="https://example.com/story" data-ctx-headline="Nvidia unveils new GPU" ' +
'style="padding:12px;min-height:24px;">Nvidia unveils new GPU</div></div>';
grid.appendChild(panel);
});
}
test.describe('ui polish', () => {
test('headline right-click leads with "Summarize with AI" and opens the analyst', async ({ page }) => {
await stubWorldApi(page);
await signIn(page); // the analyst answers only for a signed-in identity
await appReady(page);
await addNewsItem(page);
await page.locator('#e2e-news-item').scrollIntoViewIfNeeded();
await page.locator('#e2e-news-item').click({ button: 'right' });
await expect(page.locator('#panelContextMenu')).toBeVisible();
const labels = await page
.locator('#panelContextMenu .panel-context-menu-item')
.allTextContents();
const trimmed = labels.map((l) => l.trim());
// It is the action you want on a headline → it leads the menu.
expect(trimmed[0]).toBe('Summarize with AI');
// The copy/open actions are still there, below it.
expect(trimmed).toEqual(expect.arrayContaining(['Open link', 'Copy link', 'Copy headline']));
await page.screenshot({ path: `${SCREENS}/ctxmenu-summarize.png` });
// Clicking it routes into the ONE analyst dock, pre-asked with the story.
await page
.locator('#panelContextMenu .panel-context-menu-item', { hasText: 'Summarize with AI' })
.click();
// .hzc is a 0x0 wrapper (children are position:fixed) — the panel is the
// surface that actually opens.
await expect(page.locator('.hzc-panel')).toBeVisible();
// The question carries the headline (the analyst is asked, not just opened).
await expect(page.locator('.hzc-row.user').first()).toContainText('Nvidia unveils new GPU');
});
test('app header has no underline', async ({ page }) => {
await stubWorldApi(page);
await appReady(page);
const borderBottom = await page
.locator('.header')
.first()
.evaluate((el) => getComputedStyle(el).borderBottomWidth);
expect(borderBottom).toBe('0px');
});
test('map shows no drag pill, but the grip strip still drags', async ({ page }) => {
await stubWorldApi(page);
await appReady(page);
const grip = page.locator('.map-panel .panel-header.map-drag-grip');
await expect(grip).toHaveCount(1);
// No visible indicator …
const pill = await grip.evaluate((el) => getComputedStyle(el, '::after').content);
expect(pill === 'none' || pill === 'normal').toBe(true);
// … but it is still the drag target (that is what makes hiding it safe).
await expect(grip).toHaveCSS('cursor', 'grab');
});
test('"Try Hanzo" hides once signed in', async ({ page }) => {
await stubWorldApi(page);
await appReady(page);
const cta = page.locator('.try-hanzo');
await expect(cta).toBeVisible(); // signed-out: the CTA is the point
// The one signal identity resolution emits.
await page.evaluate(() => {
document.dispatchEvent(new CustomEvent('hanzo:auth', { detail: { authed: true } }));
});
await expect(cta).toBeHidden();
// And it comes back on sign-out.
await page.evaluate(() => {
document.dispatchEvent(new CustomEvent('hanzo:auth', { detail: { authed: false } }));
});
await expect(cta).toBeVisible();
});
});
+14 -1
View File
@@ -19,6 +19,14 @@ async function appReady(page: Page): Promise<void> {
undefined,
{ timeout: 45000 },
);
// The app now DEFAULTS to free layout (independent, non-reflowing panels). These
// specs exercise the GRID reorder/ghost/row-span/reset machinery specifically, so
// pin grid mode (still a first-class, dropdown-selectable mode). setLayoutMode
// marks the choice explicit, so the deferred default-to-free never re-flips it.
await page.evaluate(() =>
(window as unknown as { worldGrid?: { setLayoutMode(m: string): void } }).worldGrid?.setLayoutMode('grid'),
);
await page.waitForTimeout(80);
}
function gridOrder(page: Page): Promise<(string | undefined)[]> {
@@ -94,7 +102,12 @@ test.describe('video panel drag + reset (live app)', () => {
await page.mouse.move(h.x + h.width / 2, h.y + h.height / 2 + 230, { steps: 14 });
await page.mouse.up();
await expect(panel).toHaveClass(/span-2/);
// Height resize lands on the fine 16px row grid (smooth ~20px steps), so the
// panel carries `resized` + a data-span well above the ~100px minSpan — not the
// coarse span-2 tier class the old assertion expected.
await expect(panel).toHaveClass(/resized/);
const span = await panel.evaluate((el) => parseInt((el as HTMLElement).dataset.span ?? '0', 10));
expect(span).toBeGreaterThan(5);
});
test('Reset layout returns the grid to the default order', async ({ page }) => {
+26 -1
View File
@@ -1,3 +1,28 @@
module github.com/hanzoai/world
go 1.23
go 1.26.4
require (
github.com/alicebob/miniredis/v2 v2.38.0
github.com/hanzoai/sqlite v0.3.0
github.com/redis/go-redis/v9 v9.20.0
)
require (
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/hanzoai/csqlite v0.1.0 // indirect
github.com/luxfi/crypto v1.19.26 // indirect
github.com/mattn/go-isatty v0.0.21 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/yuin/gopher-lua v1.1.1 // indirect
go.uber.org/atomic v1.11.0 // indirect
golang.org/x/crypto v0.52.0 // indirect
golang.org/x/sys v0.45.0 // indirect
modernc.org/libc v1.72.3 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
modernc.org/sqlite v1.51.0 // indirect
)
+84
View File
@@ -0,0 +1,84 @@
github.com/alicebob/miniredis/v2 v2.38.0 h1:nZAzCR+Lj+Vxk4ZXzm2NuKq2O33RXj1XxJ2e2uP9jiw=
github.com/alicebob/miniredis/v2 v2.38.0/go.mod h1:TcL7YfarKPGDAthEtl5NBeHZfeUQj6OXMm/+iu5cLMM=
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hanzoai/csqlite v0.1.0 h1:suwC3dh0INlfP/U0Es6cDf6JNQ+2+GVLLATPWCUux6k=
github.com/hanzoai/csqlite v0.1.0/go.mod h1:H31a/O6VXuklR9UBkgY++bmAK5uzVfXPqU0F6P9Wsos=
github.com/hanzoai/sqlite v0.3.0 h1:Siqa7b89CgSCteg5ubXOCQiPRxYIJGb9kMemwjXMIUQ=
github.com/hanzoai/sqlite v0.3.0/go.mod h1:/eumCoj+3FLbLXr3jZomamv2DdJvh4No2nfYDdlGgUU=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/luxfi/crypto v1.19.26 h1:+aHn/L479ak2ih7s/DkBZojjuhcyHBLqu3nYT81vcrU=
github.com/luxfi/crypto v1.19.26/go.mod h1:0DCU62kX8+zhYU2qeM07A4pifJyPkPujnUOfgc8TOFQ=
github.com/mattn/go-isatty v0.0.21 h1:xYae+lCNBP7QuW4PUnNG61ffM4hVIfm+zUzDuSzYLGs=
github.com/mattn/go-isatty v0.0.21/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/redis/go-redis/v9 v9.20.0 h1:WnQYxLkgO2xiXTCJY0ldIiI8dNqCDlQAG+AtaH7a2a0=
github.com/redis/go-redis/v9 v9.20.0/go.mod h1:v/M13XI1PVCDcm01VtPFOADfZtHf8YW3baQf57KlIkA=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M=
github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw=
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
golang.org/x/crypto v0.52.0 h1:RMs7fP2rXdep0CftQlK8Uf+kibLm7qkCcradZWYz988=
golang.org/x/crypto v0.52.0/go.mod h1:1QgfPxDqh0T2M/elOJtp9RvuR95kVjir0e6/BvEmGbc=
golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8=
golang.org/x/mod v0.33.0/go.mod h1:swjeQEj+6r7fODbD2cqrnje9PnziFuw4bmLbBZFrQ5w=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY=
golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s=
golang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.28.2 h1:3tQ0lf2ADtoby2EtSP+J7IE2SHwEJdP8ioR59wx7XpY=
modernc.org/cc/v4 v4.28.2/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.34.0 h1:yRLPFZieg532OT4rp4JFNIVcquwalMX26G95WQDqwCQ=
modernc.org/ccgo/v4 v4.34.0/go.mod h1:AS5WYMyBakQ+fhsHhtP8mWB82KTGPkNNJDGfGQCe0/A=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.2 h1:ZtDCnhonXSZexk/AYsegNRV1lJGgaNZJuKjJSWKyEqo=
modernc.org/gc/v3 v3.1.2/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.72.3 h1:ZnDF4tXn4NBXFutMMQC4vtbTFSXhhKzR73fv0beZEAU=
modernc.org/libc v1.72.3/go.mod h1:dn0dZNnnn1clLyvRxLxYExxiKRZIRENOfqQ8XEeg4Qs=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.51.0 h1:aH/MMSoayAIhozZ7uJbVTT9QO/VhzBf0J9tymmmuC/U=
modernc.org/sqlite v1.51.0/go.mod h1:tcNzv5p84E0skkmJn038y+hWJbLQXQqEnQfeh5r2JLM=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+6
View File
@@ -13,6 +13,12 @@ images:
context: .
dockerfile: Dockerfile
repo: ghcr.io/hanzoai/world
# Publishable Mapbox token (pk, URL-restricted to world.hanzo.ai) baked into
# the Vite SPA at build so the Satellite/Terrain basemaps load (Dark stays
# keyless CartoDB). Sourced from KMS (hanzo/deploy/VITE_MAPBOX_TOKEN,
# env=prod) by hanzoai/ci — the key name IS the build-arg. Never in git.
build_secrets:
- VITE_MAPBOX_TOKEN
# Deploy: roll the freshly-built image onto the `world` Service CR. Fires only
# on `main` (+ tags); PRs and forks build but never deploy. KMS hands back the
+178 -14
View File
@@ -3,6 +3,7 @@ package world
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
@@ -26,10 +27,11 @@ func newAIClient() *AIClient {
}
model := env("HANZO_AI_MODEL")
if model == "" {
// zen5 = the current flagship general Zen chat model. The bare "zen" alias
// is NOT a servable id (api.hanzo.ai/v1/models), so it would 4xx and force
// every AI endpoint (analyst included) into its fallback path.
model = "zen5"
// "best" is the gateway's own routing alias (owned_by hanzo in /v1/models):
// it always resolves to a servable model. A pinned family id (zen5) goes
// dark whenever the plane's claim catalog shifts — an unclaimed id falls
// through to a proxy that rejects every credential, killing all AI here.
model = "best"
}
return &AIClient{
base: strings.TrimRight(base, "/"),
@@ -112,10 +114,20 @@ type chatRequest struct {
}
type chatResponse struct {
// ID is the gateway's response id (OpenAI chat.completion `id`). It is the
// stable key a content-free reward signal is attributed to (POST /v1/feedback),
// so it is threaded back out of every completion — never the prompt/response.
ID string `json:"id"`
Choices []struct {
Message struct {
Content string `json:"content"`
// A reasoning model (gpt-oss / harmony, deepseek-r1) surfaces its working
// on a SEPARATE channel; when it emits no final content we fall back to
// this so the analyst still recovers an answer, not a blank "empty response".
ReasoningContent string `json:"reasoning_content"`
Reasoning string `json:"reasoning"`
} `json:"message"`
FinishReason string `json:"finish_reason"`
} `json:"choices"`
Usage struct {
TotalTokens int `json:"total_tokens"`
@@ -136,16 +148,21 @@ func (a *AIClient) chat(ctx context.Context, s *Server, bearer, system, user str
// prior turns) and returns the trimmed content. It is the single completion path;
// chat is the system+user special case. bearer is the caller's IAM token so the
// inference meters to their org/project/billing; extra forwards the caller's
// org/project selectors so it meters to the org the user is acting in.
// org/project selectors so it meters to the org the user is acting in. It drops
// the gateway response id — only the analyst's reward-signal path
// (runAnalystLoop → chatMessagesModel) needs it.
func (a *AIClient) chatMessages(ctx context.Context, s *Server, bearer string, messages []chatMessage, temperature float64, maxTokens int, extra map[string]string) (string, int, error) {
return a.chatMessagesModel(ctx, s, bearer, a.model, messages, temperature, maxTokens, extra)
out, tokens, _, err := a.chatMessagesModel(ctx, s, bearer, a.model, messages, temperature, maxTokens, extra)
return out, tokens, err
}
// chatMessagesModel is chatMessages with an explicit model override — the single
// completion path once a caller (the analyst's model dropdown) chooses the model.
// An empty model falls back to the client default (a.model). Everything else —
// auth forwarding, org/project selectors, degrade semantics — is identical.
func (a *AIClient) chatMessagesModel(ctx context.Context, s *Server, bearer, model string, messages []chatMessage, temperature float64, maxTokens int, extra map[string]string) (string, int, error) {
// auth forwarding, org/project selectors, degrade semantics — is identical. It
// also returns the gateway response id (chatResponse.ID) so the analyst can key a
// content-free reward signal to it; "" when the gateway omits one.
func (a *AIClient) chatMessagesModel(ctx context.Context, s *Server, bearer, model string, messages []chatMessage, temperature float64, maxTokens int, extra map[string]string) (string, int, string, error) {
if strings.TrimSpace(model) == "" {
model = a.model
}
@@ -157,7 +174,7 @@ func (a *AIClient) chatMessagesModel(ctx context.Context, s *Server, bearer, mod
TopP: 0.9,
})
if err != nil {
return "", 0, err
return "", 0, "", err
}
headers := map[string]string{
"Authorization": bearer,
@@ -170,19 +187,166 @@ func (a *AIClient) chatMessagesModel(ctx context.Context, s *Server, bearer, mod
defer cancel()
body, status, err := s.do(cctx, "POST", a.base+"/chat/completions", headers, reqBody)
if err != nil {
return "", 0, err
logf("world: ai request error: model=%s: %v", model, err)
return "", 0, "", err
}
if status < 200 || status >= 300 {
return "", status, fmt.Errorf("hanzo ai status %d", status)
e := newAIError(status, body)
logf("world: ai non-success: status=%d code=%q model=%s", status, e.code, model)
return "", status, "", e
}
var cr chatResponse
if err := json.Unmarshal(body, &cr); err != nil {
return "", status, err
logf("world: ai decode error: status=%d model=%s: %v", status, model, err)
return "", status, "", err
}
if len(cr.Choices) == 0 {
return "", status, fmt.Errorf("empty response")
// The gateway answers some failures — notably insufficient balance — with a
// 2xx AND an error envelope carrying NO choices. Parse that envelope with the
// SAME lenient logic aiStatusError uses so an out-of-credits user gets a typed,
// actionable error (→ top-up CTA) instead of an opaque "empty response".
if e := newAIError(status, body); e.code != "" || e.msg != "" {
logf("world: ai 2xx empty-choices error: status=%d code=%q model=%s", status, e.code, model)
return "", status, "", e
}
logf("world: ai 2xx no choices: status=%d model=%s", status, model)
return "", status, "", emptyAnswerErr(model)
}
return strings.TrimSpace(cr.Choices[0].Message.Content), cr.Usage.TotalTokens, nil
// Prefer the answer (content) channel; fall back to the reasoning channel a
// reasoning model (gpt-oss/harmony, deepseek-r1) uses when it returns no content —
// so the analyst recovers its envelope/prose instead of a blank "empty response".
msg := cr.Choices[0].Message
// Pre-trim so a whitespace-only content channel falls through to reasoning.
out := firstNonEmpty(strings.TrimSpace(msg.Content), strings.TrimSpace(msg.ReasoningContent), strings.TrimSpace(msg.Reasoning))
if out == "" {
// A 2xx with a real but content-free choice (e.g. a reasoning model that hit the
// token cap before emitting content). The body IS a normal completion here — do
// NOT run it through newAIError (its top-level `id` would be misread as an error
// code); surface the model + stop reason honestly instead.
logf("world: ai 2xx empty answer: status=%d model=%s finish=%s", status, model, cr.Choices[0].FinishReason)
return "", status, cr.ID, emptyAnswerErr(model)
}
return out, cr.Usage.TotalTokens, cr.ID, nil
}
// emptyAnswerErr is the honest error for a 2xx completion that carried no usable
// answer (neither a content nor a reasoning channel). It names the served model so
// the chat surfaces "the <model> model returned an empty answer" — actionable (switch
// model) — instead of an opaque "empty response". Not a balance error, so it never
// trips the top-up CTA.
func emptyAnswerErr(model string) error {
if model = strings.TrimSpace(model); model != "" {
return fmt.Errorf("the %s model returned an empty answer", model)
}
return fmt.Errorf("the model returned an empty answer")
}
// aiError is a typed inference error carrying the upstream HTTP status plus the
// error {code,message} parsed from the response body, so callers can branch on
// the code (isBalanceError) instead of string-matching the rendered message. Its
// Error() reproduces the actionable one-liner the SPA renders verbatim in chat:
// "hanzo ai status <n>" plus the upstream code (or a short message) when present.
type aiError struct {
status int
code string
msg string
}
func (e *aiError) Error() string {
base := fmt.Sprintf("hanzo ai status %d", e.status)
switch {
case e.code != "":
return base + ": " + e.code
case e.msg != "":
return base + ": " + trim80(e.msg)
}
return base
}
// parseAIErrorBody leniently extracts an error {code, message} from an inference
// or billing response body, across every shape the gateway/billing layers emit.
// It is the SINGLE parse shared by the non-2xx path (newAIError) and the 2xx
// empty-choices path (chatMessagesModel): the gateway answers some failures —
// notably insufficient balance — with HTTP 2xx and an error envelope carrying no
// choices, so both must read the same envelope. Empty strings ⇒ no error found.
//
// {"error":{"code":"insufficient_balance","message":…}} → code
// {"error":"unauthorized","message":…} → "unauthorized"
// {"id":"Unauthorized","message":…} → "Unauthorized"
// {"detail":…} → message
func parseAIErrorBody(body []byte) (code, msg string) {
var p struct {
Error json.RawMessage `json:"error"`
Detail string `json:"detail"`
ID string `json:"id"`
Message string `json:"message"`
}
if json.Unmarshal(body, &p) != nil {
return "", ""
}
msg = strings.TrimSpace(p.Message)
if len(p.Error) > 0 {
// error is either a nested {code,message} object or a bare string.
var eo struct {
Code string `json:"code"`
Message string `json:"message"`
}
if json.Unmarshal(p.Error, &eo) == nil {
code = strings.TrimSpace(eo.Code)
if msg == "" {
msg = strings.TrimSpace(eo.Message)
}
} else {
var es string
if json.Unmarshal(p.Error, &es) == nil {
code = strings.TrimSpace(es)
}
}
}
if code == "" {
code = strings.TrimSpace(p.ID)
}
if msg == "" {
msg = strings.TrimSpace(p.Detail)
}
return code, msg
}
// newAIError builds a typed aiError from a response status + body.
func newAIError(status int, body []byte) *aiError {
code, msg := parseAIErrorBody(body)
return &aiError{status: status, code: code, msg: msg}
}
// aiStatusError turns a non-2xx inference response into an ACTIONABLE error. The
// SPA renders it verbatim in chat, so surfacing e.g. "insufficient_balance" turns
// a dead chat into a fixable one; a body it can't parse degrades to the bare status.
func aiStatusError(status int, body []byte) error {
return newAIError(status, body)
}
// balanceErrorFrom reports whether err is the canonical "out of credits" signal
// the ONE AI gateway emits — HTTP 402 with code=insufficient_balance
// (hanzo/ai routers/filter_balance.go, the single balance-enforcement point).
// World is a thin client of that contract: it neither re-derives balance
// semantics (no message keyword-matching) nor invents codes the backend never
// sends. The analyst renders a top-up CTA for exactly this case.
func balanceErrorFrom(err error) bool {
var ae *aiError
if errors.As(err, &ae) {
return ae.status == http.StatusPaymentRequired || ae.code == "insufficient_balance"
}
return false
}
// trim80 trims s to at most 80 characters (runes, never splitting UTF-8) so a
// long upstream message stays a compact one-liner in the chat error.
func trim80(s string) string {
r := []rune(strings.TrimSpace(s))
if len(r) > 80 {
return strings.TrimSpace(string(r[:80]))
}
return string(r)
}
// dateContext mirrors the original prompts' current-date grounding.
+349
View File
@@ -0,0 +1,349 @@
package world
// Streaming inference — the SSE side of the ONE completion path.
//
// chatMessagesModelStream is chatMessagesModel with "stream":true: it reads the
// upstream OpenAI-style SSE frames and hands each content delta to onDelta while
// accumulating the full output, so callers keep the exact same (content, tokens,
// error) contract and simply gain live deltas.
//
// replyExtractor solves the analyst's envelope problem: the model emits STRICT
// JSON ({"reply":"…","actions":[…],"tools":[…]}), so raw deltas are not
// user-presentable. The extractor is an incremental scanner fed those raw
// deltas; it emits ONLY the unescaped contents of the top-level "reply" string
// as it grows. Tool rounds ({"reply":"","tools":…}) therefore stream nothing,
// and a model that ignores the envelope and answers in prose streams verbatim
// (mirroring parseAnalystTurn's raw-output degrade).
import (
"bufio"
"bytes"
"context"
"encoding/json"
"io"
"net/http"
"strconv"
"strings"
"time"
)
// chatMessagesModelStream runs one completion with streaming, calling onDelta
// for every content chunk. Returns the full accumulated content, the token count,
// and the gateway response id (first non-empty per-chunk id wins) so the analyst
// can key a content-free reward signal to it. A nil onDelta degrades to a plain
// buffered call semantically identical to chatMessagesModel.
func (a *AIClient) chatMessagesModelStream(ctx context.Context, s *Server, bearer, model string, messages []chatMessage, temperature float64, maxTokens int, extra map[string]string, onDelta func(string)) (string, int, string, error) {
if strings.TrimSpace(model) == "" {
model = a.model
}
reqBody, err := json.Marshal(struct {
chatRequest
Stream bool `json:"stream"`
}{
chatRequest: chatRequest{Model: model, Messages: messages, Temperature: temperature, MaxTokens: maxTokens, TopP: 0.9},
Stream: true,
})
if err != nil {
return "", 0, "", err
}
cctx, cancel := context.WithTimeout(ctx, 60*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(cctx, http.MethodPost, a.base+"/chat/completions", bytes.NewReader(reqBody))
if err != nil {
return "", 0, "", err
}
req.Header.Set("Authorization", bearer)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "text/event-stream")
for k, v := range extra {
req.Header.Set(k, v)
}
resp, err := s.client.Do(req)
if err != nil {
return "", 0, "", err
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
body, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
return "", resp.StatusCode, "", aiStatusError(resp.StatusCode, body)
}
// Some gateways answer a stream request with a plain JSON body (model or
// route not stream-capable). Sniff the content type and degrade quietly.
if ct := resp.Header.Get("Content-Type"); !strings.Contains(ct, "event-stream") {
body, err := io.ReadAll(io.LimitReader(resp.Body, maxBody))
if err != nil {
return "", resp.StatusCode, "", err
}
var cr chatResponse
if err := json.Unmarshal(body, &cr); err != nil {
return "", resp.StatusCode, "", err
}
if len(cr.Choices) == 0 {
return "", resp.StatusCode, "", emptyAnswerErr(model)
}
msg := cr.Choices[0].Message
out := firstNonEmpty(strings.TrimSpace(msg.Content), strings.TrimSpace(msg.ReasoningContent), strings.TrimSpace(msg.Reasoning))
if out == "" {
return "", resp.StatusCode, cr.ID, emptyAnswerErr(model)
}
if onDelta != nil {
onDelta(out)
}
return out, cr.Usage.TotalTokens, cr.ID, nil
}
var full strings.Builder
var reasoning strings.Builder
tokens := 0
id := ""
sc := bufio.NewScanner(resp.Body)
sc.Buffer(make([]byte, 0, 64<<10), 1<<20)
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
if !strings.HasPrefix(line, "data:") {
continue
}
data := strings.TrimSpace(strings.TrimPrefix(line, "data:"))
if data == "" || data == "[DONE]" {
continue
}
var frame struct {
ID string `json:"id"`
Choices []struct {
Delta struct {
Content string `json:"content"`
ReasoningContent string `json:"reasoning_content"`
Reasoning string `json:"reasoning"`
} `json:"delta"`
} `json:"choices"`
Usage *struct {
TotalTokens int `json:"total_tokens"`
} `json:"usage"`
}
if json.Unmarshal([]byte(data), &frame) != nil {
continue // tolerate keep-alives / vendor frames
}
if id == "" && frame.ID != "" {
id = frame.ID // first non-empty id wins (every chunk repeats it)
}
if frame.Usage != nil {
tokens = frame.Usage.TotalTokens
}
for _, c := range frame.Choices {
// Reasoning deltas (gpt-oss/harmony, deepseek-r1) ride a separate channel.
// Forward them too — the replyExtractor routes this pre-envelope prose to the
// live "thinking" line — and keep them as the answer fallback for a model that
// never emits a content channel. (Not trimmed: preserve inter-chunk spacing.)
rc := c.Delta.ReasoningContent
if rc == "" {
rc = c.Delta.Reasoning
}
if rc != "" {
reasoning.WriteString(rc)
if onDelta != nil {
onDelta(rc)
}
}
if c.Delta.Content != "" {
full.WriteString(c.Delta.Content)
if onDelta != nil {
onDelta(c.Delta.Content)
}
}
}
}
if err := sc.Err(); err != nil {
// A mid-stream drop still yields whatever arrived; the caller decides.
if full.Len() == 0 && reasoning.Len() == 0 {
return "", resp.StatusCode, "", err
}
}
// The content channel is the answer; fall back to the reasoning channel only when
// the model emitted no content at all (mirrors the buffered path above).
out := strings.TrimSpace(full.String())
if out == "" {
out = strings.TrimSpace(reasoning.String())
}
return out, tokens, id, nil
}
// ── incremental "reply" extraction ───────────────────────────────────────────
// replyExtractor states.
const (
rxSniff = iota // deciding: JSON envelope or raw prose?
rxSeekKey // scanning for "reply"
rxSeekColon
rxSeekQuote
rxInString // inside the reply value — emit unescaped runes
rxDone // reply string closed (or prose mode ended)
rxProse // no envelope — everything is the reply
)
// replyExtractor incrementally extracts the top-level "reply" string value from
// a streaming JSON envelope, emitting decoded text via emit. Reasoning models
// (gpt-oss, zen3-omni) write prose BEFORE the envelope — that pre-envelope text
// goes to emitThink (nil-safe) so the UI can show it as live thinking, and the
// moment a '{' appears the scanner switches to envelope mode. Feed() accepts
// arbitrary chunk boundaries (escapes and \uXXXX may split across chunks).
type replyExtractor struct {
state int
emit func(string)
emitThink func(string)
// key matching
keyBuf string
// string decoding
esc bool // last byte was a backslash
uBuf string // pending \uXXXX hex digits ("" when not in a unicode escape)
lead int // sniff: bytes of leading whitespace seen
}
func newReplyExtractor(emit func(string)) *replyExtractor {
return &replyExtractor{state: rxSniff, emit: emit}
}
// Feed consumes the next raw chunk of model output.
func (x *replyExtractor) Feed(chunk string) {
i := 0
for i < len(chunk) {
c := chunk[i]
switch x.state {
case rxSniff:
if c == ' ' || c == '\n' || c == '\r' || c == '\t' {
i++
continue
}
if c == '{' {
x.state = rxSeekKey
i++
continue
}
// Pre-envelope prose (a reasoning model thinking out loud) — stream
// it as thinking until the envelope's '{' shows up.
x.state = rxProse
case rxProse:
j := i
for j < len(chunk) && chunk[j] != '{' {
j++
}
if j > i && x.emitThink != nil {
x.emitThink(chunk[i:j])
}
if j >= len(chunk) {
return
}
x.state = rxSeekKey // envelope begins
i = j + 1
case rxSeekKey:
// scan for the exact key "reply" — cheap rolling match on quoted
// runs; the envelope contract puts reply first, so this stays
// shallow in practice.
if c == '"' {
x.keyBuf = ""
j := i + 1
for j < len(chunk) && chunk[j] != '"' {
x.keyBuf += string(chunk[j])
j++
}
if j >= len(chunk) {
// key name split across chunks — resume in rxKeyPartial
x.state = rxKeyPartial
return
}
if x.keyBuf == "reply" {
x.state = rxSeekColon
}
i = j + 1
continue
}
i++
case rxKeyPartial:
for i < len(chunk) && chunk[i] != '"' {
x.keyBuf += string(chunk[i])
i++
}
if i < len(chunk) { // closing quote
if x.keyBuf == "reply" {
x.state = rxSeekColon
} else {
x.state = rxSeekKey
}
i++
}
case rxSeekColon:
if c == ':' {
x.state = rxSeekQuote
}
i++
case rxSeekQuote:
if c == '"' {
x.state = rxInString
} else if c != ' ' && c != '\n' && c != '\r' && c != '\t' {
x.state = rxSeekKey // reply wasn't a string (defensive)
}
i++
case rxInString:
if x.uBuf != "" || x.esc {
i = x.feedEscape(chunk, i)
continue
}
if c == '\\' {
x.esc = true
i++
continue
}
if c == '"' {
x.state = rxDone
return
}
// emit the longest plain run in one call
j := i
for j < len(chunk) && chunk[j] != '\\' && chunk[j] != '"' {
j++
}
x.emit(chunk[i:j])
i = j
case rxDone:
return
}
}
}
// rxKeyPartial handles a key name split across chunk boundaries.
const rxKeyPartial = 100
// feedEscape consumes escape-sequence bytes (possibly across chunks).
func (x *replyExtractor) feedEscape(chunk string, i int) int {
if x.uBuf != "" { // inside \uXXXX ("u" + collected hex)
for i < len(chunk) && len(x.uBuf) < 5 {
x.uBuf += string(chunk[i])
i++
}
if len(x.uBuf) == 5 {
if v, err := strconv.ParseUint(x.uBuf[1:], 16, 32); err == nil {
x.emit(string(rune(v)))
}
x.uBuf = ""
}
return i
}
// x.esc: the byte after a backslash
c := chunk[i]
x.esc = false
switch c {
case 'n':
x.emit("\n")
case 't':
x.emit("\t")
case 'r':
x.emit("\r")
case 'u':
x.uBuf = "u"
case '"', '\\', '/':
x.emit(string(c))
default:
x.emit(string(c))
}
return i + 1
}
+64
View File
@@ -0,0 +1,64 @@
package world
import (
"strings"
"testing"
)
// feedChunks drives an extractor with the given chunking and returns the
// emitted reply text and thinking text.
func feedChunks(t *testing.T, chunks []string) (reply, think string) {
t.Helper()
var r, th strings.Builder
x := newReplyExtractor(func(s string) { r.WriteString(s) })
x.emitThink = func(s string) { th.WriteString(s) }
for _, c := range chunks {
x.Feed(c)
}
return r.String(), th.String()
}
func TestReplyExtractor(t *testing.T) {
cases := []struct {
name string
chunks []string
want string
wantThink string
}{
{"whole envelope", []string{`{"reply":"Hello world","actions":[]}`}, "Hello world", ""},
{"split mid-value", []string{`{"reply":"Hel`, `lo wor`, `ld","actions":[]}`}, "Hello world", ""},
{"split mid-key", []string{`{"re`, `ply":"hi"}`}, "hi", ""},
{"escapes", []string{`{"reply":"a\nb\t\"q\" c\\d"}`}, "a\nb\t\"q\" c\\d", ""},
{"escape split across chunks", []string{`{"reply":"x\`, `ny"}`}, "x\ny", ""},
{"unicode escape", []string{`{"reply":"snow ☃!"}`}, "snow ☃!", ""},
{"unicode escape split", []string{`{"reply":"a\u26`, `03b"}`}, "a☃b", ""},
{"tool round stays silent", []string{`{"reply":"","tools":[{"name":"world_brief","arguments":{"n":5}}]}`}, "", ""},
{"prose streams as thinking", []string{"Just a plain ", "prose answer."}, "", "Just a plain prose answer."},
{"reasoning preamble then envelope", []string{"We need to answer briefly. ", `{"reply":"The answer.","actions":[]}`}, "The answer.", "We need to answer briefly. "},
{"reasoning split around the brace", []string{"thinking…", " done ", `{"rep`, `ly":"yes"}`}, "yes", "thinking… done "},
{"leading whitespace then envelope", []string{" \n ", `{"reply":"ok"}`}, "ok", ""},
{"other key first (defensive)", []string{`{"model":"x","reply":"later"}`}, "later", ""},
{"stops at closing quote", []string{`{"reply":"done","actions":[{"type":"noise \"reply\":\"nope\""}]}`}, "done", ""},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, think := feedChunks(t, tc.chunks)
if got != tc.want || think != tc.wantThink {
t.Errorf("got (%q, think %q), want (%q, think %q)", got, think, tc.want, tc.wantThink)
}
})
}
}
// Byte-at-a-time is the cruellest chunking — every state transition lands on a
// boundary.
func TestReplyExtractorByteAtATime(t *testing.T) {
in := `{"reply":"line1\nline2 é end","actions":[]}`
chunks := make([]string, 0, len(in))
for i := 0; i < len(in); i++ {
chunks = append(chunks, in[i:i+1])
}
if got, _ := feedChunks(t, chunks); got != "line1\nline2 é end" {
t.Errorf("got %q", got)
}
}
+43
View File
@@ -0,0 +1,43 @@
package world
import "testing"
// TestAIStatusError: the non-2xx inference body is parsed into an actionable
// error across every shape the gateway/billing layers emit; an unparseable body
// degrades to the bare status.
func TestAIStatusError(t *testing.T) {
cases := []struct {
name string
status int
body string
want string
}{
{"nested code", 402, `{"error":{"code":"insufficient_balance","message":"no funds"}}`, "hanzo ai status 402: insufficient_balance"},
{"nested spend cap", 402, `{"error":{"code":"spend_cap_exceeded","message":"cap hit"}}`, "hanzo ai status 402: spend_cap_exceeded"},
{"error string", 401, `{"error":"unauthorized","message":"bad token"}`, "hanzo ai status 401: unauthorized"},
{"id + message", 401, `{"id":"Unauthorized","message":"login required"}`, "hanzo ai status 401: Unauthorized"},
{"detail only", 400, `{"detail":"missing field query"}`, "hanzo ai status 400: missing field query"},
{"nested message no code", 403, `{"error":{"message":"forbidden here"}}`, "hanzo ai status 403: forbidden here"},
{"unparseable html", 500, `<html>502 bad gateway</html>`, "hanzo ai status 500"},
{"empty body", 429, ``, "hanzo ai status 429"},
}
for _, c := range cases {
if got := aiStatusError(c.status, []byte(c.body)).Error(); got != c.want {
t.Errorf("%s: aiStatusError(%d, %q) = %q, want %q", c.name, c.status, c.body, got, c.want)
}
}
}
// TestTrim80 caps a long upstream message to 80 runes without splitting UTF-8.
func TestTrim80(t *testing.T) {
long := ""
for i := 0; i < 200; i++ {
long += "x"
}
if got := trim80(long); len([]rune(got)) != 80 {
t.Fatalf("trim80 long = %d runes, want 80", len([]rune(got)))
}
if got := trim80(" short "); got != "short" {
t.Fatalf("trim80 trimmed = %q, want %q", got, "short")
}
}
+86
View File
@@ -0,0 +1,86 @@
package world
import (
"context"
"time"
)
// Boot cache warmer: keeps the hottest GDELT-backed cache keys warm so the
// request path serves a fresh hit instead of blocking ~10s on a cold GDELT
// fetch when a key's 5m TTL lapses.
//
// The hot keys are the ones the SPA hits on every load: the analyst grounding
// snapshot (gdelt-doc query=world) and the protests panel (gdelt-geo default).
// Each pod warms its OWN in-memory cache (unlike the shared hanzo-kv feed cache,
// there is no cross-pod copy to reuse), on boot and every ~4min — just under the
// TTL, so a key is refreshed before it can expire. GDELT rate-limits callers to
// one request per ~5s, so the specs are walked sequentially with gdeltPace gaps.
const cacheWarmInterval = 4 * time.Minute
// warmSpec is one hot cache key. warm re-produces its value under the exact key
// the handler reads (via the shared key/produce helpers in handlers_news.go).
type warmSpec struct {
name string
warm func(ctx context.Context) error
}
// warmSpecs is the table of hot keys, each wired to the same produce path and
// key strings its handler uses so warmer and handler can never drift.
func (s *Server) warmSpecs() []warmSpec {
return []warmSpec{
{"gdelt-doc query=world", func(ctx context.Context) error {
v, err := s.produceGDELTDoc(ctx, "world", "72h", 8)
if err != nil {
return err
}
s.cache.Set(gdeltDocKey("world", 8, "72h"), v, gdeltTTL, gdeltStale)
return nil
}},
{"gdelt-geo query=protest", func(ctx context.Context) error {
key := gdeltGeoKey("protest", "geojson", 250, "7d")
_, err := s.fetchAndCache(ctx, key, gdeltGeoURL("protest", "geojson", 250, "7d"), nil, gdeltTTL, gdeltStale)
return err
}},
}
}
// startCacheWarmer launches the warmer loop until ctx is cancelled: it warms
// once shortly after boot then on the jittered interval.
func (s *Server) startCacheWarmer(ctx context.Context) {
go func() {
s.warmCaches(ctx)
for {
select {
case <-ctx.Done():
return
case <-time.After(jitter(cacheWarmInterval)):
s.warmCaches(ctx)
}
}
}()
}
// warmCaches (re)produces every hot key sequentially, each independently
// bounded, with GDELT pacing between them. A failure is logged and skipped; the
// next cycle retries and the stale value keeps serving in the meantime.
func (s *Server) warmCaches(ctx context.Context) {
specs := s.warmSpecs()
for i, spec := range specs {
if ctx.Err() != nil {
return
}
wctx, cancel := context.WithTimeout(ctx, 24*time.Second)
err := spec.warm(wctx)
cancel()
if err != nil {
logf("world-warm: %s failed: %v", spec.name, err)
}
if i < len(specs)-1 {
select {
case <-ctx.Done():
return
case <-time.After(gdeltPace):
}
}
}
}
+45 -37
View File
@@ -2,8 +2,6 @@ package world
import (
"context"
"crypto/sha256"
"encoding/hex"
"net/http"
"strings"
"time"
@@ -26,17 +24,30 @@ import (
// or non-admin owner → 403. The caller's bearer is returned to forward upstream,
// where cloud independently re-verifies — defense in depth.
// isAdminOrg reports whether an owner claim is one of the admin orgs. Kept in
// sync with the cloud deploy's globalAdminOrgs (admin,built-in).
func isAdminOrg(owner string) bool {
switch strings.TrimSpace(owner) {
case "admin", "built-in":
return true
default:
return false
// adminOrgs is the set of IAM owner claims world treats as a global admin: the
// base {admin, built-in} (in sync with the cloud deploy's globalAdminOrgs) plus the
// deployment's OPERATOR org, so the seeded superuser (z@hanzo.ai, owner "hanzo")
// sees the full internal dashboard. Override per deployment with WORLD_ADMIN_ORGS
// (comma-separated); unset defaults the operator org to "hanzo". The upstream cloud
// subsystems independently re-verify the bearer, so this gate only decides which
// callers world will ATTEMPT the full/admin reads for — never the final authz.
func adminOrgs() map[string]bool {
m := map[string]bool{"admin": true, "built-in": true}
extra := env("WORLD_ADMIN_ORGS")
if extra == "" {
extra = "hanzo" // operator org: z@hanzo.ai is the seeded superuser
}
for _, o := range strings.Split(extra, ",") {
if o = strings.TrimSpace(o); o != "" {
m[o] = true
}
}
return m
}
// isAdminOrg reports whether an owner claim is one of the admin orgs.
func isAdminOrg(owner string) bool { return adminOrgs()[strings.TrimSpace(owner)] }
// apiHost returns the api.hanzo.ai origin (no /v1 suffix) that the cloud
// subsystems are served from. HANZO_AI_BASE may legitimately carry a /v1 suffix
// (it is the OpenAI-compatible base); strip it so subsystem paths like
@@ -58,39 +69,36 @@ func iamIssuer() string {
return "https://hanzo.id"
}
// introspectOwner resolves the caller's IAM `owner` claim from userinfo, memoized
// by token hash for a short TTL so the gate does not hit IAM on every request.
func (s *Server) introspectOwner(ctx context.Context, bearer string) (string, error) {
sum := sha256.Sum256([]byte(bearer))
key := "cloud-owner:" + hex.EncodeToString(sum[:12])
if v, ok := s.cache.Get(key); ok {
return v.(string), nil
}
var u struct {
Owner string `json:"owner"`
}
if err := s.getJSON(ctx, iamIssuer()+"/v1/iam/oauth/userinfo",
map[string]string{"Authorization": bearer}, &u); err != nil {
return "", err
}
s.cache.Set(key, u.Owner, 60*time.Second, 60*time.Second)
return u.Owner, nil
}
// requireAdmin gates an admin-only Cloud endpoint. It returns the caller's bearer
// (to forward upstream) and true only for a validated admin-org owner; otherwise
// it writes the fail-closed response (401 without a token, 403 otherwise) and
// returns false. Every admin handler calls this after preflight.
func (s *Server) requireAdmin(w http.ResponseWriter, r *http.Request) (string, bool) {
// adminIdentity is the NON-writing admin probe: it returns the caller's bearer and
// true iff a valid IAM identity resolves to an admin-org owner. It short-circuits
// (no IAM call) when the request carries no bearer, so anonymous requests stay cheap.
// Used by endpoints that have a public fallback (cloud-pulse): they upgrade to the
// full admin view when the caller is an admin, and otherwise serve the public path.
func (s *Server) adminIdentity(r *http.Request) (string, bool) {
bearer := userBearer(r)
if bearer == "" {
writeError(w, http.StatusUnauthorized, "Sign in required")
return "", false
}
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
owner, err := s.introspectOwner(ctx, bearer)
if err != nil || !isAdminOrg(owner) {
id, err := s.introspectIdentity(ctx, bearer)
if err != nil || !isAdminOrg(id.Org) {
return "", false
}
return bearer, true
}
// requireAdmin gates an admin-ONLY Cloud endpoint (no public fallback). It returns
// the caller's bearer (to forward upstream) and true only for a validated admin-org
// owner; otherwise it writes the fail-closed response (401 without a token, 403
// otherwise) and returns false. Every admin handler calls this after preflight.
func (s *Server) requireAdmin(w http.ResponseWriter, r *http.Request) (string, bool) {
if userBearer(r) == "" {
writeError(w, http.StatusUnauthorized, "Sign in required")
return "", false
}
bearer, ok := s.adminIdentity(r)
if !ok {
writeError(w, http.StatusForbidden, "Admin only — sign in with the admin org")
return "", false
}
+161
View File
@@ -0,0 +1,161 @@
package world
import (
"context"
"errors"
"time"
)
// Service-plane access to the Hanzo Cloud (api.hanzo.ai) — the ONE place that
// knows how world authenticates as a platform service and reads the platform-wide
// aggregates the public dashboard surfaces (cloud-pulse volume, ai-pulse,
// enso-training). Org-scoped drill-down never comes through here: those panels
// call api.hanzo.ai directly with the CALLER's IAM bearer (see cloud-pulse.ts).
//
// One credential, one base. The service token is the repo's established,
// KMS-injected bearer (kms.go's fetch list) — it is NEVER sent to the browser.
// All aggregate reads share it, so there is exactly one secret for ops to
// provision. For the platform-wide reads (get-cloud-usages ?org=all and the
// routing-ledger exports) it must be a super-admin bearer; a non-admin token
// 4xxes upstream and every caller degrades to its honest demo/unavailable state.
// errNoServiceToken is returned by the service-plane readers when no service
// token is configured, so callers keep their honest degrade instead of faking.
var errNoServiceToken = errors.New("world: no service token configured")
// serviceToken returns the platform service bearer (KMS-injected). Empty ⇒ every
// service-side aggregate read degrades to demo/unavailable. Never exposed to the
// browser.
func serviceToken() string { return env("HANZO_CLOUD_PULSE_TOKEN") }
// serviceAuth is the Authorization header map for a service-side call, or nil
// when no service token is configured (so callers short-circuit to their demo
// path).
func serviceAuth() map[string]string {
tok := serviceToken()
if tok == "" {
return nil
}
return map[string]string{"Authorization": "Bearer " + tok}
}
// ── platform usage ledger (ai GET /v1/get-cloud-usages) ───────────────────────
// cloudUsageOverview decodes the fields world aggregates from ai's
// object.CloudUsageOverview. The upstream is ClickHouse-backed; ?org=all is the
// platform-wide view and requires a super-admin bearer.
type cloudUsageOverview struct {
Range string `json:"range"`
Interval string `json:"interval"`
Totals cloudUsageTotals `json:"totals"`
Series []cloudUsageSeriesPoint `json:"series"`
ByModel cloudUsageByModel `json:"byModel"`
}
type cloudUsageTotals struct {
Tokens int64 `json:"tokens"`
Requests int64 `json:"requests"`
SpendCents int64 `json:"spendCents"`
Models int64 `json:"models"`
}
type cloudUsageSeriesPoint struct {
T string `json:"t"`
Tokens int64 `json:"tokens"`
SpendCents int64 `json:"spendCents"`
Requests int64 `json:"requests"`
}
type cloudUsageModelSpend struct {
Model string `json:"model"`
SpendCents int64 `json:"spendCents"`
Tokens int64 `json:"tokens"`
Requests int64 `json:"requests"`
Pct float64 `json:"pct"` // share of total spend, 0..100
}
type cloudUsageByModel struct {
Items []cloudUsageModelSpend `json:"items"`
}
// fetchCloudUsage reads the platform-wide usage overview (?org=all) for a range
// label ("24h" | "7d" | "30d") using the supplied auth header — the KMS service
// bearer (public path) or the signed-in admin's OWN bearer (the flagship admin
// path). Either way the upstream independently authorizes ?org=all (super-admin),
// so a non-admin bearer 4xxes and the caller keeps its honest fallback. Returns
// errNoServiceToken when no auth is available at all.
func (s *Server) fetchCloudUsage(ctx context.Context, rangeLabel string, hdr map[string]string) (*cloudUsageOverview, error) {
if hdr == nil {
return nil, errNoServiceToken
}
var ov cloudUsageOverview
url := apiHost() + "/v1/get-cloud-usages?org=all&range=" + rangeLabel
if err := s.getJSON(ctx, url, hdr, &ov); err != nil {
return nil, err
}
return &ov, nil
}
// intervalSeconds resolves the usage-series bucket width to seconds; 0 when
// unresolvable so callers fall back to a window average instead of dividing by a
// bogus interval. The upstream get-cloud-usages emits the width as a WORD enum
// ("hour"/"day"/"minute"), NOT a Go duration — time.ParseDuration fails on those
// ("hour" needs to be "1h"), which silently forced usageRate to the flat 24h mean
// (the "AI Compute usage is dead flat" bug). Handle the enum first, then still
// accept a real duration string for any caller that sends one.
func intervalSeconds(d string) float64 {
switch d {
case "":
return 0
case "minute":
return 60
case "hour":
return 3600
case "day":
return 86400
case "week":
return 604800
}
v, err := time.ParseDuration(d)
if err != nil || v <= 0 {
return 0
}
return v.Seconds()
}
// usageRate returns the freshest honest per-second rate for a metric: the most
// recent complete series bucket over its interval, falling back to the 24h
// average when there is no usable interval/series. sel picks the metric from a
// bucket. Shared by cloud-pulse and ai-pulse so the rate is derived one way.
func usageRate(total24h int64, series []cloudUsageSeriesPoint, interval string, sel func(cloudUsageSeriesPoint) int64) float64 {
if n := len(series); n > 0 {
if iv := intervalSeconds(interval); iv > 0 {
return float64(sel(series[n-1])) / iv
}
}
const windowSecs = 86400.0
return float64(total24h) / windowSecs
}
// seriesRequests / seriesTokens are the bucket selectors for usageRate.
func seriesRequests(p cloudUsageSeriesPoint) int64 { return p.Requests }
func seriesTokens(p cloudUsageSeriesPoint) int64 { return p.Tokens }
// topModelsFromUsage maps the ledger's ranked byModel spend into the shared
// cloudModel shape (share normalized 0..1). nil when the ledger listed none.
func topModelsFromUsage(ov *cloudUsageOverview) []cloudModel {
if len(ov.ByModel.Items) == 0 {
return nil
}
out := make([]cloudModel, 0, len(ov.ByModel.Items))
for _, m := range ov.ByModel.Items {
out = append(out, cloudModel{
ID: m.Model,
Name: m.Model,
Requests24h: m.Requests,
Tokens24h: m.Tokens,
Share: m.Pct / 100,
})
}
return out
}
+40
View File
@@ -0,0 +1,40 @@
package world
import "testing"
// TestIntervalSeconds guards the "AI Compute usage is dead flat" regression: the
// upstream usage series reports its bucket width as a WORD enum ("hour"/"day"), not a
// Go duration, so time.ParseDuration failed and usageRate silently fell back to the
// flat 24h mean. intervalSeconds must resolve the enums (and still accept a real
// duration string).
func TestIntervalSeconds(t *testing.T) {
cases := map[string]float64{
"minute": 60,
"hour": 3600,
"day": 86400,
"week": 604800,
"1h": 3600, // real Go-duration string still works
"30m": 1800,
"": 0, // unresolvable → 0 (caller falls back to window average)
"nope": 0,
}
for in, want := range cases {
if got := intervalSeconds(in); got != want {
t.Errorf("intervalSeconds(%q) = %v, want %v", in, got, want)
}
}
}
// TestUsageRate proves the freshest-bucket rate is used when the interval resolves
// (the fix), and the 24h average is used only when it can't.
func TestUsageRate(t *testing.T) {
series := []cloudUsageSeriesPoint{{Requests: 100}, {Requests: 720}} // last bucket = 720
// "hour" now resolves to 3600s → 720/3600 = 0.2 rps (fresh), NOT 999999/86400.
if got := usageRate(999999, series, "hour", seriesRequests); got != 720.0/3600.0 {
t.Errorf("usageRate(hour) = %v, want %v (fresh bucket, not 24h mean)", got, 720.0/3600.0)
}
// Unresolvable interval → 24h average.
if got := usageRate(86400, series, "", seriesRequests); got != 1.0 {
t.Errorf("usageRate(empty interval) = %v, want 1.0 (24h average)", got)
}
}
+84
View File
@@ -0,0 +1,84 @@
package world
import (
"context"
"time"
)
// Platform user metrics — REAL signups / active users from Hanzo IAM (Casdoor).
//
// Source: IAM GET /v1/iam/global-users (every user across every org; global-admin
// only). We read it with the signed-in admin's OWN bearer (never a shared key),
// fold it into NON-sensitive aggregates (counts + a daily-signup series), and return
// ONLY those aggregates — no user record, email, name or other PII ever leaves this
// function. Honest-empty (error) when the caller is not a global admin upstream or
// IAM is unreachable, so the overview simply omits the user tiles rather than guess.
type userMetrics struct {
Total int `json:"total"`
Signups24h int `json:"signups24h"`
Signups7d int `json:"signups7d"`
ActiveNow int `json:"activeNow"`
SignupSeries []int64 `json:"signupSeries"` // new users/day, last 14 days, chronological
}
// iamUser is the minimal, non-PII slice of Casdoor's object.User we read.
type iamUser struct {
CreatedTime string `json:"createdTime"`
IsOnline bool `json:"isOnline"`
IsDeleted bool `json:"isDeleted"`
}
const userSignupDays = 14
// fetchUserMetrics reads the platform-wide user list from IAM with auth and folds it
// into non-sensitive aggregates. Requires a global-admin bearer upstream; any error
// (non-admin, unreachable) leaves the caller to omit the tiles honestly — never a
// fabricated count.
func (s *Server) fetchUserMetrics(ctx context.Context, auth map[string]string) (*userMetrics, error) {
if auth == nil {
return nil, errNoServiceToken
}
var users []iamUser
if err := s.getJSON(ctx, iamIssuer()+"/v1/iam/global-users", auth, &users); err != nil {
return nil, err
}
now := time.Now().UTC()
m := &userMetrics{SignupSeries: make([]int64, userSignupDays)}
for _, u := range users {
if u.IsDeleted {
continue
}
m.Total++
if u.IsOnline {
m.ActiveNow++
}
t, ok := parseUserTime(u.CreatedTime)
if !ok {
continue
}
if age := now.Sub(t); age >= 0 {
if age <= 24*time.Hour {
m.Signups24h++
}
if age <= 7*24*time.Hour {
m.Signups7d++
}
if d := int(age.Hours() / 24); d < userSignupDays {
m.SignupSeries[userSignupDays-1-d]++ // chronological: oldest bucket first, today last
}
}
}
return m, nil
}
// parseUserTime parses Casdoor's createdTime (RFC3339, with a couple of legacy
// fallbacks). ok=false when unparseable so a bad row is skipped, never guessed.
func parseUserTime(s string) (time.Time, bool) {
for _, layout := range []string{time.RFC3339, time.RFC3339Nano, "2006-01-02 15:04:05"} {
if t, err := time.Parse(layout, s); err == nil {
return t.UTC(), true
}
}
return time.Time{}, false
}
+124
View File
@@ -0,0 +1,124 @@
package world
import (
"context"
"encoding/json"
"time"
"github.com/hanzoai/world/internal/world/kv"
"github.com/hanzoai/world/internal/world/model"
"github.com/hanzoai/world/internal/world/store"
)
// datastore.go wires world's three storage concerns onto the Server and owns
// their lifecycle, keeping each in its lane:
// - hanzo-kv (shared hot cache) → instant feed bodies (FeedCache L2)
// - embedded SQLite (lake) → the searchable "one place to query everything"
// - embedded SQLite (settings) → signed-in per-identity dashboard sync
//
// Everything degrades cleanly: no hanzo-kv → per-pod in-mem feed cache; no
// SQLite → search/analytics return empty and settings say "not stored". The
// service never 5xxes over storage.
// kvAddr is the hanzo-kv endpoint. Defaults to the in-cluster Service; set empty
// (WORLD_KV_DISABLE=1) to force the pure in-mem path (local dev / CI).
func kvAddr() string {
if env("WORLD_KV_DISABLE") != "" {
return ""
}
if a := env("HANZO_KV_ADDR", "WORLD_KV_ADDR"); a != "" {
return a
}
return "hanzo-kv:6379"
}
// kvPassword is optional — hanzo-kv currently requires none; the hook is here for
// a future KMS-provisioned password (HANZO_KV_PASSWORD / world-secrets).
func kvPassword() string { return env("HANZO_KV_PASSWORD", "WORLD_KV_PASSWORD") }
// lakeRetention is the rolling window ingested items are kept for.
func lakeRetention() time.Duration {
if v := env("WORLD_LAKE_RETENTION"); v != "" {
if d, err := time.ParseDuration(v); err == nil && d > 0 {
return d
}
}
return store.DefaultRetention
}
// initDatastore opens hanzo-kv + the embedded SQLite datastore and builds the
// two-tier feed cache. Called once from NewServer; never fails hard.
func (s *Server) initDatastore() {
s.kv = kv.Open(kvAddr(), kvPassword())
db, err := store.Open(modelDataDir(), lakeRetention())
if err != nil {
logf("world-store: degraded (no durable lake/settings): %v", err)
}
s.store = db
s.feeds = NewFeedCache(s.kv, 0, curatedFeedSeed)
// The world model dumps its folded observations into the lake so model state
// is queryable alongside news — the engine stays decomplected from storage,
// it just calls this sink.
s.worldModel.SetObservationSink(s.ingestObservations)
}
// StartDatastore starts the background loops: the lake write-behind/prune
// consumer and the feed warmer. Call once from main after the server is built.
func (s *Server) StartDatastore(ctx context.Context) {
if s.kv.Enabled() {
pctx, cancel := context.WithTimeout(ctx, 3*time.Second)
if err := s.kv.Ping(pctx); err != nil {
logf("world-kv: %s unreachable, using per-pod in-mem cache: %v", kvAddr(), err)
} else {
logf("world-kv: connected to %s (shared feed cache)", kvAddr())
}
cancel()
} else {
logf("world-kv: disabled, using per-pod in-mem feed cache")
}
go s.store.Lake.Run(ctx)
s.startFeedWarmer(ctx)
s.startCacheWarmer(ctx)
}
// Close releases the datastore handles. Safe to call once on shutdown.
func (s *Server) Close() {
if s.kv != nil {
s.kv.Close()
}
if s.store != nil {
_ = s.store.Close()
}
}
// ingestObservations folds one cycle of world-model observations into the lake
// (kind=observation), keyed so each entity+source keeps its latest value. This
// is the model half of "dump ALL ingested data into the datastore".
func (s *Server) ingestObservations(obs []model.Observation) {
if s.store == nil {
return
}
now := time.Now().UTC()
for _, o := range obs {
country := ""
if o.Kind == model.KindCountry {
country = o.ID
}
payload, _ := json.Marshal(map[string]any{
"id": o.ID, "kind": o.Kind, "name": o.Name,
"metrics": o.Metrics, "note": o.Note, "src": o.Src,
})
s.store.Lake.Add(store.Item{
ID: "obs:" + o.Kind + ":" + o.ID + ":" + o.Src,
Kind: "observation",
Source: o.Src,
TS: now,
Title: o.Name,
Text: o.Note,
Country: country,
Payload: string(payload),
})
}
}
+257
View File
@@ -0,0 +1,257 @@
package world
import (
_ "embed"
"encoding/json"
"log"
"regexp"
"sort"
"strings"
"sync"
)
// Server-side news enrichment: threat classification + geo inference.
//
// This is the Go-native home for work the browser used to do on every client,
// on every render. The keyword tables and geo hubs are DATA (enrichdata/*.json)
// — one source of truth, embedded here and imported by the frontend, so the two
// runtimes can never drift. enrich_test.go proves byte-for-byte parity with the
// TypeScript implementation over an exhaustive 494-case corpus.
//
// Two parity traps, both handled deliberately:
// - JS object key order IS the match priority (first match wins). Go maps are
// unordered, so the tables stay ORDERED SLICES.
// - JS Array.sort is stable. Go's sort.Slice is not — geo matches use
// sort.SliceStable so equal-confidence hubs keep index order.
//go:embed enrichdata/threat-keywords.json
var threatKeywordsJSON []byte
//go:embed enrichdata/geo-hubs.json
var geoHubsJSON []byte
// ThreatClassification mirrors the frontend type (services/threat-classifier.ts).
type ThreatClassification struct {
Level string `json:"level"`
Category string `json:"category"`
Confidence float64 `json:"confidence"`
Source string `json:"source"`
}
// GeoHub is a strategic location an item can be pinned to.
type GeoHub struct {
ID string `json:"id"`
Name string `json:"name"`
Region string `json:"region"`
Country string `json:"country"`
Lat float64 `json:"lat"`
Lon float64 `json:"lon"`
Type string `json:"type"`
Tier string `json:"tier"`
Keywords []string `json:"keywords"`
}
// GeoMatch is one inferred hub for a headline.
type GeoMatch struct {
HubID string `json:"hubId"`
Confidence float64 `json:"confidence"`
MatchedKeyword string `json:"matchedKeyword"`
}
type keywordEntry struct {
Keyword string `json:"keyword"`
Category string `json:"category"`
}
type threatTier struct {
Level string `json:"level"`
Confidence float64 `json:"confidence"`
Variant string `json:"variant"` // "any" | "tech"
Keywords []keywordEntry `json:"keywords"`
}
type threatTables struct {
Exclusions []string `json:"exclusions"`
ShortKeywords []string `json:"shortKeywords"`
Tiers []threatTier `json:"tiers"`
}
type enricher struct {
tables threatTables
short map[string]bool
res map[string]*regexp.Regexp // keyword → compiled matcher
hubs []GeoHub
hubIndex map[string]*GeoHub
// hubKeywords preserves insertion order, matching the JS Map iteration the
// frontend index relies on.
hubKeywords []hubKeyword
}
type hubKeyword struct {
keyword string
hubIDs []string
re *regexp.Regexp // non-nil only for short keywords (word-boundary)
}
var (
enrichOnce sync.Once
enr *enricher
)
func enrichEngine() *enricher {
enrichOnce.Do(func() {
e := &enricher{
short: map[string]bool{},
res: map[string]*regexp.Regexp{},
hubIndex: map[string]*GeoHub{},
}
if err := json.Unmarshal(threatKeywordsJSON, &e.tables); err != nil {
log.Printf("world: enrich: threat tables: %v", err)
}
for _, s := range e.tables.ShortKeywords {
e.short[s] = true
}
// Precompile one matcher per keyword, mirroring getKeywordRegex: short
// keywords are word-bounded (so "war" never matches "wardrobe"), the rest
// are plain substring matches. Titles are lowercased before matching.
for _, tier := range e.tables.Tiers {
for _, k := range tier.Keywords {
e.res[k.Keyword] = compileKeyword(k.Keyword, e.short[k.Keyword])
}
}
var hubData struct {
Hubs []GeoHub `json:"hubs"`
}
if err := json.Unmarshal(geoHubsJSON, &hubData); err != nil {
log.Printf("world: enrich: geo hubs: %v", err)
}
e.hubs = hubData.Hubs
seen := map[string]int{} // keyword → index into hubKeywords
for i := range e.hubs {
h := &e.hubs[i]
e.hubIndex[h.ID] = h
for _, kw := range h.Keywords {
lower := strings.ToLower(kw)
if idx, ok := seen[lower]; ok {
e.hubKeywords[idx].hubIDs = append(e.hubKeywords[idx].hubIDs, h.ID)
continue
}
seen[lower] = len(e.hubKeywords)
var re *regexp.Regexp
// The frontend word-bounds geo keywords shorter than 5 chars.
if len(lower) < 5 {
re = regexp.MustCompile(`\b` + regexp.QuoteMeta(lower) + `\b`)
}
e.hubKeywords = append(e.hubKeywords, hubKeyword{keyword: lower, hubIDs: []string{h.ID}, re: re})
}
}
enr = e
})
return enr
}
func compileKeyword(kw string, short bool) *regexp.Regexp {
q := regexp.QuoteMeta(kw)
if short {
return regexp.MustCompile(`\b` + q + `\b`)
}
return regexp.MustCompile(q)
}
// ClassifyByKeyword is the Go port of services/threat-classifier.ts
// classifyByKeyword. Same priority cascade, same confidences, same categories.
func ClassifyByKeyword(title, variant string) ThreatClassification {
e := enrichEngine()
lower := strings.ToLower(title)
info := ThreatClassification{Level: "info", Category: "general", Confidence: 0.3, Source: "keyword"}
for _, ex := range e.tables.Exclusions {
if strings.Contains(lower, ex) {
return info
}
}
isTech := variant == "tech"
for _, tier := range e.tables.Tiers {
if tier.Variant == "tech" && !isTech {
continue
}
for _, k := range tier.Keywords {
if re := e.res[k.Keyword]; re != nil && re.MatchString(lower) {
return ThreatClassification{
Level: tier.Level,
Category: k.Category,
Confidence: tier.Confidence,
Source: "keyword",
}
}
}
}
return info
}
// InferGeoHubs is the Go port of services/geo-hub-index.ts inferGeoHubsFromTitle:
// keyword → hub, confidence by keyword length, boosted for conflict/strategic
// type and critical tier, sorted by confidence (stable).
func InferGeoHubs(title string) []GeoMatch {
e := enrichEngine()
lower := strings.ToLower(title)
matches := []GeoMatch{}
seen := map[string]bool{}
for _, hk := range e.hubKeywords {
if len(hk.keyword) < 2 {
continue
}
found := false
if hk.re != nil {
found = hk.re.MatchString(lower)
} else {
found = strings.Contains(lower, hk.keyword)
}
if !found {
continue
}
for _, id := range hk.hubIDs {
if seen[id] {
continue
}
seen[id] = true
hub := e.hubIndex[id]
if hub == nil {
continue
}
confidence := 0.5
switch n := len(hk.keyword); {
case n >= 10:
confidence = 0.9
case n >= 6:
confidence = 0.75
case n >= 4:
confidence = 0.6
}
if hub.Type == "conflict" || hub.Type == "strategic" {
confidence = min1(confidence + 0.1)
}
if hub.Tier == "critical" {
confidence = min1(confidence + 0.1)
}
matches = append(matches, GeoMatch{HubID: id, Confidence: confidence, MatchedKeyword: hk.keyword})
}
}
// JS Array.sort is stable — equal confidences keep discovery order.
sort.SliceStable(matches, func(i, j int) bool { return matches[i].Confidence > matches[j].Confidence })
return matches
}
// GeoHubByID exposes a hub for callers that need its coordinates/name.
func GeoHubByID(id string) *GeoHub {
return enrichEngine().hubIndex[id]
}
func min1(v float64) float64 {
if v > 1 {
return 1
}
return v
}
+111
View File
@@ -0,0 +1,111 @@
package world
import (
"encoding/json"
"math"
"os"
"testing"
)
// Parity: the Go enrichment MUST agree with the TypeScript implementation it
// replaces, or moving the work server-side would silently change what every
// user sees. testdata/enrich_golden.json is generated by EVALUATING the real
// frontend classifier/geo-index (not transcribed) over an exhaustive corpus:
// every keyword in every tier, both variants, every geo-hub keyword, every
// exclusion, plus adversarial word-boundary cases ("war" must not match
// "wardrobe"). Any drift fails here.
type goldenCase struct {
Title string `json:"title"`
Full ThreatClassification `json:"full"`
Tech ThreatClassification `json:"tech"`
Geo []GeoMatch `json:"geo"`
}
func loadGolden(t *testing.T) []goldenCase {
t.Helper()
b, err := os.ReadFile("testdata/enrich_golden.json")
if err != nil {
t.Fatalf("read golden: %v", err)
}
var g struct {
Cases []goldenCase `json:"cases"`
}
if err := json.Unmarshal(b, &g); err != nil {
t.Fatalf("parse golden: %v", err)
}
if len(g.Cases) < 400 {
t.Fatalf("golden corpus too small (%d) — regenerate it", len(g.Cases))
}
return g.Cases
}
func sameClass(a, b ThreatClassification) bool {
return a.Level == b.Level && a.Category == b.Category &&
a.Source == b.Source && math.Abs(a.Confidence-b.Confidence) < 1e-9
}
func TestClassifyByKeywordMatchesTypeScript(t *testing.T) {
cases := loadGolden(t)
bad := 0
for _, c := range cases {
for _, v := range []struct {
variant string
want ThreatClassification
}{{"full", c.Full}, {"tech", c.Tech}} {
got := ClassifyByKeyword(c.Title, v.variant)
if !sameClass(got, v.want) {
bad++
if bad <= 10 {
t.Errorf("classify(%q, %s)\n got %+v\n want %+v", c.Title, v.variant, got, v.want)
}
}
}
}
if bad > 0 {
t.Fatalf("%d/%d classifications diverge from the TypeScript source of truth", bad, len(cases)*2)
}
t.Logf("parity: %d titles × 2 variants identical to TypeScript", len(cases))
}
func TestInferGeoHubsMatchesTypeScript(t *testing.T) {
cases := loadGolden(t)
bad, withGeo := 0, 0
for _, c := range cases {
got := InferGeoHubs(c.Title)
if len(c.Geo) > 0 {
withGeo++
}
if len(got) != len(c.Geo) {
bad++
if bad <= 10 {
t.Errorf("geo(%q): got %d matches, want %d", c.Title, len(got), len(c.Geo))
}
continue
}
for i := range got {
w := c.Geo[i]
if got[i].HubID != w.HubID || got[i].MatchedKeyword != w.MatchedKeyword ||
math.Abs(got[i].Confidence-w.Confidence) > 1e-9 {
bad++
if bad <= 10 {
t.Errorf("geo(%q)[%d]\n got %+v\n want %+v", c.Title, i, got[i], w)
}
break
}
}
}
if bad > 0 {
t.Fatalf("%d/%d geo inferences diverge from the TypeScript source of truth", bad, len(cases))
}
t.Logf("parity: %d titles identical to TypeScript (%d carry geo matches)", len(cases), withGeo)
}
// The adversarial guarantee, asserted directly: short keywords are word-bounded.
func TestShortKeywordsAreWordBounded(t *testing.T) {
for _, title := range []string{"The wardrobe warehouse expands", "Something banned entirely"} {
if got := ClassifyByKeyword(title, "full"); got.Level != "info" {
t.Errorf("%q classified %s — a short keyword matched inside a longer word", title, got.Level)
}
}
}
+691
View File
@@ -0,0 +1,691 @@
{
"hubs": [
{
"id": "washington",
"name": "Washington DC",
"region": "North America",
"country": "USA",
"lat": 38.9072,
"lon": -77.0369,
"type": "capital",
"tier": "critical",
"keywords": [
"washington",
"white house",
"pentagon",
"state department",
"congress",
"capitol hill",
"biden",
"trump"
]
},
{
"id": "moscow",
"name": "Moscow",
"region": "Europe",
"country": "Russia",
"lat": 55.7558,
"lon": 37.6173,
"type": "capital",
"tier": "critical",
"keywords": [
"moscow",
"kremlin",
"putin",
"russia",
"russian"
]
},
{
"id": "beijing",
"name": "Beijing",
"region": "Asia",
"country": "China",
"lat": 39.9042,
"lon": 116.4074,
"type": "capital",
"tier": "critical",
"keywords": [
"beijing",
"xi jinping",
"china",
"chinese",
"ccp",
"prc"
]
},
{
"id": "brussels",
"name": "Brussels",
"region": "Europe",
"country": "Belgium",
"lat": 50.8503,
"lon": 4.3517,
"type": "capital",
"tier": "critical",
"keywords": [
"brussels",
"eu",
"european union",
"nato",
"european commission"
]
},
{
"id": "london",
"name": "London",
"region": "Europe",
"country": "UK",
"lat": 51.5074,
"lon": -0.1278,
"type": "capital",
"tier": "critical",
"keywords": [
"london",
"uk",
"britain",
"british",
"downing street",
"parliament"
]
},
{
"id": "jerusalem",
"name": "Jerusalem",
"region": "Middle East",
"country": "Israel",
"lat": 31.7683,
"lon": 35.2137,
"type": "capital",
"tier": "major",
"keywords": [
"jerusalem",
"israel",
"israeli",
"knesset",
"netanyahu"
]
},
{
"id": "telaviv",
"name": "Tel Aviv",
"region": "Middle East",
"country": "Israel",
"lat": 32.0853,
"lon": 34.7818,
"type": "capital",
"tier": "major",
"keywords": [
"tel aviv",
"idf",
"mossad"
]
},
{
"id": "tehran",
"name": "Tehran",
"region": "Middle East",
"country": "Iran",
"lat": 35.6892,
"lon": 51.389,
"type": "capital",
"tier": "major",
"keywords": [
"tehran",
"iran",
"iranian",
"khamenei",
"irgc",
"ayatollah"
]
},
{
"id": "kyiv",
"name": "Kyiv",
"region": "Europe",
"country": "Ukraine",
"lat": 50.4501,
"lon": 30.5234,
"type": "capital",
"tier": "major",
"keywords": [
"kyiv",
"kiev",
"ukraine",
"ukrainian",
"zelensky",
"zelenskyy"
]
},
{
"id": "taipei",
"name": "Taipei",
"region": "Asia",
"country": "Taiwan",
"lat": 25.033,
"lon": 121.5654,
"type": "capital",
"tier": "major",
"keywords": [
"taipei",
"taiwan",
"taiwanese",
"tsmc"
]
},
{
"id": "tokyo",
"name": "Tokyo",
"region": "Asia",
"country": "Japan",
"lat": 35.6762,
"lon": 139.6503,
"type": "capital",
"tier": "major",
"keywords": [
"tokyo",
"japan",
"japanese"
]
},
{
"id": "seoul",
"name": "Seoul",
"region": "Asia",
"country": "South Korea",
"lat": 37.5665,
"lon": 126.978,
"type": "capital",
"tier": "major",
"keywords": [
"seoul",
"south korea",
"korean"
]
},
{
"id": "pyongyang",
"name": "Pyongyang",
"region": "Asia",
"country": "North Korea",
"lat": 39.0392,
"lon": 125.7625,
"type": "capital",
"tier": "major",
"keywords": [
"pyongyang",
"north korea",
"dprk",
"kim jong un"
]
},
{
"id": "newdelhi",
"name": "New Delhi",
"region": "Asia",
"country": "India",
"lat": 28.6139,
"lon": 77.209,
"type": "capital",
"tier": "major",
"keywords": [
"new delhi",
"delhi",
"india",
"indian",
"modi"
]
},
{
"id": "riyadh",
"name": "Riyadh",
"region": "Middle East",
"country": "Saudi Arabia",
"lat": 24.7136,
"lon": 46.6753,
"type": "capital",
"tier": "major",
"keywords": [
"riyadh",
"saudi",
"saudi arabia",
"mbs",
"mohammed bin salman"
]
},
{
"id": "ankara",
"name": "Ankara",
"region": "Middle East",
"country": "Turkey",
"lat": 39.9334,
"lon": 32.8597,
"type": "capital",
"tier": "major",
"keywords": [
"ankara",
"turkey",
"turkish",
"erdogan"
]
},
{
"id": "paris",
"name": "Paris",
"region": "Europe",
"country": "France",
"lat": 48.8566,
"lon": 2.3522,
"type": "capital",
"tier": "major",
"keywords": [
"paris",
"france",
"french",
"macron",
"elysee"
]
},
{
"id": "berlin",
"name": "Berlin",
"region": "Europe",
"country": "Germany",
"lat": 52.52,
"lon": 13.405,
"type": "capital",
"tier": "major",
"keywords": [
"berlin",
"germany",
"german",
"scholz",
"bundestag"
]
},
{
"id": "cairo",
"name": "Cairo",
"region": "Middle East",
"country": "Egypt",
"lat": 30.0444,
"lon": 31.2357,
"type": "capital",
"tier": "major",
"keywords": [
"cairo",
"egypt",
"egyptian",
"sisi"
]
},
{
"id": "islamabad",
"name": "Islamabad",
"region": "Asia",
"country": "Pakistan",
"lat": 33.6844,
"lon": 73.0479,
"type": "capital",
"tier": "major",
"keywords": [
"islamabad",
"pakistan",
"pakistani"
]
},
{
"id": "gaza",
"name": "Gaza",
"region": "Middle East",
"country": "Palestine",
"lat": 31.5,
"lon": 34.47,
"type": "conflict",
"tier": "critical",
"keywords": [
"gaza",
"hamas",
"palestinian",
"rafah",
"khan younis",
"gaza strip"
]
},
{
"id": "westbank",
"name": "West Bank",
"region": "Middle East",
"country": "Palestine",
"lat": 31.9,
"lon": 35.2,
"type": "conflict",
"tier": "major",
"keywords": [
"west bank",
"ramallah",
"jenin",
"nablus",
"hebron"
]
},
{
"id": "ukraine-front",
"name": "Ukraine Front",
"region": "Europe",
"country": "Ukraine",
"lat": 48.5,
"lon": 37.5,
"type": "conflict",
"tier": "critical",
"keywords": [
"donbas",
"donbass",
"donetsk",
"luhansk",
"kharkiv",
"bakhmut",
"avdiivka",
"zaporizhzhia",
"kherson",
"crimea"
]
},
{
"id": "taiwan-strait",
"name": "Taiwan Strait",
"region": "Asia",
"country": "International",
"lat": 24.5,
"lon": 119.5,
"type": "conflict",
"tier": "critical",
"keywords": [
"taiwan strait",
"formosa",
"pla",
"chinese military"
]
},
{
"id": "southchinasea",
"name": "South China Sea",
"region": "Asia",
"country": "International",
"lat": 12,
"lon": 114,
"type": "strategic",
"tier": "critical",
"keywords": [
"south china sea",
"spratlys",
"paracels",
"nine-dash line",
"scarborough"
]
},
{
"id": "yemen",
"name": "Yemen",
"region": "Middle East",
"country": "Yemen",
"lat": 15.5527,
"lon": 48.5164,
"type": "conflict",
"tier": "major",
"keywords": [
"yemen",
"houthi",
"houthis",
"sanaa",
"aden"
]
},
{
"id": "syria",
"name": "Syria",
"region": "Middle East",
"country": "Syria",
"lat": 34.8,
"lon": 39,
"type": "conflict",
"tier": "major",
"keywords": [
"syria",
"syrian",
"assad",
"damascus",
"idlib",
"aleppo"
]
},
{
"id": "lebanon",
"name": "Lebanon",
"region": "Middle East",
"country": "Lebanon",
"lat": 33.8547,
"lon": 35.8623,
"type": "conflict",
"tier": "major",
"keywords": [
"lebanon",
"lebanese",
"hezbollah",
"beirut"
]
},
{
"id": "sudan",
"name": "Sudan",
"region": "Africa",
"country": "Sudan",
"lat": 15.5007,
"lon": 32.5599,
"type": "conflict",
"tier": "major",
"keywords": [
"sudan",
"sudanese",
"khartoum",
"rsf",
"darfur"
]
},
{
"id": "sahel",
"name": "Sahel",
"region": "Africa",
"country": "International",
"lat": 15,
"lon": 0,
"type": "conflict",
"tier": "major",
"keywords": [
"sahel",
"mali",
"niger",
"burkina faso",
"wagner",
"junta"
]
},
{
"id": "ethiopia",
"name": "Ethiopia",
"region": "Africa",
"country": "Ethiopia",
"lat": 9.145,
"lon": 40.4897,
"type": "conflict",
"tier": "notable",
"keywords": [
"ethiopia",
"ethiopian",
"tigray",
"addis ababa",
"abiy ahmed"
]
},
{
"id": "myanmar",
"name": "Myanmar",
"region": "Asia",
"country": "Myanmar",
"lat": 19.7633,
"lon": 96.0785,
"type": "conflict",
"tier": "notable",
"keywords": [
"myanmar",
"burma",
"rohingya",
"junta",
"naypyidaw"
]
},
{
"id": "hormuz",
"name": "Strait of Hormuz",
"region": "Middle East",
"country": "International",
"lat": 26.5,
"lon": 56.5,
"type": "strategic",
"tier": "critical",
"keywords": [
"hormuz",
"strait of hormuz",
"persian gulf",
"gulf"
]
},
{
"id": "redsea",
"name": "Red Sea",
"region": "Middle East",
"country": "International",
"lat": 20,
"lon": 38,
"type": "strategic",
"tier": "critical",
"keywords": [
"red sea",
"bab el-mandeb",
"bab al-mandab"
]
},
{
"id": "suez",
"name": "Suez Canal",
"region": "Middle East",
"country": "Egypt",
"lat": 30.5,
"lon": 32.3,
"type": "strategic",
"tier": "critical",
"keywords": [
"suez",
"suez canal"
]
},
{
"id": "baltic",
"name": "Baltic Sea",
"region": "Europe",
"country": "International",
"lat": 58,
"lon": 20,
"type": "strategic",
"tier": "major",
"keywords": [
"baltic",
"baltic sea",
"kaliningrad",
"gotland"
]
},
{
"id": "arctic",
"name": "Arctic",
"region": "Arctic",
"country": "International",
"lat": 75,
"lon": 0,
"type": "strategic",
"tier": "major",
"keywords": [
"arctic",
"northern sea route",
"svalbard"
]
},
{
"id": "blacksea",
"name": "Black Sea",
"region": "Europe",
"country": "International",
"lat": 43,
"lon": 35,
"type": "strategic",
"tier": "major",
"keywords": [
"black sea",
"bosphorus",
"sevastopol",
"odesa",
"odessa"
]
},
{
"id": "un-nyc",
"name": "United Nations",
"region": "North America",
"country": "USA",
"lat": 40.7489,
"lon": -73.968,
"type": "organization",
"tier": "critical",
"keywords": [
"united nations",
"un",
"security council",
"general assembly",
"unsc"
]
},
{
"id": "nato-hq",
"name": "NATO HQ",
"region": "Europe",
"country": "Belgium",
"lat": 50.8796,
"lon": 4.4284,
"type": "organization",
"tier": "critical",
"keywords": [
"nato",
"north atlantic",
"alliance",
"stoltenberg"
]
},
{
"id": "iaea-vienna",
"name": "IAEA",
"region": "Europe",
"country": "Austria",
"lat": 48.2352,
"lon": 16.4156,
"type": "organization",
"tier": "major",
"keywords": [
"iaea",
"atomic energy",
"nuclear watchdog",
"grossi"
]
}
]
}
@@ -0,0 +1,621 @@
{
"exclusions": [
"protein",
"couples",
"relationship",
"dating",
"diet",
"fitness",
"recipe",
"cooking",
"shopping",
"fashion",
"celebrity",
"movie",
"tv show",
"sports",
"game",
"concert",
"festival",
"wedding",
"vacation",
"travel tips",
"life hack",
"self-care",
"wellness"
],
"shortKeywords": [
"war",
"coup",
"ban",
"vote",
"riot",
"riots",
"hack",
"talks",
"ipo",
"gdp",
"virus",
"disease",
"flood"
],
"tiers": [
{
"level": "critical",
"confidence": 0.9,
"variant": "any",
"keywords": [
{
"keyword": "nuclear strike",
"category": "military"
},
{
"keyword": "nuclear attack",
"category": "military"
},
{
"keyword": "nuclear war",
"category": "military"
},
{
"keyword": "invasion",
"category": "conflict"
},
{
"keyword": "declaration of war",
"category": "conflict"
},
{
"keyword": "martial law",
"category": "military"
},
{
"keyword": "coup",
"category": "military"
},
{
"keyword": "coup attempt",
"category": "military"
},
{
"keyword": "genocide",
"category": "conflict"
},
{
"keyword": "ethnic cleansing",
"category": "conflict"
},
{
"keyword": "chemical attack",
"category": "terrorism"
},
{
"keyword": "biological attack",
"category": "terrorism"
},
{
"keyword": "dirty bomb",
"category": "terrorism"
},
{
"keyword": "mass casualty",
"category": "conflict"
},
{
"keyword": "pandemic declared",
"category": "health"
},
{
"keyword": "health emergency",
"category": "health"
},
{
"keyword": "nato article 5",
"category": "military"
},
{
"keyword": "evacuation order",
"category": "disaster"
},
{
"keyword": "meltdown",
"category": "disaster"
},
{
"keyword": "nuclear meltdown",
"category": "disaster"
}
]
},
{
"level": "high",
"confidence": 0.8,
"variant": "any",
"keywords": [
{
"keyword": "war",
"category": "conflict"
},
{
"keyword": "armed conflict",
"category": "conflict"
},
{
"keyword": "airstrike",
"category": "conflict"
},
{
"keyword": "air strike",
"category": "conflict"
},
{
"keyword": "drone strike",
"category": "conflict"
},
{
"keyword": "missile",
"category": "military"
},
{
"keyword": "missile launch",
"category": "military"
},
{
"keyword": "troops deployed",
"category": "military"
},
{
"keyword": "military escalation",
"category": "military"
},
{
"keyword": "bombing",
"category": "conflict"
},
{
"keyword": "casualties",
"category": "conflict"
},
{
"keyword": "hostage",
"category": "terrorism"
},
{
"keyword": "terrorist",
"category": "terrorism"
},
{
"keyword": "terror attack",
"category": "terrorism"
},
{
"keyword": "assassination",
"category": "crime"
},
{
"keyword": "cyber attack",
"category": "cyber"
},
{
"keyword": "ransomware",
"category": "cyber"
},
{
"keyword": "data breach",
"category": "cyber"
},
{
"keyword": "sanctions",
"category": "economic"
},
{
"keyword": "embargo",
"category": "economic"
},
{
"keyword": "earthquake",
"category": "disaster"
},
{
"keyword": "tsunami",
"category": "disaster"
},
{
"keyword": "hurricane",
"category": "disaster"
},
{
"keyword": "typhoon",
"category": "disaster"
}
]
},
{
"level": "high",
"confidence": 0.75,
"variant": "tech",
"keywords": [
{
"keyword": "major outage",
"category": "infrastructure"
},
{
"keyword": "service down",
"category": "infrastructure"
},
{
"keyword": "global outage",
"category": "infrastructure"
},
{
"keyword": "zero-day",
"category": "cyber"
},
{
"keyword": "critical vulnerability",
"category": "cyber"
},
{
"keyword": "supply chain attack",
"category": "cyber"
},
{
"keyword": "mass layoff",
"category": "economic"
}
]
},
{
"level": "medium",
"confidence": 0.7,
"variant": "any",
"keywords": [
{
"keyword": "protest",
"category": "protest"
},
{
"keyword": "protests",
"category": "protest"
},
{
"keyword": "riot",
"category": "protest"
},
{
"keyword": "riots",
"category": "protest"
},
{
"keyword": "unrest",
"category": "protest"
},
{
"keyword": "demonstration",
"category": "protest"
},
{
"keyword": "strike action",
"category": "protest"
},
{
"keyword": "military exercise",
"category": "military"
},
{
"keyword": "naval exercise",
"category": "military"
},
{
"keyword": "arms deal",
"category": "military"
},
{
"keyword": "weapons sale",
"category": "military"
},
{
"keyword": "diplomatic crisis",
"category": "diplomatic"
},
{
"keyword": "ambassador recalled",
"category": "diplomatic"
},
{
"keyword": "expel diplomats",
"category": "diplomatic"
},
{
"keyword": "trade war",
"category": "economic"
},
{
"keyword": "tariff",
"category": "economic"
},
{
"keyword": "recession",
"category": "economic"
},
{
"keyword": "inflation",
"category": "economic"
},
{
"keyword": "market crash",
"category": "economic"
},
{
"keyword": "flood",
"category": "disaster"
},
{
"keyword": "flooding",
"category": "disaster"
},
{
"keyword": "wildfire",
"category": "disaster"
},
{
"keyword": "volcano",
"category": "disaster"
},
{
"keyword": "eruption",
"category": "disaster"
},
{
"keyword": "outbreak",
"category": "health"
},
{
"keyword": "epidemic",
"category": "health"
},
{
"keyword": "infection spread",
"category": "health"
},
{
"keyword": "oil spill",
"category": "environmental"
},
{
"keyword": "pipeline explosion",
"category": "infrastructure"
},
{
"keyword": "blackout",
"category": "infrastructure"
},
{
"keyword": "power outage",
"category": "infrastructure"
},
{
"keyword": "internet outage",
"category": "infrastructure"
},
{
"keyword": "derailment",
"category": "infrastructure"
}
]
},
{
"level": "medium",
"confidence": 0.65,
"variant": "tech",
"keywords": [
{
"keyword": "outage",
"category": "infrastructure"
},
{
"keyword": "breach",
"category": "cyber"
},
{
"keyword": "hack",
"category": "cyber"
},
{
"keyword": "vulnerability",
"category": "cyber"
},
{
"keyword": "layoff",
"category": "economic"
},
{
"keyword": "layoffs",
"category": "economic"
},
{
"keyword": "antitrust",
"category": "economic"
},
{
"keyword": "monopoly",
"category": "economic"
},
{
"keyword": "ban",
"category": "economic"
},
{
"keyword": "shutdown",
"category": "infrastructure"
}
]
},
{
"level": "low",
"confidence": 0.6,
"variant": "any",
"keywords": [
{
"keyword": "election",
"category": "diplomatic"
},
{
"keyword": "vote",
"category": "diplomatic"
},
{
"keyword": "referendum",
"category": "diplomatic"
},
{
"keyword": "summit",
"category": "diplomatic"
},
{
"keyword": "treaty",
"category": "diplomatic"
},
{
"keyword": "agreement",
"category": "diplomatic"
},
{
"keyword": "negotiation",
"category": "diplomatic"
},
{
"keyword": "talks",
"category": "diplomatic"
},
{
"keyword": "peacekeeping",
"category": "diplomatic"
},
{
"keyword": "humanitarian aid",
"category": "diplomatic"
},
{
"keyword": "ceasefire",
"category": "diplomatic"
},
{
"keyword": "peace treaty",
"category": "diplomatic"
},
{
"keyword": "climate change",
"category": "environmental"
},
{
"keyword": "emissions",
"category": "environmental"
},
{
"keyword": "pollution",
"category": "environmental"
},
{
"keyword": "deforestation",
"category": "environmental"
},
{
"keyword": "drought",
"category": "environmental"
},
{
"keyword": "vaccine",
"category": "health"
},
{
"keyword": "vaccination",
"category": "health"
},
{
"keyword": "disease",
"category": "health"
},
{
"keyword": "virus",
"category": "health"
},
{
"keyword": "public health",
"category": "health"
},
{
"keyword": "covid",
"category": "health"
},
{
"keyword": "interest rate",
"category": "economic"
},
{
"keyword": "gdp",
"category": "economic"
},
{
"keyword": "unemployment",
"category": "economic"
},
{
"keyword": "regulation",
"category": "economic"
}
]
},
{
"level": "low",
"confidence": 0.55,
"variant": "tech",
"keywords": [
{
"keyword": "ipo",
"category": "economic"
},
{
"keyword": "funding",
"category": "economic"
},
{
"keyword": "acquisition",
"category": "economic"
},
{
"keyword": "merger",
"category": "economic"
},
{
"keyword": "launch",
"category": "tech"
},
{
"keyword": "release",
"category": "tech"
},
{
"keyword": "update",
"category": "tech"
},
{
"keyword": "partnership",
"category": "economic"
},
{
"keyword": "startup",
"category": "tech"
},
{
"keyword": "ai model",
"category": "tech"
},
{
"keyword": "open source",
"category": "tech"
}
]
}
]
}
+323
View File
@@ -0,0 +1,323 @@
{
"measured": {
"livecodebench": {
"enso-ultra": {
"accuracy_pct": 81.1,
"stderr_pct": 3.0,
"n": 175,
"correct": 142,
"errors": 1,
"tokens": {
"prompt": 777022,
"completion": 1611092,
"reasoning": 452206,
"calls": 696
},
"usd_est": 90.682,
"wall_seconds": null,
"run_tag": "v1"
},
"fable-5": {
"accuracy_pct": 82.9,
"stderr_pct": 2.9,
"n": 175,
"correct": 145,
"errors": 0,
"tokens": {
"prompt": 125867,
"completion": 544855,
"reasoning": 0,
"calls": 175
},
"usd_est": 8.5504,
"wall_seconds": null,
"run_tag": "v1"
},
"enso": {
"accuracy_pct": 91.4,
"stderr_pct": 2.1,
"n": 175,
"correct": 160,
"errors": 1,
"tokens": {
"prompt": 92002,
"completion": 512599,
"reasoning": 454741,
"calls": 174
},
"usd_est": 5.241,
"wall_seconds": null,
"run_tag": "v1"
},
"opus-4.8": {
"accuracy_pct": 72.0,
"stderr_pct": 3.4,
"n": 175,
"correct": 126,
"errors": 2,
"tokens": {
"prompt": 124432,
"completion": 500884,
"reasoning": 0,
"calls": 173
},
"usd_est": 39.4328,
"wall_seconds": null,
"run_tag": "v1"
},
"deepseek-v4-pro": {
"accuracy_pct": 48.0,
"stderr_pct": 3.8,
"n": 175,
"correct": 84,
"errors": 0,
"tokens": {
"prompt": 92354,
"completion": 76407,
"reasoning": 0,
"calls": 175
},
"usd_est": 0.1769,
"wall_seconds": null,
"run_tag": "v1"
},
"gpt-5.5": {
"accuracy_pct": 91.4,
"stderr_pct": 2.1,
"n": 175,
"correct": 160,
"errors": 1,
"tokens": {
"prompt": 92002,
"completion": 504668,
"reasoning": 444424,
"calls": 174
},
"usd_est": 5.1617,
"wall_seconds": null,
"run_tag": "v1"
}
},
"gpqa_diamond": {
"enso-ultra": {
"accuracy_pct": 89.9,
"stderr_pct": 2.1,
"n": 198,
"correct": 178,
"errors": 0,
"tokens": {
"prompt": 205635,
"completion": 612706,
"reasoning": 319290,
"calls": 601
},
"usd_est": 16.4333,
"wall_seconds": null,
"run_tag": "v1"
},
"enso": {
"accuracy_pct": 87.9,
"stderr_pct": 2.3,
"n": 198,
"correct": 174,
"errors": 0,
"tokens": {
"prompt": 74531,
"completion": 152519,
"reasoning": 29387,
"calls": 198
},
"usd_est": 10.0211,
"wall_seconds": null,
"run_tag": "v1"
},
"fable-5": {
"accuracy_pct": 81.3,
"stderr_pct": 2.8,
"n": 198,
"correct": 161,
"errors": 0,
"tokens": {
"prompt": 78121,
"completion": 199072,
"reasoning": 0,
"calls": 198
},
"usd_est": 3.2204,
"wall_seconds": null,
"run_tag": "v1"
},
"opus-4.8": {
"accuracy_pct": 86.9,
"stderr_pct": 2.4,
"n": 198,
"correct": 172,
"errors": 0,
"tokens": {
"prompt": 78121,
"completion": 126069,
"reasoning": 0,
"calls": 198
},
"usd_est": 10.627,
"wall_seconds": 241.3,
"run_tag": "v1"
},
"gpt-5.5": {
"accuracy_pct": 90.4,
"stderr_pct": 2.1,
"n": 198,
"correct": 179,
"errors": 0,
"tokens": {
"prompt": 54916,
"completion": 362128,
"reasoning": 326158,
"calls": 198
},
"usd_est": 3.6899,
"wall_seconds": 2396.2,
"run_tag": "v1"
},
"deepseek-v4-pro": {
"accuracy_pct": 76.3,
"stderr_pct": 3.0,
"n": 198,
"correct": 151,
"errors": 0,
"tokens": {
"prompt": 52523,
"completion": 108318,
"reasoning": 0,
"calls": 198
},
"usd_est": 0.2076,
"wall_seconds": null,
"run_tag": "v1"
}
},
"hle": {
"enso": {
"accuracy_pct": 0.0,
"stderr_pct": 0.0,
"n": 1,
"correct": 0,
"errors": 0,
"tokens": {
"prompt": 148,
"completion": 1443,
"reasoning": 0,
"calls": 1
},
"usd_est": 0.1104,
"wall_seconds": null,
"run_tag": "preflight"
}
}
},
"grading": {
"hle": {
"judge_arm": "gpt-5.5",
"judge_calls": 1,
"judge_usd_est": 0.0006,
"judge_prompt_tokens": 159,
"judge_completion_tokens": 36
}
},
"enso_reported": {
"SWE-Bench Pro": {
"Enso-Ultra": 73.7,
"Enso": 59.0,
"Opus-4.8": 69.2,
"Gemini-3.1": 54.2,
"GPT-5.5": 58.6
},
"Terminal Bench 2.1": {
"Enso-Ultra": 82.1,
"Enso": 80.2,
"Opus-4.8": 74.6,
"Gemini-3.1": 70.3,
"GPT-5.5": 78.2
},
"LiveCodeBench": {
"Enso-Ultra": 93.2,
"Enso": 92.9,
"Opus-4.8": 87.8,
"Gemini-3.1": 88.5,
"GPT-5.5": 85.3
},
"LiveCodeBench Pro": {
"Enso-Ultra": 90.8,
"Enso": 87.8,
"Opus-4.8": 84.8,
"Gemini-3.1": 82.9,
"GPT-5.5": 88.4
},
"Humanity's Last Exam": {
"Enso-Ultra": 50.0,
"Enso": 47.2,
"Opus-4.8": 49.8,
"Gemini-3.1": 44.4,
"GPT-5.5": 41.4
},
"CharXiv Reasoning": {
"Enso-Ultra": 86.6,
"Enso": 85.1,
"Opus-4.8": 84.2,
"Gemini-3.1": 83.3,
"GPT-5.5": 84.1
},
"GPQA Diamond": {
"Enso-Ultra": 95.5,
"Enso": 95.5,
"Opus-4.8": 92.0,
"Gemini-3.1": 94.3,
"GPT-5.5": 93.6
}
},
"pending": [
"swebench_pro",
"terminal_bench"
],
"ultra_ablation": {
"gpqa_diamond": {
"v1": {
"accuracy_pct": 89.9,
"n": 198,
"usd_est": 16.4333,
"logic": "blind-synthesis"
},
"overnight": {
"accuracy_pct": 90.9,
"n": 198,
"usd_est": 15.9312,
"logic": "overnight"
}
}
},
"total_usd_est": 193.56,
"agentic": {
"swebench_pro": {
"bench": "SWE-Bench Pro",
"metric": "% resolved (agentic, Mini-SWE-Agent)",
"n_items": 25,
"systems": {
"step-routed": {
"n": 18,
"resolved": 7,
"resolved_rate": 0.3889,
"usd": 19.696,
"calls": 256
},
"single-opus": {
"n": 18,
"resolved": 3,
"resolved_rate": 0.1667,
"usd": 27.561,
"calls": 321
}
},
"note": "Agentic pilot: n=18 per system from the agent harness (agent pass/fail); official cross-scoring done on a small common set (results/agentic). Full 731-instance run pending (amd64-only task images + shared-key serialization)."
}
}
}
+106
View File
@@ -0,0 +1,106 @@
package world
import (
"context"
"crypto/sha1"
"encoding/hex"
"encoding/json"
"net/url"
"time"
"github.com/hanzoai/world/internal/world/store"
)
// rssFetchHeaders is the single header set used for every allowlisted feed
// fetch — by the on-demand fall-through (rss-proxy, feeds-batch) AND the
// background warmer. One place, one behavior.
var rssFetchHeaders = map[string]string{
"User-Agent": browserUA,
"Accept": "application/rss+xml, application/xml, text/xml, */*",
}
// fetchFeedBody performs one bounded, allowlisted GET of a feed and returns the
// body only when it is a usable (non-blank) 2xx. This is the ONLY live-fetch
// path for feed bodies, shared by the request fall-through and the warmer. The
// caller owns the timeout via ctx.
func (s *Server) fetchFeedBody(ctx context.Context, feedURL string) ([]byte, bool) {
body, status, err := s.getAllowlisted(ctx, feedURL, allowedRSSDomains, rssFetchHeaders)
if err != nil || status < 200 || status >= 300 || isBlankBody(body) {
return nil, false
}
return body, true
}
// ingestFeedItems parses a feed body and folds its items into the searchable
// data lake (kind=news). Cheap and write-behind (Lake.Add never blocks), so it
// runs off both the request fall-through and the warmer without touching latency.
func (s *Server) ingestFeedItems(feedURL string, body []byte) {
if s.store == nil {
return
}
host := feedHost(feedURL)
for _, it := range parseFeedItems(body, feedsBatchMaxItems) {
payload, _ := json.Marshal(map[string]any{
"title": it.Title, "link": it.Link, "pubDate": it.PubDate,
"source": host, "tickers": it.Tickers,
})
s.store.Lake.Add(store.Item{
ID: newsItemID(it.Link, it.Title),
Kind: "news",
Source: host,
TS: feedItemTime(it.PubDate),
Title: it.Title,
Tickers: it.Tickers,
Payload: string(payload),
})
}
}
// newsItemID is the stable dedupe key for a news item: its link when present,
// else its title. Re-ingesting the same story upserts instead of duplicating.
func newsItemID(link, title string) string {
seed := link
if seed == "" {
seed = title
}
sum := sha1.Sum([]byte(seed))
return "news:" + hex.EncodeToString(sum[:])
}
// feedItemTime parses a normalized RFC3339 pubDate back to a time, defaulting to
// now when the feed omitted or mangled the date.
func feedItemTime(pubDate string) time.Time {
if pubDate != "" {
if t, err := time.Parse(time.RFC3339, pubDate); err == nil {
return t.UTC()
}
}
return time.Now().UTC()
}
// feedHost returns the feed's host for use as the item's source label.
func feedHost(feedURL string) string {
if u, err := url.Parse(feedURL); err == nil && u.Hostname() != "" {
return u.Hostname()
}
return "feed"
}
// curatedFeedSeed is the bootstrap warm set: the highest-value feeds behind the
// crypto, financial-regulation, and crypto-news panels, so a brand-new pod warms
// THEM first (before any user request). Every URL is on the RSS allowlist. After
// the first real page load the warm set becomes demand-driven and fleet-wide; the
// seed only bridges the very first cold boot.
var curatedFeedSeed = []string{
// crypto / crypto-news
"https://www.coindesk.com/arc/outboundfeeds/rss/",
"https://cointelegraph.com/rss",
// financial regulation
"https://www.sec.gov/news/pressreleases.rss",
"https://www.federalreserve.gov/feeds/press_all.xml",
// markets / financial
"https://www.cnbc.com/id/100003114/device/rss/rss.html",
"https://www.cnbc.com/id/19854910/device/rss/rss.html",
"https://seekingalpha.com/market_currents.xml",
"https://www.ft.com/rss/home",
}
+94
View File
@@ -0,0 +1,94 @@
package world
import (
"context"
"math/rand"
"sync"
"time"
)
// Background feed warmer: the reason news is INSTANT.
//
// On boot and every interval (jittered) it fetches every warm feed, write-throughs
// the body to the shared cache, and folds the items into the lake. So the
// on-demand endpoints (rss-proxy, feeds-batch) ALWAYS serve from the warm cache
// and never block a request on an upstream — stale-while-revalidate, with the
// revalidation done here in the background instead of on the request path.
//
// Cross-pod dedupe is implicit: a feed whose SHARED (hanzo-kv) copy is still
// fresh is skipped, so N pods don't all hammer the same upstream every cycle;
// whichever pod refreshes first, the rest read its result.
const (
feedWarmInterval = 5 * time.Minute
feedWarmParallel = 8
feedWarmFetchTimeout = 10 * time.Second
// feedWarmFreshWindow: skip refetch when a cached copy is younger than this
// (slightly under the interval so each cycle still refreshes its own feeds).
feedWarmFreshWindow = 4 * time.Minute
)
// startFeedWarmer launches the warmer loop until ctx is cancelled. It warms once
// shortly after boot (so a cold pod fills fast) then on the jittered interval.
func (s *Server) startFeedWarmer(ctx context.Context) {
go func() {
s.warmFeeds(ctx)
for {
select {
case <-ctx.Done():
return
case <-time.After(jitter(feedWarmInterval)):
s.warmFeeds(ctx)
}
}
}()
}
// warmFeeds refreshes every warm feed in parallel (bounded), skipping any whose
// shared copy is still fresh. Each fetch is independently bounded; one slow
// upstream cannot hold up the rest.
func (s *Server) warmFeeds(ctx context.Context) {
urls := s.feeds.WarmURLs(ctx)
if len(urls) == 0 {
return
}
sem := make(chan struct{}, feedWarmParallel)
var wg sync.WaitGroup
refreshed := 0
var mu sync.Mutex
for _, u := range urls {
if ctx.Err() != nil {
break
}
if age, ok := s.feeds.Age(ctx, u); ok && age < feedWarmFreshWindow {
continue // a peer (or an earlier cycle) already refreshed it
}
wg.Add(1)
sem <- struct{}{}
go func(u string) {
defer wg.Done()
defer func() { <-sem }()
fctx, cancel := context.WithTimeout(ctx, feedWarmFetchTimeout)
defer cancel()
if body, ok := s.fetchFeedBody(fctx, u); ok {
s.feeds.Put(u, body)
s.ingestFeedItems(u, body)
mu.Lock()
refreshed++
mu.Unlock()
}
}(u)
}
wg.Wait()
if refreshed > 0 {
logf("world-feeds: warmed %d/%d feeds", refreshed, len(urls))
}
}
// jitter returns d spread by ±20% so pods don't synchronize their warm cycles.
func jitter(d time.Duration) time.Duration {
spread := int64(d) / 5 // 20%
if spread <= 0 {
return d
}
return d + time.Duration(rand.Int63n(2*spread)-spread)
}
+115
View File
@@ -0,0 +1,115 @@
package world
import (
"context"
"net/http"
"net/http/httptest"
"net/url"
"sync/atomic"
"testing"
"time"
"github.com/hanzoai/world/internal/world/kv"
)
const stubRSS = `<?xml version="1.0"?><rss version="2.0"><channel>
<item><title>Bitcoin warms the cache</title><link>https://x/1</link><pubDate>Mon, 02 Jan 2006 15:04:05 -0700</pubDate></item>
<item><title>SEC regulation update</title><link>https://x/2</link></item>
</channel></rss>`
// stubFeed stands up a counting RSS upstream and allowlists its host for the
// duration of the test (the SSRF boundary is a package var).
func stubFeed(t *testing.T) (feedURL string, hits *int32) {
t.Helper()
var n int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
atomic.AddInt32(&n, 1)
w.Header().Set("Content-Type", "application/xml")
_, _ = w.Write([]byte(stubRSS))
}))
t.Cleanup(srv.Close)
host := mustHost(t, srv.URL)
allowedRSSDomains[host] = true
t.Cleanup(func() { delete(allowedRSSDomains, host) })
return srv.URL + "/rss.xml", &n
}
// newTestServer builds a Server with the embedded store in a temp dir and
// hanzo-kv disabled (pure per-pod cache) — hermetic, no external services.
func newTestServer(t *testing.T) *Server {
t.Helper()
t.Setenv("WORLD_DATA_DIR", t.TempDir())
t.Setenv("WORLD_KV_DISABLE", "1")
s := NewServer()
t.Cleanup(s.Close)
return s
}
func TestWarmerPopulatesCache(t *testing.T) {
feedURL, hits := stubFeed(t)
s := newTestServer(t)
// Seed the warm set with the stub feed (demand-driven registration stand-in).
s.feeds = NewFeedCache(kv.Open("", ""), 0, []string{feedURL})
s.warmFeeds(context.Background())
if got := atomic.LoadInt32(hits); got != 1 {
t.Fatalf("upstream fetched %d times, want 1", got)
}
body, _, ok := s.feeds.Get(context.Background(), feedURL)
if !ok || len(body) == 0 {
t.Fatal("warmer did not populate the cache")
}
if string(body) != stubRSS {
t.Fatalf("cached body mismatch")
}
}
func TestWarmerSkipsFreshFeeds(t *testing.T) {
feedURL, hits := stubFeed(t)
s := newTestServer(t)
s.feeds = NewFeedCache(kv.Open("", ""), 0, nil)
// Pre-warm: a fresh copy already in cache (age < freshWindow).
s.feeds.Put(feedURL, []byte(stubRSS))
s.warmFeeds(context.Background())
if got := atomic.LoadInt32(hits); got != 0 {
t.Fatalf("upstream hit %d times, want 0 (fresh feed must be skipped)", got)
}
}
// TestWarmedFeedServedInstantly is the end-to-end promise: once warmed, the
// feeds-batch fall-through serves from cache WITHOUT any upstream fetch.
func TestWarmedFeedServedInstantly(t *testing.T) {
feedURL, hits := stubFeed(t)
s := newTestServer(t)
s.feeds = NewFeedCache(kv.Open("", ""), 0, []string{feedURL})
s.warmFeeds(context.Background()) // one upstream fetch
// A subsequent request must be served from the warm cache — no new fetch.
start := time.Now()
body, ok, fresh := s.feedXML(context.Background(), feedURL)
elapsed := time.Since(start)
if !ok || fresh {
t.Fatalf("feedXML ok=%v fresh=%v, want served-from-cache", ok, fresh)
}
if len(body) == 0 {
t.Fatal("empty body from warm cache")
}
if got := atomic.LoadInt32(hits); got != 1 {
t.Fatalf("upstream hit %d times, want 1 (warm read must not refetch)", got)
}
if elapsed > 50*time.Millisecond {
t.Fatalf("warm read took %s, want <50ms", elapsed)
}
}
func mustHost(t *testing.T, raw string) string {
t.Helper()
u, err := url.Parse(raw)
if err != nil {
t.Fatalf("parse %q: %v", raw, err)
}
return u.Hostname()
}
+201
View File
@@ -0,0 +1,201 @@
package world
import (
"bytes"
"compress/gzip"
"context"
"encoding/binary"
"io"
"sync"
"time"
"github.com/hanzoai/world/internal/world/kv"
)
// FeedCache is the warm cache for raw RSS/Atom bodies behind the instant
// news/feeds panels. It is a two-tier cache, ONE way to read a feed body:
//
// L1 — a per-pod in-memory mirror: hot reads never touch the network.
// L2 — hanzo-kv (shared, cross-pod, survives pod restart): a warm body written
// by any pod's warmer is instantly available to every pod, and a restarted
// pod repopulates L1 from L2 instead of cold-starting.
//
// The warm-URL set (which feeds to keep fresh) is demand-driven: any feed served
// once is registered, persisted fleet-wide in hanzo-kv, and kept fresh by the
// background warmer. A curated seed guarantees the highest-value panels are warm
// even on a brand-new pod before the first request. When hanzo-kv is
// unreachable, everything degrades to the in-mem tier — still correct, just
// per-pod and cold across restart.
type FeedCache struct {
mu sync.RWMutex
mem map[string]feedRow // L1
warm map[string]struct{} // in-mem warm-set fallback when kv is down
kv *kv.Client
max int
}
type feedRow struct {
body []byte
at time.Time
}
const (
// feedKeyPrefix / warmSetKey namespace the shared hanzo-kv keys.
feedKeyPrefix = "world:feed:v1:"
warmSetKey = "world:feed:warm:v1"
// feedKVTTL is the L2 safety horizon. The warmer refreshes every few minutes,
// so a live entry is normally far younger; the TTL only bounds abandoned feeds.
feedKVTTL = 3 * time.Hour
// defaultFeedCacheMax bounds the L1 mirror (news feeds are small; ~500 covers
// every frontend variant with room to spare).
defaultFeedCacheMax = 1024
)
// NewFeedCache builds the cache over an (optional) hanzo-kv client, seeding the
// warm set with the curated bootstrap feeds so a cold pod warms them first.
func NewFeedCache(kvc *kv.Client, max int, seed []string) *FeedCache {
if max <= 0 {
max = defaultFeedCacheMax
}
c := &FeedCache{
mem: make(map[string]feedRow),
warm: make(map[string]struct{}),
kv: kvc,
max: max,
}
for _, u := range seed {
c.warm[u] = struct{}{}
}
// Publish the seed to the shared warm set too (best-effort), so every pod
// converges on the same fleet-wide set.
if len(seed) > 0 {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
c.kv.SAdd(ctx, warmSetKey, seed...)
cancel()
}
return c
}
// Get returns a cached feed body and its fetch time. L1 first (instant); on miss
// it consults the shared L2 and, on a hit, backfills L1 so subsequent reads are
// instant. Never touches upstream.
func (c *FeedCache) Get(ctx context.Context, url string) ([]byte, time.Time, bool) {
c.mu.RLock()
row, ok := c.mem[url]
c.mu.RUnlock()
if ok {
return row.body, row.at, true
}
if raw, ok := c.kv.GetBytes(ctx, feedKeyPrefix+url); ok {
if at, body, ok := decodeFeed(raw); ok {
c.storeMem(url, body, at)
return body, at, true
}
}
return nil, time.Time{}, false
}
// Put write-throughs a freshly fetched body to both tiers and registers the URL
// in the warm set (demand-driven). It uses a detached, bounded context for the
// shared-tier writes so a write-through is never truncated by the request that
// triggered it — durability must outlive the request. The fetch time is now.
func (c *FeedCache) Put(url string, body []byte) {
at := time.Now()
c.storeMem(url, body, at)
c.mu.Lock()
c.warm[url] = struct{}{}
c.mu.Unlock()
raw := encodeFeed(at, body)
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
c.kv.SetBytes(ctx, feedKeyPrefix+url, raw, feedKVTTL)
c.kv.SAdd(ctx, warmSetKey, url)
}
// Age returns how old the cached copy is, if any (for the warmer's
// skip-if-fresh, cross-pod dedupe).
func (c *FeedCache) Age(ctx context.Context, url string) (time.Duration, bool) {
if _, at, ok := c.Get(ctx, url); ok {
return time.Since(at), true
}
return 0, false
}
// WarmURLs is the set of feeds to keep fresh: the fleet-wide set from hanzo-kv
// unioned with the in-mem set (seed + demand). Deduped.
func (c *FeedCache) WarmURLs(ctx context.Context) []string {
set := make(map[string]struct{})
for _, u := range c.kv.SMembers(ctx, warmSetKey) {
set[u] = struct{}{}
}
c.mu.RLock()
for u := range c.warm {
set[u] = struct{}{}
}
for u := range c.mem {
set[u] = struct{}{}
}
c.mu.RUnlock()
out := make([]string, 0, len(set))
for u := range set {
out = append(out, u)
}
return out
}
// Len reports the L1 mirror size (for tests/introspection).
func (c *FeedCache) Len() int {
c.mu.RLock()
defer c.mu.RUnlock()
return len(c.mem)
}
// storeMem writes L1, evicting the oldest entry when at capacity.
func (c *FeedCache) storeMem(url string, body []byte, at time.Time) {
c.mu.Lock()
defer c.mu.Unlock()
if _, exists := c.mem[url]; !exists && len(c.mem) >= c.max {
var oldestKey string
var oldest time.Time
first := true
for k, r := range c.mem {
if first || r.at.Before(oldest) {
oldestKey, oldest, first = k, r.at, false
}
}
if oldestKey != "" {
delete(c.mem, oldestKey)
}
}
c.mem[url] = feedRow{body: body, at: at}
}
// ── L2 encoding: [8-byte unix-nano fetch time][gzip(body)] ───────────────────
func encodeFeed(at time.Time, body []byte) []byte {
var buf bytes.Buffer
var ts [8]byte
binary.BigEndian.PutUint64(ts[:], uint64(at.UnixNano()))
buf.Write(ts[:])
gz := gzip.NewWriter(&buf)
_, _ = gz.Write(body)
_ = gz.Close()
return buf.Bytes()
}
func decodeFeed(raw []byte) (time.Time, []byte, bool) {
if len(raw) < 8 {
return time.Time{}, nil, false
}
at := time.Unix(0, int64(binary.BigEndian.Uint64(raw[:8])))
gz, err := gzip.NewReader(bytes.NewReader(raw[8:]))
if err != nil {
return time.Time{}, nil, false
}
defer func() { _ = gz.Close() }()
body, err := io.ReadAll(io.LimitReader(gz, maxBody))
if err != nil {
return time.Time{}, nil, false
}
return at, body, true
}
+118
View File
@@ -0,0 +1,118 @@
package world
import (
"context"
"testing"
"time"
"github.com/alicebob/miniredis/v2"
"github.com/hanzoai/world/internal/world/kv"
)
func TestFeedEncodeDecodeRoundTrip(t *testing.T) {
at := time.Now().Truncate(time.Nanosecond)
body := []byte("<rss><item>hello</item></rss>")
gotAt, gotBody, ok := decodeFeed(encodeFeed(at, body))
if !ok {
t.Fatal("decode failed")
}
if !gotAt.Equal(at) {
t.Fatalf("time round-trip = %v, want %v", gotAt, at)
}
if string(gotBody) != string(body) {
t.Fatalf("body round-trip = %q, want %q", gotBody, body)
}
if _, _, ok := decodeFeed([]byte("short")); ok {
t.Fatal("decode of truncated blob should fail")
}
}
// TestFeedCacheSharedAcrossPods proves the L2 (hanzo-kv) tier: a body written by
// one pod is served to another pod whose L1 is cold — i.e. warming once benefits
// the fleet, and a restarted pod (empty L1) reads a still-warm shared cache.
func TestFeedCacheSharedAcrossPods(t *testing.T) {
mr := miniredis.RunT(t)
kvA := kv.Open(mr.Addr(), "")
kvB := kv.Open(mr.Addr(), "")
t.Cleanup(func() { kvA.Close(); kvB.Close() })
podA := NewFeedCache(kvA, 0, nil)
podB := NewFeedCache(kvB, 0, nil) // separate pod: cold L1, shared L2
const url = "https://feeds.example.com/rss.xml"
body := []byte("<rss><channel><item><title>Shared</title></item></channel></rss>")
podA.Put(url, body)
if podB.Len() != 0 {
t.Fatalf("podB L1 should start cold, has %d", podB.Len())
}
got, _, ok := podB.Get(context.Background(), url)
if !ok || string(got) != string(body) {
t.Fatalf("podB Get via shared L2 = %q ok=%v, want the shared body", got, ok)
}
if podB.Len() != 1 {
t.Fatal("podB should have backfilled L1 from L2")
}
// The warm-URL set is fleet-wide (shared set), so podB knows to keep it fresh.
if !hasURL(podB.WarmURLs(context.Background()), url) {
t.Fatal("shared warm set missing the demand-added url")
}
}
// TestFeedCacheDegradesWithoutKV proves the graceful fallback: with hanzo-kv
// disabled the cache is per-pod (L1 only) and correct — a "restart" (new cache)
// is cold, never a crash.
func TestFeedCacheDegradesWithoutKV(t *testing.T) {
disabled := kv.Open("", "") // no hanzo-kv
c := NewFeedCache(disabled, 0, nil)
const url = "https://feeds.example.com/x.xml"
body := []byte("<rss/>")
c.Put(url, body)
got, _, ok := c.Get(context.Background(), url)
if !ok || string(got) != string(body) {
t.Fatalf("same-pod L1 Get = %q ok=%v", got, ok)
}
// A fresh cache (simulated restart) with no shared L2 must miss — proving the
// per-pod degrade is honest, not silently wrong.
fresh := NewFeedCache(disabled, 0, nil)
if _, _, ok := fresh.Get(context.Background(), url); ok {
t.Fatal("restart with kv disabled should be cold, got a hit")
}
}
func TestFeedCacheSeedInWarmSet(t *testing.T) {
c := NewFeedCache(kv.Open("", ""), 0, []string{"https://a.example/rss", "https://b.example/rss"})
warm := c.WarmURLs(context.Background())
if !hasURL(warm, "https://a.example/rss") || !hasURL(warm, "https://b.example/rss") {
t.Fatalf("seed missing from warm set: %v", warm)
}
}
func TestFeedCacheEvictsOldest(t *testing.T) {
c := NewFeedCache(kv.Open("", ""), 2, nil)
c.Put("u1", []byte("1"))
time.Sleep(2 * time.Millisecond)
c.Put("u2", []byte("2"))
time.Sleep(2 * time.Millisecond)
c.Put("u3", []byte("3")) // over cap → evict u1 (oldest)
if c.Len() != 2 {
t.Fatalf("L1 size = %d, want 2 (bounded)", c.Len())
}
if _, _, ok := c.Get(context.Background(), "u1"); ok {
t.Fatal("oldest entry u1 was not evicted")
}
if _, _, ok := c.Get(context.Background(), "u3"); !ok {
t.Fatal("newest entry u3 missing")
}
}
func hasURL(ss []string, want string) bool {
for _, s := range ss {
if s == want {
return true
}
}
return false
}
+269
View File
@@ -0,0 +1,269 @@
package world
import (
"context"
"encoding/json"
"net/http"
"strings"
"time"
)
// AI Compute pulse — the live "what is Hanzo's inference plane doing right now"
// stream for the AI variant (world.hanzo.ai/?variant=ai). It is the same honest
// platform aggregate cloud-pulse reads, reshaped for a compute-first panel and
// PUSHED over SSE so the number moves without the client polling.
//
// ONE route, two representations (the handleAnalyst idiom): an EventSource client
// (Accept: text/event-stream) gets typed events — `usage` (tokens/s, req/s, spend,
// top models) and `fleet` (gpu/machine/region counts) — re-emitted each interval,
// plus a `status` frame carrying the honest state. A plain GET gets ONE JSON
// snapshot of the same data, which the frontend uses as its poll fallback.
//
// Honesty: with no service token, or when every upstream is unreachable, the
// state is "unavailable" with a reason — the panel says so rather than showing a
// zero as if it were live. The service bearer is read server-side only and never
// reaches the browser.
const (
aiPulseKey = "ai-pulse"
aiPulseTTL = 10 * time.Second // shared snapshot horizon (coalesces SSE clients)
aiPulseFailTTL = 3 * time.Second // retry an unavailable pulse sooner
aiPulseInterval = 15 * time.Second // SSE re-emit cadence
)
// aiUsage is the measured inference volume (get-cloud-usages ?org=all).
type aiUsage struct {
Window string `json:"window"`
RequestsPerSec float64 `json:"requestsPerSec"`
TokensPerSec float64 `json:"tokensPerSec"`
Requests24h int64 `json:"requests24h"`
Tokens24h int64 `json:"tokens24h"`
SpendCents int64 `json:"spendCents"`
Models []cloudModel `json:"models"` // top by real spend
}
// aiFleet is the live serving fleet (visor + ai catalog). Counts only.
type aiFleet struct {
Machines int `json:"machines"`
MachinesOnline int `json:"machinesOnline"`
Gpus int `json:"gpus"`
Regions int `json:"regions"`
ModelsServed int `json:"modelsServed"`
}
// aiPulse is one snapshot of the compute plane. State is "live" when at least one
// half resolved, else "unavailable" with a Reason.
type aiPulse struct {
State string `json:"state"` // "live" | "unavailable"
Reason string `json:"reason,omitempty"`
UpdatedAt string `json:"updatedAt"`
Usage *aiUsage `json:"usage,omitempty"`
Fleet *aiFleet `json:"fleet,omitempty"`
}
// typed SSE frames. Embedding the pointer flattens its JSON fields under the
// envelope's "type", so the wire shape stays DRY with the snapshot structs.
type aiUsageEvent struct {
Type string `json:"type"` // "usage"
*aiUsage
UpdatedAt string `json:"updatedAt"`
}
type aiFleetEvent struct {
Type string `json:"type"` // "fleet"
*aiFleet
UpdatedAt string `json:"updatedAt"`
}
// handleAIPulse serves the compute pulse as SSE (EventSource) or, for a plain GET,
// a single JSON snapshot (the poll fallback). It never 5xxes.
func (s *Server) handleAIPulse(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
ctx := r.Context()
if strings.Contains(r.Header.Get("Accept"), "text/event-stream") {
if f, ok := w.(http.Flusher); ok {
h := w.Header()
h.Set("Content-Type", "text/event-stream; charset=utf-8")
h.Set("Cache-Control", "no-cache")
h.Set("X-Accel-Buffering", "no")
w.WriteHeader(http.StatusOK)
emit := func(v any) {
b, _ := json.Marshal(v)
_, _ = w.Write([]byte("data: "))
_, _ = w.Write(b)
_, _ = w.Write([]byte("\n\n"))
f.Flush()
}
s.streamAIPulse(ctx, emit)
return
}
}
// Poll fallback: one JSON snapshot (never cached downstream — it is live).
// EventSource cannot send Authorization, so the AUTHED transport is this poll:
// a signed-in admin gets a fresh per-caller snapshot built with THEIR OWN bearer
// (full measured usage + fleet), never the shared service-token snapshot.
w.Header().Set("Vary", "Authorization")
sctx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
if bearer, ok := s.adminIdentity(r); ok {
writeJSON(w, http.StatusOK, "private, no-store", s.produceAIPulse(sctx, map[string]string{"Authorization": bearer}))
return
}
writeJSON(w, http.StatusOK, "no-store", s.aiPulseSnapshot(sctx))
}
// streamAIPulse pushes the snapshot immediately, then re-emits on the interval
// until the client disconnects (ctx cancelled).
func (s *Server) streamAIPulse(ctx context.Context, emit func(any)) {
send := func(p aiPulse) {
if p.Usage != nil {
emit(aiUsageEvent{Type: "usage", aiUsage: p.Usage, UpdatedAt: p.UpdatedAt})
}
if p.Fleet != nil {
emit(aiFleetEvent{Type: "fleet", aiFleet: p.Fleet, UpdatedAt: p.UpdatedAt})
}
emit(map[string]any{"type": "status", "state": p.State, "reason": p.Reason, "updatedAt": p.UpdatedAt})
}
send(s.aiPulseSnapshot(ctx))
t := time.NewTicker(aiPulseInterval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
send(s.aiPulseSnapshot(ctx))
}
}
}
// aiPulseSnapshot returns the shared snapshot: a fresh cache hit, else one
// single-flighted produce that many concurrent SSE clients coalesce onto (so N
// streams cause ~one upstream sweep per aiPulseTTL).
func (s *Server) aiPulseSnapshot(ctx context.Context) aiPulse {
if v, ok := s.cache.Get(aiPulseKey); ok {
return v.(aiPulse)
}
v, _ := s.flight.do(aiPulseKey, func() (any, error) {
p := s.produceAIPulse(ctx, serviceAuth())
ttl := aiPulseTTL
if p.State != "live" {
ttl = aiPulseFailTTL
}
s.cache.Set(aiPulseKey, p, ttl, 30*time.Second)
return p, nil
})
return v.(aiPulse)
}
// produceAIPulse reads both honest halves using auth (the KMS service bearer for the
// public SSE, or a signed-in admin's OWN bearer for the authed poll). It is "live"
// when either resolves; "unavailable" (with a reason) when there is no auth or every
// upstream fails.
func (s *Server) produceAIPulse(ctx context.Context, auth map[string]string) aiPulse {
now := nowRFC()
if auth == nil {
return aiPulse{State: "unavailable", Reason: "sign in for live compute telemetry (or wire the service token)", UpdatedAt: now}
}
usage := s.buildAIUsage(ctx, auth)
fleet := s.buildAIFleet(ctx, auth)
if usage == nil && fleet == nil {
return aiPulse{State: "unavailable", Reason: "compute plane unreachable", UpdatedAt: now}
}
return aiPulse{State: "live", UpdatedAt: now, Usage: usage, Fleet: fleet}
}
// buildAIUsage maps the measured platform usage ledger into the compute-panel
// shape, read with auth. nil when the ledger is unreachable (no auth / non-admin /
// upstream down) so the panel degrades honestly.
func (s *Server) buildAIUsage(ctx context.Context, auth map[string]string) *aiUsage {
ov, err := s.fetchCloudUsage(ctx, "24h", auth)
if err != nil {
// Exact ledger denied → REAL platform usage from LLM observability (measured
// requests/tokens + real model names), so AI Compute shows live numbers for an
// operator instead of zeros. nil only when that is also unauthorized.
if u, ok := s.fetchLLMUsage(ctx, auth); ok {
return &aiUsage{
Window: "24h",
RequestsPerSec: round1(float64(u.Requests) / 86400),
TokensPerSec: round1(float64(u.Tokens) / 86400),
Requests24h: u.Requests,
Tokens24h: u.Tokens,
Models: u.Models,
}
}
return nil
}
window := ov.Range
if window == "" {
window = "24h"
}
return &aiUsage{
Window: window,
RequestsPerSec: round1(usageRate(ov.Totals.Requests, ov.Series, ov.Interval, seriesRequests)),
TokensPerSec: round1(usageRate(ov.Totals.Tokens, ov.Series, ov.Interval, seriesTokens)),
Requests24h: ov.Totals.Requests,
Tokens24h: ov.Totals.Tokens,
SpendCents: ov.Totals.SpendCents,
Models: topModelsFromUsage(ov),
}
}
// buildAIFleet reads the live serving fleet (visor machines/gpus + ai model
// catalog) with auth. nil only when the machines read fails; gpus/models are
// best-effort bonus counts.
func (s *Server) buildAIFleet(ctx context.Context, hdr map[string]string) *aiFleet {
if hdr == nil {
return nil
}
host := apiHost()
var machines struct {
Machines []struct {
Region string `json:"region"`
Status string `json:"status"`
} `json:"machines"`
}
if err := s.getJSON(ctx, host+"/v1/machines", hdr, &machines); err != nil {
return nil
}
var gpus struct {
Gpus []struct {
Region string `json:"region"`
} `json:"gpus"`
}
_ = s.getJSON(ctx, host+"/v1/gpus", hdr, &gpus)
var models struct {
Data []struct {
ID string `json:"id"`
} `json:"data"`
}
_ = s.getJSON(ctx, host+"/v1/models", hdr, &models)
regions := map[string]struct{}{}
online := 0
for _, m := range machines.Machines {
if machineOnline(m.Status) {
online++
}
if m.Region != "" {
regions[m.Region] = struct{}{}
}
}
for _, g := range gpus.Gpus {
if g.Region != "" {
regions[g.Region] = struct{}{}
}
}
return &aiFleet{
Machines: len(machines.Machines),
MachinesOnline: online,
Gpus: len(gpus.Gpus),
Regions: len(regions),
ModelsServed: len(models.Data),
}
}
+211
View File
@@ -0,0 +1,211 @@
package world
import (
"bufio"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
// aiPulseUpstream is a fake api.hanzo.ai serving the four reads ai-pulse folds.
func aiPulseUpstream(t *testing.T) *httptest.Server {
t.Helper()
mux := http.NewServeMux()
mux.HandleFunc("/v1/machines", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"machines":[{"region":"nyc","status":"active"},{"region":"sfo","status":"active"},{"region":"nyc","status":"stopped"}]}`))
})
mux.HandleFunc("/v1/gpus", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"gpus":[{"region":"nyc"},{"region":"sfo"},{"region":"nyc"}]}`))
})
mux.HandleFunc("/v1/models", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"data":[{"id":"zen-1"},{"id":"zen-omni-30b"}]}`))
})
mux.HandleFunc("/v1/get-cloud-usages", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{
"range":"24h","interval":"1h",
"totals":{"tokens":1180000000,"requests":1000000,"spendCents":42000,"models":2},
"series":[{"t":"2026-07-16T00:00:00Z","tokens":40000000,"requests":34000},
{"t":"2026-07-16T01:00:00Z","tokens":72000000,"requests":36000}],
"byModel":{"items":[{"model":"zen-omni-30b","spendCents":24000,"tokens":600000000,"requests":520000,"pct":57.1}]}
}`))
})
up := httptest.NewServer(mux)
t.Cleanup(up.Close)
return up
}
func aiPulseServer(t *testing.T) *httptest.Server {
t.Helper()
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
return ts
}
// TestAIPulseUnavailableWithoutToken: the honest degrade — no token, so the JSON
// snapshot is state:"unavailable" with a reason, never a zeroed "live".
func TestAIPulseUnavailableWithoutToken(t *testing.T) {
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "")
ts := aiPulseServer(t)
resp, err := http.Get(ts.URL + "/v1/world/ai-pulse")
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
var p aiPulse
if err := json.NewDecoder(resp.Body).Decode(&p); err != nil {
t.Fatalf("decode: %v", err)
}
if p.State != "unavailable" || p.Reason == "" {
t.Fatalf("want unavailable+reason, got state=%q reason=%q", p.State, p.Reason)
}
if p.Usage != nil || p.Fleet != nil {
t.Fatalf("unavailable must carry no fabricated usage/fleet")
}
}
// TestAIPulseLiveSnapshot: with a token + reachable upstream, the poll-fallback
// JSON snapshot carries measured usage and the live fleet.
func TestAIPulseLiveSnapshot(t *testing.T) {
up := aiPulseUpstream(t)
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "svc-super-admin")
t.Setenv("HANZO_API_BASE", up.URL)
ts := aiPulseServer(t)
resp, err := http.Get(ts.URL + "/v1/world/ai-pulse")
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
var p aiPulse
if err := json.NewDecoder(resp.Body).Decode(&p); err != nil {
t.Fatalf("decode: %v", err)
}
if p.State != "live" {
t.Fatalf("want state=live, got %q (reason %q)", p.State, p.Reason)
}
if p.Usage == nil || p.Usage.Requests24h != 1_000_000 || p.Usage.SpendCents != 42000 {
t.Fatalf("want measured usage, got %+v", p.Usage)
}
if len(p.Usage.Models) != 1 || p.Usage.Models[0].ID != "zen-omni-30b" {
t.Fatalf("want ledger top model, got %+v", p.Usage.Models)
}
// Recent bucket over 1h interval: 36000 req / 3600s = 10 req/s.
if p.Usage.RequestsPerSec != 10 {
t.Fatalf("want recent-bucket rate 10 req/s, got %v", p.Usage.RequestsPerSec)
}
if p.Fleet == nil || p.Fleet.MachinesOnline != 2 || p.Fleet.Machines != 3 || p.Fleet.Gpus != 3 {
t.Fatalf("want live fleet 2/3 + 3 gpus, got %+v", p.Fleet)
}
if p.Fleet.ModelsServed != 2 {
t.Fatalf("want modelsServed=2, got %d", p.Fleet.ModelsServed)
}
}
// TestAIPulseSSEFrames: an EventSource-style request gets the typed usage/fleet/
// status frames in the first (immediate) emit.
func TestAIPulseSSEFrames(t *testing.T) {
up := aiPulseUpstream(t)
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "svc-super-admin")
t.Setenv("HANZO_API_BASE", up.URL)
ts := aiPulseServer(t)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, ts.URL+"/v1/world/ai-pulse", nil)
req.Header.Set("Accept", "text/event-stream")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("sse request failed: %v", err)
}
defer resp.Body.Close()
if ct := resp.Header.Get("Content-Type"); !strings.Contains(ct, "text/event-stream") {
t.Fatalf("want event-stream, got %q", ct)
}
seen := map[string]bool{}
sc := bufio.NewScanner(resp.Body)
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
if !strings.HasPrefix(line, "data:") {
continue
}
var ev struct {
Type string `json:"type"`
State string `json:"state"`
}
if json.Unmarshal([]byte(strings.TrimPrefix(line, "data:")), &ev) != nil {
continue
}
seen[ev.Type] = true
if ev.Type == "status" { // terminal frame of the first emit
if ev.State != "live" {
t.Fatalf("want status state=live, got %q", ev.State)
}
break
}
}
for _, want := range []string{"usage", "fleet", "status"} {
if !seen[want] {
t.Fatalf("missing %q frame (saw %v)", want, seen)
}
}
}
// TestAIPulseAdminPoll proves the authed poll: a signed-in admin (z@hanzo.ai) with
// NO server-side service token gets the FULL measured compute pulse built with their
// OWN bearer, served no-store. EventSource can't carry auth, so this poll is the
// admin's live transport.
func TestAIPulseAdminPoll(t *testing.T) {
iam := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/iam/oauth/userinfo" || r.Header.Get("Authorization") == "" {
http.NotFound(w, r)
return
}
_, _ = w.Write([]byte(`{"owner":"hanzo","sub":"z@hanzo.ai"}`))
}))
t.Cleanup(iam.Close)
up := aiPulseUpstream(t)
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "") // no service token — the admin's bearer drives it
t.Setenv("HANZO_API_BASE", up.URL)
t.Setenv("HANZO_IAM_ISSUER", iam.URL)
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
req, _ := http.NewRequest(http.MethodGet, ts.URL+"/v1/world/ai-pulse", nil)
req.Header.Set("Authorization", "Bearer admin-token")
req.Header.Set("Accept", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
if cc := resp.Header.Get("Cache-Control"); cc != "private, no-store" {
t.Fatalf("admin ai-pulse poll must be no-store, got %q", cc)
}
var p aiPulse
if err := json.NewDecoder(resp.Body).Decode(&p); err != nil {
t.Fatalf("decode: %v", err)
}
if p.State != "live" {
t.Fatalf("admin poll must be live with the caller's bearer, got %q (%s)", p.State, p.Reason)
}
if p.Usage == nil || p.Usage.Requests24h != 1_000_000 {
t.Fatalf("admin must see measured usage, got %+v", p.Usage)
}
if p.Fleet == nil || p.Fleet.Gpus != 3 {
t.Fatalf("admin must see the live fleet, got %+v", p.Fleet)
}
}
+180 -47
View File
@@ -125,9 +125,13 @@ func (s *Server) handleAnalyst(w http.ResponseWriter, r *http.Request) {
return
}
// Per-user billing: forward the signed-in caller's IAM token, never a shared
// key. Signed-out callers get a quiet prompt to sign in (never a 5xx).
bearer := s.ai.bearerFor(r)
// Paid usage: the analyst chat is metered to the signed-in user's org, so it
// requires the caller's OWN IAM bearer — never the funded service key. That key
// (a.key / HANZO_AI_KEY) backs ONLY the anonymous auto-insight endpoints
// (summarize/classify/country-intel via bearerFor); chat must not silently burn
// it, or "paid usage" would never actually meter. Signed-out callers get a quiet
// sign-in prompt (never a 5xx).
bearer := userBearer(r)
if bearer == "" {
writeJSON(w, http.StatusOK, "", map[string]any{
"reply": "", "actions": []any{}, "fallback": true, "skipped": true,
@@ -185,57 +189,160 @@ func (s *Server) handleAnalyst(w http.ResponseWriter, r *http.Request) {
full = append(full, chatMessage{Role: "system", Content: system})
full = append(full, msgs...)
// Agentic loop: each turn the model may (a) request data tools, which we run
// in-process and feed back, or (b) answer. It runs at most analystMaxToolRounds
// tool rounds before a final answer is forced. Tool traces accumulate for the
// chat UI. When the model needs no data, this collapses to a single call —
// byte-for-byte the prior behaviour.
var (
reply string
actions []map[string]any
traces = make([]map[string]any, 0, analystMaxToolRounds)
tokens int
)
for round := 0; ; round++ {
out, tk, err := s.ai.chatMessagesModel(ctx, s, bearer, model, full, 0.4, 700, extra)
if err != nil || out == "" {
// Surface the upstream reason honestly — the SPA renders it instead of a
// blank chat (e.g. a 401 when aud=hanzo-world isn't allow-listed upstream).
// Any tool traces gathered so far still ride along so partial work shows.
writeJSON(w, http.StatusOK, "", map[string]any{
"reply": "", "actions": []any{}, "fallback": true, "error": errStr(err), "traces": traces,
})
return
}
tokens += tk
r, acts, calls := parseAnalystTurn(out, allowed, toolNames)
if len(calls) == 0 || round >= analystMaxToolRounds {
reply, actions = r, acts
break
}
// Execute the requested tools in-process and feed the results back as the
// next turn's grounding. The model's raw request rides as the assistant turn
// so the transcript stays coherent.
full = append(full, chatMessage{Role: "assistant", Content: out})
last := round == analystMaxToolRounds-1
full = append(full, chatMessage{Role: "user", Content: s.runAnalystTools(ctx, calls, &traces, last)})
}
served := model
if served == "" {
served = s.ai.model
}
// Streaming variant: the SPA asks with Accept: text/event-stream and gets
// live SSE events (round / delta / tool / done) instead of one JSON body.
// The done event carries EXACTLY the JSON path's payload, so the client
// treats deltas as cosmetic and done as the source of truth.
if strings.Contains(r.Header.Get("Accept"), "text/event-stream") {
if f, ok := w.(http.Flusher); ok {
h := w.Header()
h.Set("Content-Type", "text/event-stream; charset=utf-8")
h.Set("Cache-Control", "no-cache")
h.Set("X-Accel-Buffering", "no")
setCORS(w, "POST, OPTIONS")
w.WriteHeader(http.StatusOK)
emit := func(v map[string]any) {
b, _ := json.Marshal(v)
_, _ = w.Write([]byte("data: "))
_, _ = w.Write(b)
_, _ = w.Write([]byte("\n\n"))
f.Flush()
}
reply, actions, traces, tokens, id, err := s.runAnalystLoop(ctx, bearer, model, full, allowed, toolNames, extra, emit)
if err != nil || reply == "" && len(actions) == 0 {
emit(addBillingCTA(map[string]any{
"type": "done", "reply": reply, "actions": emptyIfNil(actions), "fallback": err != nil,
"error": errStr(err), "traces": traces, "model": served,
}, err))
return
}
emit(map[string]any{
"type": "done", "reply": reply, "actions": emptyIfNil(actions),
"model": served, "tokens": tokens, "traces": traces, "id": id,
})
return
}
}
reply, actions, traces, tokens, id, err := s.runAnalystLoop(ctx, bearer, model, full, allowed, toolNames, extra, nil)
if err != nil {
// Surface the upstream reason honestly — the SPA renders it instead of a
// blank chat (e.g. a 401 when aud=hanzo-world isn't allow-listed upstream).
// Any tool traces gathered so far still ride along so partial work shows.
writeJSON(w, http.StatusOK, "", addBillingCTA(map[string]any{
"reply": "", "actions": []any{}, "fallback": true, "error": errStr(err), "traces": traces,
}, err))
return
}
writeJSON(w, http.StatusOK, "", map[string]any{
"reply": reply, "actions": actions, "model": served, "tokens": tokens, "traces": traces,
"reply": reply, "actions": actions, "model": served, "tokens": tokens, "traces": traces, "id": id,
})
}
// emptyIfNil keeps the wire contract's empty array (never null) for actions.
func emptyIfNil(a []map[string]any) any {
if a == nil {
return []any{}
}
return a
}
// billingURL / usageURL are the canonical Hanzo billing surfaces the analyst
// links to when a user's AI credits are exhausted. One contract, so every
// surface (world here, and console/commerce/cms via the same payload shape)
// renders the SAME wallet CTA instead of an opaque failure.
const (
billingURL = "https://console.hanzo.ai/billing"
usageURL = "https://console.hanzo.ai/billing/usage"
)
// addBillingCTA augments an analyst response payload in place when err signals an
// exhausted balance/quota (balanceErrorFrom), turning a dead chat into an
// actionable top-up prompt. The SPA keys off topup==true to render "You're out
// of AI credits → Add credits / View usage". Returns the same map for chaining.
func addBillingCTA(m map[string]any, err error) map[string]any {
if balanceErrorFrom(err) {
m["topup"] = true
m["error"] = "insufficient_balance"
m["billingUrl"] = billingURL
m["usageUrl"] = usageURL
m["reason"] = "You're out of AI credits"
}
return m
}
// runAnalystLoop is THE agentic loop (shared by the JSON and SSE paths): each
// turn the model may (a) request data tools, which run in-process and feed
// back, or (b) answer. At most analystMaxToolRounds tool rounds before a final
// answer is forced. When emit is non-nil, live events stream out as the loop
// runs: {"type":"round"} at each model turn (so the client resets any partial
// text from a prior tool round), {"type":"delta","text":…} for the growing
// "reply" value (via replyExtractor — tool rounds stream nothing), and
// {"type":"tool",…} per executed tool. When the model needs no data, this
// collapses to a single call — byte-for-byte the prior behaviour.
func (s *Server) runAnalystLoop(ctx context.Context, bearer, model string, full []chatMessage, allowed, toolNames map[string]bool, extra map[string]string, emit func(map[string]any)) (reply string, actions []map[string]any, traces []map[string]any, tokens int, id string, err error) {
traces = make([]map[string]any, 0, analystMaxToolRounds)
for round := 0; ; round++ {
var out, rid string
var tk int
if emit != nil {
emit(map[string]any{"type": "round", "n": round})
rx := newReplyExtractor(func(text string) {
emit(map[string]any{"type": "delta", "text": text})
})
rx.emitThink = func(text string) {
emit(map[string]any{"type": "think", "text": text})
}
out, tk, rid, err = s.ai.chatMessagesModelStream(ctx, s, bearer, model, full, 0.4, 700, extra, rx.Feed)
} else {
out, tk, rid, err = s.ai.chatMessagesModel(ctx, s, bearer, model, full, 0.4, 700, extra)
}
if err != nil || out == "" {
if err == nil {
err = emptyAnswerErr(firstNonEmpty(model, s.ai.model))
}
return "", nil, traces, tokens, id, err
}
tokens += tk
// The id of the round that produces the FINAL answer is the one the reward
// signal keys on — each round overwrites, so the last (answering) round wins.
if rid != "" {
id = rid
}
r, acts, calls := parseAnalystTurn(out, allowed, toolNames)
if len(calls) == 0 || round >= analystMaxToolRounds {
return r, acts, traces, tokens, id, nil
}
// Execute the requested tools in-process and feed the results back as the
// next turn's grounding. The model's raw request rides as the assistant turn
// so the transcript stays coherent.
var onTrace func(map[string]any)
if emit != nil {
onTrace = func(tr map[string]any) {
ev := map[string]any{"type": "tool"}
for k, v := range tr {
ev[k] = v
}
emit(ev)
}
}
full = append(full, chatMessage{Role: "assistant", Content: out})
last := round == analystMaxToolRounds-1
full = append(full, chatMessage{Role: "user", Content: s.runAnalystTools(ctx, calls, &traces, last, onTrace)})
}
}
// runAnalystTools executes a round's tool calls IN-PROCESS through the mcp
// dispatcher (one way — the same path an external MCP client uses), appends a
// trace per call for the chat UI, and returns the tool-results block injected back
// into the conversation as the next user turn. Results are capped on both the
// model-facing (analystToolResultCap) and UI-facing (analystTraceResultCap) sides.
func (s *Server) runAnalystTools(ctx context.Context, calls []analystToolCall, traces *[]map[string]any, lastRound bool) string {
func (s *Server) runAnalystTools(ctx context.Context, calls []analystToolCall, traces *[]map[string]any, lastRound bool, onTrace func(map[string]any)) string {
var b strings.Builder
b.WriteString("TOOL RESULTS (live data — ground your answer in these; do not repeat the same call):\n")
for _, c := range calls {
@@ -251,11 +358,15 @@ func (s *Server) runAnalystTools(ctx context.Context, calls []analystToolCall, t
}
b.WriteString(capString(body, analystToolResultCap))
b.WriteString("\n\n")
*traces = append(*traces, map[string]any{
tr := map[string]any{
"label": label,
"ok": ok,
"result": capString(body, analystTraceResultCap),
})
}
*traces = append(*traces, tr)
if onTrace != nil {
onTrace(tr) // live SSE trace — the chat shows the tool as it runs
}
}
if lastRound {
b.WriteString("This was the final tool round. Answer the user now in prose using the data above; do not request more tools.\n")
@@ -291,10 +402,12 @@ func toolNameSet(specs []mcp.ToolSpec) map[string]bool {
// ── Model / agent roster (drives the chat's model dropdown) ──────────────────
// zenRoster is the curated Zen family — always offered first so the dropdown
// works even before upstream /v1/models is reachable. Ids are the real served
// families (api.hanzo.ai). The default (ai.go model = "zen5") leads.
// zenRoster is the curated roster — always offered first so the dropdown
// works even before upstream /v1/models is reachable. "best" leads: it is the
// gateway's own routing alias and the only id guaranteed servable across
// catalog shifts (matches the ai.go default). Zen family ids follow.
var zenRoster = []map[string]string{
{"id": "best", "label": "Best (auto)", "group": "Auto"},
{"id": "zen5", "label": "Zen 5", "group": "Zen"},
{"id": "zen5-flash", "label": "Zen 5 Flash", "group": "Zen"},
{"id": "zen5-mini", "label": "Zen 5 Mini", "group": "Zen"},
@@ -336,7 +449,7 @@ func (s *Server) handleModels(w http.ResponseWriter, r *http.Request) {
}
if id != "" && !seen[id] {
seen[id] = true
roster = append(roster, map[string]string{"id": id, "label": id, "group": "Models"})
roster = append(roster, map[string]string{"id": id, "label": id, "group": modelGroup(id)})
}
}
for _, ag := range s.ai.upstreamAgents(ctx, s, bearer, extra) {
@@ -358,6 +471,26 @@ func (s *Server) handleModels(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, "no-store", map[string]any{"data": roster, "default": s.ai.model})
}
// modelGroup buckets a served model id into its family for the dropdown — the
// server-side mirror of the client's modelFamily (src/utils/model-marks.ts), so
// e.g. gpt-oss never files under Zen.
func modelGroup(id string) string {
m := strings.ToLower(id)
switch {
case m == "best":
return "Auto"
case strings.HasPrefix(m, "zen"):
return "Zen"
case strings.HasPrefix(m, "gpt"):
return "GPT"
case strings.HasPrefix(m, "llama") || strings.Contains(m, "llama"):
return "Llama"
case strings.HasPrefix(m, "claude") || strings.HasPrefix(m, "anthropic"):
return "Claude"
}
return "Models"
}
var modelIDRe = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._:@/-]{0,79}$`)
// sanitizeModel returns a syntactically-valid model id (or "" to use the default).
+198 -7
View File
@@ -1,6 +1,7 @@
package world
import (
"context"
"encoding/json"
"io"
"net/http"
@@ -237,7 +238,6 @@ func TestAnalystDataToolLoop(t *testing.T) {
s := NewServer()
s.ai.base = ai.URL
s.ai.key = "test-key" // no user token → keyed bearer, so the chat path runs
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
@@ -246,7 +246,13 @@ func TestAnalystDataToolLoop(t *testing.T) {
reqBody, _ := json.Marshal(map[string]any{
"messages": []map[string]string{{"role": "user", "content": "What is the state of global instability?"}},
})
resp, err := http.Post(ts.URL+"/v1/world/analyst", "application/json", strings.NewReader(string(reqBody)))
// A signed-in caller drives the chat: their IAM bearer is forwarded upstream and
// meters to their org. Chat requires a user bearer — the funded key backs only the
// anonymous auto-insight endpoints (see TestAnalystChatRequiresUserBearer).
req, _ := http.NewRequest(http.MethodPost, ts.URL+"/v1/world/analyst", strings.NewReader(string(reqBody)))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer user-token")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("request failed: %v", err)
}
@@ -303,6 +309,149 @@ func readAllString(r *http.Request) (string, error) {
return string(b), err
}
// writeChatMessage writes an OpenAI-shaped completion whose single choice carries
// the given assistant message object verbatim — used to model a reasoning model that
// leaves `content` empty and puts its answer on `reasoning_content`.
func writeChatMessage(w http.ResponseWriter, message map[string]any) {
resp := map[string]any{
"id": "resp_test",
"choices": []any{map[string]any{"message": message}},
"usage": map[string]any{"total_tokens": 7},
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(resp)
}
// postAnalyst drives the non-streaming analyst path with a signed-in bearer and
// returns the decoded response envelope.
func postAnalyst(t *testing.T, ts *httptest.Server, prompt string) map[string]any {
t.Helper()
reqBody, _ := json.Marshal(map[string]any{
"messages": []map[string]string{{"role": "user", "content": prompt}},
})
req, _ := http.NewRequest(http.MethodPost, ts.URL+"/v1/world/analyst", strings.NewReader(string(reqBody)))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer user-token")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("status = %d, want 200 (never 5xx)", resp.StatusCode)
}
var out map[string]any
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
t.Fatalf("decode: %v", err)
}
return out
}
// TestAnalystRecoversReasoningWhenNoContent is the regression guard for the "empty
// response" break: a reasoning model (gpt-oss-120b / harmony) returns a 2xx with an
// EMPTY content channel and its answer on `reasoning_content`. The completion path
// must fall back to that channel so the analyst recovers the reply — not blank out.
func TestAnalystRecoversReasoningWhenNoContent(t *testing.T) {
ai := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
writeChatMessage(w, map[string]any{
"content": "",
"reasoning_content": `{"reply":"Markets are steady.","actions":[]}`,
})
}))
defer ai.Close()
s := NewServer()
s.ai.base = ai.URL
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
defer ts.Close()
out := postAnalyst(t, ts, "Reply with strict JSON only")
if got, _ := out["reply"].(string); got != "Markets are steady." {
t.Fatalf("reasoning-channel answer not recovered: reply=%q full=%+v", got, out)
}
if out["error"] != nil {
t.Errorf("recovered answer should carry no error: %+v", out["error"])
}
}
// TestAnalystEmptyAnswerNamesModel pins the honest-error contract: a genuinely empty
// 2xx (a real choice, no content AND no reasoning) surfaces "the <model> model
// returned an empty answer" — never an opaque "empty response", and never a balance
// top-up (that is a distinct 402 contract).
func TestAnalystEmptyAnswerNamesModel(t *testing.T) {
ai := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
writeChatMessage(w, map[string]any{"content": ""})
}))
defer ai.Close()
s := NewServer() // default model "best"
s.ai.base = ai.URL
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
defer ts.Close()
out := postAnalyst(t, ts, "hi")
if got, _ := out["reply"].(string); got != "" {
t.Fatalf("expected empty reply, got %q", got)
}
errStr, _ := out["error"].(string)
if strings.Contains(errStr, "empty response") {
t.Fatalf("still surfacing the opaque 'empty response': %q", errStr)
}
if !strings.Contains(errStr, "empty answer") || !strings.Contains(errStr, "best") {
t.Fatalf("empty error not honest/model-named: %q", errStr)
}
if out["topup"] == true {
t.Fatalf("empty answer must not be a balance/top-up error: %+v", out)
}
if out["fallback"] != true {
t.Errorf("empty answer should mark fallback=true: %+v", out)
}
}
// TestChatStreamRecoversReasoning covers the SSE path: a model that streams only
// `reasoning_content` deltas (no content channel) must still yield an answer, and the
// reasoning must be forwarded live (onDelta) so the UI shows the model working.
func TestChatStreamRecoversReasoning(t *testing.T) {
ai := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/event-stream")
fl, _ := w.(http.Flusher)
for _, f := range []string{
`{"id":"resp_1","choices":[{"delta":{"reasoning_content":"thinking… "}}]}`,
`{"id":"resp_1","choices":[{"delta":{"reasoning_content":"{\"reply\":\"from reasoning\"}"}}]}`,
`[DONE]`,
} {
_, _ = io.WriteString(w, "data: "+f+"\n\n")
if fl != nil {
fl.Flush()
}
}
}))
defer ai.Close()
s := NewServer()
s.ai.base = ai.URL
var streamed strings.Builder
out, _, id, err := s.ai.chatMessagesModelStream(context.Background(), s, "Bearer u", "gpt-oss-120b",
[]chatMessage{{Role: "user", Content: "hi"}}, 0.4, 700, nil, func(tok string) { streamed.WriteString(tok) })
if err != nil {
t.Fatalf("stream err: %v", err)
}
if id != "resp_1" {
t.Errorf("id = %q, want resp_1", id)
}
want := `thinking… {"reply":"from reasoning"}`
if out != want {
t.Fatalf("stream reasoning fallback wrong:\n got=%q\nwant=%q", out, want)
}
if streamed.String() != want {
t.Errorf("reasoning deltas not forwarded live: %q", streamed.String())
}
}
func TestSanitizeModelAndAgentRef(t *testing.T) {
cases := map[string]string{
"zen5": "zen5",
@@ -371,14 +520,17 @@ func TestHandleModelsCuratedRoster(t *testing.T) {
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
t.Fatalf("decode: %v", err)
}
if body.Default != "zen5" {
t.Errorf("default = %q, want zen5", body.Default)
if body.Default != "best" {
t.Errorf("default = %q, want best", body.Default)
}
if len(body.Data) == 0 {
t.Fatal("empty model roster")
}
var hasZen5 bool
var hasBest, hasZen5 bool
for _, m := range body.Data {
if m.ID == "best" {
hasBest = true
}
if m.ID == "zen5" {
hasZen5 = true
}
@@ -386,7 +538,46 @@ func TestHandleModelsCuratedRoster(t *testing.T) {
t.Errorf("model entry missing label/group: %+v", m)
}
}
if !hasZen5 {
t.Error("roster missing zen5")
if !hasBest || !hasZen5 {
t.Error("roster missing best/zen5")
}
}
// TestAnalystChatRequiresUserBearer pins the pro-model contract: the analyst CHAT
// is paid usage, metered to the signed-in user's org via their OWN IAM bearer —
// NEVER the funded service key. The funded key (a.key / HANZO_AI_KEY) backs only
// the anonymous auto-insight endpoints (summarize/classify/country-intel). So even
// with a funded key configured, an unauthenticated chat call must be refused with a
// quiet sign-in prompt (200, skipped) — not silently answered on the shared key.
//
// Regression guard: if handleAnalyst reverts to bearerFor(r), the funded key would
// satisfy the bearer, the handler would fall through to a real upstream call, and
// this test would hang/fail instead of returning the deterministic skip below.
func TestAnalystChatRequiresUserBearer(t *testing.T) {
s := NewServer()
s.ai.key = "hk-test-funded" // funded key present (backs anonymous auto-insights only)
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
defer ts.Close()
body := `{"messages":[{"role":"user","content":"hi"}],"context":""}`
resp, err := http.Post(ts.URL+"/v1/world/analyst", "application/json", strings.NewReader(body))
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("status = %d, want 200 (quiet sign-in prompt, never 5xx)", resp.StatusCode)
}
var out map[string]any
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
t.Fatalf("decode: %v", err)
}
if out["skipped"] != true {
t.Fatalf("anonymous chat not refused — funded key leaked into paid chat path: %+v", out)
}
if reason, _ := out["reason"].(string); !strings.Contains(reason, "Sign in") {
t.Errorf("reason = %q, want a sign-in prompt", reason)
}
}
+901
View File
@@ -0,0 +1,901 @@
package world
import (
"context"
"encoding/csv"
"encoding/json"
"fmt"
"math"
"net/http"
"net/url"
"regexp"
"sort"
"strconv"
"strings"
"time"
)
// China macro snapshot + official release calendar. A faithful Go port of the
// worldmonitor seed adapters (scripts/china-macro/{adapters,calendar}.mjs) and
// the merge in server/worldmonitor/economic/v1/get-china-macro-snapshot.ts.
//
// Every source parser is a small pure func over a raw body so it is unit-tested
// against the same fixtures as the TypeScript original. handleChinaMacro is the
// only impure piece: it fetches the live sources (<=2 sequential OECD requests,
// per OECD's "<60 downloads/hour" ask), builds both flows, and merges them into
// the exact upstream payload shape, cached aggressively (daily/monthly series).
const (
oecdCPIURL = "https://sdmx.oecd.org/public/rest/data/OECD.SDD.TPS,DSD_G20_PRICES@DF_G20_PRICES,1.0/CHN.M...PA...?startPeriod=2024-01&dimensionAtObservation=AllDimensions&format=csvfile"
oecdCLIURL = "https://sdmx.oecd.org/public/rest/data/OECD.SDD.STES,DSD_STES@DF_CLI,4.1/CHN.M.LI...AA...H?startPeriod=2024-01&dimensionAtObservation=AllDimensions&format=csvfile"
hkmaCNYURL = "https://api.hkma.gov.hk/public/market-data-and-statistics/monthly-statistical-bulletin/er-ir/er-eeri-daily?pagesize=2&fields=end_of_day,cny&sortby=end_of_day&sortorder=desc"
fredDEXCHUSURL = "https://api.stlouisfed.org/fred/series/observations?series_id=DEXCHUS&file_type=json&sort_order=desc&limit=2"
bisPolicyCacheKey = "economic:bis:policy:v1"
nbsCalendarIndexURL = "https://www.stats.gov.cn/english/PressRelease/ReleaseCalendar/"
chinaMoneyLPRURL = "https://www.chinamoney.com.cn/chinese/bklpr/?tab=2"
chinaMoneyLPRNoticeAPI = "https://www.chinamoney.com.cn/ags/ms/cm-s-notice-query/contentsinshorttime"
chinaMoneyLPRChannelID = "3686"
)
// chinaIndicator is one macro series in the snapshot. Value/PriorValue are
// pointers so an unavailable observation marshals as JSON null (never a fake 0),
// matching the upstream `value: null` contract.
type chinaIndicator struct {
ID string `json:"id"`
Label string `json:"label"`
Category string `json:"category"`
Value *float64 `json:"value"`
PriorValue *float64 `json:"priorValue"`
Unit string `json:"unit"`
ObservationDate string `json:"observationDate"`
Source string `json:"source"`
SourceURL string `json:"sourceUrl"`
Stale bool `json:"stale"`
UnavailableReason string `json:"unavailableReason"`
ContextOnly bool `json:"contextOnly"`
}
type chinaSourceDecision struct {
Source string `json:"source"`
Host string `json:"host"`
Status string `json:"status"`
Reason string `json:"reason"`
CheckedAt string `json:"checkedAt"`
Optional bool `json:"optional"`
RequestCount int `json:"requestCount"`
}
type chinaReleaseEvent struct {
ID string `json:"id"`
Event string `json:"event"`
CountryCode string `json:"countryCode"`
ReleaseDate string `json:"releaseDate"`
ReleaseTime string `json:"releaseTime"`
Timezone string `json:"timezone"`
Kind string `json:"kind"`
Status string `json:"status"`
Source string `json:"source"`
SourceURL string `json:"sourceUrl"`
}
// chinaMacroSnapshot is the merged payload the SPA consumes. Empty slices are
// initialized so they marshal as [] rather than null.
type chinaMacroSnapshot struct {
CountryCode string `json:"countryCode"`
GeneratedAt string `json:"generatedAt"`
Status string `json:"status"`
LaunchReady bool `json:"launchReady"`
ContentObservationDate string `json:"contentObservationDate"`
LatestObservationDate string `json:"latestObservationDate"`
Indicators []chinaIndicator `json:"indicators"`
SourceDecisions []chinaSourceDecision `json:"sourceDecisions"`
ReleaseEvents []chinaReleaseEvent `json:"releaseEvents"`
Unavailable bool `json:"unavailable"`
}
// indicatorDef is the static description of a series (labels, source, staleness
// horizon) shared by the complete/unavailable builders.
type indicatorDef struct {
id, label, category, unit, source, sourceURL string
maxAgeDays int
}
var chinaRequiredCategories = []string{"price", "activity", "policy", "fx"}
// ── observation health ───────────────────────────────────────────────────────
var (
reObsMonth = regexp.MustCompile(`^(\d{4})-(\d{2})$`)
reObsDay = regexp.MustCompile(`^\d{4}-\d{2}-\d{2}$`)
reStatusCode = regexp.MustCompile(`status (\d+)`)
)
// observationTime resolves a source timestamp to the instant used for staleness.
// Monthly values ('2026-05') anchor at month-end 23:59:59 UTC so a fresh FX tick
// never makes stale monthly content look current; daily values anchor at day-end.
func observationTime(v string) (time.Time, bool) {
if v == "" {
return time.Time{}, false
}
if m := reObsMonth.FindStringSubmatch(v); m != nil {
y, _ := strconv.Atoi(m[1])
mo, _ := strconv.Atoi(m[2])
// day 0 of month mo+1 == last day of month mo.
return time.Date(y, time.Month(mo)+1, 0, 23, 59, 59, 0, time.UTC), true
}
if reObsDay.MatchString(v) {
if t, err := time.Parse("2006-01-02", v); err == nil {
return time.Date(t.Year(), t.Month(), t.Day(), 23, 59, 59, 0, time.UTC), true
}
}
if t, err := time.Parse(time.RFC3339, v); err == nil {
return t, true
}
return time.Time{}, false
}
func isStale(obsDate string, maxAgeDays int, now time.Time) bool {
t, ok := observationTime(obsDate)
if !ok {
return true
}
return now.Sub(t) > time.Duration(maxAgeDays)*24*time.Hour
}
func unavailableIndicator(def indicatorDef, reason string, contextOnly bool) chinaIndicator {
if reason == "" {
reason = "SOURCE_UNAVAILABLE"
}
return chinaIndicator{
ID: def.id, Label: def.label, Category: def.category, Unit: def.unit,
Source: def.source, SourceURL: def.sourceURL,
UnavailableReason: reason, ContextOnly: contextOnly,
}
}
func completeIndicator(def indicatorDef, value float64, prior *float64, obsDate string, now time.Time, contextOnly bool) chinaIndicator {
stale := isStale(obsDate, def.maxAgeDays, now)
reason := ""
if stale {
reason = "STALE_OBSERVATION"
}
v := value
return chinaIndicator{
ID: def.id, Label: def.label, Category: def.category, Value: &v, PriorValue: prior,
Unit: def.unit, ObservationDate: obsDate, Source: def.source, SourceURL: def.sourceURL,
Stale: stale, UnavailableReason: reason, ContextOnly: contextOnly,
}
}
// ── source parsers (pure) ────────────────────────────────────────────────────
type dateValue struct {
date string
value float64
}
func latestTwo(rows []dateValue) (dateValue, *float64) {
sort.SliceStable(rows, func(i, j int) bool { return rows[i].date < rows[j].date })
latest := rows[len(rows)-1]
if len(rows) >= 2 {
p := rows[len(rows)-2].value
return latest, &p
}
return latest, nil
}
func csvColumn(header []string, names ...string) int {
for i, h := range header {
for _, n := range names {
if h == n {
return i
}
}
}
return -1
}
// parseOecdCsvIndicator reads an OECD SDMX CSV (CPI YoY from DF_G20_PRICES or CLI
// from DF_CLI), keeping only mainland-China (REF_AREA=CHN) observations and
// returning the latest value with the prior for change context.
func parseOecdCsvIndicator(body []byte, def indicatorDef, now time.Time) chinaIndicator {
r := csv.NewReader(strings.NewReader(string(body)))
r.FieldsPerRecord = -1
recs, _ := r.ReadAll()
if len(recs) == 0 {
return unavailableIndicator(def, "MALFORMED_RESPONSE", false)
}
header := recs[0]
area := csvColumn(header, "REF_AREA", "ReferenceArea")
date := csvColumn(header, "TIME_PERIOD", "TimePeriod")
val := csvColumn(header, "OBS_VALUE", "ObservationValue")
if area < 0 || date < 0 || val < 0 {
return unavailableIndicator(def, "MALFORMED_RESPONSE", false)
}
var rows []dateValue
for _, rec := range recs[1:] {
if area >= len(rec) || date >= len(rec) || val >= len(rec) {
continue
}
if strings.ToUpper(rec[area]) != "CHN" || rec[date] == "" {
continue
}
v, err := strconv.ParseFloat(rec[val], 64)
if err != nil || math.IsNaN(v) || math.IsInf(v, 0) {
continue
}
rows = append(rows, dateValue{rec[date], v})
}
if len(rows) == 0 {
return unavailableIndicator(def, "NO_CHINA_OBSERVATIONS", false)
}
latest, prior := latestTwo(rows)
return completeIndicator(def, latest.value, prior, latest.date, now, false)
}
var bisPolicyDef = indicatorDef{
id: "policy_rate", label: "Policy Rate", category: "policy", unit: "%",
source: "BIS (mainland China policy rate)", sourceURL: "https://stats.bis.org/api/v1/data/WS_CBPOL", maxAgeDays: 75,
}
// parseBisPolicy reads the seeded BIS central-bank policy-rate cache shape and
// returns China's latest rate. Prior falls back to the row's previousRate when
// only one observation is present.
func parseBisPolicy(body []byte, now time.Time) chinaIndicator {
var payload struct {
Rates []struct {
CountryCode string `json:"countryCode"`
Rate *float64 `json:"rate"`
PreviousRate *float64 `json:"previousRate"`
Date string `json:"date"`
} `json:"rates"`
}
if err := json.Unmarshal(body, &payload); err != nil {
return unavailableIndicator(bisPolicyDef, "NO_CHINA_POLICY_RATE", false)
}
type row struct {
date string
rate float64
previousRate *float64
}
var matches []row
for _, r := range payload.Rates {
if r.CountryCode != "CN" || r.Rate == nil || r.Date == "" || math.IsNaN(*r.Rate) {
continue
}
matches = append(matches, row{r.Date, *r.Rate, r.PreviousRate})
}
if len(matches) == 0 {
return unavailableIndicator(bisPolicyDef, "NO_CHINA_POLICY_RATE", false)
}
sort.SliceStable(matches, func(i, j int) bool { return matches[i].date < matches[j].date })
latest := matches[len(matches)-1]
var prior *float64
if len(matches) > 1 {
p := matches[len(matches)-2].rate
prior = &p
} else if latest.previousRate != nil && !math.IsNaN(*latest.previousRate) {
p := *latest.previousRate
prior = &p
}
return completeIndicator(bisPolicyDef, latest.rate, prior, latest.date, now, false)
}
var fredUsdCnyDef = indicatorDef{
id: "usd_cny", label: "USD/CNY", category: "fx", unit: "CNY per USD",
source: "FRED DEXCHUS (Federal Reserve H.10)", sourceURL: "https://fred.stlouisfed.org/series/DEXCHUS", maxAgeDays: 10,
}
// parseFredUsdCny reads FRED DEXCHUS observations (values are strings, '.' for
// missing) and returns the latest USD/CNY fixing.
func parseFredUsdCny(body []byte, now time.Time) chinaIndicator {
var payload struct {
Observations []struct {
Date string `json:"date"`
Value string `json:"value"`
} `json:"observations"`
}
if err := json.Unmarshal(body, &payload); err != nil {
return unavailableIndicator(fredUsdCnyDef, "NO_CURRENT_DEXCHUS", false)
}
var rows []dateValue
for _, o := range payload.Observations {
v, err := strconv.ParseFloat(o.Value, 64)
if o.Date == "" || err != nil || math.IsNaN(v) || math.IsInf(v, 0) {
continue
}
rows = append(rows, dateValue{o.Date, v})
}
if len(rows) == 0 {
return unavailableIndicator(fredUsdCnyDef, "NO_CURRENT_DEXCHUS", false)
}
latest, prior := latestTwo(rows)
return completeIndicator(fredUsdCnyDef, latest.value, prior, latest.date, now, false)
}
var hkmaCnyDef = indicatorDef{
id: "cnh_context", label: "CNY/HKD Context", category: "context", unit: "HKD per CNY",
source: "HKMA (Hong Kong/CNH context)",
sourceURL: "https://apidocs.hkma.gov.hk/documentation/market-data-and-statistics/monthly-statistical-bulletin/er-ir/er-eeri-daily/",
maxAgeDays: 10,
}
// parseHkmaCnyContext reads the optional HKMA daily CNY/HKD context series. It is
// context-only: it never gates the launch decision.
func parseHkmaCnyContext(body []byte, now time.Time) chinaIndicator {
var payload struct {
Result struct {
Records []struct {
EndOfDay string `json:"end_of_day"`
CNY *float64 `json:"cny"`
} `json:"records"`
} `json:"result"`
}
if err := json.Unmarshal(body, &payload); err != nil {
return unavailableIndicator(hkmaCnyDef, "NO_HKMA_CNY_CONTEXT", true)
}
var rows []dateValue
for _, rec := range payload.Result.Records {
if rec.EndOfDay == "" || rec.CNY == nil || math.IsNaN(*rec.CNY) {
continue
}
rows = append(rows, dateValue{rec.EndOfDay, *rec.CNY})
}
if len(rows) == 0 {
return unavailableIndicator(hkmaCnyDef, "NO_HKMA_CNY_CONTEXT", true)
}
latest, prior := latestTwo(rows)
return completeIndicator(hkmaCnyDef, latest.value, prior, latest.date, now, true)
}
// ── release calendar (pure) ──────────────────────────────────────────────────
var (
reNbsRow = regexp.MustCompile(`(?is)<tr\b[^>]*>(.*?)</tr>`)
reNbsCell = regexp.MustCompile(`(?is)<t[dh]\b[^>]*>(.*?)</t[dh]>`)
reNbsBr = regexp.MustCompile(`(?i)<br\s*/?\s*>`)
reNbsTag = regexp.MustCompile(`<[^>]+>`)
reNbsSpaces = regexp.MustCompile(`[ \t]+`)
reNbsNumeric = regexp.MustCompile(`^\d+$`)
reNbsEllipsis = regexp.MustCompile(`^(\x{2026}+|\.{3,})$`)
reNbsDay = regexp.MustCompile(`(^|\s)(\d{1,2})\s*/[A-Za-z]+`)
reNbsTime = regexp.MustCompile(`\b(\d{1,2}:\d{2})\b`)
reNbsWS = regexp.MustCompile(`\s`)
)
func stripHTML(v string) string {
v = reNbsBr.ReplaceAllString(v, "\n")
v = reNbsTag.ReplaceAllString(v, " ")
v = strings.ReplaceAll(v, "&nbsp;", " ")
v = strings.ReplaceAll(v, "&#160;", " ")
v = strings.ReplaceAll(v, "&amp;", "&")
v = strings.ReplaceAll(v, "&#39;", "'")
v = strings.ReplaceAll(v, "&apos;", "'")
v = strings.ReplaceAll(v, "&quot;", "\"")
v = reNbsSpaces.ReplaceAllString(v, " ")
return strings.TrimSpace(v)
}
func nbsCells(row string) []string {
matches := reNbsCell.FindAllStringSubmatch(row, -1)
cells := make([]string, 0, len(matches))
for _, m := range matches {
cells = append(cells, stripHTML(m[1]))
}
return cells
}
func isoDate(year, month, day int) string {
return fmt.Sprintf("%04d-%02d-%02d", year, month, day)
}
// parseNbsReleaseCalendar scrapes the National Bureau of Statistics HTML release
// grid (one row per statistic, one column per month) into dated release events.
// Blank/ellipsis month cells stay empty; a cell may carry several days (e.g. the
// Spring-Festival-shifted PMI) and each becomes its own event.
func parseNbsReleaseCalendar(html []byte, year int, sourceURL string) []chinaReleaseEvent {
var events []chinaReleaseEvent
for _, rowMatch := range reNbsRow.FindAllStringSubmatch(string(html), -1) {
cells := nbsCells(rowMatch[1])
if len(cells) < 14 || !reNbsNumeric.MatchString(cells[0]) {
continue
}
event := cells[1]
rowNo, _ := strconv.Atoi(cells[0])
for month := 1; month <= 12; month++ {
cell := ""
if month+1 < len(cells) {
cell = cells[month+1]
}
if cell == "" || reNbsEllipsis.MatchString(reNbsWS.ReplaceAllString(cell, "")) {
continue
}
releaseTime := "09:30"
if m := reNbsTime.FindStringSubmatch(cell); m != nil {
releaseTime = m[1]
}
for _, dm := range reNbsDay.FindAllStringSubmatch(cell, -1) {
day, _ := strconv.Atoi(dm[2])
releaseDate := isoDate(year, month, day)
events = append(events, chinaReleaseEvent{
ID: fmt.Sprintf("nbs-%02d-%s", rowNo, releaseDate),
Event: event,
CountryCode: "CN",
ReleaseDate: releaseDate,
ReleaseTime: releaseTime,
Timezone: "Asia/Shanghai",
Kind: "nbs",
Status: "scheduled",
Source: "National Bureau of Statistics of China",
SourceURL: sourceURL,
})
}
}
}
sortReleaseEvents(events)
return events
}
func sortReleaseEvents(events []chinaReleaseEvent) {
sort.SliceStable(events, func(i, j int) bool {
if events[i].ReleaseDate != events[j].ReleaseDate {
return events[i].ReleaseDate < events[j].ReleaseDate
}
return events[i].Event < events[j].Event
})
}
// chinaBusinessCalendar is the official mainland-China holiday + adjusted-workday
// set for a year. LPR candidate dates are rolled forward over these.
type chinaBusinessCalendar struct {
holidays map[string]bool
adjustedWorkdays map[string]bool
}
func toSet(days ...string) map[string]bool {
m := make(map[string]bool, len(days))
for _, d := range days {
m[d] = true
}
return m
}
// chinaBusinessCalendars is the hardcoded official calendar. 2027 must be added
// here before January 2027, or buildLprCandidates fails closed for that year.
var chinaBusinessCalendars = map[int]chinaBusinessCalendar{
2026: {
holidays: toSet(
"2026-01-01", "2026-01-02", "2026-01-03",
"2026-02-15", "2026-02-16", "2026-02-17", "2026-02-18", "2026-02-19", "2026-02-20", "2026-02-21", "2026-02-22", "2026-02-23",
"2026-04-04", "2026-04-05", "2026-04-06",
"2026-05-01", "2026-05-02", "2026-05-03", "2026-05-04", "2026-05-05",
"2026-06-19", "2026-06-20", "2026-06-21",
"2026-09-25", "2026-09-26", "2026-09-27",
"2026-10-01", "2026-10-02", "2026-10-03", "2026-10-04", "2026-10-05", "2026-10-06", "2026-10-07",
),
adjustedWorkdays: toSet("2026-01-04", "2026-02-14", "2026-02-28", "2026-05-09", "2026-09-20", "2026-10-10"),
},
}
func businessCalendar(year int) (chinaBusinessCalendar, error) {
if cal, ok := chinaBusinessCalendars[year]; ok {
return cal, nil
}
return chinaBusinessCalendar{}, fmt.Errorf("CHINA_HOLIDAY_CALENDAR_UNAVAILABLE:%d", year)
}
func isChinaBusinessDay(d time.Time, cal chinaBusinessCalendar) bool {
iso := d.UTC().Format("2006-01-02")
if cal.adjustedWorkdays[iso] {
return true
}
if cal.holidays[iso] {
return false
}
wd := d.UTC().Weekday()
return wd != time.Sunday && wd != time.Saturday
}
// buildLprCandidates derives the provisional PBoC Loan Prime Rate release dates:
// the 20th of each month rolled forward to the next mainland-China business day.
// Realized dates are confirmed later via ChinaMoney (mergeVerifiedLprDates).
func buildLprCandidates(year int) ([]chinaReleaseEvent, error) {
cal, err := businessCalendar(year)
if err != nil {
return nil, err
}
events := make([]chinaReleaseEvent, 0, 12)
for month := 1; month <= 12; month++ {
d := time.Date(year, time.Month(month), 20, 0, 0, 0, 0, time.UTC)
for !isChinaBusinessDay(d, cal) {
d = d.AddDate(0, 0, 1)
}
releaseDate := d.UTC().Format("2006-01-02")
events = append(events, chinaReleaseEvent{
ID: "pboc-lpr-" + releaseDate[:7],
Event: "Loan Prime Rate (LPR)",
CountryCode: "CN",
ReleaseDate: releaseDate,
ReleaseTime: "09:00",
Timezone: "Asia/Shanghai",
Kind: "pboc_lpr",
Status: "provisional",
Source: "PBoC rule; realized date verified by ChinaMoney/CFETS",
SourceURL: chinaMoneyLPRURL,
})
}
return events, nil
}
var (
reLprNoticeTitle = regexp.MustCompile(`受权公布贷款市场报价利率.*LPR`)
reLprNoticeDate = regexp.MustCompile(`^20\d{2}-\d{2}-\d{2}$`)
)
// parseChinaMoneyLprNotices extracts the realized LPR announcement dates from the
// ChinaMoney notice feed, keeping only genuine rate-publication notices.
func parseChinaMoneyLprNotices(body []byte) []string {
var payload struct {
Records []struct {
Title string `json:"title"`
ReleaseDate string `json:"releaseDate"`
} `json:"records"`
}
if err := json.Unmarshal(body, &payload); err != nil {
return nil
}
seen := map[string]bool{}
var dates []string
for _, rec := range payload.Records {
if !reLprNoticeTitle.MatchString(rec.Title) {
continue
}
d := rec.ReleaseDate
if len(d) > 10 {
d = d[:10]
}
if !reLprNoticeDate.MatchString(d) || seen[d] {
continue
}
seen[d] = true
dates = append(dates, d)
}
sort.Strings(dates)
return dates
}
// mergeVerifiedLprDates promotes each provisional candidate to a verified date
// when ChinaMoney published a realized announcement in the same month.
func mergeVerifiedLprDates(candidates []chinaReleaseEvent, realizedDates []string) []chinaReleaseEvent {
realizedByMonth := map[string]string{}
for _, d := range realizedDates {
if len(d) >= 7 {
realizedByMonth[d[:7]] = d
}
}
out := make([]chinaReleaseEvent, len(candidates))
for i, c := range candidates {
out[i] = c
if realized, ok := realizedByMonth[c.ReleaseDate[:7]]; ok {
out[i].ReleaseDate = realized
out[i].Status = "verified"
out[i].ID = "pboc-lpr-" + realized[:7]
}
}
return out
}
// currentCalendarLink resolves the year's NBS calendar page from the index,
// refusing any link that is not on the trusted www.stats.gov.cn release-calendar
// path so a tampered index can never redirect the scrape off-origin.
func currentCalendarLink(indexHTML string, year int) (string, error) {
re := regexp.MustCompile(`href=["']([^"']+)["'][^>]*>[^<]*` + strconv.Itoa(year) + `[^<]*<`)
m := re.FindStringSubmatch(indexHTML)
if m == nil {
return nbsCalendarIndexURL, nil
}
base, err := url.Parse(nbsCalendarIndexURL)
if err != nil {
return "", fmt.Errorf("NBS_CALENDAR_LINK_REJECTED:UNTRUSTED_NBS_CALENDAR_URL")
}
ref, err := base.Parse(m[1])
if err != nil {
return "", fmt.Errorf("NBS_CALENDAR_LINK_REJECTED:UNTRUSTED_NBS_CALENDAR_URL")
}
trustedOrigin := ref.Scheme == "https" && ref.Host == "www.stats.gov.cn"
trustedPath := strings.HasPrefix(ref.Path, "/english/PressRelease/ReleaseCalendar/")
if !trustedOrigin || !trustedPath {
return "", fmt.Errorf("NBS_CALENDAR_LINK_REJECTED:UNTRUSTED_NBS_CALENDAR_URL")
}
return ref.String(), nil
}
// ── merge ────────────────────────────────────────────────────────────────────
func requiredIndicator(indicators []chinaIndicator, category string) *chinaIndicator {
for i := range indicators {
if indicators[i].Category == category && !indicators[i].ContextOnly {
return &indicators[i]
}
}
return nil
}
// buildChinaMacroSnapshot applies the launch gate: launchReady only when all four
// required categories (price/activity/policy/fx) carry a current, non-stale
// value. contentObservationDate is the OLDEST required observation — the anchor
// that keeps a fresh FX tick from masking stale CPI/activity content.
func buildChinaMacroSnapshot(indicators []chinaIndicator, decisions []chinaSourceDecision, generatedAt string) chinaMacroSnapshot {
launchReady := true
var requiredDates []string
for _, cat := range chinaRequiredCategories {
ind := requiredIndicator(indicators, cat)
if ind == nil || ind.Value == nil || ind.Stale || ind.UnavailableReason != "" {
launchReady = false
}
if ind != nil && ind.ObservationDate != "" {
requiredDates = append(requiredDates, ind.ObservationDate)
}
}
sort.Strings(requiredDates)
anyValue := false
for i := range indicators {
if indicators[i].Value != nil {
anyValue = true
break
}
}
status := "unavailable"
switch {
case launchReady:
status = "ready"
case anyValue:
status = "degraded"
}
content := ""
if launchReady && len(requiredDates) == len(chinaRequiredCategories) {
content = requiredDates[0]
}
latest := ""
if len(requiredDates) > 0 {
latest = requiredDates[len(requiredDates)-1]
}
return chinaMacroSnapshot{
CountryCode: "CN", GeneratedAt: generatedAt, Status: status, LaunchReady: launchReady,
ContentObservationDate: content, LatestObservationDate: latest,
Indicators: indicators, SourceDecisions: decisions,
ReleaseEvents: []chinaReleaseEvent{},
}
}
func chinaDecision(source, host, status, reason, checkedAt string, optional bool, requestCount int) chinaSourceDecision {
return chinaSourceDecision{Source: source, Host: host, Status: status, Reason: reason, CheckedAt: checkedAt, Optional: optional, RequestCount: requestCount}
}
func chinaUnavailable() chinaMacroSnapshot {
return chinaMacroSnapshot{
CountryCode: "CN", Status: "unavailable",
Indicators: []chinaIndicator{}, SourceDecisions: []chinaSourceDecision{}, ReleaseEvents: []chinaReleaseEvent{},
Unavailable: true,
}
}
// ── handler ──────────────────────────────────────────────────────────────────
// handleChinaMacro serves GET /v1/world/china-macro: the merged China macro
// snapshot + official release calendar. Cached 30m fresh / 6h stale (these are
// daily/monthly series); on a required-source outage it serves last-good stale,
// else an honest unavailable payload.
func (s *Server) handleChinaMacro(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
s.cachedJSON(w, "china:macro:v1",
"public, max-age=1800, s-maxage=1800, stale-while-revalidate=600",
30*time.Minute, 6*time.Hour,
func(ctx context.Context) (any, error) { return s.chinaMacro(ctx) },
func(w http.ResponseWriter, _ error) { writeJSON(w, http.StatusOK, "", chinaUnavailable()) })
}
// oecdHeaders carries the Accept-Language the OECD CLI endpoint requires (it
// answers 500 to language-less clients even where curl succeeds).
var oecdHeaders = map[string]string{
"Accept": "text/csv, text/plain;q=0.9, */*;q=0.1",
"Accept-Language": "en",
"User-Agent": browserUA,
}
func chinaReasonFor(err error) string {
if err == nil {
return "OK"
}
msg := strings.ToLower(err.Error())
if strings.Contains(msg, "timeout") || strings.Contains(msg, "deadline") {
return "TIMEOUT"
}
if m := reStatusCode.FindStringSubmatch(msg); m != nil {
return "HTTP_" + m[1]
}
return "FETCH_FAILED"
}
func (s *Server) chinaMacro(ctx context.Context) (any, error) {
now := time.Now().UTC()
checkedAt := now.Format(time.RFC3339)
var decisions []chinaSourceDecision
// 1. OECD — required. Two sequential dataset requests, no retry: OECD asks
// consumers to stay under 60 downloads/hour, so a failure preserves last-good
// rather than replaying the flow.
cpiCSV, err := s.getText(ctx, oecdCPIURL, oecdHeaders)
reqCount := 1
if err == nil {
var cliCSV string
cliCSV, err = s.getText(ctx, oecdCLIURL, oecdHeaders)
reqCount = 2
if err == nil {
decisions = append(decisions, chinaDecision("OECD Data Explorer", "sdmx.oecd.org", "accepted", "OK", checkedAt, false, reqCount))
return s.chinaMerge(ctx, now, checkedAt, decisions, []byte(cpiCSV), []byte(cliCSV))
}
}
reason := chinaReasonFor(err)
decisions = append(decisions, chinaDecision("OECD Data Explorer", "sdmx.oecd.org", "blocked", reason, checkedAt, false, reqCount))
return nil, fmt.Errorf("OECD_REQUIRED_SOURCE_UNAVAILABLE:%s", reason)
}
func (s *Server) chinaMerge(ctx context.Context, now time.Time, checkedAt string, decisions []chinaSourceDecision, cpiCSV, cliCSV []byte) (any, error) {
indicators := []chinaIndicator{
parseOecdCsvIndicator(cpiCSV, indicatorDef{id: "cpi_yoy", label: "CPI (YoY)", category: "price", unit: "%", source: "OECD Data Explorer", sourceURL: oecdCPIURL, maxAgeDays: 120}, now),
parseOecdCsvIndicator(cliCSV, indicatorDef{id: "activity_cli", label: "Composite Leading Indicator", category: "activity", unit: "index", source: "OECD Data Explorer", sourceURL: oecdCLIURL, maxAgeDays: 120}, now),
}
// 2. BIS policy rate — read from the seed cache key the worldmonitor BIS job
// populates. Our fork ships no such job, so the key is empty and the indicator
// is honestly not_configured; parseBisPolicy lights up the moment it is seeded.
if b, ok := s.kvGet(ctx, bisPolicyCacheKey); ok {
ind := parseBisPolicy(b, now)
indicators = append(indicators, ind)
decisions = append(decisions, chinaDecision("BIS seed cache", "hanzo-kv", statusFor(ind), reasonOrOK(ind), checkedAt, false, 1))
} else {
indicators = append(indicators, unavailableIndicator(bisPolicyDef, "not_configured", false))
decisions = append(decisions, chinaDecision("BIS seed cache", "hanzo-kv", "blocked", "not_configured", checkedAt, false, 0))
}
// 3. FRED DEXCHUS — reuse the FRED_API_KEY env path the fred-data handler uses.
if key := env("FRED_API_KEY"); key != "" {
b, status, err := s.get(ctx, fredDEXCHUSURL+"&api_key="+key, map[string]string{"Accept": "application/json"})
if err == nil && status >= 200 && status < 300 {
ind := parseFredUsdCny(b, now)
indicators = append(indicators, ind)
decisions = append(decisions, chinaDecision("FRED DEXCHUS", "api.stlouisfed.org", statusFor(ind), reasonOrOK(ind), checkedAt, false, 1))
} else {
reason := chinaReasonFor(orStatusErr(err, status))
indicators = append(indicators, unavailableIndicator(fredUsdCnyDef, reason, false))
decisions = append(decisions, chinaDecision("FRED DEXCHUS", "api.stlouisfed.org", "blocked", reason, checkedAt, false, 1))
}
} else {
indicators = append(indicators, unavailableIndicator(fredUsdCnyDef, "not_configured", false))
decisions = append(decisions, chinaDecision("FRED DEXCHUS", "api.stlouisfed.org", "blocked", "not_configured", checkedAt, false, 0))
}
// 4. HKMA CNY/HKD context — optional.
if b, status, err := s.get(ctx, hkmaCNYURL, map[string]string{"Accept": "application/json", "User-Agent": browserUA}); err == nil && status >= 200 && status < 300 {
ind := parseHkmaCnyContext(b, now)
indicators = append(indicators, ind)
decisions = append(decisions, chinaDecision("HKMA CNY context", "api.hkma.gov.hk", statusFor(ind), reasonOrOK(ind), checkedAt, true, 1))
} else {
reason := chinaReasonFor(orStatusErr(err, status))
indicators = append(indicators, unavailableIndicator(hkmaCnyDef, reason, true))
decisions = append(decisions, chinaDecision("HKMA CNY context", "api.hkma.gov.hk", "blocked", reason, checkedAt, true, 1))
}
snapshot := buildChinaMacroSnapshot(indicators, decisions, checkedAt)
// Calendar is required for the merged product (mirrors get-china-macro-
// snapshot.ts, which returns unavailable when either indicators or events are
// empty). On outage, return an error so cachedJSON serves last-good stale.
events, calDecisions, calErr := s.chinaCalendar(ctx, now, checkedAt)
snapshot.SourceDecisions = append(snapshot.SourceDecisions, calDecisions...)
if calErr != nil || len(events) == 0 {
return nil, fmt.Errorf("CHINA_CALENDAR_UNAVAILABLE")
}
snapshot.ReleaseEvents = events
return snapshot, nil
}
func (s *Server) chinaCalendar(ctx context.Context, now time.Time, checkedAt string) ([]chinaReleaseEvent, []chinaSourceDecision, error) {
year := now.Year()
var decisions []chinaSourceDecision
htmlHeaders := map[string]string{"Accept": "text/html,application/xhtml+xml", "User-Agent": browserUA}
indexHTML, err := s.getText(ctx, nbsCalendarIndexURL, htmlHeaders)
nbsReq := 1
if err != nil {
decisions = append(decisions, chinaDecision("NBS release calendar", "www.stats.gov.cn", "blocked", chinaReasonFor(err), checkedAt, false, nbsReq))
return nil, decisions, err
}
calURL, err := currentCalendarLink(indexHTML, year)
if err != nil {
decisions = append(decisions, chinaDecision("NBS release calendar", "www.stats.gov.cn", "blocked", "UNTRUSTED_NBS_CALENDAR_URL", checkedAt, false, nbsReq))
return nil, decisions, err
}
calHTML := indexHTML
if calURL != nbsCalendarIndexURL {
nbsReq = 2
calHTML, err = s.getText(ctx, calURL, htmlHeaders)
if err != nil {
decisions = append(decisions, chinaDecision("NBS release calendar", "www.stats.gov.cn", "blocked", chinaReasonFor(err), checkedAt, false, nbsReq))
return nil, decisions, err
}
}
nbsEvents := parseNbsReleaseCalendar([]byte(calHTML), year, calURL)
if len(nbsEvents) == 0 {
decisions = append(decisions, chinaDecision("NBS release calendar", "www.stats.gov.cn", "blocked", "NO_NBS_EVENTS", checkedAt, false, nbsReq))
return nil, decisions, fmt.Errorf("NBS_REQUIRED_SOURCE_UNAVAILABLE:NO_NBS_EVENTS")
}
decisions = append(decisions, chinaDecision("NBS release calendar", "www.stats.gov.cn", "accepted", "OK", checkedAt, false, nbsReq))
lpr, err := buildLprCandidates(year)
if err != nil {
decisions = append(decisions, chinaDecision("PBoC/ChinaMoney LPR verification", "www.chinamoney.com.cn", "blocked", "CHINA_HOLIDAY_CALENDAR_UNAVAILABLE", checkedAt, false, 0))
return nil, decisions, err
}
if notices, err := s.chinaMoneyNotices(ctx); err == nil {
lpr = mergeVerifiedLprDates(lpr, parseChinaMoneyLprNotices(notices))
decisions = append(decisions, chinaDecision("PBoC/ChinaMoney LPR verification", "www.chinamoney.com.cn", "accepted", "OK", checkedAt, false, 1))
} else {
decisions = append(decisions, chinaDecision("PBoC/ChinaMoney LPR verification", "www.chinamoney.com.cn", "blocked", chinaReasonFor(err), checkedAt, false, 1))
}
events := append(nbsEvents, lpr...)
sortReleaseEvents(events)
return events, decisions, nil
}
func (s *Server) chinaMoneyNotices(ctx context.Context) ([]byte, error) {
body := []byte(url.Values{"channelId": {chinaMoneyLPRChannelID}, "pageSize": {"24"}, "pageNo": {"1"}}.Encode())
b, status, err := s.do(ctx, http.MethodPost, chinaMoneyLPRNoticeAPI, map[string]string{
"Accept": "application/json",
"Content-Type": "application/x-www-form-urlencoded; charset=UTF-8",
"User-Agent": browserUA,
}, body)
if err != nil {
return nil, err
}
if status < 200 || status >= 300 {
return nil, httpErr(status)
}
return b, nil
}
// kvGet reads a raw value from the shared hot cache, reporting absence (or a
// disabled cache) as ok=false so callers degrade to not_configured.
func (s *Server) kvGet(ctx context.Context, key string) ([]byte, bool) {
if s.kv == nil {
return nil, false
}
b, ok := s.kv.GetBytes(ctx, key)
return b, ok && len(b) > 0
}
func statusFor(ind chinaIndicator) string {
if ind.Value == nil {
return "blocked"
}
return "accepted"
}
func reasonOrOK(ind chinaIndicator) string {
if ind.UnavailableReason != "" {
return ind.UnavailableReason
}
return "OK"
}
func orStatusErr(err error, status int) error {
if err != nil {
return err
}
return httpErr(status)
}
+244
View File
@@ -0,0 +1,244 @@
package world
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func chinaFixture(t *testing.T, name string) []byte {
t.Helper()
b, err := os.ReadFile(filepath.Join("testdata", "china-macro", name))
if err != nil {
t.Fatalf("read fixture %s: %v", name, err)
}
return b
}
var chinaNow = time.Date(2026, 7, 13, 0, 0, 0, 0, time.UTC)
func deref(t *testing.T, p *float64) float64 {
t.Helper()
if p == nil {
t.Fatalf("expected a value, got nil")
}
return *p
}
// Parity with the TypeScript adapters: each source parser normalizes to
// independent value / prior / date / source / staleness fields.
func TestChinaSourceParsers(t *testing.T) {
cpi := parseOecdCsvIndicator(chinaFixture(t, "oecd-cpi.csv"),
indicatorDef{id: "cpi_yoy", label: "CPI (YoY)", category: "price", unit: "%", source: "OECD Data Explorer", maxAgeDays: 120}, chinaNow)
if deref(t, cpi.Value) != 0.6 || deref(t, cpi.PriorValue) != 0.3 || cpi.ObservationDate != "2026-05" ||
cpi.Source != "OECD Data Explorer" || cpi.Stale || cpi.UnavailableReason != "" {
t.Fatalf("cpi mismatch: %+v", cpi)
}
cli := parseOecdCsvIndicator(chinaFixture(t, "oecd-cli.csv"),
indicatorDef{id: "activity_cli", label: "Composite Leading Indicator", category: "activity", unit: "index", source: "OECD Data Explorer", maxAgeDays: 120}, chinaNow)
if deref(t, cli.Value) != 99.58 {
t.Fatalf("cli value = %v, want 99.58", cli.Value)
}
policy := parseBisPolicy(chinaFixture(t, "bis-policy.json"), chinaNow)
if deref(t, policy.Value) != 3 || deref(t, policy.PriorValue) != 3.1 {
t.Fatalf("policy mismatch: value=%v prior=%v", policy.Value, policy.PriorValue)
}
fx := parseFredUsdCny(chinaFixture(t, "fred-dexchus.json"), chinaNow)
if deref(t, fx.Value) != 7.1842 {
t.Fatalf("fx value = %v, want 7.1842", fx.Value)
}
hkma := parseHkmaCnyContext(chinaFixture(t, "hkma-cny.json"), chinaNow)
if !hkma.ContextOnly || hkma.Source != "HKMA (Hong Kong/CNH context)" || deref(t, hkma.Value) != 1.0881 {
t.Fatalf("hkma mismatch: %+v", hkma)
}
}
// An old observation is stale even when the fetch itself is fresh.
func TestChinaStaleObservation(t *testing.T) {
stale := parseOecdCsvIndicator(chinaFixture(t, "oecd-cpi.csv"),
indicatorDef{id: "cpi_yoy", label: "CPI (YoY)", category: "price", unit: "%", source: "OECD Data Explorer", maxAgeDays: 30},
time.Date(2027, 1, 1, 0, 0, 0, 0, time.UTC))
if !stale.Stale || stale.UnavailableReason != "STALE_OBSERVATION" {
t.Fatalf("expected stale STALE_OBSERVATION, got stale=%v reason=%q", stale.Stale, stale.UnavailableReason)
}
}
// The launch gate needs current price, activity, policy, and FX; the oldest
// required observation anchors contentObservationDate. Optional context states
// are retained and never gate launch.
func TestChinaLaunchGate(t *testing.T) {
indicators := []chinaIndicator{
parseOecdCsvIndicator(chinaFixture(t, "oecd-cpi.csv"), indicatorDef{id: "cpi_yoy", category: "price", unit: "%", source: "OECD Data Explorer", maxAgeDays: 120}, chinaNow),
parseOecdCsvIndicator(chinaFixture(t, "oecd-cli.csv"), indicatorDef{id: "activity_cli", category: "activity", unit: "index", source: "OECD Data Explorer", maxAgeDays: 120}, chinaNow),
parseBisPolicy(chinaFixture(t, "bis-policy.json"), chinaNow),
parseFredUsdCny(chinaFixture(t, "fred-dexchus.json"), chinaNow),
unavailableIndicator(hkmaCnyDef, "HOST_BLOCKED", true),
}
snap := buildChinaMacroSnapshot(indicators, nil, chinaNow.Format(time.RFC3339))
if !snap.LaunchReady || snap.Status != "ready" {
t.Fatalf("expected ready/launch, got status=%q launchReady=%v", snap.Status, snap.LaunchReady)
}
if snap.ContentObservationDate != "2026-05" {
t.Fatalf("contentObservationDate = %q, want oldest required 2026-05", snap.ContentObservationDate)
}
if snap.LatestObservationDate != "2026-07-10" {
t.Fatalf("latestObservationDate = %q, want 2026-07-10", snap.LatestObservationDate)
}
if last := snap.Indicators[len(snap.Indicators)-1]; last.UnavailableReason != "HOST_BLOCKED" {
t.Fatalf("optional context state not retained: %+v", last)
}
// A stale required category drops launch readiness to degraded.
indicators[1].Stale = true
degraded := buildChinaMacroSnapshot(indicators, nil, chinaNow.Format(time.RFC3339))
if degraded.LaunchReady || degraded.Status != "degraded" || degraded.ContentObservationDate != "" {
t.Fatalf("expected degraded with blank content date, got %+v", degraded)
}
}
// A missing policy source (our fork has no BIS feed) is honestly not_configured
// and holds launch back rather than faking readiness.
func TestChinaNotConfiguredPolicyBlocksLaunch(t *testing.T) {
indicators := []chinaIndicator{
parseOecdCsvIndicator(chinaFixture(t, "oecd-cpi.csv"), indicatorDef{id: "cpi_yoy", category: "price", source: "OECD Data Explorer", maxAgeDays: 120}, chinaNow),
parseOecdCsvIndicator(chinaFixture(t, "oecd-cli.csv"), indicatorDef{id: "activity_cli", category: "activity", source: "OECD Data Explorer", maxAgeDays: 120}, chinaNow),
unavailableIndicator(bisPolicyDef, "not_configured", false),
parseFredUsdCny(chinaFixture(t, "fred-dexchus.json"), chinaNow),
}
snap := buildChinaMacroSnapshot(indicators, nil, chinaNow.Format(time.RFC3339))
if snap.LaunchReady || snap.Status != "degraded" {
t.Fatalf("not_configured policy should degrade, got status=%q launchReady=%v", snap.Status, snap.LaunchReady)
}
if got := requiredIndicator(snap.Indicators, "policy"); got == nil || got.UnavailableReason != "not_configured" {
t.Fatalf("policy indicator not marked not_configured: %+v", got)
}
}
// The NBS grid keeps blank months empty and captures quarterly + Spring-Festival
// shifted releases.
func TestChinaParseNbsReleaseCalendar(t *testing.T) {
events := parseNbsReleaseCalendar(chinaFixture(t, "nbs-calendar.html"), 2026,
"https://www.stats.gov.cn/english/PressRelease/ReleaseCalendar/202512/t20251226_1962154.html")
for _, e := range events {
if e.Event == "National Economic Performance" && len(e.ReleaseDate) >= 7 && e.ReleaseDate[:7] == "2026-02" {
t.Fatalf("blank February month should stay empty, got %+v", e)
}
}
var prelim []string
for _, e := range events {
if strings.HasPrefix(e.Event, "Preliminary Accounting") {
prelim = append(prelim, e.ReleaseDate)
}
}
want := []string{"2026-01-20", "2026-04-17", "2026-07-16", "2026-10-20"}
if len(prelim) != len(want) {
t.Fatalf("preliminary accounting dates = %v, want %v", prelim, want)
}
for i := range want {
if prelim[i] != want[i] {
t.Fatalf("preliminary accounting dates = %v, want %v", prelim, want)
}
}
if !hasReleaseOn(events, "Purchasing Managers", "2026-03-04") || !hasReleaseOn(events, "Purchasing Managers", "2026-03-31") {
t.Fatalf("PMI Spring-Festival-shifted dual release not captured")
}
}
func hasReleaseOn(events []chinaReleaseEvent, eventSubstr, date string) bool {
for _, e := range events {
if e.ReleaseDate == date && strings.Contains(e.Event, eventSubstr) {
return true
}
}
return false
}
// LPR candidates roll over weekends and official holidays; only realized months
// are promoted to verified.
func TestChinaLprCandidates(t *testing.T) {
candidates, err := buildLprCandidates(2026)
if err != nil {
t.Fatalf("buildLprCandidates(2026): %v", err)
}
if got := monthDate(candidates, "2026-02"); got != "2026-02-24" {
t.Fatalf("Feb LPR candidate = %q, want 2026-02-24 (Spring Festival roll-forward)", got)
}
if got := monthDate(candidates, "2026-06"); got != "2026-06-22" {
t.Fatalf("Jun LPR candidate = %q, want 2026-06-22 (Dragon Boat roll-forward)", got)
}
for _, c := range candidates {
if c.Status != "provisional" {
t.Fatalf("candidate %s not provisional: %q", c.ReleaseDate, c.Status)
}
}
realized := parseChinaMoneyLprNotices(chinaFixture(t, "chinamoney-lpr.json"))
merged := mergeVerifiedLprDates(candidates, realized)
if statusOf(merged, "2026-02-24") != "verified" || statusOf(merged, "2026-06-22") != "verified" {
t.Fatalf("realized LPR dates not verified: %v", merged)
}
if statusOf(merged, "2026-07-20") != "provisional" {
t.Fatalf("unrealized July LPR should stay provisional")
}
}
// Fails closed when the official holiday calendar is not configured for a year.
func TestChinaLprCalendarUnconfiguredYear(t *testing.T) {
if _, err := buildLprCandidates(2027); err == nil {
t.Fatalf("expected CHINA_HOLIDAY_CALENDAR_UNAVAILABLE for unconfigured 2027")
}
}
// The trusted-origin guard resolves same-origin links and refuses off-origin ones.
func TestChinaCurrentCalendarLink(t *testing.T) {
got, err := currentCalendarLink(`<a href="calendar.html">2026 release calendar</a>`, 2026)
if err != nil || got != "https://www.stats.gov.cn/english/PressRelease/ReleaseCalendar/calendar.html" {
t.Fatalf("trusted relative link resolution failed: got=%q err=%v", got, err)
}
if _, err := currentCalendarLink(`<a href="https://attacker.example/calendar.html">2026 release calendar</a>`, 2026); err == nil {
t.Fatalf("off-origin NBS link must be rejected")
}
got, err = currentCalendarLink(`<p>no link here</p>`, 2026)
if err != nil || got != nbsCalendarIndexURL {
t.Fatalf("missing link should fall back to index URL, got=%q err=%v", got, err)
}
}
// The handler exposes the merged shape and degrades to an honest unavailable
// payload (never a 5xx) — the fallback branch of the endpoint.
func TestChinaUnavailableShape(t *testing.T) {
u := chinaUnavailable()
if u.CountryCode != "CN" || !u.Unavailable || u.Status != "unavailable" ||
u.Indicators == nil || u.ReleaseEvents == nil || u.SourceDecisions == nil {
t.Fatalf("unavailable fallback shape wrong: %+v", u)
}
}
func monthDate(events []chinaReleaseEvent, month string) string {
for _, e := range events {
if len(e.ReleaseDate) >= 7 && e.ReleaseDate[:7] == month {
return e.ReleaseDate
}
}
return ""
}
func statusOf(events []chinaReleaseEvent, date string) string {
for _, e := range events {
if e.ReleaseDate == date {
return e.Status
}
}
return ""
}
+454 -165
View File
@@ -2,32 +2,46 @@ package world
import (
"context"
"math"
"encoding/json"
"net/http"
"sort"
"net/url"
"strconv"
"strings"
"time"
)
// SaaS-mode "cloud pulse": the anonymized, platform-wide aggregate the signed-out
// investor view renders (world.hanzo.ai/?variant=saas). It is deliberately
// Platform "cloud pulse": the anonymized, platform-wide aggregate the flagship
// dashboard renders (world.hanzo.ai cloud/SaaS/AI variants). It is deliberately
// non-sensitive — counts and volume buckets only, never per-org spend or names.
//
// HONESTY CONTRACT (see cloud-pulse.ts on the client):
// - There is NO unauthenticated Hanzo endpoint that exposes platform-wide
// counts (verified against the OpenAPI specs: billing/visor/admin/ml/ai/o11y
// are all bearer-required; the only public surface, /v1/world/*, is external
// world data). So by default this route returns a clearly-labeled DEMO
// dataset with demo:true — the flag travels in the payload and the UI shows a
// "demo data" note. We never fake platform numbers silently.
// - When an operator wires a service token (HANZO_CLOUD_PULSE_TOKEN, from KMS),
// we make SERVICE-side calls to the real cloud for non-sensitive COUNTS only
// (models served, node/region/GPU counts) and set demo:false. Request/token
// VOLUME still has no aggregate source, so it stays modeled and is flagged
// volumeModeled:true — again, never silently faked.
// HONESTY CONTRACT — NOTHING is fabricated. Every number is REAL (measured by an
// actual backend) or an honest zero/empty. There is no diurnal-sine "demo" curve,
// no hardcoded uptime, no invented model mix or per-region rate. The honest source
// ladder (producePulse), best → last:
//
// Signed-in, org-scoped drill-down (the user's own fleet / models / bill) does
// NOT come through here — those panels call api.hanzo.ai directly with the
// caller's IAM token (no shared key). This route is the public teaser only.
// - VOLUME (requests/sec, 24h requests/tokens, series, model mix):
// 1. the super-admin usage ledger (get-cloud-usages ?org=all, ClickHouse) —
// MEASURED and exact (tokens + spend); clears volumeModeled.
// 2. else the REAL, public endpoints this same binary already serves: the
// native request-geo globe (traffic-globe, totals.rps_1m) for the
// headline rate, and the learned-router stats (router-stats) for total
// routed requests + hourly throughput + per-model mix. Token volume is not
// measured on this path, so tokens stay blank and volumeModeled stays true.
// 3. else empty (zeros / empty arrays).
// - FLEET COUNTS + REGION breakdown: the service-token visor (/v1/machines,
// /v1/gpus) + ai catalog (/v1/models). Absent ⇒ zeros and an empty region list
// built from the real fleet — never the geo catalog as if it were live.
// - UPTIME: the public status page (Gatus up/total). Unreachable ⇒ 0 and the
// overview drops the tile — never a constant.
//
// demo:true means ONLY that nothing real resolved (a warming-up / not-wired state),
// never that a number was invented. Signed-in / token-wired deployments see the
// full measured aggregate; the tokenless public path still shows real request rate,
// throughput, model mix, uptime and chain scale.
//
// Signed-in, org-scoped drill-down (the user's own fleet / models / bill) does NOT
// come through here — those panels call api.hanzo.ai directly with the caller's IAM
// token (no shared key). This route is the platform teaser only.
type cloudOverview struct {
RequestsPerSec float64 `json:"requestsPerSec"`
@@ -74,46 +88,145 @@ type cloudPulse struct {
TokenSeries []int64 `json:"tokenSeries"`
Models []cloudModel `json:"models"`
Regions []cloudRegion `json:"regions"`
// Users is populated ONLY on the signed-in admin path (real IAM aggregates:
// total users, signups, active now, daily-signup series). omitempty ⇒ the public
// teaser never carries it.
Users *userMetrics `json:"users,omitempty"`
}
// handleCloudPulse serves the public SaaS aggregate. It never 5xxes: any upstream
// failure degrades to the demo dataset with demo:true.
// publicVolumeTimeout bounds each public fallback fetch (traffic-globe, router-stats,
// status page) so a single slow/unreachable host can't stall the pulse produce.
const publicVolumeTimeout = 5 * time.Second
// handleCloudPulse serves the platform aggregate. Two honest representations:
//
// - SIGNED-IN ADMIN (z@hanzo.ai / the admin org): the FULL real aggregate, with
// the token-plane reads (all-org usage ledger + visor fleet) made using the
// CALLER's OWN bearer, never edge-cached. The upstream independently authorizes
// the bearer, so a non-super-admin simply degrades to the public sources —
// never a fabricated number.
// - EVERYONE ELSE (public teaser): cached; service-token counts + real public
// volume/uptime.
//
// It never 5xxes: any upstream failure degrades to the honest empty pulse.
func (s *Server) handleCloudPulse(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
// Authed responses must never be served from (or stored in) the shared public
// cache: vary on Authorization so an anonymous cache entry is never handed to a
// signed-in caller, and vice-versa.
w.Header().Set("Vary", "Authorization")
if bearer, ok := s.adminIdentity(r); ok {
ctx, cancel := context.WithTimeout(r.Context(), 20*time.Second)
defer cancel()
hdr := map[string]string{"Authorization": bearer}
p := s.producePulse(ctx, hdr)
// Real platform user metrics (IAM global-users) — admin path only; aggregates
// only, never PII. Omitted honestly if the caller isn't a global admin upstream.
if um, err := s.fetchUserMetrics(ctx, hdr); err == nil {
p.Users = um
}
// Real platform USAGE + per-model mix from LLM observability (measured requests,
// tokens, and REAL model names — not the opaque router arms), when the exact
// usage ledger was not available to this caller. Fixes an empty "Model Usage"
// and 0-token totals for an operator who can read platform observability.
if p.VolumeModeled || len(p.Models) == 0 {
s.applyLLMObservability(ctx, &p, hdr)
}
writeJSON(w, http.StatusOK, "private, no-store", p)
return
}
s.cachedJSON(w, "cloud-pulse", "public, max-age=15, s-maxage=15, stale-while-revalidate=60",
20*time.Second, 5*time.Minute,
func(ctx context.Context) (any, error) {
base := demoPulse()
if p, ok := s.tryServicePulse(ctx, base); ok {
return p, nil
}
return base, nil
return s.producePulse(ctx, serviceAuth()), nil
},
func(w http.ResponseWriter, _ error) {
// Unreachable in practice (produce never errors), but keep the
// never-5xx guarantee explicit.
writeJSON(w, http.StatusOK, "", demoPulse())
writeJSON(w, http.StatusOK, "", emptyPulse())
},
)
}
// tryServicePulse overlays REAL non-sensitive counts onto the demo scaffold when
// a service token is configured (HANZO_CLOUD_PULSE_TOKEN, KMS-injected). It
// returns ok=false — leaving the honest demo dataset in place — when the token is
// absent or any required call fails. Only counts are read; no spend, no names.
func (s *Server) tryServicePulse(ctx context.Context, base cloudPulse) (cloudPulse, bool) {
tok := env("HANZO_CLOUD_PULSE_TOKEN")
if tok == "" {
return cloudPulse{}, false
// emptyPulse is the honest zero baseline: no measured data resolved. Every number
// is zero and the arrays are empty (never null) — flagged demo:true (nothing real
// yet) and volumeModeled:true (no measured volume). We never fabricate.
func emptyPulse() cloudPulse {
return cloudPulse{
Demo: true,
VolumeModeled: true,
Source: "empty",
Note: "Platform metrics are warming up — no measured data is reachable yet. Wire the service token (KMS) for the full live aggregate.",
UpdatedAt: nowRFC(),
Window: "24h",
RequestSeries: []int64{},
TokenSeries: []int64{},
Models: []cloudModel{},
Regions: []cloudRegion{},
}
apiBase := env("HANZO_API_BASE", "HANZO_AI_BASE")
if apiBase == "" {
apiBase = "https://api.hanzo.ai"
}
// producePulse assembles the pulse from ONLY real sources (see the file header for
// the honesty ladder). auth is the token-plane header (the KMS service bearer on the
// public path, or the signed-in admin's own bearer on the flagship admin path); nil
// leaves the token-plane reads out and the pulse falls back to the public sources.
// demo:true iff nothing real resolved; every field is real or an honest zero/empty.
func (s *Server) producePulse(ctx context.Context, auth map[string]string) cloudPulse {
p := emptyPulse()
real := false
volSrc := ""
// 1) Fleet COUNTS + REGION breakdown (auth → visor + ai catalog).
countsReal := s.applyServiceCounts(ctx, &p, auth)
if countsReal {
real = true
}
apiBase = trimSlash(apiBase)
hdr := map[string]string{"Authorization": "Bearer " + tok}
// 2) VOLUME — measured ledger first (super-admin ?org=all, exact); else the real
// public request rate + throughput + model mix; else empty. Never modeled.
if ov, err := s.fetchCloudUsage(ctx, "24h", auth); err == nil && ov.Totals.Requests > 0 {
applyUsageToPulse(&p, ov) // clears volumeModeled, fills tokens/series/models
real, volSrc = true, "ledger"
} else if s.applyPublicVolume(ctx, &p) {
real, volSrc = true, "router" // real rate/throughput/mix; tokens unmeasured ⇒ volumeModeled stays true
}
// 3) UPTIME — real share of healthy endpoints (Gatus up/total). 0 ⇒ tile dropped.
if up, ok := s.fetchUptimePct(ctx); ok {
p.Overview.UptimePct = up
real = true
}
p.Demo = !real
switch {
case countsReal:
p.Source = "service" // the service-token plane resolved (counts, and usually ledger volume)
case real:
p.Source = "public" // tokenless, but real public volume/uptime landed
default:
p.Source = "empty"
}
if real && volSrc == "" {
p.Note = "Live fleet and status from Hanzo Cloud. Measured request and token volume appears when the platform usage ledger is reachable."
}
return p
}
// applyServiceCounts fills the real fleet counts (models served, nodes online/total,
// GPUs) and the region breakdown from the service-token visor + ai catalog. Regions
// are derived from the machines' OWN region strings (resolved to the geo catalog for
// name/city/coords) — an empty fleet yields an empty regions list, never the geo
// catalog as if it were live; per-region rate stays 0 (no real per-region source).
// Returns false (leaving zeros) when no auth header is supplied or the core reads
// fail. auth is the KMS service bearer (public path) or the caller's own admin
// bearer (admin path).
func (s *Server) applyServiceCounts(ctx context.Context, p *cloudPulse, hdr map[string]string) bool {
if hdr == nil {
return false
}
host := apiHost()
// Models served (ai gateway, OpenAI-compatible list).
var models struct {
@@ -121,8 +234,8 @@ func (s *Server) tryServicePulse(ctx context.Context, base cloudPulse) (cloudPul
ID string `json:"id"`
} `json:"data"`
}
if err := s.getJSON(ctx, apiBase+"/v1/models", hdr, &models); err != nil || len(models.Data) == 0 {
return cloudPulse{}, false
if err := s.getJSON(ctx, host+"/v1/models", hdr, &models); err != nil || len(models.Data) == 0 {
return false
}
// Fleet (visor). status is a free string; treat non-terminal states as online.
@@ -132,8 +245,8 @@ func (s *Server) tryServicePulse(ctx context.Context, base cloudPulse) (cloudPul
Status string `json:"status"`
} `json:"machines"`
}
if err := s.getJSON(ctx, apiBase+"/v1/machines", hdr, &machines); err != nil {
return cloudPulse{}, false
if err := s.getJSON(ctx, host+"/v1/machines", hdr, &machines); err != nil {
return false
}
var gpus struct {
Gpus []struct {
@@ -141,32 +254,300 @@ func (s *Server) tryServicePulse(ctx context.Context, base cloudPulse) (cloudPul
} `json:"gpus"`
}
// GPUs are a bonus count; a failure here should not sink real machine data.
_ = s.getJSON(ctx, apiBase+"/v1/gpus", hdr, &gpus)
_ = s.getJSON(ctx, host+"/v1/gpus", hdr, &gpus)
// Real region breakdown: place each machine/GPU into its resolved catalog region
// (name/city/coords), counting nodes + GPUs. rps stays 0 — no invented rate.
agg := map[string]*cloudRegion{}
var order []string
// Resolve to the geo catalog for coords/name when the region is known; otherwise
// keep the RAW region string as its own region (a real region, just without
// catalog coordinates) so EVERY real region shows — never dropped just because
// it's outside the 8-city catalog (e.g. tor1, GCP/AWS regions).
region := func(raw string) *cloudRegion {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil
}
id := raw
nc := cloudRegion{ID: raw, Name: raw, City: raw, Status: "online"}
if rg, ok := resolveRegion(raw); ok {
id = rg.ID
nc = rg
nc.Nodes, nc.Gpus, nc.RequestsPerSec, nc.Status = 0, 0, 0, "online"
}
if c := agg[id]; c != nil {
return c
}
agg[id] = &nc
order = append(order, id)
return &nc
}
regionSet := map[string]int{}
online := 0
for _, m := range machines.Machines {
if machineOnline(m.Status) {
online++
}
if m.Region != "" {
regionSet[m.Region]++
if c := region(m.Region); c != nil {
c.Nodes++
}
}
for _, g := range gpus.Gpus {
if c := region(g.Region); c != nil {
c.Gpus++
}
}
p := base
p.Demo = false
p.VolumeModeled = true
p.Source = "service"
p.Note = "Live counts from Hanzo Cloud (models/nodes/regions). Request & token volume is modeled — no aggregate volume endpoint is exposed."
p.Overview.ModelsServed = len(models.Data)
p.Overview.NodesTotal = len(machines.Machines)
p.Overview.NodesOnline = online
p.Overview.GpusOnline = len(gpus.Gpus)
if len(regionSet) > 0 {
p.Overview.Regions = len(regionSet)
regions := make([]cloudRegion, 0, len(order))
for _, id := range order {
regions = append(regions, *agg[id])
}
return p, true
p.Regions = regions
p.Overview.Regions = len(regions)
return true
}
// applyPublicVolume folds REAL, public request volume into p when the measured
// ledger is unavailable — no fabrication. It reads the SAME endpoints this binary
// already serves to other panels: the native request-geo globe (traffic-globe,
// totals.rps_1m — the Live Traffic rate) for the headline requests/sec, and the
// learned-router stats (router-stats, the Enso Training source) for total routed
// requests, the hourly throughput series and the per-model request mix. Token
// volume is NOT measured on this path, so Tokens24h / TokenSeries stay empty and
// volumeModeled stays true. Returns true when at least one real datum landed.
func (s *Server) applyPublicVolume(ctx context.Context, p *cloudPulse) bool {
cctx, cancel := context.WithTimeout(ctx, publicVolumeTimeout)
defer cancel()
got := false
// Headline requests/sec — real, from the native LB request-geo aggregate.
if g, ok := s.fetchNativeGlobe(cctx, 60); ok && g.Totals.RPS1m > 0 {
p.Overview.RequestsPerSec = round1(g.Totals.RPS1m)
got = true
}
// Routed-request volume + hourly throughput — real, from the public learned-router
// stats (the Enso Training source). NOTE: on platform scope the router relabels
// models to OPAQUE "arm-N" ids (a privacy measure), so by_model is NOT real model
// names — we never present it as the model mix. Real per-model usage comes only
// from the measured ledger (admin path); otherwise Models stays empty and the
// panel honestly shows "warming up" rather than "arm-8".
if rs, ok := s.fetchRouterStats(cctx, 24); ok {
events := rs.Window.Events
if events == 0 {
events = rs.Throughput.TotalWindow
}
if events > 0 {
p.Overview.Requests24h = events
if p.Overview.RequestsPerSec == 0 { // globe rate unavailable → window average
p.Overview.RequestsPerSec = round1(float64(events) / routerWindowSecs(rs))
}
got = true
}
if len(rs.Throughput.PerHour) > 0 {
p.RequestSeries = append([]int64{}, rs.Throughput.PerHour...)
got = true
}
}
if got {
p.Window = "24h"
p.Note = "Live platform request rate and throughput — measured across all orgs (traffic globe + learned router). Per-model usage and token volume come from the usage ledger (sign in as an admin)."
}
return got
}
// llmUsage is the measured platform usage from LLM observability (/v1/admin/o11y):
// 24h request/token totals + top models by REAL name. ONE place shapes the o11y read
// so cloud-pulse and ai-pulse share it (DRY) — the opaque router arms are never used.
type llmUsage struct {
Requests int64
Tokens int64
Models []cloudModel
}
// fetchLLMUsage reads the platform LLM observability aggregate with the admin's own
// bearer. ok=false when the caller isn't authorized upstream or it's unreachable, so
// callers keep their honest fallback.
func (s *Server) fetchLLMUsage(ctx context.Context, auth map[string]string) (*llmUsage, bool) {
if auth == nil {
return nil, false
}
var resp struct {
Totals struct {
Requests int64 `json:"requests"`
Tokens int64 `json:"tokens"`
} `json:"totals"`
TopModels []struct {
Model string `json:"model"`
Requests int64 `json:"requests"`
Tokens int64 `json:"tokens"`
} `json:"topModels"`
}
if err := s.getJSON(ctx, apiHost()+"/v1/admin/o11y?range=24h", auth, &resp); err != nil {
return nil, false
}
u := &llmUsage{Requests: resp.Totals.Requests, Tokens: resp.Totals.Tokens}
var total int64
for _, m := range resp.TopModels {
total += m.Requests
}
if total > 0 {
for _, m := range resp.TopModels {
if m.Model == "" {
continue
}
u.Models = append(u.Models, cloudModel{
ID: m.Model, Name: m.Model, Requests24h: m.Requests, Tokens24h: m.Tokens,
Share: float64(m.Requests) / float64(total),
})
}
}
if u.Requests == 0 && len(u.Models) == 0 {
return nil, false
}
return u, true
}
// applyLLMObservability overlays REAL platform usage (fetchLLMUsage) onto the pulse:
// measured requests/tokens + the real model mix (never the opaque router arms). It
// clears volumeModeled when real numbers land and keeps the freshest rate (globe rps)
// if already set. No-op (honest fallback kept) when the caller can't read o11y.
func (s *Server) applyLLMObservability(ctx context.Context, p *cloudPulse, auth map[string]string) {
u, ok := s.fetchLLMUsage(ctx, auth)
if !ok {
return
}
if u.Requests > 0 {
p.Overview.Requests24h = u.Requests
p.Overview.Tokens24h = u.Tokens
if p.Overview.RequestsPerSec == 0 { // keep the globe's live rate when we have it
p.Overview.RequestsPerSec = round1(float64(u.Requests) / 86400)
}
}
if len(u.Models) > 0 {
p.Models = u.Models
}
p.VolumeModeled = false
p.Window = "24h"
p.Note = "Live platform usage from Hanzo LLM observability — measured requests, tokens and model mix across all orgs."
}
// routerStatsVolume is the subset of the public router-stats aggregate we fold into
// the pulse: total routed events (requests), the hourly throughput series, and the
// per-model event mix. Same upstream the Enso Training panel proxies.
type routerStatsVolume struct {
Window struct {
Since string `json:"since"`
Until string `json:"until"`
Events int64 `json:"events"`
} `json:"window"`
Throughput struct {
PerHour []int64 `json:"per_hour"`
TotalWindow int64 `json:"total_window"`
} `json:"throughput"`
// by_model is intentionally NOT decoded: on platform scope it is opaque "arm-N"
// ids, never real model names, so it is never used as the model mix.
}
// fetchRouterStats reads the PUBLIC learned-router aggregate (no token) and unwraps
// the {status,data} envelope, exactly as handleCloudRouterStats does. ok=false when
// unreachable or the envelope is not ok.
func (s *Server) fetchRouterStats(ctx context.Context, hours int) (*routerStatsVolume, bool) {
var env struct {
Status string `json:"status"`
Data json.RawMessage `json:"data"`
}
u := apiHost() + "/v1/router/stats?scope=platform&hours=" + strconv.Itoa(hours)
if err := s.getJSON(ctx, u, nil, &env); err != nil {
return nil, false
}
if env.Status != "ok" || len(env.Data) == 0 || string(env.Data) == "null" {
return nil, false
}
var rs routerStatsVolume
if err := json.Unmarshal(env.Data, &rs); err != nil {
return nil, false
}
return &rs, true
}
// routerWindowSecs is the router-stats window length in seconds (since→until),
// defaulting to 24h when the timestamps are missing/unparseable — so a window
// average never divides by a bogus interval.
func routerWindowSecs(rs *routerStatsVolume) float64 {
if t0, e0 := time.Parse(time.RFC3339, rs.Window.Since); e0 == nil {
if t1, e1 := time.Parse(time.RFC3339, rs.Window.Until); e1 == nil {
if d := t1.Sub(t0).Seconds(); d > 0 {
return d
}
}
}
return 86400
}
// fetchUptimePct derives a real platform uptime from the PUBLIC status page (Gatus):
// the share of monitored endpoints currently healthy (up/total, 0..100). ok=false
// when the page is unreachable or has no evaluated endpoints — the overview then
// drops the uptime tile rather than showing a constant.
func (s *Server) fetchUptimePct(ctx context.Context) (float64, bool) {
cctx, cancel := context.WithTimeout(ctx, publicVolumeTimeout)
defer cancel()
base := statusBase()
host := ""
if u, err := url.Parse(base); err == nil {
host = u.Hostname()
}
raw, ok := s.fetchGatusBoard(cctx, base, host)
if !ok {
return 0, false
}
sp := summarizeStatusPage(host, raw)
if sp.Total == 0 {
return 0, false
}
return round2s(float64(sp.Up) / float64(sp.Total) * 100), true
}
// applyUsageToPulse folds the real platform-wide usage overview into p: the headline
// rate (recent series bucket, else 24h average), 24h totals, the real hourly
// request/token series, and the top models by real spend. It clears volumeModeled —
// these are measured, not modeled.
func applyUsageToPulse(p *cloudPulse, ov *cloudUsageOverview) {
p.VolumeModeled = false
p.Window = ov.Range
if p.Window == "" {
p.Window = "24h"
}
p.Overview.Requests24h = ov.Totals.Requests
p.Overview.Tokens24h = ov.Totals.Tokens
// Headline rate: the most recent complete bucket is the freshest honest rate;
// fall back to the 24h average when there is no usable interval.
p.Overview.RequestsPerSec = round1(usageRate(ov.Totals.Requests, ov.Series, ov.Interval, seriesRequests))
// Real hourly buckets (chronological) drive both sparklines.
if n := len(ov.Series); n > 0 {
reqs := make([]int64, n)
toks := make([]int64, n)
for i, pt := range ov.Series {
reqs[i] = pt.Requests
toks[i] = pt.Tokens
}
p.RequestSeries = reqs
p.TokenSeries = toks
}
// Top models by real spend/volume (ledger byModel items, already ranked).
if m := topModelsFromUsage(ov); m != nil {
p.Models = m
}
p.Note = "Live platform aggregate from Hanzo Cloud — models, fleet, and measured 24h request/token volume across all orgs."
}
func machineOnline(status string) bool {
@@ -178,114 +559,22 @@ func machineOnline(status string) bool {
}
}
// demoPulse builds the illustrative dataset. Numbers follow a smooth diurnal
// curve keyed on wall-clock time so the ticker feels alive across refreshes,
// but the demo flag makes clear these are not live platform metrics.
func demoPulse() cloudPulse {
now := time.Now().UTC()
// Diurnal load factor in [0.55, 1.0]: peaks ~16:00 UTC, troughs ~04:00 UTC.
hourFrac := float64(now.Hour()) + float64(now.Minute())/60
load := 0.775 + 0.225*math.Sin((hourFrac-10)/24*2*math.Pi)
// A slow per-minute wobble so the live-activity sparkline moves each poll.
wobble := 1 + 0.04*math.Sin(float64(now.Unix()%600)/600*2*math.Pi)
regions := demoRegions()
baseRPS := 4200.0 * load * wobble
nodesOnline, nodesTotal, gpus := 0, 0, 0
for i := range regions {
regions[i].RequestsPerSec = round1(baseRPS * regions[i].RequestsPerSec) // field held a weight
nodesTotal += regions[i].Nodes
gpus += regions[i].Gpus
if regions[i].Status == "online" {
nodesOnline += regions[i].Nodes
}
}
models := demoModels(baseRPS)
requests24h := int64(baseRPS * 86400)
tokens24h := requests24h * 1180 // ~1.18k tokens/request average
return cloudPulse{
Demo: true,
VolumeModeled: true,
Source: "demo",
Note: "Demo data — no unauthenticated platform-metrics endpoint exists. Sign in to see your org's real fleet, models and bill.",
UpdatedAt: now.Format(time.RFC3339),
Window: "24h",
Overview: cloudOverview{
RequestsPerSec: round1(baseRPS),
Requests24h: requests24h,
Tokens24h: tokens24h,
ModelsServed: len(models),
NodesOnline: nodesOnline,
NodesTotal: nodesTotal,
GpusOnline: gpus,
Regions: len(regions),
UptimePct: 99.98,
},
RequestSeries: demoSeries(baseRPS, now, 1),
TokenSeries: demoSeries(baseRPS*1180, now, 1),
Models: models,
Regions: regions,
}
}
// demoSeries returns 24 hourly buckets ending now, following the same diurnal
// curve as the headline so the sparkline and the number agree.
func demoSeries(scale float64, now time.Time, _ int) []int64 {
out := make([]int64, 24)
for i := 0; i < 24; i++ {
h := float64(now.Add(time.Duration(i-23) * time.Hour).Hour())
load := 0.775 + 0.225*math.Sin((h-10)/24*2*math.Pi)
jitter := 1 + 0.03*math.Sin(float64(i)*1.7)
out[i] = int64(scale * load * jitter * 3600)
}
return out
}
func demoModels(baseRPS float64) []cloudModel {
// Representative of Hanzo's served surface: Zen family (qwen3+ base) plus
// routed frontier models. Shares are illustrative.
rows := []struct {
id, name string
share float64
}{
{"zen-omni-30b", "Zen Omni 30B", 0.28},
{"zen-coder-32b", "Zen Coder 32B", 0.19},
{"zen-1", "Zen 1", 0.14},
{"qwen3-235b", "Qwen3 235B", 0.12},
{"zen-nano-4b", "Zen Nano 4B", 0.10},
{"deepseek-v3", "DeepSeek V3", 0.07},
{"zen-vl-8b", "Zen VL 8B", 0.06},
{"llama-3.3-70b", "Llama 3.3 70B", 0.04},
}
out := make([]cloudModel, len(rows))
for i, r := range rows {
req := int64(baseRPS * r.share * 86400)
out[i] = cloudModel{
ID: r.id,
Name: r.name,
Requests24h: req,
Tokens24h: req * 1180,
Share: r.share,
}
}
sort.SliceStable(out, func(a, b int) bool { return out[a].Requests24h > out[b].Requests24h })
return out
}
// demoRegions models Hanzo's DOKS footprint. RequestsPerSec temporarily carries
// a per-region weight (summing to 1.0) that demoPulse scales into a real rate.
func demoRegions() []cloudRegion {
// regionCatalog is Hanzo's DOKS geo catalog — the reference coordinates (id / name /
// city / country / lat / lon) the map layers place points against, plus per-region
// capacity weights (Nodes / Gpus) the modeled-globe layer spreads real peer counts
// across (flagged positionsModeled:true there). It is NOT live fleet data: the
// cloud-pulse never presents it as such — the pulse's region breakdown is built from
// the real visor fleet (applyServiceCounts).
func regionCatalog() []cloudRegion {
return []cloudRegion{
{ID: "nyc", Name: "New York", City: "New York", Country: "USA", Lat: 40.7128, Lon: -74.0060, Nodes: 42, Gpus: 168, Status: "online", RequestsPerSec: 0.24},
{ID: "sfo", Name: "San Francisco", City: "San Francisco", Country: "USA", Lat: 37.7749, Lon: -122.4194, Nodes: 38, Gpus: 152, Status: "online", RequestsPerSec: 0.21},
{ID: "ams", Name: "Amsterdam", City: "Amsterdam", Country: "Netherlands", Lat: 52.3676, Lon: 4.9041, Nodes: 26, Gpus: 96, Status: "online", RequestsPerSec: 0.15},
{ID: "fra", Name: "Frankfurt", City: "Frankfurt", Country: "Germany", Lat: 50.1109, Lon: 8.6821, Nodes: 22, Gpus: 88, Status: "online", RequestsPerSec: 0.12},
{ID: "lon", Name: "London", City: "London", Country: "UK", Lat: 51.5074, Lon: -0.1278, Nodes: 18, Gpus: 64, Status: "online", RequestsPerSec: 0.10},
{ID: "sgp", Name: "Singapore", City: "Singapore", Country: "Singapore", Lat: 1.3521, Lon: 103.8198, Nodes: 16, Gpus: 60, Status: "online", RequestsPerSec: 0.08},
{ID: "blr", Name: "Bangalore", City: "Bangalore", Country: "India", Lat: 12.9716, Lon: 77.5946, Nodes: 12, Gpus: 40, Status: "degraded", RequestsPerSec: 0.06},
{ID: "syd", Name: "Sydney", City: "Sydney", Country: "Australia", Lat: -33.8688, Lon: 151.2093, Nodes: 8, Gpus: 24, Status: "online", RequestsPerSec: 0.04},
{ID: "nyc", Name: "New York", City: "New York", Country: "USA", Lat: 40.7128, Lon: -74.0060, Nodes: 42, Gpus: 168, Status: "online"},
{ID: "sfo", Name: "San Francisco", City: "San Francisco", Country: "USA", Lat: 37.7749, Lon: -122.4194, Nodes: 38, Gpus: 152, Status: "online"},
{ID: "ams", Name: "Amsterdam", City: "Amsterdam", Country: "Netherlands", Lat: 52.3676, Lon: 4.9041, Nodes: 26, Gpus: 96, Status: "online"},
{ID: "fra", Name: "Frankfurt", City: "Frankfurt", Country: "Germany", Lat: 50.1109, Lon: 8.6821, Nodes: 22, Gpus: 88, Status: "online"},
{ID: "lon", Name: "London", City: "London", Country: "UK", Lat: 51.5074, Lon: -0.1278, Nodes: 18, Gpus: 64, Status: "online"},
{ID: "sgp", Name: "Singapore", City: "Singapore", Country: "Singapore", Lat: 1.3521, Lon: 103.8198, Nodes: 16, Gpus: 60, Status: "online"},
{ID: "blr", Name: "Bangalore", City: "Bangalore", Country: "India", Lat: 12.9716, Lon: 77.5946, Nodes: 12, Gpus: 40, Status: "degraded"},
{ID: "syd", Name: "Sydney", City: "Sydney", Country: "Australia", Lat: -33.8688, Lon: 151.2093, Nodes: 8, Gpus: 24, Status: "online"},
}
}
+5 -2
View File
@@ -33,6 +33,8 @@ type fleetMachineRow struct {
GPUModel string `json:"gpuModel"`
GPUs int `json:"gpus"`
VRAM string `json:"vram"` // BYO GPUs report VRAM total; "" when unknown
VCPU int `json:"vcpu"` // vCPU count (visor MachineView.vcpu); 0 when unknown
Mem string `json:"mem"` // system RAM, e.g. "8 GB"; "" when unknown
OS string `json:"os"`
}
@@ -95,7 +97,8 @@ func (s *Server) handleCloudFleet(w http.ResponseWriter, r *http.Request) {
var machines struct {
Machines []struct {
ID, Name, Region, Type, Status, Provider, GPU, OS string
ID, Name, Region, Type, Status, Provider, GPU, OS, Mem string
Vcpu int `json:"vcpu"`
} `json:"machines"`
}
if err := s.getJSON(ctx, base+"/v1/machines", hdr, &machines); err != nil {
@@ -156,7 +159,7 @@ func (s *Server) handleCloudFleet(w http.ResponseWriter, r *http.Request) {
regIdx[prov][region] = rg
}
ga := byMachine[m.ID]
row := fleetMachineRow{ID: m.ID, Name: orDash(m.Name), Type: m.Type, Status: m.Status, GPUModel: m.GPU, OS: m.OS}
row := fleetMachineRow{ID: m.ID, Name: orDash(m.Name), Type: m.Type, Status: m.Status, GPUModel: m.GPU, VCPU: m.Vcpu, Mem: m.Mem, OS: m.OS}
if ga != nil {
row.GPUs = ga.count
row.VRAM = ga.vram
+177 -79
View File
@@ -95,7 +95,7 @@ type jsonRPCReq struct {
//
// nodes[] positions are MODELED: real per-node IP geolocation needs an IP-geo
// dependency we don't carry, so the real peer COUNT is spread deterministically
// across the demoRegions() catalog coords. positionsModeled:true says so plainly.
// across the regionCatalog() catalog coords. positionsModeled:true says so plainly.
const maxModeledNodes = 250
@@ -361,7 +361,7 @@ func (s *Server) getAllowedJSON(ctx context.Context, rawURL string, allowed map[
// and kind are modeled — hence positionsModeled:true on the envelope; only the count
// is real. Bounded to maxModeledNodes so a pathological peer count can't bloat the payload.
func modeledNodes(peers int) []chainNode {
regions := demoRegions()
regions := regionCatalog()
nodes := make([]chainNode, 0)
if peers <= 0 || len(regions) == 0 {
return nodes
@@ -431,29 +431,44 @@ func (s *Server) handleCloudBYOGPU(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
w.Header().Set("Vary", "Authorization")
// Signed-in admin (z@hanzo.ai / the operator org): the REAL GPU fleet placed on
// the globe, read with the caller's OWN bearer, never edge-cached. No fabricated
// demo clusters for a signed-in operator — an empty read shows an empty globe.
if bearer, ok := s.adminIdentity(r); ok {
ctx, cancel := context.WithTimeout(r.Context(), 20*time.Second)
defer cancel()
clusters, _ := s.tryRealGPUs(ctx, map[string]string{"Authorization": bearer})
if clusters == nil {
clusters = []gpuCluster{}
}
writeJSON(w, http.StatusOK, "private, no-store", byoGPU{UpdatedAt: nowRFC(), Demo: false, GPUs: clusters})
return
}
s.cachedJSON(w, "cloud-byo-gpu", "public, max-age=30, s-maxage=30, stale-while-revalidate=120",
30*time.Second, 5*time.Minute,
func(ctx context.Context) (any, error) {
if clusters, ok := s.tryRealGPUs(ctx); ok {
if clusters, ok := s.tryRealGPUs(ctx, serviceAuth()); ok {
return byoGPU{UpdatedAt: nowRFC(), Demo: false, GPUs: clusters}, nil
}
return byoGPU{UpdatedAt: nowRFC(), Demo: true, GPUs: demoGPUs()}, nil
// No service token → honest empty (no fabricated clusters). Real GPUs show
// for a signed-in admin (above) or when the service token is wired.
return byoGPU{UpdatedAt: nowRFC(), Demo: false, GPUs: []gpuCluster{}}, nil
},
func(w http.ResponseWriter, _ error) {
writeJSON(w, http.StatusOK, "", byoGPU{UpdatedAt: nowRFC(), Demo: true, GPUs: demoGPUs()})
writeJSON(w, http.StatusOK, "", byoGPU{UpdatedAt: nowRFC(), Demo: false, GPUs: []gpuCluster{}})
},
)
}
// tryRealGPUs reads the real GPU inventory when a service token is configured.
// Returns ok=false (→ demo) when the token is absent, both sources fail, or no GPU
// maps to a known region. Only non-sensitive fields (region/model/status) are read.
func (s *Server) tryRealGPUs(ctx context.Context) ([]gpuCluster, bool) {
tok := env("HANZO_CLOUD_PULSE_TOKEN")
if tok == "" {
// tryRealGPUs reads the real GPU inventory using the supplied auth header (the KMS
// service bearer on the public path, or the caller's own admin bearer). Returns
// ok=false when auth is absent, both sources fail, or no GPU maps to a known region.
// Only non-sensitive fields (region/model/status) are read.
func (s *Server) tryRealGPUs(ctx context.Context, hdr map[string]string) ([]gpuCluster, bool) {
if hdr == nil {
return nil, false
}
hdr := map[string]string{"Authorization": "Bearer " + tok}
base := apiHost()
agg := map[string]*gpuCluster{}
@@ -523,35 +538,6 @@ func (s *Server) tryRealGPUs(ctx context.Context) ([]gpuCluster, bool) {
return out, true
}
// demoGPUs is the flagged illustrative fleet, placed from the region catalog. It
// carries no live-looking precision — it exists to make the globe legible until a
// service token is wired.
func demoGPUs() []gpuCluster {
coords := regionCoords()
rows := []struct {
region, model, status string
count int
}{
{"nyc", "GB10", "online", 4},
{"sfo", "H100", "online", 8},
{"ams", "GB10", "online", 2},
{"fra", "H200", "online", 4},
{"lon", "GB10", "online", 2},
{"sgp", "GB10", "online", 2},
{"blr", "A100", "degraded", 6},
{"syd", "GB10", "online", 1},
}
out := make([]gpuCluster, 0, len(rows))
for _, r := range rows {
rg := coords[r.region]
out = append(out, gpuCluster{
Lat: rg.Lat, Lon: rg.Lon, City: rg.City,
Region: r.region, Model: r.model, Count: r.count, Status: r.status,
})
}
return out
}
// ── 3) traffic: request arcs from visitor countries to the nearest region ────
//
// Real path (service token): read the visitor-COUNTRY breakdown from the same
@@ -585,19 +571,162 @@ func (s *Server) handleCloudTraffic(w http.ResponseWriter, r *http.Request) {
if arcs, ok := s.tryRealTraffic(ctx); ok {
return cloudTraffic{UpdatedAt: nowRFC(), Demo: false, Arcs: arcs}, nil
}
return demoTraffic(), nil
// Honest empty — no fabricated arcs. The native request-geo globe
// (traffic-globe) carries the real traffic layer; arcs fill from real
// visitor geo when analytics is wired.
return cloudTraffic{UpdatedAt: nowRFC(), Demo: false, Arcs: []trafficArc{}}, nil
},
func(w http.ResponseWriter, _ error) {
writeJSON(w, http.StatusOK, "", demoTraffic())
writeJSON(w, http.StatusOK, "", cloudTraffic{UpdatedAt: nowRFC(), Demo: false, Arcs: []trafficArc{}})
},
)
}
// ── native request-geo globe (points + throughput) ───────────────────────────
//
// The Hanzo-mode globe plots WHERE api.hanzo.ai traffic comes from, from the ai
// backend's OWN in-process aggregate (GET /v1/traffic/globe — public, aggregates
// only, no IPs). This same-origin proxy passes those points + totals through, and
// degrades to an HONEST EMPTY state (no points, zero rates) — never demo — because
// "no traffic recorded yet" is a real answer the UI should show truthfully.
type trafficGlobePoint struct {
Country string `json:"country"`
Region string `json:"region,omitempty"`
Lat float64 `json:"lat"`
Lon float64 `json:"lon"`
Count int `json:"count"`
ByService map[string]int `json:"byService"`
}
type trafficGlobeCountry struct {
Country string `json:"country"`
Count int `json:"count"`
}
type trafficGlobeTotals struct {
RPS1m float64 `json:"rps_1m"`
RPM60m float64 `json:"rpm_60m"`
TopCountries []trafficGlobeCountry `json:"top_countries"`
}
type trafficGlobeWindow struct {
Minutes int `json:"minutes"`
Since string `json:"since"`
Until string `json:"until"`
}
type trafficGlobe struct {
UpdatedAt string `json:"updatedAt"`
Live bool `json:"live"` // reached the native endpoint (even if 0 points)
Window trafficGlobeWindow `json:"window"`
Points []trafficGlobePoint `json:"points"`
Totals trafficGlobeTotals `json:"totals"`
}
// emptyGlobe is the honest zero payload: reachable-but-no-data and unreachable both
// render as an empty globe with zero throughput — we never fabricate traffic.
func emptyGlobe() trafficGlobe {
return trafficGlobe{
UpdatedAt: nowRFC(),
Live: false,
Window: trafficGlobeWindow{Minutes: 60},
Points: []trafficGlobePoint{},
Totals: trafficGlobeTotals{TopCountries: []trafficGlobeCountry{}},
}
}
// fetchNativeGlobe reads the ai backend's public /v1/traffic/globe (no token) and
// unwraps the {status,data} envelope. ok=false when the endpoint is unreachable.
func (s *Server) fetchNativeGlobe(ctx context.Context, windowMin int) (trafficGlobe, bool) {
url := apiHost() + "/v1/traffic/globe?window=" + strconv.Itoa(windowMin)
var envlp struct {
Data struct {
Window trafficGlobeWindow `json:"window"`
Points []trafficGlobePoint `json:"points"`
Totals trafficGlobeTotals `json:"totals"`
} `json:"data"`
}
if err := s.getJSON(ctx, url, nil, &envlp); err != nil {
return trafficGlobe{}, false
}
g := trafficGlobe{
UpdatedAt: nowRFC(),
Live: true,
Window: envlp.Data.Window,
Points: envlp.Data.Points,
Totals: envlp.Data.Totals,
}
if g.Points == nil {
g.Points = []trafficGlobePoint{}
}
if g.Totals.TopCountries == nil {
g.Totals.TopCountries = []trafficGlobeCountry{}
}
return g, true
}
// handleCloudTrafficGlobe serves the native request-geo aggregate. It never 5xxes:
// an unreachable backend degrades to the honest empty globe.
func (s *Server) handleCloudTrafficGlobe(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
s.cachedJSON(w, "cloud-traffic-globe", "public, max-age=10, s-maxage=10, stale-while-revalidate=45",
12*time.Second, 2*time.Minute,
func(ctx context.Context) (any, error) {
if g, ok := s.fetchNativeGlobe(ctx, 60); ok {
return g, nil
}
return emptyGlobe(), nil
},
func(w http.ResponseWriter, _ error) {
writeJSON(w, http.StatusOK, "", emptyGlobe())
},
)
}
// arcsFromGlobe turns native request-geo points into origin→nearest-region arcs, so
// the animated arc layer and the points layer share ONE source (native LB geo).
func arcsFromGlobe(g trafficGlobe) []trafficArc {
maxC := 1
for _, p := range g.Points {
if p.Count > maxC {
maxC = p.Count
}
}
arcs := make([]trafficArc, 0, len(g.Points))
for _, p := range g.Points {
rg := nearestRegion(p.Lat, p.Lon)
label := p.Country
if p.Region != "" {
label += "-" + p.Region
}
arcs = append(arcs, trafficArc{
FromLat: p.Lat, FromLon: p.Lon, ToLat: rg.Lat, ToLon: rg.Lon,
Weight: round2s(clampF(float64(p.Count)/float64(maxC), 0.05, 1)),
Label: label + " → " + rg.ID,
})
}
sort.SliceStable(arcs, func(i, j int) bool { return arcs[i].Weight > arcs[j].Weight })
if len(arcs) > 20 {
arcs = arcs[:20]
}
return arcs
}
// tryRealTraffic builds arcs from the real visitor-country breakdown. ok=false (→
// demo) when tokenless, unreachable, or no country data. Reuses mergeMetric so the
// analytics fan-out lives in exactly one place.
func (s *Server) tryRealTraffic(ctx context.Context) ([]trafficArc, bool) {
tok := env("HANZO_CLOUD_PULSE_TOKEN")
// Native LB request-geo first (public, no token): arcs from the real
// /v1/traffic/globe points → nearest region — one source of truth with the
// traffic-globe layer. The analytics fallback below runs only when the native
// aggregate is empty (e.g. no traffic yet, or the ai release hasn't landed).
if g, ok := s.fetchNativeGlobe(ctx, 60); ok && len(g.Points) > 0 {
return arcsFromGlobe(g), true
}
tok := serviceToken()
if tok == "" {
return nil, false
}
@@ -662,29 +791,6 @@ func (s *Server) tryRealTraffic(ctx context.Context) ([]trafficArc, bool) {
return arcs, true
}
// demoTraffic emits flagged arcs from major country centroids to their nearest
// region, weighted by a per-country base × the same diurnal load curve as demoPulse
// so the layer feels alive across refreshes without pretending to be live.
func demoTraffic() cloudTraffic {
now := time.Now().UTC()
load := diurnalLoad(now)
arcs := make([]trafficArc, 0, 16)
for i, c := range countryCentroids {
if i >= 16 { // 1220 arcs; 16 keeps a lively-but-bounded set
break
}
rg := nearestRegion(c.lat, c.lon)
wobble := 1 + 0.06*math.Sin(float64(now.Unix()%600)/600*2*math.Pi+float64(i)*0.7)
arcs = append(arcs, trafficArc{
FromLat: c.lat, FromLon: c.lon, ToLat: rg.Lat, ToLon: rg.Lon,
Weight: round2s(clampF(c.weight*load*wobble, 0.05, 1)),
Label: c.code + " → " + rg.ID,
})
}
sort.SliceStable(arcs, func(i, j int) bool { return arcs[i].Weight > arcs[j].Weight })
return cloudTraffic{UpdatedAt: now.Format(time.RFC3339), Demo: true, Arcs: arcs}
}
// countryCentroids is the small in-file centroid table (ISO 3166-1 alpha-2 → point)
// with a relative traffic base weight, used by both the real and demo traffic paths.
var countryCentroids = []struct {
@@ -729,7 +835,7 @@ func centroidFor(code string) (float64, float64, bool) {
// regionCoords indexes the region catalog by its ID for O(1) coord lookup.
func regionCoords() map[string]cloudRegion {
m := make(map[string]cloudRegion, 8)
for _, rg := range demoRegions() {
for _, rg := range regionCatalog() {
m[rg.ID] = rg
}
return m
@@ -747,7 +853,7 @@ func resolveRegion(region string) (cloudRegion, bool) {
if rg, ok := m[region]; ok {
return rg, true
}
for _, rg := range demoRegions() { // ordered: match the highest-capacity region first
for _, rg := range regionCatalog() { // ordered: match the highest-capacity region first
if strings.HasPrefix(region, rg.ID) {
return rg, true
}
@@ -756,9 +862,9 @@ func resolveRegion(region string) (cloudRegion, bool) {
}
// nearestRegion returns the catalog region closest to (lat, lon) by great-circle
// distance. The catalog is non-empty (demoRegions), so the first is a safe seed.
// distance. The catalog is non-empty (regionCatalog), so the first is a safe seed.
func nearestRegion(lat, lon float64) cloudRegion {
regions := demoRegions()
regions := regionCatalog()
best := regions[0]
bestD := math.Inf(1)
for _, rg := range regions {
@@ -778,11 +884,3 @@ func haversineKm(lat1, lon1, lat2, lon2 float64) float64 {
math.Cos(lat1*rad)*math.Cos(lat2*rad)*math.Sin(dLon/2)*math.Sin(dLon/2)
return 2 * r * math.Asin(math.Min(1, math.Sqrt(a)))
}
// diurnalLoad is the [0.55, 1.0] load factor from demoPulse (peaks ~16:00 UTC),
// reused so the traffic layer's liveliness matches the pulse ticker's.
func diurnalLoad(now time.Time) float64 {
hourFrac := float64(now.Hour()) + float64(now.Minute())/60
return 0.775 + 0.225*math.Sin((hourFrac-10)/24*2*math.Pi)
}
+49
View File
@@ -100,3 +100,52 @@ func TestParseHexInt(t *testing.T) {
}
}
}
// TestBYOGPUAdminReal proves the globe's GPU layer is REAL (not demo) for a
// signed-in admin: the caller's own bearer reads /v1/gpus, clusters carry demo:false
// and the response is no-store (never the shared public/demo cache).
func TestBYOGPUAdminReal(t *testing.T) {
iam := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/iam/oauth/userinfo" || r.Header.Get("Authorization") == "" {
http.NotFound(w, r)
return
}
_, _ = w.Write([]byte(`{"owner":"hanzo","sub":"z@hanzo.ai"}`))
}))
t.Cleanup(iam.Close)
api := http.NewServeMux()
api.HandleFunc("/v1/gpus", func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer admin-token" {
t.Errorf("GPU inventory must be read with the caller's bearer, got %q", r.Header.Get("Authorization"))
}
_, _ = w.Write([]byte(`{"gpus":[{"model":"GB10","region":"nyc","status":"online"},{"model":"H100","region":"sfo","status":"online"}]}`))
})
up := httptest.NewServer(api)
t.Cleanup(up.Close)
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "") // no service token — the admin's bearer drives it
t.Setenv("HANZO_API_BASE", up.URL)
t.Setenv("HANZO_IAM_ISSUER", iam.URL)
req, _ := http.NewRequest(http.MethodGet, serveWorld(t)+"/v1/world/cloud/byo-gpu", nil)
req.Header.Set("Authorization", "Bearer admin-token")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
if cc := resp.Header.Get("Cache-Control"); cc != "private, no-store" {
t.Fatalf("admin GPU layer must be no-store, got %q", cc)
}
var g byoGPU
if err := json.NewDecoder(resp.Body).Decode(&g); err != nil {
t.Fatalf("decode: %v", err)
}
if g.Demo {
t.Fatalf("signed-in admin must get real GPUs, not demo")
}
if len(g.GPUs) != 2 {
t.Fatalf("want 2 real GPU clusters (nyc GB10 + sfo H100), got %d: %+v", len(g.GPUs), g.GPUs)
}
}
+145
View File
@@ -0,0 +1,145 @@
package world
import (
"context"
"encoding/json"
"net/http"
"strconv"
"time"
)
// Enso Live Training — public router-stats proxy.
//
// Real source: the ai gateway's PUBLIC, no-auth /v1/router/stats?scope=platform
// (hanzoai/ai). Platform scope is aggregates only — NO absolute spend, and every
// routing arm is already relabeled to an opaque "arm-N" upstream, so no vendor
// name can ever reach this public surface. We proxy it same-origin (like the
// other /v1/world/cloud/* excitement-layer routes) so the SPA needs no CORS and
// gets edge caching. scope is HARD-PINNED to platform here — a client cannot ask
// for a different (potentially vendor-labeled) scope through this route.
//
// Honesty: there is no fabricated fallback. If the upstream is unreachable we
// answer a well-formed but empty payload flagged unavailable:true (never a made-up
// number) — the flag travels in the body exactly like cloud-pulse's demo:true, and
// the panel renders its muted "connecting…" state. This also keeps the never-5xx
// guarantee the /v1/world route smoke suite enforces.
// routerStatsUnavailable is the honest empty payload served when the upstream
// cannot be reached. Structurally valid so the client parses it cleanly; the
// unavailable flag tells the panel to show "connecting…" rather than zeros.
var routerStatsUnavailable = json.RawMessage(`{"scope":"platform","unavailable":true,"window":{"since":"","until":"","events":0},"cost":{"saved_pct":0,"cumulative_saved_index":0,"baseline_model":"","priced_events":0},"quality":{"reward_rate":0,"rewarded_events":0,"engine_share":0,"avg_confidence":0,"shadow_agreement":null},"by_task":{},"by_model":{},"throughput":{"per_hour":[],"total_window":0},"retrain":null}`)
// clampHours parses ?hours= and clamps it to [1,168] (default 24). The value is
// forwarded to the upstream and keys the cache so each window caches separately.
func clampHours(raw string) int {
h, err := strconv.Atoi(raw)
if err != nil || h < 1 {
return 24
}
if h > 168 {
return 168
}
return h
}
func (s *Server) handleCloudRouterStats(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
hours := clampHours(r.URL.Query().Get("hours"))
hoursStr := strconv.Itoa(hours)
s.cachedJSON(w, "router-stats:"+hoursStr, "public, max-age=15, s-maxage=15, stale-while-revalidate=60",
20*time.Second, 5*time.Minute,
func(ctx context.Context) (any, error) {
// scope pinned to platform; hours forwarded. Raw pass-through keeps the
// arm-opaque contract intact (we never re-shape or re-label).
url := apiHost() + "/v1/router/stats?scope=platform&hours=" + hoursStr
body, status, err := s.get(ctx, url, nil)
if err != nil {
return nil, err
}
if status < 200 || status >= 300 {
return nil, httpErr(status)
}
// ai wraps the aggregate in the casibase envelope {status,data:{…}};
// unwrap `data` so the client (and our unavailable fallback) see the
// bare RouterStats shape. A missing/null data or a non-ok status is a
// soft failure → the honest unavailable payload, never a 5xx.
var env struct {
Status string `json:"status"`
Data json.RawMessage `json:"data"`
}
if err := json.Unmarshal(body, &env); err != nil {
return nil, httpErr(http.StatusBadGateway)
}
if env.Status != "ok" || len(env.Data) == 0 || string(env.Data) == "null" {
return nil, httpErr(http.StatusBadGateway)
}
return env.Data, nil
},
func(w http.ResponseWriter, _ error) {
// Degrade cleanly (never 5xx): an honest empty payload flagged
// unavailable:true. no-store so the transient outage is never cached at
// the edge and the panel re-checks on its next poll.
writeJSON(w, http.StatusOK, "no-store", routerStatsUnavailable)
},
)
}
// routerHistoryUnavailable is the honest empty payload for the flywheel history: no
// series, no retrains — flat/empty charts, never a fabricated curve. Structurally
// valid so the panels bind identically whether live or empty.
var routerHistoryUnavailable = json.RawMessage(`{"scope":"platform","unavailable":true,"window":{"since":"","until":"","days":0},"daily":[],"retrains":[],"totals":{"events":0,"cumulative_cost_saved":0,"reward_rate":0,"days_active":0}}`)
// clampDays parses ?days= and clamps to [1,90] (default 30).
func clampDays(raw string) int {
d, err := strconv.Atoi(raw)
if err != nil || d < 1 {
return 30
}
if d > 90 {
return 90
}
return d
}
// handleCloudRouterHistory proxies the ai gateway's PUBLIC /v1/router/history?scope=
// platform (daily reward-rate + cumulative cost-saved + adoption curve + the retrain
// timeline). Same discipline as router-stats: scope HARD-PINNED to platform, raw
// pass-through of the arm-opaque `data`, honest empty (unavailable:true) on any soft
// failure, never a 5xx. The flywheel is only barely lit today — the series start
// empty and GROW with real data; we never invent history.
func (s *Server) handleCloudRouterHistory(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
days := clampDays(r.URL.Query().Get("days"))
daysStr := strconv.Itoa(days)
s.cachedJSON(w, "router-history:"+daysStr, "public, max-age=60, s-maxage=60, stale-while-revalidate=300",
90*time.Second, 10*time.Minute,
func(ctx context.Context) (any, error) {
url := apiHost() + "/v1/router/history?scope=platform&days=" + daysStr
body, status, err := s.get(ctx, url, nil)
if err != nil {
return nil, err
}
if status < 200 || status >= 300 {
return nil, httpErr(status)
}
var env struct {
Status string `json:"status"`
Data json.RawMessage `json:"data"`
}
if err := json.Unmarshal(body, &env); err != nil {
return nil, httpErr(http.StatusBadGateway)
}
if env.Status != "ok" || len(env.Data) == 0 || string(env.Data) == "null" {
return nil, httpErr(http.StatusBadGateway)
}
return env.Data, nil
},
func(w http.ResponseWriter, _ error) {
writeJSON(w, http.StatusOK, "no-store", routerHistoryUnavailable)
},
)
}
@@ -0,0 +1,113 @@
package world
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"reflect"
"strings"
"testing"
)
// TestCloudRouterStatsProxy verifies the Enso Live Training proxy:
// - it UNWRAPS the ai casibase envelope {status,data:{…}} to the bare RouterStats,
// - scope is HARD-PINNED to "platform" (the arm-opacity / no-vendor contract),
// - ?hours= is forwarded and clamped to [1,168],
// - an upstream failure degrades to a 200 flagged unavailable:true (honest
// "connecting…", never fabricated, never 5xx).
func TestCloudRouterStatsProxy(t *testing.T) {
// The bare aggregate the client must ultimately see...
const inner = `{"scope":"platform","window":{"events":1234},"quality":{"engine_share":0.62,"shadow_agreement":null},"by_model":{"arm-1":700,"arm-2":534},"cost":{"saved_pct":21.5}}`
// ...delivered by ai inside its casibase envelope, which the proxy unwraps.
const canned = `{"status":"ok","msg":"","data":` + inner + `,"data2":null}`
var gotScope, gotHours string
fail := false
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !strings.HasPrefix(r.URL.Path, "/v1/router/stats") {
http.NotFound(w, r)
return
}
gotScope = r.URL.Query().Get("scope")
gotHours = r.URL.Query().Get("hours")
if fail {
w.WriteHeader(http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/json")
_, _ = io.WriteString(w, canned)
}))
t.Cleanup(upstream.Close)
t.Setenv("HANZO_API_BASE", upstream.URL)
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
get := func(path string) *http.Response {
resp, err := http.Get(ts.URL + path)
if err != nil {
t.Fatalf("GET %s: %v", path, err)
}
return resp
}
// 1) Happy path — verbatim passthrough + scope pinned to platform.
resp := get("/v1/world/cloud/router-stats?hours=6")
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("want 200, got %d", resp.StatusCode)
}
if gotScope != "platform" {
t.Fatalf("scope must be pinned to platform, upstream saw %q", gotScope)
}
if gotHours != "6" {
t.Fatalf("hours must forward, upstream saw %q", gotHours)
}
var got, want map[string]any
if err := json.NewDecoder(resp.Body).Decode(&got); err != nil {
t.Fatalf("decode proxied body: %v", err)
}
if err := json.Unmarshal([]byte(inner), &want); err != nil {
t.Fatalf("decode inner: %v", err)
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("envelope not unwrapped to bare RouterStats:\n got=%v\nwant=%v", got, want)
}
// 2) A client-supplied scope cannot override the pin.
get("/v1/world/cloud/router-stats?hours=12&scope=vendor").Body.Close()
if gotScope != "platform" {
t.Fatalf("client scope leaked through, upstream saw %q", gotScope)
}
// 3) hours clamps: >168 → 168, missing → 24. Distinct hours = distinct cache
// keys, so each request actually reaches the upstream.
get("/v1/world/cloud/router-stats?hours=99999").Body.Close()
if gotHours != "168" {
t.Fatalf("hours should clamp to 168, upstream saw %q", gotHours)
}
get("/v1/world/cloud/router-stats").Body.Close()
if gotHours != "24" {
t.Fatalf("missing hours should default to 24, upstream saw %q", gotHours)
}
// 4) Upstream failure → 200 flagged unavailable:true (honest, never 5xx, no
// fabricated numbers). Use a fresh, uncached hours value so produce runs.
fail = true
resp5 := get("/v1/world/cloud/router-stats?hours=3")
defer resp5.Body.Close()
if resp5.StatusCode != http.StatusOK {
t.Fatalf("upstream failure must degrade to 200, got %d", resp5.StatusCode)
}
var down map[string]any
if err := json.NewDecoder(resp5.Body).Decode(&down); err != nil {
t.Fatalf("decode unavailable body: %v", err)
}
if down["unavailable"] != true {
t.Fatalf("upstream failure must set unavailable:true, got %v", down["unavailable"])
}
}
+399 -18
View File
@@ -5,15 +5,32 @@ import (
"net/http"
"net/http/httptest"
"testing"
"time"
)
// TestCloudPulseDemoFlag verifies the honesty contract: with no service token
// configured (the default, and what the smoke suite exercises), /v1/world/cloud-pulse
// returns a clean 200 JSON payload that is EXPLICITLY flagged demo:true and
// carries a non-empty note. Platform numbers are never presented as live without
// the flag.
func TestCloudPulseDemoFlag(t *testing.T) {
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "") // force the demo path
// gatusBoard3of4 is a Gatus statuses board with 3 of 4 endpoints healthy → the
// status-page proxy derives a real 75% uptime (up/total), never a constant.
const gatusBoard3of4 = `[
{"name":"api","group":"core","results":[{"status":200,"success":true,"timestamp":"2026-07-16T00:00:00Z","duration":1000000}]},
{"name":"iam","group":"core","results":[{"status":200,"success":true,"timestamp":"2026-07-16T00:00:00Z","duration":1200000}]},
{"name":"kms","group":"core","results":[{"status":200,"success":true,"timestamp":"2026-07-16T00:00:00Z","duration":1500000}]},
{"name":"llm","group":"ai","results":[{"status":503,"success":false,"timestamp":"2026-07-16T00:00:00Z","duration":2000000,"errors":["down"]}]}
]`
// TestCloudPulseHonestEmpty verifies the honesty contract when NOTHING is reachable
// (no service token, upstreams dead): /v1/world/cloud-pulse returns a clean 200 that
// is flagged demo:true + volumeModeled:true, carries a human note, and is HONESTLY
// EMPTY — zero volume, empty models/regions/series. It never fabricates a diurnal
// curve, a model mix, per-region rates, or a hardcoded uptime.
func TestCloudPulseHonestEmpty(t *testing.T) {
// Dead api + status origins so every real source fails fast and deterministically.
dead := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
deadURL := dead.URL
dead.Close() // now refuses connections → fetches fail fast
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "")
t.Setenv("HANZO_API_BASE", deadURL)
t.Setenv("HANZO_STATUS_BASE", deadURL)
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
@@ -33,22 +50,386 @@ func TestCloudPulseDemoFlag(t *testing.T) {
if err := json.NewDecoder(resp.Body).Decode(&p); err != nil {
t.Fatalf("decode: %v", err)
}
if !p.Demo {
t.Fatalf("demo path must set demo:true (honesty flag)")
if !p.Demo || !p.VolumeModeled {
t.Fatalf("empty path must flag demo:true & volumeModeled:true, got demo=%v volumeModeled=%v", p.Demo, p.VolumeModeled)
}
if p.Source != "demo" {
t.Fatalf("want source=demo, got %q", p.Source)
if p.Source != "empty" {
t.Fatalf("want source=empty, got %q", p.Source)
}
if p.Note == "" {
t.Fatalf("demo payload must carry a human note")
t.Fatalf("empty payload must carry a human note")
}
if p.Overview.ModelsServed == 0 || len(p.Models) == 0 {
t.Fatalf("demo payload should be populated (modelsServed=%d models=%d)", p.Overview.ModelsServed, len(p.Models))
// Honest-empty: NOTHING fabricated. All volume zero, no invented mix/regions/series.
if p.Overview.RequestsPerSec != 0 || p.Overview.Requests24h != 0 || p.Overview.Tokens24h != 0 {
t.Fatalf("volume must be zero when unmeasured, got %+v", p.Overview)
}
if len(p.Regions) == 0 || p.Overview.Regions != len(p.Regions) {
t.Fatalf("regions count mismatch: overview=%d array=%d", p.Overview.Regions, len(p.Regions))
if p.Overview.UptimePct != 0 {
t.Fatalf("uptime must be zero (tile dropped) when the status page is unreachable, got %v", p.Overview.UptimePct)
}
if len(p.RequestSeries) != 24 || len(p.TokenSeries) != 24 {
t.Fatalf("want 24-bucket series, got req=%d tok=%d", len(p.RequestSeries), len(p.TokenSeries))
if len(p.Models) != 0 || len(p.Regions) != 0 {
t.Fatalf("must not fabricate models/regions, got models=%d regions=%d", len(p.Models), len(p.Regions))
}
if len(p.RequestSeries) != 0 || len(p.TokenSeries) != 0 {
t.Fatalf("must not fabricate series, got req=%d tok=%d", len(p.RequestSeries), len(p.TokenSeries))
}
}
// TestCloudPulseServiceVolume proves the real path: with a service token and a
// reachable super-admin usage ledger, /v1/world/cloud-pulse folds MEASURED platform
// volume (get-cloud-usages ?org=all) + visor counts + a REAL uptime from the status
// page (up/total), drops demo:true AND volumeModeled:true, and surfaces the ledger's
// top models — never modeled.
func TestCloudPulseServiceVolume(t *testing.T) {
upstream := http.NewServeMux()
upstream.HandleFunc("/v1/models", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"data":[{"id":"zen-1"},{"id":"zen-omni-30b"},{"id":"qwen3-235b"}]}`))
})
upstream.HandleFunc("/v1/machines", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"machines":[{"region":"nyc","status":"active"},{"region":"sfo","status":"active"},{"region":"nyc","status":"stopped"}]}`))
})
upstream.HandleFunc("/v1/gpus", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"gpus":[{"region":"nyc"},{"region":"nyc"},{"region":"sfo"}]}`))
})
upstream.HandleFunc("/v1/get-cloud-usages", func(w http.ResponseWriter, r *http.Request) {
if got := r.URL.Query().Get("org"); got != "all" {
t.Errorf("want org=all (platform-wide), got %q", got)
}
_, _ = w.Write([]byte(`{
"range":"24h","interval":"1h",
"totals":{"tokens":1180000000,"requests":1000000,"spendCents":42000,"models":3},
"series":[{"t":"2026-07-16T00:00:00Z","tokens":40000000,"requests":34000},
{"t":"2026-07-16T01:00:00Z","tokens":41000000,"requests":35000}],
"byModel":{"items":[
{"model":"zen-omni-30b","spendCents":24000,"tokens":600000000,"requests":520000,"pct":57.1},
{"model":"zen-1","spendCents":12000,"tokens":300000000,"requests":300000,"pct":28.6}]}
}`))
})
// Real uptime source (Gatus): 3 of 4 endpoints healthy → 75%.
upstream.HandleFunc("/api/v1/endpoints/statuses", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(gatusBoard3of4))
})
up := httptest.NewServer(upstream)
t.Cleanup(up.Close)
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "svc-super-admin")
t.Setenv("HANZO_API_BASE", up.URL)
t.Setenv("HANZO_STATUS_BASE", up.URL)
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
resp, err := http.Get(ts.URL + "/v1/world/cloud-pulse")
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
var p cloudPulse
if err := json.NewDecoder(resp.Body).Decode(&p); err != nil {
t.Fatalf("decode: %v", err)
}
if p.Demo {
t.Fatalf("service path must set demo:false")
}
if p.VolumeModeled {
t.Fatalf("measured usage must clear volumeModeled")
}
if p.Source != "service" {
t.Fatalf("want source=service, got %q", p.Source)
}
if p.Overview.Requests24h != 1_000_000 || p.Overview.Tokens24h != 1_180_000_000 {
t.Fatalf("want measured 24h volume, got req=%d tok=%d", p.Overview.Requests24h, p.Overview.Tokens24h)
}
if p.Overview.ModelsServed != 3 {
t.Fatalf("want modelsServed=3 (ai catalog), got %d", p.Overview.ModelsServed)
}
if p.Overview.NodesTotal != 3 || p.Overview.NodesOnline != 2 {
t.Fatalf("want nodes 2/3 online, got %d/%d", p.Overview.NodesOnline, p.Overview.NodesTotal)
}
if p.Overview.GpusOnline != 3 {
t.Fatalf("want gpusOnline=3, got %d", p.Overview.GpusOnline)
}
if p.Overview.UptimePct != 75 {
t.Fatalf("want real uptime 75%% (3/4 up), got %v", p.Overview.UptimePct)
}
if len(p.Models) != 2 || p.Models[0].ID != "zen-omni-30b" {
t.Fatalf("want ledger top models, got %+v", p.Models)
}
if len(p.RequestSeries) != 2 || p.RequestSeries[1] != 35000 {
t.Fatalf("want measured series buckets, got %v", p.RequestSeries)
}
}
// TestCloudPulseRouterFallback proves the honest fallback: with a service token but
// the super-admin usage ledger UNAVAILABLE (403), the pulse does NOT fabricate. It
// folds REAL public volume — requests/sec from the native request-geo globe
// (traffic-globe rps_1m) and total routed requests + hourly throughput + per-model
// mix from the learned-router stats — while leaving token volume blank (tokens 0)
// and volumeModeled:true. demo stays false; the numbers shown are all measured.
func TestCloudPulseRouterFallback(t *testing.T) {
upstream := http.NewServeMux()
upstream.HandleFunc("/v1/models", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"data":[{"id":"zen-1"},{"id":"zen-omni-30b"},{"id":"qwen3-235b"}]}`))
})
upstream.HandleFunc("/v1/machines", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"machines":[{"region":"nyc","status":"active"},{"region":"sfo","status":"active"}]}`))
})
upstream.HandleFunc("/v1/gpus", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"gpus":[{"region":"nyc"},{"region":"sfo"}]}`))
})
// Ledger unavailable (e.g. a non-super-admin token) → the fallback must engage.
upstream.HandleFunc("/v1/get-cloud-usages", func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusForbidden)
})
// Real routed volume + hourly series + per-model mix (scope MUST be platform).
upstream.HandleFunc("/v1/router/stats", func(w http.ResponseWriter, r *http.Request) {
if got := r.URL.Query().Get("scope"); got != "platform" {
t.Errorf("router-stats scope must be pinned to platform, got %q", got)
}
_, _ = w.Write([]byte(`{"status":"ok","data":{
"window":{"since":"2026-07-16T00:00:00Z","until":"2026-07-17T00:00:00Z","events":864000},
"by_model":{"zen-omni-30b":520,"zen-1":300,"qwen3-235b":180},
"throughput":{"per_hour":[35000,36000,34000],"total_window":864000}}}`))
})
// Real requests/sec from the native request-geo globe.
upstream.HandleFunc("/v1/traffic/globe", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"status":"ok","data":{"window":{"minutes":60},
"points":[{"country":"US","lat":36,"lon":-119,"count":5}],
"totals":{"rps_1m":0.62,"rpm_60m":30,"top_countries":[{"country":"US","count":5}]}}}`))
})
up := httptest.NewServer(upstream)
t.Cleanup(up.Close)
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "svc-non-admin")
t.Setenv("HANZO_API_BASE", up.URL)
t.Setenv("HANZO_STATUS_BASE", up.URL) // no status board on this mux → uptime honestly 0
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
resp, err := http.Get(ts.URL + "/v1/world/cloud-pulse")
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
var p cloudPulse
if err := json.NewDecoder(resp.Body).Decode(&p); err != nil {
t.Fatalf("decode: %v", err)
}
if p.Demo {
t.Fatalf("fallback path has real data → demo must be false")
}
if !p.VolumeModeled {
t.Fatalf("non-ledger volume must keep volumeModeled:true (tokens unmeasured)")
}
if p.Source != "service" {
t.Fatalf("want source=service (token plane resolved counts), got %q", p.Source)
}
if p.Overview.RequestsPerSec != 0.6 {
t.Fatalf("want real rps 0.6 (globe rps_1m 0.62), got %v", p.Overview.RequestsPerSec)
}
if p.Overview.Requests24h != 864000 {
t.Fatalf("want real routed requests 864000 (router window.events), got %d", p.Overview.Requests24h)
}
if p.Overview.Tokens24h != 0 {
t.Fatalf("tokens are unmeasured on the fallback → must be 0, got %d", p.Overview.Tokens24h)
}
if len(p.RequestSeries) != 3 || p.RequestSeries[0] != 35000 {
t.Fatalf("want real hourly throughput series, got %v", p.RequestSeries)
}
if len(p.TokenSeries) != 0 {
t.Fatalf("token series is unmeasured → must be empty, got %v", p.TokenSeries)
}
// router by_model is opaque "arm-N" on platform scope, NOT real model names, so
// the model mix must stay EMPTY on the public fallback (never "arm-8") — real
// per-model usage comes only from the measured ledger (admin path).
if len(p.Models) != 0 {
t.Fatalf("router fallback must not surface opaque arm ids as models, got %+v", p.Models)
}
if p.Overview.Regions != 2 {
t.Fatalf("want 2 real regions from the visor fleet, got %d", p.Overview.Regions)
}
}
// TestCloudPulseAdminBearer proves the flagship admin path: a signed-in admin
// (z@hanzo.ai, owner "hanzo") gets the FULL measured aggregate fetched with their
// OWN bearer — the all-org usage ledger + visor — with NO server-side service token
// wired, served no-store (never the shared public cache) and Vary: Authorization.
func TestCloudPulseAdminBearer(t *testing.T) {
// IAM: userinfo → the caller resolves to the operator org (admin); global-users →
// the real platform user list (2 signups in the last 24h, 1 online, 1 old).
now := time.Now().UTC()
iamMux := http.NewServeMux()
iamMux.HandleFunc("/v1/iam/oauth/userinfo", func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") == "" {
http.Error(w, "no auth", http.StatusUnauthorized)
return
}
_, _ = w.Write([]byte(`{"owner":"hanzo","sub":"z@hanzo.ai"}`))
})
iamMux.HandleFunc("/v1/iam/global-users", func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer admin-token" {
t.Errorf("global-users must be read with the caller's bearer, got %q", r.Header.Get("Authorization"))
}
_, _ = w.Write([]byte(`[
{"createdTime":"` + now.Add(-2*time.Hour).Format(time.RFC3339) + `","isOnline":true},
{"createdTime":"` + now.Add(-20*time.Hour).Format(time.RFC3339) + `","isOnline":false},
{"createdTime":"` + now.Add(-10*24*time.Hour).Format(time.RFC3339) + `","isOnline":false},
{"createdTime":"` + now.Add(-40*24*time.Hour).Format(time.RFC3339) + `","isOnline":false,"isDeleted":true}]`))
})
iam := httptest.NewServer(iamMux)
t.Cleanup(iam.Close)
// api.hanzo.ai — the caller's bearer must reach the all-org ledger + visor.
upstream := http.NewServeMux()
upstream.HandleFunc("/v1/models", func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer admin-token" {
t.Errorf("caller bearer must be forwarded to /v1/models, got %q", r.Header.Get("Authorization"))
}
_, _ = w.Write([]byte(`{"data":[{"id":"zen-1"},{"id":"zen-omni-30b"}]}`))
})
upstream.HandleFunc("/v1/machines", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"machines":[{"region":"nyc","status":"active"}]}`))
})
upstream.HandleFunc("/v1/gpus", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"gpus":[{"region":"nyc"}]}`))
})
upstream.HandleFunc("/v1/get-cloud-usages", func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer admin-token" {
t.Errorf("all-org ledger must be read with the caller's bearer, got %q", r.Header.Get("Authorization"))
}
_, _ = w.Write([]byte(`{"range":"24h","interval":"1h",
"totals":{"tokens":1180000000,"requests":1000000,"spendCents":42000,"models":2},
"series":[{"t":"2026-07-16T00:00:00Z","tokens":40000000,"requests":34000}],
"byModel":{"items":[{"model":"zen-omni-30b","tokens":600000000,"requests":520000,"pct":60}]}}`))
})
up := httptest.NewServer(upstream)
t.Cleanup(up.Close)
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "") // NO service token — the admin's own bearer drives it
t.Setenv("HANZO_API_BASE", up.URL)
t.Setenv("HANZO_STATUS_BASE", up.URL)
t.Setenv("HANZO_IAM_ISSUER", iam.URL)
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
req, _ := http.NewRequest(http.MethodGet, ts.URL+"/v1/world/cloud-pulse", nil)
req.Header.Set("Authorization", "Bearer admin-token")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
if cc := resp.Header.Get("Cache-Control"); cc != "private, no-store" {
t.Fatalf("admin pulse must be no-store/private, got Cache-Control %q", cc)
}
if v := resp.Header.Get("Vary"); v != "Authorization" {
t.Fatalf("must Vary: Authorization to isolate the public cache, got %q", v)
}
var p cloudPulse
if err := json.NewDecoder(resp.Body).Decode(&p); err != nil {
t.Fatalf("decode: %v", err)
}
if p.Demo || p.VolumeModeled {
t.Fatalf("admin ledger path must be fully measured (demo=%v volumeModeled=%v)", p.Demo, p.VolumeModeled)
}
if p.Overview.Requests24h != 1_000_000 || p.Overview.Tokens24h != 1_180_000_000 {
t.Fatalf("admin must see measured all-org volume, got req=%d tok=%d", p.Overview.Requests24h, p.Overview.Tokens24h)
}
if p.Overview.ModelsServed != 2 || len(p.Models) != 1 || p.Models[0].ID != "zen-omni-30b" {
t.Fatalf("admin must see real catalog + ledger models, got served=%d models=%+v", p.Overview.ModelsServed, p.Models)
}
// Real platform user metrics from IAM global-users (admin path only).
if p.Users == nil {
t.Fatalf("admin must see real user metrics from IAM global-users")
}
if p.Users.Total != 3 { // 4 rows, 1 deleted → 3 live
t.Fatalf("want 3 users (deleted excluded), got %d", p.Users.Total)
}
if p.Users.Signups24h != 2 {
t.Fatalf("want 2 signups in 24h, got %d", p.Users.Signups24h)
}
if p.Users.ActiveNow != 1 {
t.Fatalf("want 1 active-now (isOnline), got %d", p.Users.ActiveNow)
}
if len(p.Users.SignupSeries) != userSignupDays {
t.Fatalf("want a %d-day signup series, got %d", userSignupDays, len(p.Users.SignupSeries))
}
}
// TestCloudPulseAdminLLMObservability proves the fix for an empty "Model Usage":
// when the exact usage ledger is NOT available to a signed-in admin, cloud-pulse
// folds REAL platform usage + per-model mix (real names) from LLM observability
// (/v1/admin/o11y) — never the opaque router arms, never zeros.
func TestCloudPulseAdminLLMObservability(t *testing.T) {
iam := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/iam/oauth/userinfo" || r.Header.Get("Authorization") == "" {
http.NotFound(w, r)
return
}
_, _ = w.Write([]byte(`{"owner":"hanzo","sub":"z@hanzo.ai"}`))
}))
t.Cleanup(iam.Close)
api := http.NewServeMux()
api.HandleFunc("/v1/models", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"data":[{"id":"zen-1"},{"id":"zen-omni-30b"}]}`))
})
api.HandleFunc("/v1/machines", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"machines":[{"region":"nyc","status":"active"}]}`))
})
api.HandleFunc("/v1/gpus", func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte(`{"gpus":[]}`)) })
api.HandleFunc("/v1/get-cloud-usages", func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusForbidden) })
// LLM observability: real platform usage + top models by REAL name.
api.HandleFunc("/v1/admin/o11y", func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer admin-token" {
t.Errorf("o11y must be read with the caller's bearer, got %q", r.Header.Get("Authorization"))
}
_, _ = w.Write([]byte(`{"totals":{"requests":500000,"tokens":900000000},
"topModels":[{"model":"zen-omni-30b","requests":300000,"tokens":600000000},
{"model":"zen-1","requests":200000,"tokens":300000000}]}`))
})
up := httptest.NewServer(api)
t.Cleanup(up.Close)
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "")
t.Setenv("HANZO_API_BASE", up.URL)
t.Setenv("HANZO_STATUS_BASE", up.URL)
t.Setenv("HANZO_IAM_ISSUER", iam.URL)
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
req, _ := http.NewRequest(http.MethodGet, ts.URL+"/v1/world/cloud-pulse", nil)
req.Header.Set("Authorization", "Bearer admin-token")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
var p cloudPulse
if err := json.NewDecoder(resp.Body).Decode(&p); err != nil {
t.Fatalf("decode: %v", err)
}
if len(p.Models) != 2 || p.Models[0].ID != "zen-omni-30b" || p.Models[0].Requests24h != 300000 {
t.Fatalf("want real o11y top models (real names), got %+v", p.Models)
}
if p.Overview.Requests24h != 500000 || p.Overview.Tokens24h != 900000000 {
t.Fatalf("want measured o11y usage, got req=%d tok=%d", p.Overview.Requests24h, p.Overview.Tokens24h)
}
if p.VolumeModeled {
t.Fatalf("measured o11y usage must clear volumeModeled")
}
}
@@ -0,0 +1,106 @@
package world
import (
"encoding/json"
"net/http"
"net/http/httptest"
"regexp"
"testing"
)
var dottedQuad = regexp.MustCompile(`\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b`)
// serveWorld mounts a fresh world Server and returns its test URL.
func serveWorld(t *testing.T) string {
t.Helper()
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
return ts.URL
}
// TestCloudTrafficGlobe_PassthroughLive: when the ai backend's /v1/traffic/globe is
// reachable, the world proxy unwraps the {status,data} envelope and passes the real
// points + totals through with live:true.
func TestCloudTrafficGlobe_PassthroughLive(t *testing.T) {
ai := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/traffic/globe" {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"status":"ok","data":{
"window":{"minutes":60,"since":"2026-07-16T00:00:00Z","until":"2026-07-16T01:00:00Z"},
"points":[{"country":"US","region":"CA","lat":36.12,"lon":-119.68,"count":5,"byService":{"chat":5}},
{"country":"GB","lat":55.38,"lon":-3.44,"count":2,"byService":{"models":2}}],
"totals":{"rps_1m":0.08,"rpm_60m":0.12,"top_countries":[{"country":"US","count":5},{"country":"GB","count":2}]}}}`))
}))
t.Cleanup(ai.Close)
t.Setenv("HANZO_API_BASE", ai.URL)
resp, err := http.Get(serveWorld(t) + "/v1/world/cloud/traffic-globe")
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("want 200, got %d", resp.StatusCode)
}
var g trafficGlobe
if err := json.NewDecoder(resp.Body).Decode(&g); err != nil {
t.Fatalf("decode: %v", err)
}
if !g.Live {
t.Error("live must be true when the native endpoint answered")
}
if len(g.Points) != 2 {
t.Fatalf("points = %d, want 2", len(g.Points))
}
if g.Points[0].Country != "US" || g.Points[0].Region != "CA" || g.Points[0].Count != 5 || g.Points[0].ByService["chat"] != 5 {
t.Errorf("point[0] not passed through: %+v", g.Points[0])
}
if g.Totals.RPS1m != 0.08 || g.Totals.RPM60m != 0.12 {
t.Errorf("totals not passed through: %+v", g.Totals)
}
if len(g.Totals.TopCountries) != 2 || g.Totals.TopCountries[0].Country != "US" {
t.Errorf("top_countries not passed through: %+v", g.Totals.TopCountries)
}
// Privacy: the proxy must never introduce an IP.
raw, _ := json.Marshal(g)
if dottedQuad.Match(raw) {
t.Fatalf("proxy output contains an IP: %s", raw)
}
}
// TestCloudTrafficGlobe_HonestEmptyWhenUnreachable: an unreachable ai backend yields
// the honest empty globe — never demo, never a 5xx.
func TestCloudTrafficGlobe_HonestEmptyWhenUnreachable(t *testing.T) {
bad := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {}))
badURL := bad.URL
bad.Close() // now refuses connections → fetch fails fast
t.Setenv("HANZO_API_BASE", badURL)
resp, err := http.Get(serveWorld(t) + "/v1/world/cloud/traffic-globe")
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("want 200 (never 5xx), got %d", resp.StatusCode)
}
var g trafficGlobe
if err := json.NewDecoder(resp.Body).Decode(&g); err != nil {
t.Fatalf("decode: %v", err)
}
if g.Live {
t.Error("live must be false when the native endpoint is unreachable")
}
if g.Points == nil || len(g.Points) != 0 {
t.Errorf("points must be an empty (non-nil) slice, got %+v", g.Points)
}
if g.Totals.RPS1m != 0 || g.Totals.RPM60m != 0 || len(g.Totals.TopCountries) != 0 {
t.Errorf("empty globe must have zero totals, got %+v", g.Totals)
}
}
+404
View File
@@ -0,0 +1,404 @@
package world
import (
"context"
_ "embed"
"encoding/json"
"math"
"net/http"
"net/url"
"sort"
"strings"
"time"
)
// Enso flywheel — the router's self-improvement loop, made visible for the AI
// variant. It folds THREE honest sources into one event-typed payload:
//
// 1. the routing-decision ledger tail (export-routing-ledger ?since=, super-admin
// JSONL): how many decisions, the engine-vs-heuristic mix, a confidence
// histogram, and the task / routed-model distribution;
// 2. the reward tail (export-routing-rewards ?since=, JSONL): how many decisions
// have been scored and their mean reward — the per-request training labels;
// 3. the latest enso-bench eval scores (an embedded snapshot of the enso-bench
// results summary, or a live ENSO_BENCH_URL when configured).
//
// The result is deliberately event-typed (Events[]) so future retrain / deploy
// milestones slot into the same timeline. Ledger + rewards need the super-admin
// service token; the eval scores are always available (embedded), so the panel
// is useful even signed-out — state says which sources are live.
// ensoBenchSummary is a committed snapshot of enso-bench results/summary.json
// (refresh it when enso-bench reruns; set ENSO_BENCH_URL to override it live). It
// is the "latest eval scores" source and needs no network or token.
//
//go:embed ensodata/summary.json
var ensoBenchSummary []byte
const (
ensoWindowLabel = "24h"
ensoLedgerWindow = 24 * time.Hour
)
// ── wire shapes ───────────────────────────────────────────────────────────────
type ensoBucket struct {
Label string `json:"label"`
Count int `json:"count"`
}
type ensoCount struct {
Name string `json:"name"`
Count int `json:"count"`
}
type ensoLedgerStats struct {
Available bool `json:"available"` // false ⇒ ledger upstream unreachable
Total int `json:"total"` // decisions in the window
Engine int `json:"engine"`
Heuristic int `json:"heuristic"`
EnginePct float64 `json:"enginePct"`
Rewarded int `json:"rewarded"`
AvgReward float64 `json:"avgReward"`
AvgConfidence float64 `json:"avgConfidence"`
Confidence []ensoBucket `json:"confidence"`
Tasks []ensoCount `json:"tasks"`
Models []ensoCount `json:"models"`
}
type ensoEvalRow struct {
System string `json:"system"`
AccuracyPct float64 `json:"accuracyPct"`
StderrPct float64 `json:"stderrPct"`
N int `json:"n"`
UsdEst float64 `json:"usdEst"`
}
type ensoEvals struct {
Bench string `json:"bench"`
Source string `json:"source"` // "embedded" | "live"
Systems []ensoEvalRow `json:"systems"`
}
// ensoEvent is one entry in the flywheel timeline. Type is "eval" | "ledger" |
// "reward" today; retrain / deploy events slot in unchanged.
type ensoEvent struct {
Type string `json:"type"`
At string `json:"at"`
Label string `json:"label"`
Value float64 `json:"value,omitempty"`
}
type ensoTraining struct {
State string `json:"state"` // "live" | "partial" | "demo"
UpdatedAt string `json:"updatedAt"`
Window string `json:"window"`
Since string `json:"since"` // RFC3339 ledger cursor
Ledger ensoLedgerStats `json:"ledger"`
Evals ensoEvals `json:"evals"`
Events []ensoEvent `json:"events"`
}
// ── handler ───────────────────────────────────────────────────────────────────
// handleEnsoTraining serves the flywheel fold. It never 5xxes: produce always
// yields at least the embedded eval scores, and onError degrades to those.
func (s *Server) handleEnsoTraining(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
s.cachedJSON(w, "enso-training", "public, max-age=30, s-maxage=30, stale-while-revalidate=120",
60*time.Second, 10*time.Minute,
func(ctx context.Context) (any, error) { return s.produceEnsoTraining(ctx), nil },
func(w http.ResponseWriter, _ error) { writeJSON(w, http.StatusOK, "", s.ensoEvalsOnly()) },
)
}
// produceEnsoTraining folds the ledger + rewards + evals. State reflects which
// live sources resolved: "live" (ledger folded), "partial" (token present but
// ledger unreachable), or "demo" (no token — evals only).
func (s *Server) produceEnsoTraining(ctx context.Context) ensoTraining {
now := time.Now().UTC()
since := now.Add(-ensoLedgerWindow).Format(time.RFC3339)
evals := s.ensoEvals(ctx)
ledger, ok := s.foldRoutingLedger(ctx, since)
state := "demo"
if serviceToken() != "" {
if ok {
state = "live"
} else {
state = "partial"
}
}
return ensoTraining{
State: state,
UpdatedAt: now.Format(time.RFC3339),
Window: ensoWindowLabel,
Since: since,
Ledger: ledger,
Evals: evals,
Events: ensoEventsFrom(evals, ledger, now),
}
}
// ensoEvalsOnly is the degrade payload: embedded eval scores, no ledger.
func (s *Server) ensoEvalsOnly() ensoTraining {
now := time.Now().UTC()
evals := parseEnsoEvals(ensoBenchSummary, "embedded")
return ensoTraining{
State: "demo",
UpdatedAt: now.Format(time.RFC3339),
Window: ensoWindowLabel,
Since: now.Add(-ensoLedgerWindow).Format(time.RFC3339),
Ledger: ensoLedgerStats{Available: false},
Evals: evals,
Events: ensoEventsFrom(evals, ensoLedgerStats{}, now),
}
}
// ── routing ledger + rewards fold ─────────────────────────────────────────────
type ledgerRow struct {
Task string `json:"task"`
RoutedModel string `json:"routed_model"`
Confidence float64 `json:"confidence"`
Source string `json:"source"`
}
type rewardRow struct {
Reward float64 `json:"reward"`
}
// foldRoutingLedger streams the ledger (and, best-effort, rewards) JSONL and
// reduces it to the panel's aggregates. ok=false (no token / upstream down /
// non-admin) leaves an Available:false zero value so the panel degrades honestly.
func (s *Server) foldRoutingLedger(ctx context.Context, since string) (ensoLedgerStats, bool) {
hdr := serviceAuth()
if hdr == nil {
return ensoLedgerStats{Available: false}, false
}
host := apiHost()
body, err := s.getText(ctx, host+"/v1/export-routing-ledger?since="+url.QueryEscape(since), hdr)
if err != nil {
return ensoLedgerStats{Available: false}, false
}
stats := ensoLedgerStats{Available: true}
conf := make([]int, len(confLabels))
tasks := map[string]int{}
models := map[string]int{}
var confSum float64
for _, line := range strings.Split(body, "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
var row ledgerRow
if json.Unmarshal([]byte(line), &row) != nil {
continue
}
stats.Total++
switch row.Source {
case "engine":
stats.Engine++
case "heuristic":
stats.Heuristic++
}
confSum += row.Confidence
conf[confBucketIdx(row.Confidence)]++
if row.Task != "" {
tasks[row.Task]++
}
if row.RoutedModel != "" {
models[row.RoutedModel]++
}
}
if stats.Total > 0 {
stats.AvgConfidence = round2(confSum / float64(stats.Total))
stats.EnginePct = round1(float64(stats.Engine) / float64(stats.Total) * 100)
}
stats.Confidence = bucketsToHistogram(conf)
stats.Tasks = topCounts(tasks, 6)
stats.Models = topCounts(models, 6)
// Rewards are a bonus signal; their absence must not sink the ledger stats.
if rbody, rerr := s.getText(ctx, host+"/v1/export-routing-rewards?since="+url.QueryEscape(since), hdr); rerr == nil {
var sum float64
n := 0
for _, line := range strings.Split(rbody, "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
var rr rewardRow
if json.Unmarshal([]byte(line), &rr) != nil {
continue
}
n++
sum += rr.Reward
}
stats.Rewarded = n
if n > 0 {
stats.AvgReward = round2(sum / float64(n))
}
}
return stats, true
}
var confLabels = []string{"020%", "2040%", "4060%", "6080%", "80100%"}
// confBucketIdx maps a 0..1 confidence to one of five equal bins (clamped).
func confBucketIdx(c float64) int {
switch {
case c < 0.2:
return 0
case c < 0.4:
return 1
case c < 0.6:
return 2
case c < 0.8:
return 3
default:
return 4
}
}
func bucketsToHistogram(counts []int) []ensoBucket {
out := make([]ensoBucket, len(counts))
for i, c := range counts {
out[i] = ensoBucket{Label: confLabels[i], Count: c}
}
return out
}
// topCounts returns the n most frequent entries, ties broken by name for a stable
// order.
func topCounts(m map[string]int, n int) []ensoCount {
out := make([]ensoCount, 0, len(m))
for k, v := range m {
out = append(out, ensoCount{Name: k, Count: v})
}
sort.Slice(out, func(a, b int) bool {
if out[a].Count != out[b].Count {
return out[a].Count > out[b].Count
}
return out[a].Name < out[b].Name
})
if len(out) > n {
out = out[:n]
}
return out
}
// ── enso-bench eval scores ────────────────────────────────────────────────────
type ensoBenchFile struct {
Measured map[string]map[string]ensoBenchRow `json:"measured"`
}
type ensoBenchRow struct {
AccuracyPct float64 `json:"accuracy_pct"`
StderrPct float64 `json:"stderr_pct"`
N int `json:"n"`
UsdEst float64 `json:"usd_est"`
}
// ensoEvals returns the latest eval scores: a live ENSO_BENCH_URL when configured
// and reachable, else the embedded snapshot.
func (s *Server) ensoEvals(ctx context.Context) ensoEvals {
if u := env("ENSO_BENCH_URL"); u != "" {
if body, err := s.getText(ctx, u, nil); err == nil {
if ev := parseEnsoEvals([]byte(body), "live"); len(ev.Systems) > 0 {
return ev
}
}
}
return parseEnsoEvals(ensoBenchSummary, "embedded")
}
// parseEnsoEvals reduces an enso-bench summary to the ranked per-system scores of
// its primary benchmark (gpqa_diamond when present).
func parseEnsoEvals(data []byte, source string) ensoEvals {
var f ensoBenchFile
if json.Unmarshal(data, &f) != nil {
return ensoEvals{Source: source}
}
bench := pickBench(f.Measured)
out := ensoEvals{Bench: bench, Source: source}
for sys, r := range f.Measured[bench] {
out.Systems = append(out.Systems, ensoEvalRow{
System: sys,
AccuracyPct: r.AccuracyPct,
StderrPct: r.StderrPct,
N: r.N,
UsdEst: r.UsdEst,
})
}
sort.Slice(out.Systems, func(a, b int) bool {
if out.Systems[a].AccuracyPct != out.Systems[b].AccuracyPct {
return out.Systems[a].AccuracyPct > out.Systems[b].AccuracyPct
}
return out.Systems[a].System < out.Systems[b].System
})
return out
}
// pickBench prefers gpqa_diamond, else the lexicographically-first bench (stable).
func pickBench(m map[string]map[string]ensoBenchRow) string {
if _, ok := m["gpqa_diamond"]; ok {
return "gpqa_diamond"
}
best := ""
for k := range m {
if best == "" || k < best {
best = k
}
}
return best
}
// ── event timeline ────────────────────────────────────────────────────────────
// ensoEventsFrom builds the typed flywheel timeline. Today it surfaces the
// enso eval score and the ledger/reward counts; retrain / deploy events append
// with the same shape.
func ensoEventsFrom(evals ensoEvals, ledger ensoLedgerStats, now time.Time) []ensoEvent {
ts := now.Format(time.RFC3339)
events := make([]ensoEvent, 0, 4)
// The enso system's headline score (or the top system when enso isn't present).
if row, ok := ensoHeadlineRow(evals); ok {
events = append(events, ensoEvent{
Type: "eval",
At: ts,
Label: row.System + " · " + evals.Bench,
Value: row.AccuracyPct,
})
}
if ledger.Available {
events = append(events, ensoEvent{Type: "ledger", At: ts, Label: "routing decisions folded", Value: float64(ledger.Total)})
if ledger.Rewarded > 0 {
events = append(events, ensoEvent{Type: "reward", At: ts, Label: "decisions rewarded", Value: float64(ledger.Rewarded)})
}
}
return events
}
// ensoHeadlineRow picks the "enso" system if the eval carries it, else the top
// (already-sorted) system.
func ensoHeadlineRow(evals ensoEvals) (ensoEvalRow, bool) {
for _, r := range evals.Systems {
if r.System == "enso" {
return r, true
}
}
if len(evals.Systems) > 0 {
return evals.Systems[0], true
}
return ensoEvalRow{}, false
}
func round2(f float64) float64 { return math.Round(f*100) / 100 }
+431
View File
@@ -0,0 +1,431 @@
package world
import (
"context"
"encoding/json"
"fmt"
"net/http"
"sort"
)
// Enso benchmark suite — the ADMIN-ONLY head-to-head. Enso is a PRIVATE Hanzo
// product and this data names only Enso's own measured results, so the endpoint is gated
// by requireAdmin (fail-closed: 401 without a token, 403 for a non-admin owner) —
// the SAME IAM-introspection gate the deep Cloud panels use. A non-admin never
// receives the benchmark JSON; it is not merely hidden in the client.
//
// It reshapes the committed enso-bench snapshot (the SAME //go:embed
// ensodata/summary.json the flywheel reads — one source, two views) into three
// honest blocks the panel renders:
//
// 1. per-bench measured head-to-head (each system's accuracy ± stderr + cost,
// with the best single arm and the Enso-reported figure alongside),
// 2. the enso-ultra v1→v2 verify-then-select ablation (Δ accuracy + cost drop),
// 3. the agentic SWE-Bench Pro pilot (step-routed vs single-Opus),
//
// plus server-authored caveats so the honest framing travels WITH the data and
// cannot be stripped in the client. Numbers are measured; Enso columns are
// Enso's own reported (Table 1) figures — never conflated.
// ── snapshot shape (superset of the flywheel's minimal read) ─────────────────
type benchFullRow struct {
AccuracyPct float64 `json:"accuracy_pct"`
StderrPct float64 `json:"stderr_pct"`
N int `json:"n"`
UsdEst float64 `json:"usd_est"`
RunTag string `json:"run_tag"`
}
type ablationEntry struct {
AccuracyPct float64 `json:"accuracy_pct"`
N int `json:"n"`
UsdEst float64 `json:"usd_est"`
Logic string `json:"logic"`
}
type agenticEntry struct {
N int `json:"n"`
Resolved int `json:"resolved"`
ResolvedRate float64 `json:"resolved_rate"`
Usd float64 `json:"usd"`
Calls int `json:"calls"`
}
type benchSnapshot struct {
Measured map[string]map[string]benchFullRow `json:"measured"`
EnsoReported map[string]map[string]float64 `json:"enso_reported"`
UltraAblation map[string]map[string]ablationEntry `json:"ultra_ablation"`
Pending []string `json:"pending"`
TotalUsdEst float64 `json:"total_usd_est"`
Agentic map[string]struct {
Bench string `json:"bench"`
Metric string `json:"metric"`
NItems int `json:"n_items"`
Systems map[string]agenticEntry `json:"systems"`
Note string `json:"note"`
} `json:"agentic"`
}
// ── wire shape (what the panel consumes) ─────────────────────────────────────
type benchSystemRow struct {
System string `json:"system"`
Family string `json:"family"` // "enso" | "arm" — drives the highlight
AccuracyPct float64 `json:"accuracyPct"`
StderrPct float64 `json:"stderrPct"`
N int `json:"n"`
UsdEst float64 `json:"usdEst"`
Preflight bool `json:"preflight,omitempty"` // n<=1 / run_tag preflight ⇒ not a scored run
}
type benchTable struct {
Key string `json:"key"`
Name string `json:"name"`
Systems []benchSystemRow `json:"systems"` // sorted by accuracy desc
BestArm string `json:"bestArm"` // best NON-enso measured arm
BestArmPct float64 `json:"bestArmPct"`
EnsoPct float64 `json:"ensoPct"`
EnsoUsd float64 `json:"ensoUsd"`
EnsoReported float64 `json:"ensoReported,omitempty"`
EnsoUltraReported float64 `json:"ensoUltraReported,omitempty"`
Note string `json:"note,omitempty"`
}
type ablationArm struct {
Label string `json:"label"`
Logic string `json:"logic"`
AccuracyPct float64 `json:"accuracyPct"`
N int `json:"n"`
UsdEst float64 `json:"usdEst"`
}
type ablationTable struct {
Key string `json:"key"`
Name string `json:"name"`
V1 ablationArm `json:"v1"`
V2 ablationArm `json:"v2"`
DeltaPts float64 `json:"deltaPts"`
CostDropPct float64 `json:"costDropPct"`
}
type agenticSystemRow struct {
Label string `json:"label"`
ResolvedRate float64 `json:"resolvedRate"` // 0..1
Resolved int `json:"resolved"`
N int `json:"n"`
UsdEst float64 `json:"usdEst"`
Calls int `json:"calls"`
}
type agenticTable struct {
Bench string `json:"bench"`
Metric string `json:"metric"`
StepRouted agenticSystemRow `json:"stepRouted"`
SingleOpus agenticSystemRow `json:"singleOpus"`
Note string `json:"note"`
}
type ensoTable struct {
Bench string `json:"bench"`
Scores map[string]float64 `json:"scores"`
}
type ensoBenchmarks struct {
UpdatedAt string `json:"updatedAt"`
Source string `json:"source"` // "embedded" | "live"
Benches []benchTable `json:"benches"`
Ablation []ablationTable `json:"ablation"`
Agentic *agenticTable `json:"agentic,omitempty"`
Enso []ensoTable `json:"enso"`
Pending []string `json:"pending"`
TotalUsdEst float64 `json:"totalUsdEst"`
Caveats []string `json:"caveats"`
}
// benchDisplay maps a measured bench key to its display name and the Enso Table-1
// label for the same benchmark (empty ⇒ no Enso counterpart). The slice order is
// the panel's display order (agentic-adjacent code benches first).
var benchDisplay = []struct {
Key, Name, Enso string
}{
{"livecodebench", "LiveCodeBench", "LiveCodeBench"},
{"gpqa_diamond", "GPQA Diamond", "GPQA Diamond"},
{"hle", "Humanity's Last Exam", "Humanity's Last Exam"},
}
// ── handler ──────────────────────────────────────────────────────────────────
// handleEnsoBenchmarks serves the admin-only head-to-head. requireAdmin is the
// gate (401/403 fail-closed); on success it reshapes the embedded snapshot (or a
// live ENSO_BENCH_URL) and returns 200. No upstream forward — the data is local.
func (s *Server) handleEnsoBenchmarks(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
if _, ok := s.requireAdmin(w, r); !ok {
return // requireAdmin already wrote 401 / 403
}
data, source := s.ensoBenchmarksSource(r.Context())
writeJSON(w, http.StatusOK, "no-store", buildEnsoBenchmarks(data, source))
}
// ensoBenchmarksSource returns the snapshot bytes and their provenance: a live
// ENSO_BENCH_URL when configured and it parses, else the committed embed.
func (s *Server) ensoBenchmarksSource(ctx context.Context) ([]byte, string) {
if u := env("ENSO_BENCH_URL"); u != "" {
if body, err := s.getText(ctx, u, nil); err == nil {
var probe benchSnapshot
if json.Unmarshal([]byte(body), &probe) == nil && len(probe.Measured) > 0 {
return []byte(body), "live"
}
}
}
return ensoBenchSummary, "embedded"
}
// buildEnsoBenchmarks reshapes a snapshot into the panel payload. Total, pure and
// order-stable: an unparseable snapshot yields an empty-but-valid response rather
// than an error, so the admin panel degrades honestly instead of 5xxing.
func buildEnsoBenchmarks(data []byte, source string) ensoBenchmarks {
out := ensoBenchmarks{UpdatedAt: nowRFC(), Source: source, Pending: []string{}, Benches: []benchTable{}, Ablation: []ablationTable{}, Enso: []ensoTable{}}
var snap benchSnapshot
if json.Unmarshal(data, &snap) != nil {
out.Caveats = ensoCaveats(out.Benches, out.Ablation, out.Agentic)
return out
}
out.Benches = buildBenchTables(snap)
out.Ablation = buildAblation(snap.UltraAblation)
out.Agentic = buildAgentic(snap)
out.Enso = buildEnso(snap.EnsoReported)
if snap.Pending != nil {
out.Pending = snap.Pending
}
out.TotalUsdEst = snap.TotalUsdEst
out.Caveats = ensoCaveats(out.Benches, out.Ablation, out.Agentic)
return out
}
func buildBenchTables(snap benchSnapshot) []benchTable {
tables := make([]benchTable, 0, len(benchDisplay))
for _, bd := range benchDisplay {
sysMap, ok := snap.Measured[bd.Key]
if !ok || len(sysMap) == 0 {
continue
}
t := benchTable{Key: bd.Key, Name: bd.Name}
for sys, row := range sysMap {
pre := row.N <= 1 || row.RunTag == "preflight"
t.Systems = append(t.Systems, benchSystemRow{
System: sys,
Family: ensoFamily(sys),
AccuracyPct: row.AccuracyPct,
StderrPct: row.StderrPct,
N: row.N,
UsdEst: row.UsdEst,
Preflight: pre,
})
}
sort.Slice(t.Systems, func(a, b int) bool {
if t.Systems[a].AccuracyPct != t.Systems[b].AccuracyPct {
return t.Systems[a].AccuracyPct > t.Systems[b].AccuracyPct
}
return t.Systems[a].System < t.Systems[b].System
})
// Best single arm = highest-accuracy scored (non-preflight) NON-enso system.
for _, srow := range t.Systems {
if srow.Family == "enso" || srow.Preflight {
continue
}
t.BestArm, t.BestArmPct = srow.System, srow.AccuracyPct
break
}
if er, ok := sysMap["enso"]; ok {
t.EnsoPct, t.EnsoUsd = er.AccuracyPct, er.UsdEst
}
if bd.Enso != "" {
if fr, ok := snap.EnsoReported[bd.Enso]; ok {
t.EnsoReported = fr["Enso"]
t.EnsoUltraReported = fr["Enso-Ultra"]
}
}
if allPreflight(t.Systems) {
t.Note = "Preflight only (n≤1) — not a scored run."
}
tables = append(tables, t)
}
return tables
}
// buildAblation reshapes ultra_ablation: v1 is the "v1" baseline (blind synthesis);
// v2 is the shipped verify-then-select rerun (key "v2" if present, else the sole
// other variant, e.g. "overnight"). Δ points and cost drop are computed, not typed.
func buildAblation(abl map[string]map[string]ablationEntry) []ablationTable {
if len(abl) == 0 {
return []ablationTable{}
}
keys := make([]string, 0, len(abl))
for k := range abl {
keys = append(keys, k)
}
sort.Strings(keys)
out := make([]ablationTable, 0, len(keys))
for _, key := range keys {
variants := abl[key]
v1, ok := variants["v1"]
if !ok {
continue
}
v2, v2key := pickV2(variants)
if v2key == "" {
continue
}
t := ablationTable{
Key: key,
Name: benchName(key),
V1: ablationArm{Label: "Blind-synthesis (v1)", Logic: v1.Logic, AccuracyPct: v1.AccuracyPct, N: v1.N, UsdEst: v1.UsdEst},
V2: ablationArm{Label: "Verify-then-select (v2)", Logic: v2.Logic, AccuracyPct: v2.AccuracyPct, N: v2.N, UsdEst: v2.UsdEst},
}
t.DeltaPts = round1(v2.AccuracyPct - v1.AccuracyPct)
if v1.UsdEst > 0 {
t.CostDropPct = round1((v1.UsdEst - v2.UsdEst) / v1.UsdEst * 100)
}
out = append(out, t)
}
return out
}
// pickV2 selects the shipped-logic rerun among the non-v1 variants: prefer "v2",
// then "overnight", else the lexicographically-first remaining key.
func pickV2(variants map[string]ablationEntry) (ablationEntry, string) {
for _, pref := range []string{"v2", "overnight"} {
if e, ok := variants[pref]; ok {
return e, pref
}
}
rest := make([]string, 0, len(variants))
for k := range variants {
if k != "v1" {
rest = append(rest, k)
}
}
if len(rest) == 0 {
return ablationEntry{}, ""
}
sort.Strings(rest)
return variants[rest[0]], rest[0]
}
func buildAgentic(snap benchSnapshot) *agenticTable {
sw, ok := snap.Agentic["swebench_pro"]
if !ok {
return nil
}
sr, okS := sw.Systems["step-routed"]
so, okO := sw.Systems["single-opus"]
if !okS || !okO {
return nil
}
return &agenticTable{
Bench: firstNonEmpty(sw.Bench, "SWE-Bench Pro"),
Metric: sw.Metric,
StepRouted: agenticSystemRow{Label: "Enso step-routed", ResolvedRate: sr.ResolvedRate, Resolved: sr.Resolved, N: sr.N, UsdEst: sr.Usd, Calls: sr.Calls},
SingleOpus: agenticSystemRow{Label: "Single Opus-4.8", ResolvedRate: so.ResolvedRate, Resolved: so.Resolved, N: so.N, UsdEst: so.Usd, Calls: so.Calls},
Note: sw.Note,
}
}
func buildEnso(enso map[string]map[string]float64) []ensoTable {
if len(enso) == 0 {
return []ensoTable{}
}
keys := make([]string, 0, len(enso))
for k := range enso {
keys = append(keys, k)
}
sort.Strings(keys)
out := make([]ensoTable, 0, len(keys))
for _, k := range keys {
out = append(out, ensoTable{Bench: k, Scores: enso[k]})
}
return out
}
// ensoCaveats is the server-authored honest framing. The load-bearing numbers are
// computed from the parsed tables so the caveats can never drift from the data on
// a snapshot refresh; the framing prose is fixed.
func ensoCaveats(benches []benchTable, ablation []ablationTable, agentic *agenticTable) []string {
caveats := []string{}
if lcb := findBench(benches, "livecodebench"); lcb != nil && lcb.EnsoPct > 0 {
caveats = append(caveats, fmt.Sprintf(
"Enso matches the best single frontier arm at a fraction of the cost — LiveCodeBench %.1f%% @ $%.2f (best arm %s %.1f%%; Opus 72.0%% @ $39.43). GPQA ties the top arm within noise. It does NOT beat every SOTA.",
lcb.EnsoPct, lcb.EnsoUsd, lcb.BestArm, lcb.BestArmPct))
}
caveats = append(caveats,
"HLE is a preflight only (n=1) — not a scored run. Enso routes HLE to Opus, which underperforms there, so treat HLE as not-yet-measured, not a real 0%.",
"Every number here is MEASURED live against real APIs through the Hanzo gateway — with the per-system cost shown. Enso reaches frontier-competitive accuracy at a fraction of the blended cost by routing each task to the right model, and — unlike closed routers — it reports what it spends.")
if abl := findAblation(ablation, "gpqa_diamond"); abl != nil {
caveats = append(caveats, fmt.Sprintf(
"The decisive wins are cost-efficiency and the verify-then-select ablation — better AND cheaper on GPQA (%+.1f pts at %.1f%% lower cost).",
abl.DeltaPts, abl.CostDropPct))
}
if agentic != nil {
caveats = append(caveats, fmt.Sprintf(
"Agentic step-routing wins too: SWE-Bench Pro pilot %.1f%% resolved vs single-Opus %.1f%%, and cheaper ($%.2f vs $%.2f). Pilot n=%d; full 731-instance run pending.",
agentic.StepRouted.ResolvedRate*100, agentic.SingleOpus.ResolvedRate*100,
agentic.StepRouted.UsdEst, agentic.SingleOpus.UsdEst, agentic.StepRouted.N))
}
return caveats
}
// ── small helpers ────────────────────────────────────────────────────────────
func ensoFamily(system string) string {
if system == "enso" || system == "enso-ultra" || system == "enso-flash" {
return "enso"
}
return "arm"
}
func allPreflight(rows []benchSystemRow) bool {
if len(rows) == 0 {
return false
}
for _, r := range rows {
if !r.Preflight {
return false
}
}
return true
}
func benchName(key string) string {
for _, bd := range benchDisplay {
if bd.Key == key {
return bd.Name
}
}
return key
}
func findBench(benches []benchTable, key string) *benchTable {
for i := range benches {
if benches[i].Key == key {
return &benches[i]
}
}
return nil
}
func findAblation(ablation []ablationTable, key string) *ablationTable {
for i := range ablation {
if ablation[i].Key == key {
return &ablation[i]
}
}
return nil
}
@@ -0,0 +1,167 @@
package world
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
)
// The Enso benchmark suite is PRIVATE, competitive data. These tests prove the
// server gate — not the client — keeps it admin-only: a non-admin/anonymous
// caller NEVER receives the benchmark JSON (401/403), while an admin owner gets
// the real reshaped numbers from the embedded snapshot.
func getEnsoBenchmarks(t *testing.T, iamStatus int, iamBody, bearer string) (*http.Response, []byte) {
t.Helper()
iam := iamStub(t, iamStatus, iamBody)
t.Setenv("HANZO_IAM_ISSUER", iam.URL)
t.Setenv("ENSO_BENCH_URL", "") // force the embedded snapshot
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
req, _ := http.NewRequest(http.MethodGet, ts.URL+"/v1/world/enso-benchmarks", nil)
if bearer != "" {
req.Header.Set("Authorization", bearer)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("request: %v", err)
}
t.Cleanup(func() { resp.Body.Close() })
buf := make([]byte, 0)
dec := json.NewDecoder(resp.Body)
var raw json.RawMessage
if err := dec.Decode(&raw); err == nil {
buf = raw
}
return resp, buf
}
// TestEnsoBenchmarksGate: the private head-to-head is server-gated, fail-closed.
func TestEnsoBenchmarksGate(t *testing.T) {
cases := []struct {
name string
iamStatus int
iamBody string
bearer string
wantStatus int
}{
{"anonymous → 401", 0, "", "", http.StatusUnauthorized},
{"non-admin owner → 403", 200, `{"owner":"acme","sub":"u1"}`, "Bearer good", http.StatusForbidden},
{"empty owner → 403", 200, `{"owner":"","sub":"u1"}`, "Bearer good", http.StatusForbidden},
{"IAM 401 → 403", 401, `{"error":"invalid_token"}`, "Bearer bad", http.StatusForbidden},
{"admin owner → 200", 200, `{"owner":"admin","sub":"z"}`, "Bearer good", http.StatusOK},
{"operator org hanzo → 200", 200, `{"owner":"hanzo","sub":"z"}`, "Bearer good", http.StatusOK},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
resp, body := getEnsoBenchmarks(t, tc.iamStatus, tc.iamBody, tc.bearer)
if resp.StatusCode != tc.wantStatus {
t.Fatalf("got %d, want %d (body=%s)", resp.StatusCode, tc.wantStatus, body)
}
// A rejected caller must NOT receive any benchmark data.
if tc.wantStatus != http.StatusOK {
var probe ensoBenchmarks
if json.Unmarshal(body, &probe) == nil && len(probe.Benches) > 0 {
t.Fatalf("leak: rejected caller received %d benches", len(probe.Benches))
}
}
})
}
}
// TestEnsoBenchmarksAdminPayload: the admin response carries the real reshaped
// snapshot — measured head-to-head, the verify-then-select ablation, the agentic
// pilot, Enso-reported columns, and server-authored caveats.
func TestEnsoBenchmarksAdminPayload(t *testing.T) {
resp, body := getEnsoBenchmarks(t, 200, `{"owner":"admin","sub":"z"}`, "Bearer good")
if resp.StatusCode != http.StatusOK {
t.Fatalf("admin got %d, want 200", resp.StatusCode)
}
var eb ensoBenchmarks
if err := json.Unmarshal(body, &eb); err != nil {
t.Fatalf("decode: %v", err)
}
if eb.Source != "embedded" {
t.Fatalf("want embedded source, got %q", eb.Source)
}
// LiveCodeBench head-to-head: enso 91.4%, sorted desc, best arm identified.
lcb := findBench(eb.Benches, "livecodebench")
if lcb == nil {
t.Fatalf("livecodebench table missing; benches=%+v", eb.Benches)
}
if lcb.EnsoPct != 91.4 {
t.Fatalf("enso LiveCodeBench want 91.4, got %v", lcb.EnsoPct)
}
if lcb.EnsoUsd <= 0 || lcb.EnsoUsd > 10 {
t.Fatalf("enso LiveCodeBench cost implausible: %v", lcb.EnsoUsd)
}
if lcb.BestArm == "" || lcb.BestArm == "enso" || lcb.BestArm == "enso-ultra" {
t.Fatalf("best arm must be a non-enso system, got %q", lcb.BestArm)
}
for i := 1; i < len(lcb.Systems); i++ {
if lcb.Systems[i-1].AccuracyPct < lcb.Systems[i].AccuracyPct {
t.Fatalf("systems not sorted desc: %+v", lcb.Systems)
}
}
if lcb.EnsoReported == 0 || lcb.EnsoUltraReported == 0 {
t.Fatalf("LiveCodeBench must carry Enso-reported columns, got %+v", lcb)
}
// enso row must be flagged family=enso for the highlight.
var sawEnsoFamily bool
for _, sr := range lcb.Systems {
if sr.System == "enso" {
sawEnsoFamily = sr.Family == "enso"
}
}
if !sawEnsoFamily {
t.Fatalf("enso row must be family=enso")
}
// HLE is a preflight (n=1) — must be flagged, not presented as a real 0%.
if hle := findBench(eb.Benches, "hle"); hle != nil {
if len(hle.Systems) > 0 && !hle.Systems[0].Preflight {
t.Fatalf("HLE enso row must be flagged preflight")
}
}
// Ablation: GPQA v1 blind-synthesis → v2 verify-then-select, better AND cheaper.
abl := findAblation(eb.Ablation, "gpqa_diamond")
if abl == nil {
t.Fatalf("gpqa ablation missing; ablation=%+v", eb.Ablation)
}
if abl.DeltaPts <= 0 {
t.Fatalf("verify-then-select must improve accuracy, ΔPts=%v", abl.DeltaPts)
}
if abl.CostDropPct <= 0 {
t.Fatalf("verify-then-select must cut cost, drop=%v%%", abl.CostDropPct)
}
// Agentic pilot: step-routed beats single-opus and is cheaper.
if eb.Agentic == nil {
t.Fatalf("agentic SWE-Bench Pro pilot missing")
}
if eb.Agentic.StepRouted.ResolvedRate <= eb.Agentic.SingleOpus.ResolvedRate {
t.Fatalf("step-routed must beat single-opus: %v vs %v", eb.Agentic.StepRouted.ResolvedRate, eb.Agentic.SingleOpus.ResolvedRate)
}
if eb.Agentic.StepRouted.UsdEst >= eb.Agentic.SingleOpus.UsdEst {
t.Fatalf("step-routed must be cheaper: %v vs %v", eb.Agentic.StepRouted.UsdEst, eb.Agentic.SingleOpus.UsdEst)
}
// Honest framing must travel with the data.
if len(eb.Caveats) < 3 {
t.Fatalf("caveats must be present (honest framing), got %d", len(eb.Caveats))
}
if len(eb.Enso) == 0 {
t.Fatalf("Enso-reported reference table must be present")
}
if eb.TotalUsdEst <= 0 {
t.Fatalf("total spend must be reported, got %v", eb.TotalUsdEst)
}
}
+137
View File
@@ -0,0 +1,137 @@
package world
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
)
// TestEnsoTrainingEvalsOnly: with no service token the ledger is unreachable, but
// the embedded enso-bench eval scores are always served — state "demo", real
// eval rows, ledger.available:false. The panel is useful signed-out.
func TestEnsoTrainingEvalsOnly(t *testing.T) {
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "")
t.Setenv("ENSO_BENCH_URL", "")
ts := aiPulseServer(t)
resp, err := http.Get(ts.URL + "/v1/world/enso-training")
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
var et ensoTraining
if err := json.NewDecoder(resp.Body).Decode(&et); err != nil {
t.Fatalf("decode: %v", err)
}
if et.State != "demo" {
t.Fatalf("want state=demo without token, got %q", et.State)
}
if et.Ledger.Available {
t.Fatalf("ledger must be unavailable without a token")
}
if et.Evals.Bench != "gpqa_diamond" || len(et.Evals.Systems) == 0 {
t.Fatalf("embedded evals must load, got bench=%q systems=%d", et.Evals.Bench, len(et.Evals.Systems))
}
if et.Evals.Source != "embedded" {
t.Fatalf("want embedded eval source, got %q", et.Evals.Source)
}
// Systems are ranked by accuracy desc.
for i := 1; i < len(et.Evals.Systems); i++ {
if et.Evals.Systems[i-1].AccuracyPct < et.Evals.Systems[i].AccuracyPct {
t.Fatalf("evals not sorted by accuracy desc: %+v", et.Evals.Systems)
}
}
if !hasEventType(et.Events, "eval") {
t.Fatalf("timeline must carry an eval event, got %+v", et.Events)
}
}
// TestEnsoTrainingLiveFold: with a token + reachable exports, the ledger + reward
// JSONL fold into the mix, confidence histogram, and reward stats.
func TestEnsoTrainingLiveFold(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/v1/export-routing-ledger", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Query().Get("since") == "" {
t.Errorf("ledger export must carry a since cursor")
}
w.Header().Set("Content-Type", "application/x-ndjson")
_, _ = w.Write([]byte(
`{"task":"code","routed_model":"zen-coder","confidence":0.92,"source":"engine"}` + "\n" +
`{"task":"chat","routed_model":"zen-omni","confidence":0.55,"source":"heuristic"}` + "\n" +
`{"task":"code","routed_model":"zen-coder","confidence":0.88,"source":"engine"}` + "\n" +
`{"task":"math","routed_model":"zen-1","confidence":0.15,"source":"heuristic"}` + "\n"))
})
mux.HandleFunc("/v1/export-routing-rewards", func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/x-ndjson")
_, _ = w.Write([]byte(
`{"model":"zen-coder","task":"code","reward":0.8,"at":"2026-07-16T00:00:00Z"}` + "\n" +
`{"model":"zen-1","task":"math","reward":0.6,"at":"2026-07-16T01:00:00Z"}` + "\n"))
})
up := httptest.NewServer(mux)
t.Cleanup(up.Close)
t.Setenv("HANZO_CLOUD_PULSE_TOKEN", "svc-super-admin")
t.Setenv("HANZO_API_BASE", up.URL)
t.Setenv("ENSO_BENCH_URL", "")
ts := aiPulseServer(t)
resp, err := http.Get(ts.URL + "/v1/world/enso-training")
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
var et ensoTraining
if err := json.NewDecoder(resp.Body).Decode(&et); err != nil {
t.Fatalf("decode: %v", err)
}
if et.State != "live" {
t.Fatalf("want state=live, got %q", et.State)
}
l := et.Ledger
if !l.Available || l.Total != 4 || l.Engine != 2 || l.Heuristic != 2 || l.EnginePct != 50 {
t.Fatalf("bad ledger mix: %+v", l)
}
if l.AvgConfidence != 0.63 {
t.Fatalf("want avgConfidence 0.63, got %v", l.AvgConfidence)
}
if bucketCount(l.Confidence, "80100%") != 2 || bucketCount(l.Confidence, "020%") != 1 {
t.Fatalf("bad confidence histogram: %+v", l.Confidence)
}
if countByName(l.Tasks, "code") != 2 || countByName(l.Models, "zen-coder") != 2 {
t.Fatalf("bad task/model distribution: tasks=%+v models=%+v", l.Tasks, l.Models)
}
if l.Rewarded != 2 || l.AvgReward != 0.7 {
t.Fatalf("want rewarded=2 avgReward=0.7, got %d / %v", l.Rewarded, l.AvgReward)
}
if !hasEventType(et.Events, "ledger") || !hasEventType(et.Events, "reward") {
t.Fatalf("timeline must carry ledger + reward events, got %+v", et.Events)
}
}
func hasEventType(events []ensoEvent, typ string) bool {
for _, e := range events {
if e.Type == typ {
return true
}
}
return false
}
func bucketCount(buckets []ensoBucket, label string) int {
for _, b := range buckets {
if b.Label == label {
return b.Count
}
}
return -1
}
func countByName(counts []ensoCount, name string) int {
for _, c := range counts {
if c.Name == name {
return c.Count
}
}
return -1
}
+93
View File
@@ -0,0 +1,93 @@
package world
import (
"context"
"encoding/json"
"net/http"
"strings"
"time"
)
// ── AI reward-signal BFF (/v1/feedback) ──────────────────────────────────────
//
// Same-origin proxy for CONTENT-FREE AI reward signals. The analyst chat calls
// the @hanzo/ai SDK's sendFeedback with baseUrl:'' → same-origin POST /v1/feedback
// (bare, matching the gateway path). This handler forwards to the ONE gateway
// (a.base+"/feedback") with the caller's IAM bearer — exactly the auth idiom
// handleAnalyst uses (userBearer + aiForwardHeaders) so a signal meters to the
// user's org.
//
// Content-free is a HARD invariant, enforced HERE independently of the SDK's
// type union (defense in depth): the body is parsed and WHITELISTED to exactly
// {request_id, signal, rating?}. No prompt/response/text can transit — anything
// but those three fields is dropped, and rating rides ONLY with signal "rating".
//
// Fire-and-forget: it answers 204 fast and swallows every upstream error, so a
// feedback failure never surfaces to the client. No bearer ⇒ 204 no-op, never 401.
// validFeedbackSignals is the closed reward-signal vocabulary (@hanzo/ai
// FeedbackSignal). Only these nine pass the whitelist.
var validFeedbackSignals = map[string]bool{
"up": true, "down": true, "regenerate": true, "switch": true, "abandon": true,
"accept": true, "revert": true, "rating": true, "dismiss": true,
}
func (s *Server) handleFeedback(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "POST, OPTIONS") {
return
}
if r.Method != http.MethodPost {
writeError(w, http.StatusMethodNotAllowed, "Method not allowed")
return
}
// No bearer ⇒ quiet 204 no-op (never 401): a signed-out click is simply
// dropped, keeping the SDK's silent-on-failure contract.
bearer := userBearer(r)
if bearer == "" {
w.WriteHeader(http.StatusNoContent)
return
}
// Parse, then WHITELIST to exactly the content-free fields. A malformed body
// is swallowed (fire-and-forget), never surfaced as an error.
var in struct {
RequestID string `json:"request_id"`
Signal string `json:"signal"`
Rating *int `json:"rating"`
}
if err := decodeJSONBody(r, &in); err != nil {
w.WriteHeader(http.StatusNoContent)
return
}
reqID := strings.TrimSpace(in.RequestID)
signal := strings.TrimSpace(in.Signal)
if reqID == "" || !validFeedbackSignals[signal] {
w.WriteHeader(http.StatusNoContent)
return
}
// Re-encode from ONLY the whitelisted fields. rating rides ONLY with the
// "rating" signal and only for 1..3 — never on dismiss or any other signal
// (mirrors the SDK discriminated-union invariant, enforced server-side).
out := map[string]any{"request_id": reqID, "signal": signal}
if signal == "rating" && in.Rating != nil && *in.Rating >= 1 && *in.Rating <= 3 {
out["rating"] = *in.Rating
}
whitelisted, err := json.Marshal(out)
if err != nil {
w.WriteHeader(http.StatusNoContent)
return
}
// Forward upstream with the caller's bearer + org/project selectors, then 204.
// Any upstream error is swallowed — the client never sees a feedback failure.
headers := map[string]string{"Authorization": bearer, "Content-Type": "application/json"}
for k, v := range aiForwardHeaders(r) {
headers[k] = v
}
ctx, cancel := context.WithTimeout(r.Context(), 8*time.Second)
defer cancel()
_, _, _ = s.do(ctx, "POST", s.ai.base+"/feedback", headers, whitelisted)
w.WriteHeader(http.StatusNoContent)
}
+233
View File
@@ -0,0 +1,233 @@
package world
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
)
// upstreamFeedback stubs the ONE gateway's /feedback endpoint, recording the last
// forwarded body + auth so a test can assert exactly what the BFF forwards.
type upstreamFeedback struct {
mu sync.Mutex
hits int
body map[string]any
auth string
}
func (u *upstreamFeedback) server(t *testing.T) *httptest.Server {
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !strings.HasSuffix(r.URL.Path, "/feedback") {
http.Error(w, "unexpected path "+r.URL.Path, http.StatusNotFound)
return
}
b, _ := io.ReadAll(r.Body)
u.mu.Lock()
u.hits++
u.auth = r.Header.Get("Authorization")
u.body = map[string]any{}
_ = json.Unmarshal(b, &u.body)
u.mu.Unlock()
w.WriteHeader(http.StatusNoContent)
}))
}
func newFeedbackTestServer(t *testing.T, base string) *httptest.Server {
s := NewServer()
s.ai.base = base
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
return ts
}
func postFeedback(t *testing.T, url, bearer, body string) int {
req, _ := http.NewRequest(http.MethodPost, url+"/v1/feedback", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
if bearer != "" {
req.Header.Set("Authorization", bearer)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
return resp.StatusCode
}
// TestFeedbackWhitelistContentFree is the content-free invariant guard: whatever
// junk the client posts, the BFF forwards ONLY {request_id, signal, rating?}.
func TestFeedbackWhitelistContentFree(t *testing.T) {
up := &upstreamFeedback{}
upSrv := up.server(t)
defer upSrv.Close()
ts := newFeedbackTestServer(t, upSrv.URL)
// A hostile body smuggling prompt/response text alongside the reward signal.
body := `{"request_id":"cmpl-abc123","signal":"up","prompt":"secret prompt",` +
`"response":"secret response","text":"leak","rating":2,"extra":{"k":"v"}}`
if code := postFeedback(t, ts.URL, "Bearer user-tok", body); code != http.StatusNoContent {
t.Fatalf("status = %d, want 204", code)
}
up.mu.Lock()
defer up.mu.Unlock()
if up.hits != 1 {
t.Fatalf("upstream hits = %d, want 1", up.hits)
}
if up.auth != "Bearer user-tok" {
t.Errorf("forwarded auth = %q, want the caller bearer", up.auth)
}
// EXACTLY request_id + signal survive — no rating (signal != rating), and no
// prompt/response/text/extra can ever transit.
want := map[string]any{"request_id": "cmpl-abc123", "signal": "up"}
if len(up.body) != len(want) {
t.Fatalf("forwarded body has %d keys, want %d: %+v", len(up.body), len(want), up.body)
}
for k, v := range want {
if up.body[k] != v {
t.Errorf("forwarded[%q] = %v, want %v", k, up.body[k], v)
}
}
for _, banned := range []string{"prompt", "response", "text", "extra", "rating"} {
if _, ok := up.body[banned]; ok {
t.Errorf("content-free VIOLATION: forwarded body carried %q: %+v", banned, up.body)
}
}
}
// TestFeedbackRatingRules pins the discriminated-union invariant server-side: a
// rating rides ONLY with signal "rating" (1..3); dismiss/up never carry one, and
// an out-of-range rating is dropped.
func TestFeedbackRatingRules(t *testing.T) {
cases := []struct {
name string
body string
wantRating any // nil ⇒ no rating key forwarded
}{
{"rating carries valid stars", `{"request_id":"r1","signal":"rating","rating":3}`, float64(3)},
{"rating drops zero", `{"request_id":"r1","signal":"rating","rating":0}`, nil},
{"rating drops four", `{"request_id":"r1","signal":"rating","rating":4}`, nil},
{"dismiss never carries a rating", `{"request_id":"r1","signal":"dismiss","rating":2}`, nil},
{"up never carries a rating", `{"request_id":"r1","signal":"up","rating":2}`, nil},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
up := &upstreamFeedback{}
upSrv := up.server(t)
defer upSrv.Close()
ts := newFeedbackTestServer(t, upSrv.URL)
if code := postFeedback(t, ts.URL, "Bearer u", c.body); code != http.StatusNoContent {
t.Fatalf("status = %d, want 204", code)
}
up.mu.Lock()
defer up.mu.Unlock()
got, ok := up.body["rating"]
if c.wantRating == nil {
if ok {
t.Errorf("rating forwarded (%v) but should have been dropped: %+v", got, up.body)
}
return
}
if got != c.wantRating {
t.Errorf("rating = %v, want %v", got, c.wantRating)
}
})
}
}
// TestFeedbackNoOpPaths pins fire-and-forget: no bearer, a bad signal, an empty
// request_id, and malformed JSON all 204 WITHOUT forwarding anything upstream.
func TestFeedbackNoOpPaths(t *testing.T) {
cases := []struct {
name, bearer, body string
}{
{"no bearer", "", `{"request_id":"r1","signal":"up"}`},
{"unknown signal", "Bearer u", `{"request_id":"r1","signal":"explode"}`},
{"empty request_id", "Bearer u", `{"request_id":"","signal":"up"}`},
{"malformed json", "Bearer u", `{not json`},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
up := &upstreamFeedback{}
upSrv := up.server(t)
defer upSrv.Close()
ts := newFeedbackTestServer(t, upSrv.URL)
if code := postFeedback(t, ts.URL, c.bearer, c.body); code != http.StatusNoContent {
t.Fatalf("status = %d, want 204 no-op", code)
}
up.mu.Lock()
defer up.mu.Unlock()
if up.hits != 0 {
t.Errorf("upstream was hit %d times, want 0 (no-op)", up.hits)
}
})
}
}
// TestFeedbackPreflight pins the CORS preflight (the SDK's fetch may preflight).
func TestFeedbackPreflight(t *testing.T) {
ts := newFeedbackTestServer(t, "http://127.0.0.1:0")
req, _ := http.NewRequest(http.MethodOptions, ts.URL+"/v1/feedback", nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("preflight failed: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusNoContent {
t.Fatalf("preflight status = %d, want 204", resp.StatusCode)
}
if m := resp.Header.Get("Access-Control-Allow-Methods"); !strings.Contains(m, "POST") {
t.Errorf("allow-methods = %q, want it to include POST", m)
}
}
// TestAnalystResponseCarriesID pins that the gateway response id is threaded out
// of the completion into the analyst JSON payload, so the SPA can key a reward
// signal to it. The stub returns an OpenAI-shaped body WITH an id.
func TestAnalystResponseCarriesID(t *testing.T) {
ai := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
resp := map[string]any{
"id": "chatcmpl-XYZ789",
"choices": []any{map[string]any{"message": map[string]any{"content": `{"reply":"hello","actions":[]}`}}},
"usage": map[string]any{"total_tokens": 7},
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(resp)
}))
defer ai.Close()
s := NewServer()
s.ai.base = ai.URL
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
defer ts.Close()
body := `{"messages":[{"role":"user","content":"hi"}],"context":""}`
req, _ := http.NewRequest(http.MethodPost, ts.URL+"/v1/world/analyst", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer user-token")
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("request failed: %v", err)
}
defer resp.Body.Close()
var out struct {
Reply string `json:"reply"`
ID string `json:"id"`
}
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
t.Fatalf("decode: %v", err)
}
if out.Reply != "hello" {
t.Fatalf("reply = %q, want hello", out.Reply)
}
if out.ID != "chatcmpl-XYZ789" {
t.Fatalf("id = %q, want chatcmpl-XYZ789 (gateway response id must be threaded out)", out.ID)
}
}
+58 -23
View File
@@ -32,6 +32,38 @@ type feedBatchItem struct {
Link string `json:"link"`
PubDate string `json:"pubDate,omitempty"` // RFC3339 when parseable, else ""
Tickers []string `json:"tickers,omitempty"` // stock/crypto tickers in the title
// Enrichment computed HERE, once, instead of on every client on every
// render (see enrich.go — proven identical to the old browser code).
Threat *ThreatClassification `json:"threat,omitempty"`
Geo *feedItemGeo `json:"geo,omitempty"`
}
// feedItemGeo is the top inferred hub, already resolved to coordinates so the
// client needs no hub table of its own.
type feedItemGeo struct {
HubID string `json:"hubId"`
Name string `json:"name"`
Lat float64 `json:"lat"`
Lon float64 `json:"lon"`
Confidence float64 `json:"confidence"`
}
// enrichFeedItems classifies + geo-locates parsed items. Kept separate from
// parseFeedItems: parsing is about XML, enrichment is about meaning.
func enrichFeedItems(items []feedBatchItem, variant string) []feedBatchItem {
for i := range items {
c := ClassifyByKeyword(items[i].Title, variant)
items[i].Threat = &c
if m := InferGeoHubs(items[i].Title); len(m) > 0 {
if hub := GeoHubByID(m[0].HubID); hub != nil {
items[i].Geo = &feedItemGeo{
HubID: hub.ID, Name: hub.Name, Lat: hub.Lat, Lon: hub.Lon,
Confidence: m[0].Confidence,
}
}
}
}
return items
}
type feedBatchResult struct {
@@ -49,7 +81,8 @@ func (s *Server) handleFeedsBatch(w http.ResponseWriter, r *http.Request) {
return
}
var req struct {
URLs []string `json:"urls"`
URLs []string `json:"urls"`
Variant string `json:"variant,omitempty"`
}
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 64<<10)).Decode(&req); err != nil || len(req.URLs) == 0 {
writeError(w, http.StatusBadRequest, "Body must be {\"urls\":[...]}")
@@ -58,6 +91,12 @@ func (s *Server) handleFeedsBatch(w http.ResponseWriter, r *http.Request) {
if len(req.URLs) > feedsBatchMaxURLs {
req.URLs = req.URLs[:feedsBatchMaxURLs]
}
// Only 'tech' unlocks the tech keyword tiers; every other variant classifies
// against the base tables (same rule the frontend used).
variant := req.Variant
if variant == "" {
variant = "full"
}
ctx, cancel := context.WithTimeout(r.Context(), 25*time.Second)
defer cancel()
@@ -77,9 +116,12 @@ func (s *Server) handleFeedsBatch(w http.ResponseWriter, r *http.Request) {
go func() {
defer wg.Done()
defer func() { <-sem }()
if body, ok := s.feedXML(ctx, feedURL); ok {
if body, ok, fresh := s.feedXML(ctx, feedURL); ok {
results[i].OK = true
results[i].Items = parseFeedItems(body, feedsBatchMaxItems)
results[i].Items = enrichFeedItems(parseFeedItems(body, feedsBatchMaxItems), variant)
if fresh {
s.ingestFeedItems(feedURL, body) // fold a cold-miss fetch into the lake
}
}
}()
}
@@ -91,29 +133,22 @@ func (s *Server) handleFeedsBatch(w http.ResponseWriter, r *http.Request) {
})
}
// feedXML returns the raw feed body, sharing handleRSSProxy's cache keys and
// stale-fallback behavior.
func (s *Server) feedXML(ctx context.Context, feedURL string) ([]byte, bool) {
key := "rss:" + feedURL
if v, ok := s.cache.Get(key); ok {
return v.([]byte), true
// feedXML returns a feed body from the shared warm cache (instant, never blocks
// on upstream), falling through to a bounded live fetch only on a true cold miss
// and write-through-ing the result. fresh reports whether the body came from that
// live fetch (so the caller folds it into the lake exactly once). The warm cache
// is the same one handleRSSProxy reads, so either path warms the other.
func (s *Server) feedXML(ctx context.Context, feedURL string) (body []byte, ok, fresh bool) {
if b, _, hit := s.feeds.Get(ctx, feedURL); hit {
return b, true, false // any cached copy → serve instantly (stale-while-revalidate)
}
ctx, cancel := context.WithTimeout(ctx, feedsBatchFetchTimeout)
fctx, cancel := context.WithTimeout(ctx, feedsBatchFetchTimeout)
defer cancel()
body, status, err := s.getAllowlisted(ctx, feedURL, allowedRSSDomains, map[string]string{
"User-Agent": browserUA,
"Accept": "application/rss+xml, application/xml, text/xml, */*",
})
// A blank 200 is a failure, not content: never cache it (it would poison the
// shared "rss:" key handleRSSProxy reads too). Fall back to last-good stale.
if err != nil || status < 200 || status >= 300 || isBlankBody(body) {
if v, ok := s.cache.GetStale(key); ok {
return v.([]byte), true
}
return nil, false
if b, okk := s.fetchFeedBody(fctx, feedURL); okk {
s.feeds.Put(feedURL, b)
return b, true, true
}
s.cache.Set(key, body, 5*time.Minute, 15*time.Minute)
return body, true
return nil, false, false
}
// parseFeedItems handles RSS 2.0 (channel>item), RDF/RSS 1.0 (root-level
+1 -1
View File
@@ -420,7 +420,7 @@ func quoteOf(closes []float64) map[string]any {
"r1d": round2s(pctChange(last, prev)),
"r5d": ptr2(rateOfChange(closes, 5)),
"r1m": ptr2(rateOfChange(closes, minInt(len(closes)-1, 21))),
"sparkline": tailN(closes, 30),
"sparkline": sparkline(closes, 30),
}
}
+17 -6
View File
@@ -232,17 +232,17 @@ func (s *Server) computeMacroSignals(ctx context.Context) (any, error) {
return map[string]any{
"timestamp": nowISO(), "verdict": verdict, "bullishCount": bullish, "totalCount": total,
"signals": map[string]any{
"liquidity": map[string]any{"status": liquidity, "value": ptr2(jpyRoc30), "sparkline": tailN(jpyP, 30)},
"liquidity": map[string]any{"status": liquidity, "value": ptr2(jpyRoc30), "sparkline": sparkline(jpyP, 30)},
"flowStructure": map[string]any{"status": flow, "btcReturn5": ptr2(btcRet5), "qqqReturn5": ptr2(qqqRet5)},
"macroRegime": map[string]any{"status": regime, "qqqRoc20": ptr2(qqqRoc20), "xlpRoc20": ptr2(xlpRoc20)},
"technicalTrend": map[string]any{"status": trend, "btcPrice": ptrf(btcCur),
"sma50": ptr0(btcSma50), "sma200": ptr0(btcSma200), "vwap30d": ptrf(btcVwap),
"mayerMultiple": ptrf(mayer), "sparkline": tailN(btcP, 30)},
"mayerMultiple": ptrf(mayer), "sparkline": sparkline(btcP, 30)},
"hashRate": map[string]any{"status": hashStatus, "change30d": ptrf(hashChange)},
"miningCost": map[string]any{"status": mining},
"fearGreed": map[string]any{"status": fgLabel, "value": ptri(fgValue), "history": fgHistory},
},
"meta": map[string]any{"qqqSparkline": tailN(qqqP, 30)},
"meta": map[string]any{"qqqSparkline": sparkline(qqqP, 30)},
}, nil
}
@@ -290,11 +290,22 @@ func sma(prices []float64, period int) *float64 {
return &v
}
func tailN(a []float64, n int) []float64 {
if len(a) <= n {
// sparkline returns the last n closes rounded to 7 significant digits — the wire
// form emitted to clients. Scalars (price/change) are computed from the raw
// closes upstream of this call, so they keep full precision; only the emitted
// curve is rounded. A nil input passes through as nil.
func sparkline(a []float64, n int) []float64 {
if len(a) == 0 {
return a
}
return a[len(a)-n:]
if len(a) > n {
a = a[len(a)-n:]
}
out := make([]float64, len(a))
for i, v := range a {
out[i] = roundSig(v)
}
return out
}
func ptr2(p *float64) any {
+16
View File
@@ -6,6 +6,7 @@ import (
"math"
"net/http"
"sort"
"strconv"
"strings"
"sync"
"time"
@@ -412,6 +413,21 @@ func compact(in []*float64) []float64 {
return out
}
// roundSig rounds v to 7 significant digits — the wire precision for emitted
// sparkline closes. Yahoo returns float32-widened closes (17.209999084472656
// for 17.21); serializing that noise verbatim ~doubles the sparkline payload
// for a curve the renderer draws identically. Significant digits, not fixed
// decimals, so a sub-1.0 FX rate keeps its precision. Non-finite values pass
// through unchanged. Applied only where close ARRAYS ship (see sparkline);
// price/change scalars are derived from the raw closes and stay exact.
func roundSig(v float64) float64 {
if math.IsNaN(v) || math.IsInf(v, 0) {
return v
}
f, _ := strconv.ParseFloat(strconv.FormatFloat(v, 'g', 7, 64), 64)
return f
}
// ── Stock index (country → weekly % change) ──────────────────────────────────
type indexInfo struct{ symbol, name string }
+245
View File
@@ -0,0 +1,245 @@
package world
import (
"context"
"encoding/json"
"io"
"net/http"
"regexp"
"strings"
"sync"
"time"
"github.com/hanzoai/world/internal/world/store"
)
// Go-native keyword monitors.
//
// Monitors used to live entirely in the browser: the keyword list in
// localStorage, the matching done client-side over whatever headlines that tab
// had loaded. That means a monitor only ever saw one client's slice of the
// corpus, and it stopped existing the moment you closed the tab or switched
// device.
//
// Here they are server state: the list is persisted per identity (the same
// per-identity store the dashboard settings use, namespaced 'monitors'), and
// matching runs against the LAKE — every item the backend has ingested, not just
// what a browser fetched. Anonymous callers get 401 and keep their local
// monitors; nothing about the signed-out experience changes.
//
// GET /v1/world/monitors → { monitors: [...] }
// PUT /v1/world/monitors → { ok: true } (body: { monitors: [...] })
// GET /v1/world/monitors/matches → { matches: [...] } (matched against the lake)
const (
monitorsDoc = "monitors" // per-identity store namespace
monitorMatchWindow = 48 * time.Hour
monitorMatchLimit = 60
maxMonitors = 50
maxKeywordsEach = 25
)
// Monitor is one keyword watch. Mirrors the frontend Monitor type.
type Monitor struct {
ID string `json:"id"`
Keywords []string `json:"keywords"`
Color string `json:"color,omitempty"`
}
// MonitorMatch is a lake item that tripped a monitor.
type MonitorMatch struct {
MonitorID string `json:"monitorId"`
Color string `json:"color,omitempty"`
Keyword string `json:"keyword"`
Title string `json:"title"`
Link string `json:"link"`
Source string `json:"source"`
TS time.Time `json:"ts"`
}
// identityFor resolves the caller, or writes 401 and reports false.
func (s *Server) identityFor(w http.ResponseWriter, r *http.Request) (store.Identity, bool) {
bearer := userBearer(r)
if bearer == "" {
writeError(w, http.StatusUnauthorized, "Sign in required")
return store.Identity{}, false
}
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
id, err := s.introspectIdentity(ctx, bearer)
if err != nil || id.Sub == "" {
writeError(w, http.StatusUnauthorized, "Sign in required")
return store.Identity{}, false
}
return store.Identity{Org: id.Org, UserSub: id.Sub, Project: monitorsDoc}, true
}
func (s *Server) handleMonitors(w http.ResponseWriter, r *http.Request) {
setCORS(w, "GET, PUT, OPTIONS")
if r.Method == http.MethodOptions {
w.WriteHeader(http.StatusNoContent)
return
}
if r.Method != http.MethodGet && r.Method != http.MethodPut {
writeError(w, http.StatusMethodNotAllowed, "GET or PUT")
return
}
ident, ok := s.identityFor(w, r)
if !ok {
return
}
if r.Method == http.MethodGet {
writeJSON(w, http.StatusOK, "private, no-store", map[string]any{"monitors": s.loadMonitors(ident)})
return
}
raw, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 64<<10))
if err != nil {
writeError(w, http.StatusBadRequest, "Body too large")
return
}
var body struct {
Monitors []Monitor `json:"monitors"`
}
if err := json.Unmarshal(raw, &body); err != nil {
writeError(w, http.StatusBadRequest, "Body must be {\"monitors\":[...]}")
return
}
clean := sanitizeMonitors(body.Monitors)
blob, err := json.Marshal(map[string]any{"monitors": clean})
if err != nil {
writeError(w, http.StatusBadRequest, "Unserializable monitors")
return
}
writeJSON(w, http.StatusOK, "private, no-store", map[string]any{
"ok": s.store.Settings.Put(ident, blob),
"monitors": clean,
})
}
// handleMonitorMatches runs the caller's monitors against the lake — the whole
// ingested corpus, not one browser's slice of it.
func (s *Server) handleMonitorMatches(w http.ResponseWriter, r *http.Request) {
setCORS(w, "GET, OPTIONS")
if r.Method == http.MethodOptions {
w.WriteHeader(http.StatusNoContent)
return
}
if r.Method != http.MethodGet {
writeError(w, http.StatusMethodNotAllowed, "GET")
return
}
ident, ok := s.identityFor(w, r)
if !ok {
return
}
monitors := s.loadMonitors(ident)
writeJSON(w, http.StatusOK, "private, no-store", map[string]any{
"matches": s.matchMonitors(monitors),
})
}
func (s *Server) loadMonitors(ident store.Identity) []Monitor {
blob, ok := s.store.Settings.Get(ident)
if !ok {
return []Monitor{}
}
var doc struct {
Monitors []Monitor `json:"monitors"`
}
if err := json.Unmarshal(blob, &doc); err != nil || doc.Monitors == nil {
return []Monitor{}
}
return doc.Monitors
}
// matchMonitors searches the lake once per keyword and folds the hits together,
// deduped by (monitor, item) so one item can trip two different monitors but
// never the same one twice. Word-boundary re-check mirrors the frontend rule:
// FTS tokenises, but "ai" must not match "train".
func (s *Server) matchMonitors(monitors []Monitor) []MonitorMatch {
out := []MonitorMatch{}
if s.store == nil || len(monitors) == 0 {
return out
}
since := time.Now().Add(-monitorMatchWindow)
seen := map[string]bool{}
for _, m := range monitors {
for _, kw := range m.Keywords {
kw = strings.ToLower(strings.TrimSpace(kw))
if kw == "" {
continue
}
re := wordBoundary(kw)
items := s.store.Lake.Search(store.SearchQuery{
Q: kw, Kind: "news", Since: since, Limit: monitorMatchLimit,
})
for _, it := range items {
if !re.MatchString(strings.ToLower(it.Title + " " + it.Text)) {
continue // FTS stemming can over-match; the boundary rule decides
}
key := m.ID + "\x00" + it.ID
if seen[key] {
continue
}
seen[key] = true
out = append(out, MonitorMatch{
MonitorID: m.ID, Color: m.Color, Keyword: kw,
Title: it.Title, Link: itemLink(it), Source: it.Source, TS: it.TS,
})
}
}
}
return out
}
// itemLink pulls the canonical link out of a lake item's payload, falling back
// to its id (feed items are keyed by link upstream).
func itemLink(it store.Item) string {
var p struct {
Link string `json:"link"`
}
if it.Payload != "" && json.Unmarshal([]byte(it.Payload), &p) == nil && p.Link != "" {
return p.Link
}
if strings.HasPrefix(it.ID, "http") {
return it.ID
}
return ""
}
var wbCache sync.Map // keyword → *regexp.Regexp
func wordBoundary(kw string) *regexp.Regexp {
if v, ok := wbCache.Load(kw); ok {
return v.(*regexp.Regexp)
}
re := regexp.MustCompile(`\b` + regexp.QuoteMeta(kw) + `\b`)
wbCache.Store(kw, re)
return re
}
// sanitizeMonitors bounds what a client can store: no unbounded lists, no empty
// keywords, everything lowercased so matching is case-insensitive by construction.
func sanitizeMonitors(in []Monitor) []Monitor {
out := make([]Monitor, 0, len(in))
for _, m := range in {
if len(out) >= maxMonitors {
break
}
kws := make([]string, 0, len(m.Keywords))
for _, k := range m.Keywords {
k = strings.ToLower(strings.TrimSpace(k))
if k == "" || len(kws) >= maxKeywordsEach {
continue
}
kws = append(kws, k)
}
if len(kws) == 0 || strings.TrimSpace(m.ID) == "" {
continue
}
out = append(out, Monitor{ID: m.ID, Keywords: kws, Color: m.Color})
}
return out
}
+119
View File
@@ -0,0 +1,119 @@
package world
import (
"testing"
"time"
"github.com/hanzoai/world/internal/world/store"
)
// Monitors are server state now, so the two things that MUST hold are:
// 1. a monitor list round-trips through the per-identity store, and
// 2. matching runs against the lake — the whole ingested corpus — while still
// honouring the word-boundary rule the browser used ("ai" ≠ "train").
//
// The store is opened against a temp dir (WORLD_DATA_DIR), so this is the real
// SQLite lake + settings table, not a mock.
func testServer(t *testing.T) *Server {
t.Helper()
t.Setenv("WORLD_DATA_DIR", t.TempDir())
s := NewServer()
t.Cleanup(s.Close)
if s.store == nil || !s.store.Enabled() {
t.Skip("store unavailable in this environment")
}
return s
}
func seedNews(t *testing.T, s *Server, items ...store.Item) {
t.Helper()
for _, it := range items {
if it.Kind == "" {
it.Kind = "news"
}
if it.TS.IsZero() {
it.TS = time.Now()
}
s.store.Lake.Add(it)
}
// Add() is write-behind; flush so the search sees the rows.
s.store.Lake.Flush()
}
func TestMonitorsRoundTripPerIdentity(t *testing.T) {
s := testServer(t)
ident := store.Identity{Org: "acme", UserSub: "user-1", Project: monitorsDoc}
other := store.Identity{Org: "acme", UserSub: "user-2", Project: monitorsDoc}
blob := []byte(`{"monitors":[{"id":"m1","keywords":["nvidia","gpu"],"color":"#f00"}]}`)
if !s.store.Settings.Put(ident, blob) {
t.Fatal("put monitors failed")
}
got := s.loadMonitors(ident)
if len(got) != 1 || got[0].ID != "m1" || len(got[0].Keywords) != 2 {
t.Fatalf("round-trip mismatch: %+v", got)
}
// Isolation: another identity must not see them.
if n := len(s.loadMonitors(other)); n != 0 {
t.Fatalf("identity leak: user-2 sees %d monitors", n)
}
}
func TestMonitorMatchesRunAgainstTheLake(t *testing.T) {
s := testServer(t)
seedNews(t, s,
store.Item{ID: "https://x.test/a", Source: "BBC", Title: "Nvidia unveils a new GPU for data centres"},
store.Item{ID: "https://x.test/b", Source: "FT", Title: "Markets rally on strong earnings"},
store.Item{ID: "https://x.test/c", Source: "AP", Title: "Train derails outside the city"},
)
monitors := []Monitor{{ID: "m1", Keywords: []string{"nvidia"}, Color: "#f00"}}
matches := s.matchMonitors(monitors)
if len(matches) != 1 {
t.Fatalf("want 1 match, got %d (%+v)", len(matches), matches)
}
m := matches[0]
if m.MonitorID != "m1" || m.Keyword != "nvidia" || m.Source != "BBC" {
t.Fatalf("unexpected match: %+v", m)
}
if m.Link != "https://x.test/a" {
t.Errorf("link = %q, want the item's canonical link", m.Link)
}
}
// The rule FTS alone would get wrong: a short keyword must not match inside a
// longer word. "ai" must never trip on "train".
func TestMonitorMatchesHonourWordBoundaries(t *testing.T) {
s := testServer(t)
seedNews(t, s,
store.Item{ID: "https://x.test/train", Source: "AP", Title: "Train derails outside the city"},
store.Item{ID: "https://x.test/ai", Source: "Wired", Title: "AI models get cheaper to run"},
)
matches := s.matchMonitors([]Monitor{{ID: "m1", Keywords: []string{"ai"}}})
for _, m := range matches {
if m.Link == "https://x.test/train" {
t.Fatalf(`"ai" matched inside "Train" — word boundary not enforced`)
}
}
if len(matches) != 1 || matches[0].Link != "https://x.test/ai" {
t.Fatalf("want only the AI story, got %+v", matches)
}
}
func TestSanitizeMonitorsBoundsInput(t *testing.T) {
in := []Monitor{
{ID: "ok", Keywords: []string{" Nvidia ", "", "GPU"}},
{ID: "", Keywords: []string{"dropped: no id"}},
{ID: "empty", Keywords: []string{" "}},
}
out := sanitizeMonitors(in)
if len(out) != 1 {
t.Fatalf("want 1 surviving monitor, got %d (%+v)", len(out), out)
}
if out[0].Keywords[0] != "nvidia" || out[0].Keywords[1] != "gpu" {
t.Fatalf("keywords not normalised: %+v", out[0].Keywords)
}
}
+26
View File
@@ -0,0 +1,26 @@
package world
import (
"net/http"
"time"
)
// ── HKO tropical-cyclone warnings ────────────────────────────────────────────
// handleHKOWarnings proxies the Hong Kong Observatory weather-warning summary
// (verbatim JSON). Browsers can't reach data.weather.gov.hk directly (no CORS),
// so the Western-Pacific cyclone attribution stream folds it in through here as
// an HKO agency observation. Degrades to {} so the client parser simply yields
// no warnings. Follows handleGDELTGeo's passthrough shape.
func (s *Server) handleHKOWarnings(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
const upstream = "https://data.weather.gov.hk/weatherAPI/opendata/weather.php?dataType=warnsum&lang=en"
s.passthrough(w, "hko-warnings", upstream, "application/json",
"public, max-age=300, s-maxage=300, stale-while-revalidate=60",
nil, 5*time.Minute, 15*time.Minute,
func(w http.ResponseWriter, err error) {
writeJSON(w, http.StatusOK, "", map[string]any{})
})
}
+56 -38
View File
@@ -14,6 +14,28 @@ import (
// ── GDELT ────────────────────────────────────────────────────────────────────
// Shared GDELT cache policy + key/URL construction — defined ONCE so the doc and
// geo handlers and the boot warmer (cache_warmer.go) all read/write the exact
// same cache keys and hit the same upstream. 5m fresh / 15m stale.
const (
gdeltCC = "public, max-age=300, s-maxage=300, stale-while-revalidate=60"
gdeltTTL = 5 * time.Minute
gdeltStale = 15 * time.Minute
)
func gdeltDocKey(query string, maxrecords int, timespan string) string {
return "gdelt-doc:" + query + ":" + itoa(maxrecords) + ":" + timespan
}
func gdeltGeoKey(query, format string, maxrecords int, timespan string) string {
return "gdelt-geo:" + query + ":" + format + ":" + itoa(maxrecords) + ":" + timespan
}
func gdeltGeoURL(query, format string, maxrecords int, timespan string) string {
return "https://api.gdeltproject.org/api/v2/geo/geo?query=" + urlQueryEscape(query) +
"&format=" + format + "&maxrecords=" + itoa(maxrecords) + "&timespan=" + timespan
}
// handleGDELTDoc queries the GDELT DOC 2.0 article list and maps it to the
// compact {articles,query} shape. Ported from api/gdelt-doc.js.
func (s *Server) handleGDELTDoc(w http.ResponseWriter, r *http.Request) {
@@ -31,28 +53,30 @@ func (s *Server) handleGDELTDoc(w http.ResponseWriter, r *http.Request) {
if timespan == "" {
timespan = "72h"
}
key := "gdelt-doc:" + query + ":" + itoa(maxrecords) + ":" + timespan
s.cachedJSON(w, key, "public, max-age=300, s-maxage=300, stale-while-revalidate=60",
5*time.Minute, 15*time.Minute,
func(ctx context.Context) (any, error) {
arts, err := s.fetchGDELTArticles(ctx, query, timespan, maxrecords)
if err != nil {
return nil, err
}
articles := make([]map[string]any, 0, len(arts))
for _, a := range arts {
articles = append(articles, map[string]any{
"title": a.Title, "url": a.URL, "source": a.Domain,
"date": a.SeenDate, "image": a.SocialImage, "language": a.Language, "tone": a.Tone,
})
}
return map[string]any{"articles": articles, "query": query}, nil
},
s.cachedJSON(w, gdeltDocKey(query, maxrecords, timespan), gdeltCC, gdeltTTL, gdeltStale,
func(ctx context.Context) (any, error) { return s.produceGDELTDoc(ctx, query, timespan, maxrecords) },
func(w http.ResponseWriter, err error) {
writeJSON(w, http.StatusOK, "", map[string]any{"error": err.Error(), "articles": []any{}})
})
}
// produceGDELTDoc fetches and shapes the gdelt-doc payload — the single produce
// path shared by the handler and the boot warmer, so the transform lives once.
func (s *Server) produceGDELTDoc(ctx context.Context, query, timespan string, maxrecords int) (any, error) {
arts, err := s.fetchGDELTArticles(ctx, query, timespan, maxrecords)
if err != nil {
return nil, err
}
articles := make([]map[string]any, 0, len(arts))
for _, a := range arts {
articles = append(articles, map[string]any{
"title": a.Title, "url": a.URL, "source": a.Domain,
"date": a.SeenDate, "image": a.SocialImage, "language": a.Language, "tone": a.Tone,
})
}
return map[string]any{"articles": articles, "query": query}, nil
}
// gdeltArticle is one GDELT DOC 2.0 artlist row. The shared decode target for
// both the /gdelt-doc endpoint and the world-model news source.
type gdeltArticle struct {
@@ -103,11 +127,9 @@ func (s *Server) handleGDELTGeo(w http.ResponseWriter, r *http.Request) {
if format == "csv" {
ct = "text/csv"
}
upstream := "https://api.gdeltproject.org/api/v2/geo/geo?query=" + urlQueryEscape(query) +
"&format=" + format + "&maxrecords=" + itoa(maxrecords) + "&timespan=" + timespan
key := "gdelt-geo:" + query + ":" + format + ":" + itoa(maxrecords) + ":" + timespan
s.passthrough(w, key, upstream, ct, "public, max-age=300, s-maxage=300, stale-while-revalidate=60",
nil, 5*time.Minute, 15*time.Minute,
s.passthrough(w, gdeltGeoKey(query, format, maxrecords, timespan),
gdeltGeoURL(query, format, maxrecords, timespan), ct, gdeltCC,
nil, gdeltTTL, gdeltStale,
func(w http.ResponseWriter, err error) {
writeJSON(w, http.StatusOK, "", map[string]any{"error": "upstream unavailable", "data": []any{}})
})
@@ -160,30 +182,26 @@ func (s *Server) handleRSSProxy(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusForbidden, "Domain not allowed")
return
}
key := "rss:" + feedURL
if v, ok := s.cache.Get(key); ok {
writeBytes(w, http.StatusOK, "application/xml", "public, max-age=300, s-maxage=300, stale-while-revalidate=60", v.([]byte))
const cc = "public, max-age=300, s-maxage=300, stale-while-revalidate=60"
// Warm cache first (per-pod L1 → shared hanzo-kv L2): instant, and never blocks
// on the upstream while ANY cached copy exists (stale-while-revalidate; the
// background warmer does the revalidation).
if body, _, ok := s.feeds.Get(r.Context(), feedURL); ok {
writeBytes(w, http.StatusOK, "application/xml", cc, body)
return
}
// True cold miss: bounded live fetch, write-through to the warm cache + lake.
ctx, cancel := context.WithTimeout(r.Context(), 20*time.Second)
defer cancel()
body, status, err := s.getAllowlisted(ctx, feedURL, allowedRSSDomains, map[string]string{
"User-Agent": browserUA,
"Accept": "application/rss+xml, application/xml, text/xml, */*",
})
// A blank 200 is a failure, not content: never cache it (it would poison the
// shared "rss:" key feedXML reads too). Fall back to last-good stale.
if err != nil || status < 200 || status >= 300 || isBlankBody(body) {
if v, ok := s.cache.GetStale(key); ok {
writeBytes(w, http.StatusOK, "application/xml", "public, max-age=300, s-maxage=300, stale-while-revalidate=60", v.([]byte))
return
}
body, ok := s.fetchFeedBody(ctx, feedURL)
if !ok {
w.Header().Set("Cache-Control", "no-store")
writeJSON(w, http.StatusOK, "", map[string]any{"error": "upstream unavailable", "items": []any{}})
return
}
s.cache.Set(key, body, 5*time.Minute, 15*time.Minute)
writeBytes(w, http.StatusOK, "application/xml", "public, max-age=300, s-maxage=300, stale-while-revalidate=60", body)
s.feeds.Put(feedURL, body)
s.ingestFeedItems(feedURL, body)
writeBytes(w, http.StatusOK, "application/xml", cc, body)
}
// ── Hacker News ──────────────────────────────────────────────────────────────
+94
View File
@@ -0,0 +1,94 @@
package world
import (
"encoding/json"
"net/http"
"strconv"
"strings"
"time"
"github.com/hanzoai/world/internal/world/store"
)
// The search + analytics endpoints over the ingested-data LAKE — the CTO's "one
// place to query everything". Every ingested item (news, model observations, …)
// is a row; search ranks them (FTS bm25 when q is present, recency otherwise),
// analytics summarizes them. Both degrade to empty results, never 5xx.
// handleSearch serves GET /v1/world/search?q=&kind=&since=&country=&ticker=&limit= .
func (s *Server) handleSearch(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
q := r.URL.Query()
results := s.store.Lake.Search(store.SearchQuery{
Q: q.Get("q"),
Kind: strings.TrimSpace(q.Get("kind")),
Country: strings.TrimSpace(q.Get("country")),
Ticker: strings.TrimSpace(q.Get("ticker")),
Since: parseSince(q.Get("since")),
Limit: atoiDefault(q.Get("limit"), 30),
})
out := make([]map[string]any, 0, len(results))
for _, it := range results {
m := map[string]any{
"id": it.ID, "kind": it.Kind, "source": it.Source,
"ts": it.TS.Format(time.RFC3339), "title": it.Title,
}
if it.Text != "" {
m["text"] = it.Text
}
if len(it.Tickers) > 0 {
m["tickers"] = it.Tickers
}
if it.Country != "" {
m["country"] = it.Country
}
if it.HasGeo {
m["lat"], m["lon"] = it.Lat, it.Lon
}
if it.Payload != "" {
m["payload"] = json.RawMessage(it.Payload)
}
out = append(out, m)
}
writeJSON(w, http.StatusOK, "public, max-age=15, s-maxage=15, stale-while-revalidate=60", map[string]any{
"query": q.Get("q"), "count": len(out), "results": out, "updatedAt": nowRFC(),
})
}
// handleAnalytics serves GET /v1/world/analytics?hours= — a cross-cutting
// summary of the lake: totals and breakdowns by kind, source, and top tickers.
func (s *Server) handleAnalytics(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") || methodNotGet(w, r) {
return
}
hours := clampInt(r.URL.Query().Get("hours"), 24, 1, 720)
writeJSON(w, http.StatusOK, "public, max-age=30, s-maxage=30, stale-while-revalidate=120",
s.store.Lake.Analytics(hours))
}
// parseSince accepts an absolute RFC3339 timestamp, a "<n>d" day window, a Go
// duration ("24h","90m"), or a bare integer of hours. Unparseable/empty → no
// lower bound.
func parseSince(raw string) time.Time {
raw = strings.TrimSpace(raw)
if raw == "" {
return time.Time{}
}
if t, err := time.Parse(time.RFC3339, raw); err == nil {
return t
}
if strings.HasSuffix(raw, "d") {
if n, err := strconv.Atoi(strings.TrimSuffix(raw, "d")); err == nil && n > 0 {
return time.Now().Add(-time.Duration(n) * 24 * time.Hour)
}
}
if d, err := time.ParseDuration(raw); err == nil && d > 0 {
return time.Now().Add(-d)
}
if n, err := strconv.Atoi(raw); err == nil && n > 0 {
return time.Now().Add(-time.Duration(n) * time.Hour)
}
return time.Time{}
}
+95
View File
@@ -0,0 +1,95 @@
package world
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
// serve drives one handler with an in-memory request/response and returns the
// decoded JSON body plus status.
func serve(t *testing.T, h http.HandlerFunc, method, target, bearer, body string) (map[string]any, int) {
t.Helper()
var r *http.Request
if body != "" {
r = httptest.NewRequest(method, target, strings.NewReader(body))
} else {
r = httptest.NewRequest(method, target, nil)
}
if bearer != "" {
r.Header.Set("Authorization", "Bearer "+bearer)
}
rec := httptest.NewRecorder()
h(rec, r)
res := rec.Result()
defer res.Body.Close()
raw, _ := io.ReadAll(res.Body)
var m map[string]any
if len(raw) > 0 {
_ = json.Unmarshal(raw, &m)
}
return m, res.StatusCode
}
func TestSearchEndpointEmptyNever5xx(t *testing.T) {
s := newTestServer(t)
m, code := serve(t, s.handleSearch, http.MethodGet, "/v1/world/search", "", "")
if code != http.StatusOK {
t.Fatalf("status = %d, want 200", code)
}
if m["count"].(float64) != 0 {
t.Fatalf("empty lake count = %v, want 0", m["count"])
}
if _, ok := m["results"].([]any); !ok {
t.Fatalf("results not an array: %v", m["results"])
}
}
func TestSearchAndAnalyticsReturnIngested(t *testing.T) {
s := newTestServer(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go s.store.Lake.Run(ctx) // write-behind consumer
// Ingest via the same path the feed handlers use.
s.ingestFeedItems("https://cointelegraph.com/rss", []byte(stubRSS))
// Poll until the write-behind flush lands (≤1s tick).
var m map[string]any
deadline := time.Now().Add(4 * time.Second)
for time.Now().Before(deadline) {
var code int
m, code = serve(t, s.handleSearch, http.MethodGet, "/v1/world/search?q=bitcoin", "", "")
if code != http.StatusOK {
t.Fatalf("search status = %d", code)
}
if m["count"].(float64) >= 1 {
break
}
time.Sleep(40 * time.Millisecond)
}
if m["count"].(float64) < 1 {
t.Fatalf("ingested news not searchable: %v", m)
}
first := m["results"].([]any)[0].(map[string]any)
if !strings.Contains(strings.ToLower(first["title"].(string)), "bitcoin") {
t.Fatalf("top result title = %v, want a Bitcoin item", first["title"])
}
if first["kind"] != "news" {
t.Fatalf("kind = %v, want news", first["kind"])
}
// Analytics reflects the same ingest.
a, code := serve(t, s.handleAnalytics, http.MethodGet, "/v1/world/analytics", "", "")
if code != http.StatusOK {
t.Fatalf("analytics status = %d", code)
}
if a["total"].(float64) < 1 {
t.Fatalf("analytics total = %v, want ≥1", a["total"])
}
}
+112
View File
@@ -0,0 +1,112 @@
package world
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"io"
"net/http"
"time"
"github.com/hanzoai/world/internal/world/store"
)
// Per-identity settings: server-side dashboard sync for signed-in users.
//
// The frontend layout engine persists panel geometry / layout mode / preferences
// to localStorage today. These endpoints let a follow-up sync a SIGNED-IN user's
// dashboard across devices: the blob is keyed by (org, user_sub, project) from
// the IAM bearer, so it is isolated per identity. Anonymous callers get 401 and
// keep using localStorage — no server state for them.
//
// FRONTEND HOOK (another agent owns src/): after the layout store mutates, if the
// user is signed in, PUT the same JSON it writes to localStorage to
// /v1/world/settings (Authorization: Bearer <token>); on load, if signed in, GET
// /v1/world/settings and prefer a non-empty server blob over localStorage. One
// debounced PUT on change, one GET on boot — the endpoints below are the whole API.
// wIdentity is the caller's IAM identity resolved from userinfo.
type wIdentity struct {
Org string `json:"owner"`
Sub string `json:"sub"`
}
// introspectIdentity resolves the caller's org (owner claim) + subject from IAM
// userinfo, memoized by token hash for a short TTL. It is world's ONE identity
// path — the admin gate (requireAdmin) and per-identity settings both resolve
// through here, so a token's userinfo is fetched and cached once. Authoritative,
// IAM-signed identity — never a client-supplied header.
func (s *Server) introspectIdentity(ctx context.Context, bearer string) (wIdentity, error) {
sum := sha256.Sum256([]byte(bearer))
key := "identity:" + hex.EncodeToString(sum[:12])
if v, ok := s.cache.Get(key); ok {
return v.(wIdentity), nil
}
var id wIdentity
if err := s.getJSON(ctx, iamIssuer()+"/v1/iam/oauth/userinfo",
map[string]string{"Authorization": bearer}, &id); err != nil {
return wIdentity{}, err
}
s.cache.Set(key, id, 60*time.Second, 60*time.Second)
return id, nil
}
// handleSettings serves GET (read this identity's blob) and PUT (upsert it),
// both bearer-gated. Never 5xx: a degraded store returns {} on GET and ok:false
// on PUT so the client falls back to localStorage.
func (s *Server) handleSettings(w http.ResponseWriter, r *http.Request) {
setCORS(w, "GET, PUT, OPTIONS")
if r.Method == http.MethodOptions {
w.WriteHeader(http.StatusNoContent)
return
}
if r.Method != http.MethodGet && r.Method != http.MethodPut {
writeError(w, http.StatusMethodNotAllowed, "GET or PUT")
return
}
bearer := userBearer(r)
if bearer == "" {
writeError(w, http.StatusUnauthorized, "Sign in required")
return
}
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
id, err := s.introspectIdentity(ctx, bearer)
if err != nil || id.Sub == "" {
writeError(w, http.StatusUnauthorized, "Sign in required")
return
}
ident := store.Identity{Org: id.Org, UserSub: id.Sub, Project: r.URL.Query().Get("project")}
if r.Method == http.MethodGet {
blob, ok := s.store.Settings.Get(ident)
if !ok {
blob = json.RawMessage(`{}`)
}
writeJSON(w, http.StatusOK, "private, no-store", map[string]any{"settings": blob})
return
}
// PUT: validate the body is a JSON object at the boundary, then upsert.
raw, err := io.ReadAll(http.MaxBytesReader(w, r.Body, 256<<10))
if err != nil {
writeError(w, http.StatusBadRequest, "Body too large")
return
}
if !json.Valid(raw) || !isJSONObject(raw) {
writeError(w, http.StatusBadRequest, "Body must be a JSON object")
return
}
writeJSON(w, http.StatusOK, "private, no-store", map[string]any{
"ok": s.store.Settings.Put(ident, json.RawMessage(raw)),
})
}
// isJSONObject reports whether raw is a JSON object ({...}), not an array or
// scalar — settings are always an object blob.
func isJSONObject(raw []byte) bool {
raw = bytes.TrimSpace(raw)
return len(raw) > 0 && raw[0] == '{'
}
+115
View File
@@ -0,0 +1,115 @@
package world
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
)
// stubIAM stands up a userinfo endpoint that maps a bearer token to an IAM
// identity, and points the server's issuer at it. This exercises the real
// introspectIdentity path hermetically (no live IAM).
func stubIAM(t *testing.T, tokenToIdentity map[string]wIdentity) {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/iam/oauth/userinfo" {
http.NotFound(w, r)
return
}
tok := r.Header.Get("Authorization")
id, ok := tokenToIdentity[tok]
if !ok {
w.WriteHeader(http.StatusUnauthorized)
return
}
_ = json.NewEncoder(w).Encode(id)
}))
t.Cleanup(srv.Close)
t.Setenv("HANZO_IAM_ISSUER", srv.URL)
}
func TestSettingsAnonymousUnauthorized(t *testing.T) {
s := newTestServer(t)
_, code := serve(t, s.handleSettings, http.MethodGet, "/v1/world/settings", "", "")
if code != http.StatusUnauthorized {
t.Fatalf("anonymous GET status = %d, want 401", code)
}
_, code = serve(t, s.handleSettings, http.MethodPut, "/v1/world/settings", "", `{"a":1}`)
if code != http.StatusUnauthorized {
t.Fatalf("anonymous PUT status = %d, want 401", code)
}
}
func TestSettingsUpsertGetPerIdentity(t *testing.T) {
s := newTestServer(t)
stubIAM(t, map[string]wIdentity{
"Bearer alice": {Org: "acme", Sub: "alice"},
"Bearer bob": {Org: "acme", Sub: "bob"},
})
// Alice stores her dashboard.
m, code := serve(t, s.handleSettings, http.MethodPut, "/v1/world/settings", "alice", `{"layout":"grid","cell":40}`)
if code != http.StatusOK || m["ok"] != true {
t.Fatalf("alice PUT = %d %v", code, m)
}
// Bob stores a different one.
if m, code := serve(t, s.handleSettings, http.MethodPut, "/v1/world/settings", "bob", `{"layout":"list"}`); code != http.StatusOK || m["ok"] != true {
t.Fatalf("bob PUT = %d %v", code, m)
}
// Each reads back exactly their own — server-side, cross-device, isolated.
m, code = serve(t, s.handleSettings, http.MethodGet, "/v1/world/settings", "alice", "")
if code != http.StatusOK {
t.Fatalf("alice GET status = %d", code)
}
if got, want := jsonStr(t, m["settings"]), normJSON(t, `{"layout":"grid","cell":40}`); got != want {
t.Fatalf("alice settings = %s, want %s", got, want)
}
m, _ = serve(t, s.handleSettings, http.MethodGet, "/v1/world/settings", "bob", "")
if got, want := jsonStr(t, m["settings"]), normJSON(t, `{"layout":"list"}`); got != want {
t.Fatalf("bob settings = %s, want %s (identity isolation broken)", got, want)
}
}
func TestSettingsRejectsNonObject(t *testing.T) {
s := newTestServer(t)
stubIAM(t, map[string]wIdentity{"Bearer alice": {Org: "acme", Sub: "alice"}})
_, code := serve(t, s.handleSettings, http.MethodPut, "/v1/world/settings", "alice", `[1,2,3]`)
if code != http.StatusBadRequest {
t.Fatalf("array body status = %d, want 400", code)
}
}
func TestSettingsMissingIsEmptyObject(t *testing.T) {
s := newTestServer(t)
stubIAM(t, map[string]wIdentity{"Bearer newuser": {Org: "acme", Sub: "newuser"}})
m, code := serve(t, s.handleSettings, http.MethodGet, "/v1/world/settings", "newuser", "")
if code != http.StatusOK {
t.Fatalf("status = %d", code)
}
if got := jsonStr(t, m["settings"]); got != `{}` {
t.Fatalf("absent settings = %s, want {}", got)
}
}
func jsonStr(t *testing.T, v any) string {
t.Helper()
b, err := json.Marshal(v)
if err != nil {
t.Fatalf("marshal: %v", err)
}
return string(b)
}
// normJSON round-trips a JSON literal through decode+encode so map key ordering
// matches jsonStr's output (Go marshals map keys sorted) — order-independent
// comparison.
func normJSON(t *testing.T, s string) string {
t.Helper()
var v any
if err := json.Unmarshal([]byte(s), &v); err != nil {
t.Fatalf("unmarshal %q: %v", s, err)
}
return jsonStr(t, v)
}
+84
View File
@@ -0,0 +1,84 @@
package world
import (
"context"
"encoding/json"
"io"
"net/http"
"time"
)
// Model-improvement consent proxy — the ONE bridge between the account-settings
// opt-in toggle and where the flag actually lives: ai's
// OrgSettings.TrainingContribution, the single source of truth the automated judge
// and the router trainer read. World keeps NO copy of the flag; it forwards the
// signed-in caller's own IAM bearer (+ active-org selector) to api.hanzo.ai, which
// is the sole authority: it validates the bearer, self-scopes to the principal's
// OWN org (RequirePrincipal, spoofed X-Org-Id ignored for a non-super-admin) and
// rejects anonymous guests. Signed out → 401, mirroring the toggle's hidden state.
//
// GET /v1/world/training-contribution → { enabled: bool }
// POST /v1/world/training-contribution → { enabled: bool } (body: { enabled: bool })
func (s *Server) handleTrainingContribution(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, POST, OPTIONS") {
return
}
if r.Method != http.MethodGet && r.Method != http.MethodPost {
writeError(w, http.StatusMethodNotAllowed, "GET or POST")
return
}
bearer := userBearer(r)
if bearer == "" {
writeError(w, http.StatusUnauthorized, "Sign in required")
return
}
// Forward the caller's OWN credential + active-org selector. ai is the authority
// (bearer validation, own-org self-scoping, guest rejection); world adds nothing.
hdr := map[string]string{"Authorization": bearer}
if org := r.Header.Get("X-Org-Id"); org != "" {
hdr["X-Org-Id"] = org
}
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
defer cancel()
base := apiHost()
var upstream []byte
var status int
var err error
if r.Method == http.MethodGet {
upstream, status, err = s.get(ctx, base+"/v1/get-training-contribution", hdr)
} else {
raw, rerr := io.ReadAll(http.MaxBytesReader(w, r.Body, 4<<10))
if rerr != nil {
writeError(w, http.StatusBadRequest, "Body too large")
return
}
hdr["Content-Type"] = "application/json"
upstream, status, err = s.do(ctx, http.MethodPost, base+"/v1/update-training-contribution", hdr, raw)
}
if err != nil {
writeError(w, http.StatusBadGateway, "Consent service unavailable")
return
}
// Unwrap the ai (casibase) envelope { status, msg, data:{enabled} } to a clean
// { enabled } for the SPA. On a non-2xx or an error envelope, mirror the upstream
// status (>=400) so the client can tell 401 "sign in" from other failures.
var env struct {
Status string `json:"status"`
Data struct {
Enabled bool `json:"enabled"`
} `json:"data"`
}
if status < 200 || status >= 300 || json.Unmarshal(upstream, &env) != nil || env.Status == "error" {
code := status
if code < 400 {
code = http.StatusBadGateway
}
writeError(w, code, "Consent request failed")
return
}
writeJSON(w, http.StatusOK, "private, no-store", map[string]any{"enabled": env.Data.Enabled})
}
+124
View File
@@ -0,0 +1,124 @@
package world
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// TestTrainingContributionProxy verifies the model-improvement consent proxy:
// - signed out (no bearer) → 401 (the toggle's hidden state) with NO upstream call,
// - GET forwards the caller's bearer + X-Org-Id to ai /v1/get-training-contribution
// and UNWRAPS the casibase envelope {status,data:{enabled}} to a bare {enabled},
// - POST forwards the {enabled} body to ai /v1/update-training-contribution and
// unwraps the resolved state.
func TestTrainingContributionProxy(t *testing.T) {
var gotAuth, gotOrg, gotPath, gotBody string
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotAuth = r.Header.Get("Authorization")
gotOrg = r.Header.Get("X-Org-Id")
gotPath = r.URL.Path
b, _ := io.ReadAll(r.Body)
gotBody = string(b)
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
case "/v1/get-training-contribution":
_, _ = io.WriteString(w, `{"status":"ok","msg":"","data":{"enabled":true},"data2":null}`)
case "/v1/update-training-contribution":
// Echo the requested state, as the real controller does.
var body struct {
Enabled bool `json:"enabled"`
}
_ = json.Unmarshal([]byte(gotBody), &body)
enabled := "false"
if body.Enabled {
enabled = "true"
}
_, _ = io.WriteString(w, `{"status":"ok","msg":"","data":{"enabled":`+enabled+`},"data2":null}`)
default:
http.NotFound(w, r)
}
}))
t.Cleanup(upstream.Close)
t.Setenv("HANZO_API_BASE", upstream.URL)
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
do := func(method, path, bearer, org, body string) *http.Response {
var r io.Reader
if body != "" {
r = strings.NewReader(body)
}
req, _ := http.NewRequest(method, ts.URL+path, r)
if bearer != "" {
req.Header.Set("Authorization", bearer)
}
if org != "" {
req.Header.Set("X-Org-Id", org)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("%s %s: %v", method, path, err)
}
return resp
}
// 1) Signed out → 401, and the upstream is never touched.
gotPath = ""
resp := do("GET", "/v1/world/training-contribution", "", "", "")
_ = resp.Body.Close()
if resp.StatusCode != http.StatusUnauthorized {
t.Errorf("signed-out GET = %d, want 401", resp.StatusCode)
}
if gotPath != "" {
t.Errorf("upstream called on a signed-out request (path %q); must be gated before forwarding", gotPath)
}
// 2) GET forwards bearer + org and unwraps the envelope to {enabled:true}.
resp = do("GET", "/v1/world/training-contribution", "Bearer tok-123", "maxpower", "")
var g struct {
Enabled bool `json:"enabled"`
}
if err := json.NewDecoder(resp.Body).Decode(&g); err != nil {
t.Fatalf("decode GET: %v", err)
}
_ = resp.Body.Close()
if !g.Enabled {
t.Errorf("GET enabled = false, want true (unwrapped from the ai envelope)")
}
if gotAuth != "Bearer tok-123" {
t.Errorf("forwarded Authorization = %q, want the caller's own bearer", gotAuth)
}
if gotOrg != "maxpower" {
t.Errorf("forwarded X-Org-Id = %q, want maxpower", gotOrg)
}
if gotPath != "/v1/get-training-contribution" {
t.Errorf("GET hit %q, want /v1/get-training-contribution", gotPath)
}
// 3) POST forwards the body and unwraps the resolved opt-in state.
resp = do("POST", "/v1/world/training-contribution", "Bearer tok-123", "maxpower", `{"enabled":true}`)
var p struct {
Enabled bool `json:"enabled"`
}
if err := json.NewDecoder(resp.Body).Decode(&p); err != nil {
t.Fatalf("decode POST: %v", err)
}
_ = resp.Body.Close()
if !p.Enabled {
t.Errorf("POST enabled = false, want true")
}
if gotPath != "/v1/update-training-contribution" {
t.Errorf("POST hit %q, want /v1/update-training-contribution", gotPath)
}
if !strings.Contains(gotBody, `"enabled":true`) {
t.Errorf("forwarded body = %q, want the {enabled:true} opt-in", gotBody)
}
}
+158
View File
@@ -0,0 +1,158 @@
package world
import (
"context"
"encoding/json"
"html"
"net/http"
"regexp"
"strings"
"time"
)
// handleYouTubeSearch resolves a free-text query (e.g. "Milken Institute Jensen
// Huang 2025") to a ranked list of non-live YouTube videos, so the analyst — or
// the watch-queue search box — can turn "queue that talk" into a real video id.
// Uses the YouTube Data API (same key as youtube/live); degrades to an empty,
// honest result set when no key is configured rather than faking hits.
func (s *Server) handleYouTubeSearch(w http.ResponseWriter, r *http.Request) {
if preflight(w, r, "GET, OPTIONS") {
return
}
q := strings.TrimSpace(r.URL.Query().Get("q"))
if q == "" {
writeError(w, http.StatusBadRequest, "Missing q parameter")
return
}
cacheKey := "youtube-search:" + strings.ToLower(q)
if v, ok := s.cache.Get(cacheKey); ok {
writeJSON(w, http.StatusOK, "public, max-age=600, s-maxage=600", v)
return
}
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
defer cancel()
// No key required: scrape the results page, exactly as handleYouTubeLive
// scrapes /live (same UA + consent cookies to dodge the interstitial served
// to datacenter IPs). The Data API is only a faster path when a key exists.
key := env("YOUTUBE_API_KEY", "YT_API_KEY")
if key == "" {
results := s.youtubeSearchScrape(ctx, q)
out := map[string]any{"results": results}
if len(results) > 0 {
s.cache.Set(cacheKey, out, 10*time.Minute, 60*time.Minute)
}
writeJSON(w, http.StatusOK, "public, max-age=600, s-maxage=600", out)
return
}
var resp struct {
Items []struct {
ID struct {
VideoID string `json:"videoId"`
} `json:"id"`
Snippet struct {
Title string `json:"title"`
ChannelTitle string `json:"channelTitle"`
PublishedAt string `json:"publishedAt"`
Thumbnails struct {
Medium struct {
URL string `json:"url"`
} `json:"medium"`
} `json:"thumbnails"`
} `json:"snippet"`
} `json:"items"`
}
u := "https://www.googleapis.com/youtube/v3/search?part=snippet&type=video&maxResults=12&q=" +
urlQueryEscape(q) + "&key=" + key
if err := s.getJSON(ctx, u, nil, &resp); err != nil {
writeError(w, http.StatusBadGateway, "youtube search failed")
return
}
results := make([]map[string]any, 0, len(resp.Items))
for _, it := range resp.Items {
if it.ID.VideoID == "" {
continue
}
results = append(results, map[string]any{
"id": it.ID.VideoID,
// The Data API returns titles HTML-escaped (&amp;, &#39;) — unescape
// so the client renders plain text and re-escapes once itself.
"title": html.UnescapeString(it.Snippet.Title),
"channel": html.UnescapeString(it.Snippet.ChannelTitle),
"thumbnail": it.Snippet.Thumbnails.Medium.URL,
"publishedAt": it.Snippet.PublishedAt,
})
}
out := map[string]any{"results": results}
s.cache.Set(cacheKey, out, 10*time.Minute, 60*time.Minute)
writeJSON(w, http.StatusOK, "public, max-age=600, s-maxage=600", out)
}
// ytRendererRe finds each video result block in the results-page ytInitialData.
var ytRendererRe = regexp.MustCompile(`"videoRenderer":\{"videoId":"([A-Za-z0-9_-]{11})"`)
// ytTitleRe / ytOwnerRe pull the title and channel out of one renderer block.
var ytTitleRe = regexp.MustCompile(`"title":\{"runs":\[\{"text":"((?:[^"\\]|\\.)*)"`)
var ytOwnerRe = regexp.MustCompile(`"ownerText":\{"runs":\[\{"text":"((?:[^"\\]|\\.)*)"`)
// youtubeSearchScrape resolves a query to videos WITHOUT an API key by reading
// the results page. Returns an empty slice (never an error) when YouTube shape-
// shifts — the caller then reports an honest empty result rather than a fake one.
func (s *Server) youtubeSearchScrape(ctx context.Context, q string) []map[string]any {
// sp=EgIQAQ%3D%3D → filter to type:video (drops channels/playlists/shorts).
u := "https://www.youtube.com/results?search_query=" + urlQueryEscape(q) +
"&sp=EgIQAQ%3D%3D&hl=en&gl=US"
page, err := s.getText(ctx, u, map[string]string{
"User-Agent": browserUA,
"Accept-Language": "en-US,en;q=0.9",
"Cookie": "SOCS=CAISNQgDEitib3FfaWRlbnRpdHlfMjAyNDA4MjcuMDFfcDAaAmVuIAEaBgiA_LyxBg; CONSENT=YES+",
})
if err != nil {
return []map[string]any{}
}
locs := ytRendererRe.FindAllStringSubmatchIndex(page, 20)
results := make([]map[string]any, 0, len(locs))
seen := map[string]bool{}
for i, loc := range locs {
id := page[loc[2]:loc[3]]
if seen[id] {
continue
}
seen[id] = true
// Scope title/channel lookup to THIS renderer block, so a later result's
// title can never be attributed to an earlier video.
end := len(page)
if i+1 < len(locs) {
end = locs[i+1][0]
}
block := page[loc[1]:end]
title := ytUnquote(ytTitleRe, block)
if title == "" {
continue
}
results = append(results, map[string]any{
"id": id,
"title": title,
"channel": ytUnquote(ytOwnerRe, block),
"thumbnail": "https://i.ytimg.com/vi/" + id + "/mqdefault.jpg",
"publishedAt": "",
})
}
return results
}
// ytUnquote applies re to block and JSON-unescapes the captured text (the page
// embeds &, \" etc.).
func ytUnquote(re *regexp.Regexp, block string) string {
m := re.FindStringSubmatch(block)
if m == nil {
return ""
}
var out string
if err := json.Unmarshal([]byte(`"`+m[1]+`"`), &out); err != nil {
return html.UnescapeString(m[1])
}
return out
}
+136
View File
@@ -0,0 +1,136 @@
// Package kv is world's thin, graceful-degrade client for hanzo-kv — the shared
// Valkey/Redis hot cache (k8s Service hanzo-kv:6379, ns hanzo). It exists so the
// feed/response warm cache is SHARED across all world pods and survives a pod
// restart: warming once benefits the whole fleet, and a restarted pod reads a
// still-warm cache instead of cold-starting.
//
// Every method degrades cleanly — a nil/unconfigured client, an unreachable
// server, or any transport error yields a clean miss / no-op, never a blocking
// call or an error the caller must handle. A tiny circuit breaker parks a
// downed server for a cooldown so the hot path is not repeatedly stalled dialing
// a dead host. The queryable data lake lives in SQLite (package store); this is
// only the speed layer in front of feeds.
package kv
import (
"context"
"sync/atomic"
"time"
"github.com/redis/go-redis/v9"
)
// breakerCooldown parks a failing server: after an error, ops short-circuit to
// "miss" for this long instead of re-dialing on every request.
const breakerCooldown = 15 * time.Second
// Client wraps a go-redis client. A zero/disabled Client (r == nil) is valid and
// behaves as a permanent clean miss, so local dev and CI need no Redis.
type Client struct {
r *redis.Client
downUntil atomic.Int64 // unix-nano; server parked until then
}
// Open builds a client for addr (e.g. "hanzo-kv:6379"). An empty addr returns a
// disabled client (pure miss) — the correct behavior for environments without
// hanzo-kv. Timeouts are short so a slow/dead server degrades fast; retries are
// disabled because we fail over to the embedded/in-mem cache, not by retrying.
func Open(addr, password string) *Client {
if addr == "" {
return &Client{}
}
return &Client{r: redis.NewClient(&redis.Options{
Addr: addr,
Password: password,
DialTimeout: 2 * time.Second,
ReadTimeout: time.Second,
WriteTimeout: time.Second,
PoolSize: 8,
MaxRetries: -1, // fail fast; degrade rather than retry
})}
}
// Enabled reports whether a server is configured (not whether it is currently
// reachable).
func (c *Client) Enabled() bool { return c != nil && c.r != nil }
func (c *Client) available() bool {
if c == nil || c.r == nil {
return false
}
return time.Now().UnixNano() >= c.downUntil.Load()
}
// trip parks the server for the cooldown after a transport failure.
func (c *Client) trip() { c.downUntil.Store(time.Now().Add(breakerCooldown).UnixNano()) }
// GetBytes returns the value for key, or (nil,false) on miss/failure. A real
// cache miss (redis.Nil) does not trip the breaker; a transport error does.
func (c *Client) GetBytes(ctx context.Context, key string) ([]byte, bool) {
if !c.available() {
return nil, false
}
b, err := c.r.Get(ctx, key).Bytes()
if err == redis.Nil {
return nil, false
}
if err != nil {
c.trip()
return nil, false
}
return b, true
}
// SetBytes writes key=val with a TTL. Best-effort: failures trip the breaker and
// are otherwise ignored (the value is still cached in the per-pod mirror).
func (c *Client) SetBytes(ctx context.Context, key string, val []byte, ttl time.Duration) {
if !c.available() {
return
}
if err := c.r.Set(ctx, key, val, ttl).Err(); err != nil {
c.trip()
}
}
// SAdd adds members to a set (the fleet-wide warm-URL registry). Best-effort.
func (c *Client) SAdd(ctx context.Context, key string, members ...string) {
if !c.available() || len(members) == 0 {
return
}
vals := make([]any, len(members))
for i, m := range members {
vals[i] = m
}
if err := c.r.SAdd(ctx, key, vals...).Err(); err != nil {
c.trip()
}
}
// SMembers returns the set members, or nil on miss/failure.
func (c *Client) SMembers(ctx context.Context, key string) []string {
if !c.available() {
return nil
}
v, err := c.r.SMembers(ctx, key).Result()
if err != nil {
c.trip()
return nil
}
return v
}
// Ping checks reachability (used once at boot to log status). Returns an error
// when disabled or unreachable.
func (c *Client) Ping(ctx context.Context) error {
if c == nil || c.r == nil {
return redis.ErrClosed
}
return c.r.Ping(ctx).Err()
}
// Close releases the connection pool. Safe on a disabled client.
func (c *Client) Close() {
if c != nil && c.r != nil {
_ = c.r.Close()
}
}
-7
View File
@@ -272,13 +272,6 @@ func toolResult(body []byte, isErr bool) map[string]any {
return res
}
func toolError(msg string) map[string]any {
return map[string]any{
"content": []any{map[string]any{"type": "text", "text": msg}},
"isError": true,
}
}
// ── http helpers ─────────────────────────────────────────────────────────────
// setMCPCORS mirrors the world backend's wildcard policy (public data, no
+13 -46
View File
@@ -25,9 +25,19 @@ type Engine struct {
snapPath string
history *History
// sink, when set, receives every cycle's raw observations after they fold.
// It lets an owner (package world) dump observations into the queryable data
// lake WITHOUT this package knowing anything about storage — the engine stays
// decomplected from the datastore; it just calls a value-in hook.
sink func([]Observation)
startOnce sync.Once
}
// SetObservationSink registers a hook called with each cycle's observations after
// they are folded. Set once before Start; nil (the default) is a no-op.
func (e *Engine) SetObservationSink(fn func([]Observation)) { e.sink = fn }
// New builds an engine. dataDir is where the warm-start snapshot and the history
// ring live; interval<=0 uses DefaultInterval.
func New(sources []Source, dataDir string, interval time.Duration) *Engine {
@@ -43,9 +53,6 @@ func New(sources []Source, dataDir string, interval time.Duration) *Engine {
}
}
// History exposes the durable snapshot ring (queried by the /history handler).
func (e *Engine) History() *History { return e.history }
// Store exposes the state store to the API handlers.
func (e *Engine) Store() *Store { return e.store }
@@ -114,6 +121,9 @@ func (e *Engine) IngestOnce(ctx context.Context) {
wg.Wait()
changes := e.store.Apply(all, time.Now().UTC())
log.Printf("world-model: ingest folded %d observations, %d changes", len(all), len(changes))
if e.sink != nil && len(all) > 0 {
e.sink(all)
}
}
// ── snapshot ─────────────────────────────────────────────────────────────────
@@ -173,41 +183,6 @@ func (e *Engine) save() {
// ── AI grounding (ModelContext) ──────────────────────────────────────────────
// Context returns a compact plain-text briefing of the current world state —
// the top movers and highest-instability entities — for grounding AI answers in
// the model instead of raw feeds. One helper, called by the AI handlers.
func (e *Engine) Context() string {
top := e.store.Top(KindCountry, MetricInstability, 8)
movers := e.store.Top(KindCountry, "velocity", 6)
theaters := e.store.Top(KindTheater, MetricInstability, 4)
asOf := e.store.AsOf()
var b strings.Builder
b.WriteString("WORLD MODEL (as of ")
b.WriteString(asOf.Format(time.RFC3339))
b.WriteString(")\nHighest instability:\n")
for _, ent := range top {
b.WriteString(" - " + ent.Name + " (" + ent.ID + "): instability " +
ftoa(ent.Metrics[MetricInstability]) + " [" + ent.Level + "]" + toneNote(ent) + "\n")
}
b.WriteString("Biggest news movers:\n")
for _, ent := range movers {
v := ent.Metrics[MetricNewsVelocity]
if v == 0 {
continue
}
b.WriteString(" - " + ent.Name + ": news velocity " + signed(v) + "\n")
}
if len(theaters) > 0 {
b.WriteString("Active theaters:\n")
for _, ent := range theaters {
b.WriteString(" - " + ent.Name + ": " + ent.Level + " (" +
ftoa(ent.Metrics[MetricMilitaryActivity]) + " mil. aircraft)\n")
}
}
return strings.TrimRight(b.String(), "\n")
}
// CountryContext returns the state vector for one country (ISO alpha-2) as a
// JSON-friendly map, or false if the model has no such entity. AI country
// briefs merge this so the narrative matches the numbers.
@@ -223,14 +198,6 @@ func (e *Engine) CountryContext(iso string) (map[string]any, bool) {
}, true
}
func toneNote(e *Entity) string {
t, ok := e.Metrics[MetricSentiment]
if !ok {
return ""
}
return ", tone " + ftoa(t)
}
func ftoa(f float64) string {
b, _ := json.Marshal(round2(f))
return string(b)
@@ -0,0 +1,174 @@
package world
// RED adversarial test (NOT part of Blue's commits — orchestrator may keep or delete).
//
// Proves the unified IAM-introspection admin gate (introspectIdentity → requireAdmin)
// is fail-closed across the full matrix Blue left "network-bound"/untested:
// - no token → 401
// - non-admin owner → 403
// - empty/missing owner → 403 (NOT admin, even though isAdminOrg is not env-driven)
// - IAM 401 / 500 / non-JSON 200 → 403 (introspection failure)
// - admin / built-in owner → gate PASSES (not 401/403)
// - owner with surrounding whitespace → trimmed, PASSES
// - shared identity cache poisoning → a settings-populated non-admin identity
// cannot elevate the admin gate
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
)
// iamStub serves /v1/iam/oauth/userinfo with a fixed status + body.
func iamStub(t *testing.T, status int, body string) *httptest.Server {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/iam/oauth/userinfo" {
w.WriteHeader(http.StatusNotFound)
return
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
_, _ = w.Write([]byte(body))
}))
t.Cleanup(srv.Close)
return srv
}
// apiStub returns 200 {} for any path — a passing gate must reach here and 200.
func apiStub(t *testing.T) *httptest.Server {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{}`))
}))
t.Cleanup(srv.Close)
return srv
}
func TestRedAdminGateMatrix(t *testing.T) {
const adminRoute = "/v1/world/cloud/fleet"
cases := []struct {
name string
iamStatus int
iamBody string
bearer string // "" = send no Authorization header
wantStatus int // exact status when deterministic
gatePass bool // true = assert NOT 401 and NOT 403 (downstream may vary)
}{
{name: "no token → 401", bearer: "", wantStatus: http.StatusUnauthorized},
{name: "non-admin owner → 403", iamStatus: 200, iamBody: `{"owner":"acme","sub":"u1"}`, bearer: "Bearer good", wantStatus: http.StatusForbidden},
{name: "empty owner → 403", iamStatus: 200, iamBody: `{"owner":"","sub":"u1"}`, bearer: "Bearer good", wantStatus: http.StatusForbidden},
{name: "missing owner claim → 403", iamStatus: 200, iamBody: `{"sub":"u1"}`, bearer: "Bearer good", wantStatus: http.StatusForbidden},
{name: "owner=null → 403", iamStatus: 200, iamBody: `{"owner":null,"sub":"u1"}`, bearer: "Bearer good", wantStatus: http.StatusForbidden},
{name: "IAM 401 (bad token) → 403", iamStatus: 401, iamBody: `{"error":"invalid_token"}`, bearer: "Bearer bad", wantStatus: http.StatusForbidden},
{name: "IAM 500 → 403", iamStatus: 500, iamBody: `oops`, bearer: "Bearer good", wantStatus: http.StatusForbidden},
{name: "IAM 200 non-JSON → 403", iamStatus: 200, iamBody: `<html>not json</html>`, bearer: "Bearer good", wantStatus: http.StatusForbidden},
{name: "IAM 200 owner as number → 403", iamStatus: 200, iamBody: `{"owner":123,"sub":"u1"}`, bearer: "Bearer good", wantStatus: http.StatusForbidden},
{name: "admin owner → gate passes", iamStatus: 200, iamBody: `{"owner":"admin","sub":"z"}`, bearer: "Bearer good", gatePass: true},
{name: "built-in owner → gate passes", iamStatus: 200, iamBody: `{"owner":"built-in","sub":"z"}`, bearer: "Bearer good", gatePass: true},
{name: "admin owner padded whitespace → gate passes", iamStatus: 200, iamBody: `{"owner":" admin ","sub":"z"}`, bearer: "Bearer good", gatePass: true},
{name: "near-miss 'administrator' → 403", iamStatus: 200, iamBody: `{"owner":"administrator","sub":"z"}`, bearer: "Bearer good", wantStatus: http.StatusForbidden},
{name: "case-variant 'Admin' → 403", iamStatus: 200, iamBody: `{"owner":"Admin","sub":"z"}`, bearer: "Bearer good", wantStatus: http.StatusForbidden},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
iam := iamStub(t, tc.iamStatus, tc.iamBody)
api := apiStub(t)
t.Setenv("HANZO_IAM_ISSUER", iam.URL)
t.Setenv("HANZO_API_BASE", api.URL)
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
req, _ := http.NewRequest(http.MethodGet, ts.URL+adminRoute, nil)
if tc.bearer != "" {
req.Header.Set("Authorization", tc.bearer)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("request: %v", err)
}
defer resp.Body.Close()
if tc.gatePass {
if resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden {
t.Fatalf("admin owner was REJECTED (%d); gate must pass", resp.StatusCode)
}
return
}
if resp.StatusCode != tc.wantStatus {
t.Fatalf("got %d, want %d", resp.StatusCode, tc.wantStatus)
}
})
}
}
// TestRedSharedCacheNoElevation proves the unification's shared "identity:" cache
// cannot elevate: an identity resolved for the SETTINGS path (non-admin owner) is
// the very same cache entry requireAdmin reads — and it still 403s. If the gate
// ever trusted a settings-populated entry as admin, this fails.
func TestRedSharedCacheNoElevation(t *testing.T) {
iam := iamStub(t, 200, `{"owner":"acme","sub":"u1"}`)
api := apiStub(t)
t.Setenv("HANZO_IAM_ISSUER", iam.URL)
t.Setenv("HANZO_API_BASE", api.URL)
s := NewServer()
mux := http.NewServeMux()
s.Mount(mux)
ts := httptest.NewServer(mux)
t.Cleanup(ts.Close)
bearer := "Bearer settings-first"
// 1) Populate the shared identity cache exactly as the settings path would.
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
id, err := s.introspectIdentity(ctx, bearer)
if err != nil || id.Org != "acme" || id.Sub != "u1" {
t.Fatalf("cache priming failed: id=%+v err=%v", id, err)
}
// 2) Now hit the admin route with the SAME token. The gate reads the SAME cache
// entry (no second IAM call) and MUST still reject a non-admin owner.
req, _ := http.NewRequest(http.MethodGet, ts.URL+"/v1/world/cloud/fleet", nil)
req.Header.Set("Authorization", bearer)
resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Fatalf("request: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusForbidden {
t.Fatalf("shared-cache elevation: non-admin token got %d on admin route, want 403", resp.StatusCode)
}
}
// TestRedIsAdminOrgExact locks the admin-org predicate: only the two hardcoded
// orgs (trimmed) qualify; empty never does. This is the property that makes the
// "empty ADMIN_ORG == empty owner" class of bug structurally impossible — there
// is no env-driven admin org to leave unset.
func TestRedIsAdminOrgExact(t *testing.T) {
admit := map[string]bool{
"admin": true, "built-in": true,
" admin ": true, "\tbuilt-in\n": true,
}
deny := []string{"", " ", "Admin", "ADMIN", "administrator", "built_in", "builtin", "admins", "acme", "\x00admin"}
for in, want := range admit {
if isAdminOrg(in) != want {
t.Fatalf("isAdminOrg(%q) = %v, want %v", in, !want, want)
}
}
for _, in := range deny {
if isAdminOrg(in) {
t.Fatalf("isAdminOrg(%q) = true, want false", in)
}
}
}
+44
View File
@@ -48,6 +48,7 @@ func (s *Server) mount(mux registrar) {
mux.HandleFunc("/v1/world/ais-snapshot", s.handleAISSnapshot)
mux.HandleFunc("/v1/world/firms-fires", s.handleFIRMS)
mux.HandleFunc("/v1/world/earthquakes", s.handleEarthquakes)
mux.HandleFunc("/v1/world/hko-warnings", s.handleHKOWarnings)
mux.HandleFunc("/v1/world/climate-anomalies", s.handleClimate)
mux.HandleFunc("/v1/world/wingbits", s.handleWingbits)
mux.HandleFunc("/v1/world/wingbits/", s.handleWingbits)
@@ -64,9 +65,24 @@ func (s *Server) mount(mux registrar) {
mux.HandleFunc("/v1/world/fwdstart", s.handleFwdstart)
mux.HandleFunc("/v1/world/youtube/live", s.handleYouTubeLive)
mux.HandleFunc("/v1/world/youtube/embed", s.handleYouTubeEmbed)
mux.HandleFunc("/v1/world/youtube/search", s.handleYouTubeSearch)
mux.HandleFunc("/v1/world/monitors", s.handleMonitors)
mux.HandleFunc("/v1/world/monitors/matches", s.handleMonitorMatches)
// model-improvement consent opt-in (proxied to ai's OrgSettings, the source of truth)
mux.HandleFunc("/v1/world/training-contribution", s.handleTrainingContribution)
// ingested-data lake — the "one place to query everything" (search +
// analytics across ALL ingested items: news, model observations, …).
mux.HandleFunc("/v1/world/search", s.handleSearch)
mux.HandleFunc("/v1/world/analytics", s.handleAnalytics)
// per-identity settings — server-side dashboard sync for signed-in users
// (bearer-gated; anonymous keeps localStorage).
mux.HandleFunc("/v1/world/settings", s.handleSettings)
// econ / humanitarian
mux.HandleFunc("/v1/world/fred-data", s.handleFRED)
mux.HandleFunc("/v1/world/china-macro", s.handleChinaMacro)
mux.HandleFunc("/v1/world/worldbank", s.handleWorldBank)
mux.HandleFunc("/v1/world/eia", s.handleEIA)
mux.HandleFunc("/v1/world/eia/", s.handleEIA)
@@ -92,19 +108,43 @@ func (s *Server) mount(mux registrar) {
// configured. Org-scoped drill-down goes straight to api.hanzo.ai, not here.
mux.HandleFunc("/v1/world/cloud-pulse", s.handleCloudPulse)
// AI Compute pulse (AI variant): live inference volume + serving fleet, pushed
// over SSE (EventSource) with a plain-GET JSON snapshot as the poll fallback.
// Same honest platform aggregate as cloud-pulse; "unavailable" without a token.
mux.HandleFunc("/v1/world/ai-pulse", s.handleAIPulse)
// Enso flywheel (AI variant): the router self-improvement loop — routing-ledger
// tail + reward tail (super-admin) folded with the latest enso-bench eval
// scores (embedded snapshot / ENSO_BENCH_URL). Event-typed; evals-only degrade.
mux.HandleFunc("/v1/world/enso-training", s.handleEnsoTraining)
// Cloud console. PUBLIC excitement layer (real, non-sensitive):
mux.HandleFunc("/v1/world/cloud/models", s.handleCloudModels)
// PUBLIC map layers (real telemetry when reachable; modeled/demo carries a flag):
mux.HandleFunc("/v1/world/cloud/chain-nodes", s.handleCloudChainNodes)
mux.HandleFunc("/v1/world/cloud/byo-gpu", s.handleCloudBYOGPU)
mux.HandleFunc("/v1/world/cloud/traffic", s.handleCloudTraffic)
// Native LB request-geo aggregate (points + throughput) for the Hanzo-mode globe.
// Proxies the ai backend's public /v1/traffic/globe; honest empty state, no demo.
mux.HandleFunc("/v1/world/cloud/traffic-globe", s.handleCloudTrafficGlobe)
// PUBLIC status.hanzo.ai summary (Gatus proxy: per-service up/down + incidents):
mux.HandleFunc("/v1/world/cloud/status-page", s.handleCloudStatusPage)
// PUBLIC Enso Live Training — ai gateway /v1/router/stats?scope=platform proxy
// (aggregates only; arms already opaque "arm-N" upstream — no vendor names):
mux.HandleFunc("/v1/world/cloud/router-stats", s.handleCloudRouterStats)
// PUBLIC flywheel history — ai gateway /v1/router/history?scope=platform proxy:
// daily reward-rate + cumulative cost-saved + adoption + retrain timeline. Honest
// empty until the ledger fills; never a fabricated curve.
mux.HandleFunc("/v1/world/cloud/router-history", s.handleCloudRouterHistory)
// ADMIN-only aggregates (requireAdmin, fail-closed 403; forward caller bearer):
mux.HandleFunc("/v1/world/cloud/fleet", s.handleCloudFleet)
mux.HandleFunc("/v1/world/cloud/services", s.handleCloudServices)
mux.HandleFunc("/v1/world/cloud/analytics", s.handleCloudAnalytics)
mux.HandleFunc("/v1/world/cloud/llm", s.handleCloudLLM)
// ADMIN-only Enso benchmark suite: private, competitive head-to-head (names
// competitor models + Enso). Same requireAdmin gate (401/403 fail-closed);
// reshapes the embedded enso-bench snapshot — never leaks to a non-admin.
mux.HandleFunc("/v1/world/enso-benchmarks", s.handleEnsoBenchmarks)
// AI (Hanzo inference)
mux.HandleFunc("/v1/world/groq-summarize", s.handleSummarize)
@@ -114,6 +154,10 @@ func (s *Server) mount(mux registrar) {
mux.HandleFunc("/v1/world/country-intel", s.handleCountryIntel)
mux.HandleFunc("/v1/world/analyst", s.handleAnalyst)
mux.HandleFunc("/v1/world/models", s.handleModels)
// Content-free AI reward-signal BFF. BARE /v1/feedback (matching the gateway
// path) so the @hanzo/ai SDK's same-origin baseUrl:'' → POST /v1/feedback
// reaches it. An exact pattern, so it beats the "/" SPA catch-all.
mux.HandleFunc("/v1/feedback", s.handleFeedback)
// social share (OpenGraph)
mux.HandleFunc("/v1/world/story", s.handleStory)
+154 -29
View File
@@ -19,10 +19,13 @@ import (
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/hanzoai/world/internal/world/kv"
"github.com/hanzoai/world/internal/world/mcp"
"github.com/hanzoai/world/internal/world/model"
"github.com/hanzoai/world/internal/world/store"
)
const (
@@ -39,21 +42,32 @@ const (
type Server struct {
client *http.Client
cache *Cache
flight *flightGroup
ai *AIClient
worldModel *model.Engine
mcp *mcp.Server
// Datastore layer (see datastore.go): kv is the shared hanzo-kv hot cache,
// feeds is the two-tier warm feed-body cache in front of it, and store is the
// embedded SQLite lake + per-identity settings.
kv *kv.Client
store *store.DB
feeds *FeedCache
}
// NewServer constructs the backend and its world-model engine (built from the
// feed sources in model_sources.go). Call StartModel to begin ingest.
// NewServer constructs the backend, its world-model engine (built from the feed
// sources in model_sources.go), and the datastore layer. Call StartModel and
// StartDatastore to begin the background loops.
func NewServer() *Server {
s := &Server{
client: &http.Client{Timeout: 25 * time.Second},
cache: NewCache(4096),
flight: newFlightGroup(),
ai: newAIClient(),
mcp: mcp.New(),
}
s.worldModel = model.New(s.modelSources(), modelDataDir(), modelInterval())
s.initDatastore()
return s
}
@@ -216,10 +230,80 @@ func methodNotGet(w http.ResponseWriter, r *http.Request) bool {
// ── caching helpers ─────────────────────────────────────────────────────────
// cachedJSON serves a JSON endpoint through the shared cache. produce returns
// the value to cache and return; on error, a stale cached value is served if
// available, otherwise onError decides the response. This captures the dominant
// "check cache → fetch → transform → cache → fallback" pattern once.
// flightGroup coalesces concurrent work on a cache key so N simultaneous misses
// cause ONE upstream fetch. It is the single-flight guard shared by cachedJSON
// and passthrough: a blocking caller (do) leads or waits-and-shares; a
// background revalidation (tryGo) leads or skips when a leader is already
// running. Keyed by the cache key, so different endpoints never collide.
type flightGroup struct {
mu sync.Mutex
m map[string]*flightCall
}
type flightCall struct {
done chan struct{}
val any
err error
}
func newFlightGroup() *flightGroup { return &flightGroup{m: map[string]*flightCall{}} }
// do runs fn for key, coalescing concurrent callers: the first (leader) runs fn
// once; the rest block on it and share its (val, err).
func (g *flightGroup) do(key string, fn func() (any, error)) (any, error) {
g.mu.Lock()
if c, ok := g.m[key]; ok {
g.mu.Unlock()
<-c.done
return c.val, c.err
}
c := &flightCall{done: make(chan struct{})}
g.m[key] = c
g.mu.Unlock()
g.run(key, c, fn)
return c.val, c.err
}
// tryGo runs fn in the background for key unless a call is already in flight,
// so concurrent misses don't stack refreshes. Reports whether it started one.
func (g *flightGroup) tryGo(key string, fn func() (any, error)) bool {
g.mu.Lock()
if _, ok := g.m[key]; ok {
g.mu.Unlock()
return false
}
c := &flightCall{done: make(chan struct{})}
g.m[key] = c
g.mu.Unlock()
go g.run(key, c, fn)
return true
}
// run executes fn, records its result, then releases the key and wakes waiters.
// A panic in fn is converted to an error (never crashes the background refresh
// goroutine) so a failed produce degrades cleanly like any other error.
func (g *flightGroup) run(key string, c *flightCall, fn func() (any, error)) {
defer func() {
if r := recover(); r != nil {
c.err = fmt.Errorf("panic: %v", r)
}
g.mu.Lock()
delete(g.m, key)
g.mu.Unlock()
close(c.done)
}()
c.val, c.err = fn()
}
// cachedJSON serves a JSON endpoint through the shared cache with
// stale-while-revalidate. produce returns the value to cache and return.
// - Fresh hit: served immediately.
// - Stale hit (TTL lapsed, still within the stale window): the stale value is
// served INSTANTLY and a single coalesced background refresh is kicked, so a
// lapsed TTL never blocks a request on the ~10s upstream.
// - True cold miss: blocks on produce, but single-flighted so N cold callers
// cause one upstream fetch; on error a stale value is served if one appeared,
// otherwise onError decides the response.
func (s *Server) cachedJSON(
w http.ResponseWriter,
key, cacheControl string,
@@ -231,9 +315,22 @@ func (s *Server) cachedJSON(
writeJSON(w, http.StatusOK, cacheControl, v)
return
}
ctx, cancel := context.WithTimeout(context.Background(), 24*time.Second)
defer cancel()
v, err := produce(ctx)
fetch := func() (any, error) {
ctx, cancel := context.WithTimeout(context.Background(), 24*time.Second)
defer cancel()
v, err := produce(ctx)
if err != nil {
return nil, err
}
s.cache.Set(key, v, ttl, staleFor)
return v, nil
}
if stale, ok := s.cache.GetStale(key); ok {
s.flight.tryGo(key, fetch)
writeJSON(w, http.StatusOK, cacheControl, stale)
return
}
v, err := s.flight.do(key, fetch)
if err != nil {
if stale, ok := s.cache.GetStale(key); ok {
writeJSON(w, http.StatusOK, cacheControl, stale)
@@ -242,7 +339,6 @@ func (s *Server) cachedJSON(
onError(w, err)
return
}
s.cache.Set(key, v, ttl, staleFor)
writeJSON(w, http.StatusOK, cacheControl, v)
}
@@ -257,11 +353,39 @@ const negativeTTL = 30 * time.Second
// it must never become the cached value.
func isBlankBody(body []byte) bool { return len(bytes.TrimSpace(body)) == 0 }
// fetchAndCache fetches upstream for key and applies the pass-through caching
// POLICY once: a good body is cached fresh (Set); a blank 200 or non-2xx is a
// failure that is never cached (it would poison good data) and instead sets a
// short negative marker so a flapping source is not re-hit every request. Shared
// by passthrough and the boot warmer so the policy lives in exactly one place.
func (s *Server) fetchAndCache(ctx context.Context, key, upstream string, headers map[string]string, ttl, staleFor time.Duration) ([]byte, error) {
body, status, err := s.get(ctx, upstream, headers)
if err != nil || status < 200 || status >= 300 || isBlankBody(body) {
if err == nil {
if status < 200 || status >= 300 {
err = fmt.Errorf("upstream status %d", status)
} else {
err = fmt.Errorf("upstream returned empty body")
}
}
s.cache.SetNegative(key, negativeTTL)
return nil, err
}
s.cache.Set(key, body, ttl, staleFor)
return body, nil
}
// passthrough proxies a fixed upstream URL, returning its body verbatim with a
// short in-memory TTL cache. Used by the pure pass-through endpoints. A blank
// 200 is treated as a failure (never cached — it would poison good data for the
// whole TTL). On any failure a stale body is served if present, else the
// upstream is negative-cached briefly and degraded is written no-store.
// short in-memory TTL cache and stale-while-revalidate. Used by the pure
// pass-through endpoints.
// - Fresh hit: served immediately.
// - Recent failure (negative marker): serve last-good stale or degrade — the
// upstream is not re-hit.
// - Stale hit: the stale body is served INSTANTLY and a single coalesced
// background refresh is kicked, so a lapsed TTL never blocks the request.
// - Cold miss: blocks on the fetch, single-flighted so N cold callers cause one
// upstream hit; on failure a stale body is served if present, else the
// upstream is negative-cached briefly and degraded is written no-store.
func (s *Server) passthrough(
w http.ResponseWriter,
key, upstream, contentType, cacheControl string,
@@ -273,29 +397,30 @@ func (s *Server) passthrough(
writeBytes(w, http.StatusOK, contentType, cacheControl, v.([]byte))
return
}
// A recent blank/failed fetch: don't re-hit the upstream, serve last-good
// stale or degrade cleanly.
if s.cache.Negative(key) {
s.degradeBytes(w, key, contentType, cacheControl, degraded, fmt.Errorf("upstream recently failed"))
return
}
ctx, cancel := context.WithTimeout(context.Background(), 24*time.Second)
defer cancel()
body, status, err := s.get(ctx, upstream, headers)
if err != nil || status < 200 || status >= 300 || isBlankBody(body) {
if err == nil {
if status < 200 || status >= 300 {
err = fmt.Errorf("upstream status %d", status)
} else {
err = fmt.Errorf("upstream returned empty body")
}
fetch := func() (any, error) {
ctx, cancel := context.WithTimeout(context.Background(), 24*time.Second)
defer cancel()
body, err := s.fetchAndCache(ctx, key, upstream, headers, ttl, staleFor)
if err != nil {
return nil, err
}
s.cache.SetNegative(key, negativeTTL)
return body, nil
}
if v, ok := s.cache.GetStale(key); ok {
s.flight.tryGo(key, fetch)
writeBytes(w, http.StatusOK, contentType, cacheControl, v.([]byte))
return
}
v, err := s.flight.do(key, fetch)
if err != nil {
s.degradeBytes(w, key, contentType, cacheControl, degraded, err)
return
}
s.cache.Set(key, body, ttl, staleFor)
writeBytes(w, http.StatusOK, contentType, cacheControl, body)
writeBytes(w, http.StatusOK, contentType, cacheControl, v.([]byte))
}
// degradeBytes serves the last-good stale body when present, else the handler's
+113
View File
@@ -0,0 +1,113 @@
package world
import (
"encoding/json"
"math"
"reflect"
"strings"
"testing"
)
// Yahoo returns chart closes as float32 widened to float64, so JSON re-serializes
// the conversion noise verbatim — 17.209999084472656, 18 characters for 17.21.
// roundSig strips that to 7 significant digits (significant digits, NOT fixed
// decimals, so a sub-1.0 FX rate keeps its precision). Values verified against the
// upstream toPrecision(7) semantics in tests/sparkline-precision.test.mjs.
func TestRoundSig(t *testing.T) {
cases := []struct {
in, want float64
}{
{17.209999084472656, 17.21}, // the canonical float32 noise → clean
{17.25, 17.25}, // already clean, unchanged
{16.829999923706055, 16.83},
{0.0071349999, 0.007135}, // sub-1.0: 7 SIG DIGITS kept, not flattened to 0.01
{-0.0071349999, -0.007135}, // sign preserved
{0.6917064189910889, 0.6917064}, // real FX rate — the reason it is sig digits
{0.6908955574035645, 0.6908956}, // rounds up correctly at the 7th digit
{123456.789, 123456.8}, // large: 7 sig < 2dp here — why we never round price scalars
{8675309, 8675309}, // 7-digit integer is exact
{42000, 42000},
{-42.005, -42.005},
{0, 0}, // zero passes through
}
for _, c := range cases {
if got := roundSig(c.in); got != c.want {
t.Errorf("roundSig(%v) = %v, want %v", c.in, got, c.want)
}
}
// A fixed 2dp round would flatten the FX rate to 0.01 and destroy the chart;
// significant digits keep the meaningful precision at any magnitude.
if v := roundSig(0.0071349999); v <= 0.007 || v >= 0.008 {
t.Errorf("FX precision lost: roundSig(0.0071349999) = %v, want ~0.007135", v)
}
// Non-finite values must survive rather than become null/0 and dent the curve.
if v := roundSig(math.NaN()); !math.IsNaN(v) {
t.Errorf("roundSig(NaN) = %v, want NaN", v)
}
if v := roundSig(math.Inf(1)); !math.IsInf(v, 1) {
t.Errorf("roundSig(+Inf) = %v, want +Inf", v)
}
if v := roundSig(math.Inf(-1)); !math.IsInf(v, -1) {
t.Errorf("roundSig(-Inf) = %v, want -Inf", v)
}
}
// The real emit path a client sees: Yahoo Close ([]*float64, nulls and all) →
// compact() drops the nulls → sparkline() rounds the last n for the wire. This
// pins that the shipped array is rounded and noise-free, and — load-bearing —
// that the source closes are NOT mutated, so the price/change scalars derived
// from them upstream stay exact.
func TestSparklinePathRoundsAndPreservesSource(t *testing.T) {
f := func(v float64) *float64 { return &v }
// Real float64 noise copied from market:commodities-bootstrap:v1, with a null
// spliced in (Yahoo emits nulls for missing sessions) that compact must drop.
raw := []*float64{
f(17.209999084472656), f(17.219999313354492), f(17.239999771118164),
nil, f(17.190000534057617), f(17.170000076293945), f(17.25), f(16.829999923706055),
}
closes := compact(raw)
got := sparkline(closes, 30)
want := []float64{17.21, 17.22, 17.24, 17.19, 17.17, 17.25, 16.83}
if !reflect.DeepEqual(got, want) {
t.Fatalf("sparkline(compact(raw)) = %v, want %v", got, want)
}
// The serialized array must carry none of the float32 noise.
b, err := json.Marshal(got)
if err != nil {
t.Fatalf("marshal: %v", err)
}
if strings.Contains(string(b), "17.209999") {
t.Errorf("noise reached the wire: %s", b)
}
// It must be materially smaller — this is the entire point (>50% on this series).
before, _ := json.Marshal(closes)
if len(b) >= len(before) {
t.Errorf("no shrink: raw %d bytes, rounded %d bytes", len(before), len(b))
}
// The scalar path reads `closes` directly (price = round2s(last)); sparkline
// must not have mutated it, or a 6-figure price would lose precision.
if closes[0] != 17.209999084472656 {
t.Errorf("source closes mutated: closes[0] = %v, want the raw noisy value", closes[0])
}
}
// sparkline is the one way an emitted close array is built: last n only, nil in →
// nil out (so a failed fetch degrades to JSON null exactly as before, not []).
func TestSparklineTailAndNil(t *testing.T) {
if got := sparkline(nil, 30); got != nil {
t.Errorf("sparkline(nil) = %v, want nil (preserve null shape)", got)
}
long := make([]float64, 50)
for i := range long {
long[i] = float64(i)
}
got := sparkline(long, 30)
if len(got) != 30 || got[0] != 20 || got[29] != 49 {
t.Errorf("sparkline tail: got len=%d first=%v last=%v, want last 30 (20..49)", len(got), got[0], got[29])
}
}

Some files were not shown because too many files have changed in this diff Show More