Compare commits

..
Author SHA1 Message Date
hanzo-dev 5324854c44 docs(db/base): record security invariants from blue+red review 2026-07-04 12:09:18 -07:00
hanzo-dev 61192a55fe fix(db/base): address red adversarial review (C1/H1/M1/M2/L1)
- C1 CRITICAL: block prototype pollution — getPath/setPath/unsetPath reject
  __proto__/constructor/prototype segments (dotted + bare). Attacker update
  keys (saveConvo, conversation import) can no longer poison Object.prototype.
- H1 HIGH: stop pushing Date predicates to Base filter DSL. Base normalizes
  date columns to a space separator; the ISO literal never matched, silently
  excluding records (superset-invariant violation). JS matcher handles dates.
- M1: partial UNIQUE indexes (WHERE != '') on sparse unique keys
  (googleId/openidId/username/…) — DB-level OAuth race safety without null
  conflicts. Live-verified on a fresh Base (user collection).
- M2: document.save() merges onto the stored record instead of full-replace,
  so a projected load (secrets hidden) can't erase select:false fields.
- L1: literal() drops non-finite numbers (no `= NaN` invalid DSL).
- meiliSearch applies deselected projection for read-path consistency.
- +8 regression specs (37 green): pollution guard, date-no-pushdown,
  date-equality read, save() secret preservation, non-finite guard.
2026-07-04 12:08:53 -07:00
hanzo-dev 362ce9a26e docs(db/base): update phased status — Phases 1-3 done, remaining tail documented 2026-07-04 11:49:10 -07:00
hanzo-dev c33aa4aaf4 feat(db/base): Phase 3 — self-contained facade, drop mongoose from app code
- Facade no longer requires the mongoose driver: own ObjectId (isValid/
  createFromHexString/equals/toJSON), Schema.Types shim, no-op session/
  connection. api non-spec code has ZERO require('mongoose').
- Convert the 6 remaining app files (Agent, inviteUser, PermissionService,
  PermissionsController, initializeMCPs, migration) to the Base facade.
- Base UNIQUE indexes on always-present unique keys (conversationId, messageId,
  email, _id) for race-safety; sparse unique fields stay plain-indexed.
  Verified live: duplicate conversationId insert is rejected by Base.
- Query.session() no-op (data-schemas userGroup passes sessions).
- 29 jest green; boot-smoke + live-check (login+convo+msg+search+unique) pass.
2026-07-04 11:47:15 -07:00
hanzo-dev 6dc3d9b6fe feat(db/base): Phase 2 — Base/SQLite full-text search, drop MeiliSearch
- BaseModel.meiliSearch now performs real user-scoped substring search over
  content fields (title, text) flagged meiliIndex, returning Meili-shaped
  { hits }. Conversation + Message search work on Base/SQLite.
- schema.js captures `searchable` content fields (meiliIndex string, minus ids).
- indexSync.js → no-op (no external index; drops mongoose + meilisearch imports).
- Remove dead Meili helper api/db/utils.js (+specs) and indexSync.spec.js.
- /search/enable reports availability from Base (SEARCH=false to disable).
- +1 unit spec (29 green); live-check asserts real title+text hits on Base.
2026-07-04 11:38:58 -07:00
hanzo-dev 2886fce18f fix(db/base): honor Mongoose select:false + hydrate all Date fields (blue review)
- select:false secrets (password, totpSecret, backupCodes, keyHash, _meiliIndex)
  are now excluded from reads by default; only returned via explicit +field.
  Login path (findUser +password) still works. Prevents secret-hash exposure.
- Hydrate every Date-typed schema path (not just timestamps) to Date objects so
  callers can safely call .getTime()/date methods (e.g. session.expiration);
  cast epoch-number defaults (Date.now) to Date too.
- Projection engine reworked to Mongoose semantics (plain/+plus/-exclude).
- +5 specs (28 total green); live-check asserts default view hides the hash.
2026-07-04 11:34:19 -07:00
hanzo-dev c1ef90baa2 feat(db): port data layer off MongoDB to Hanzo Base (Phase 1, #48)
Drop-mongo directive: replace MongoDB with a Mongoose-compatible adapter
over @hanzo/base. One facade fed to @librechat/data-schemas' createModels/
createMethods makes every model Base-backed with zero per-model porting.

- api/db/base/{objectId,query,schema,model,store,mongoose,index}.js — adapter
- Wire api/db/{connect,models,index}.js + api/models/index.js off mongoose
- query.spec.js + model.spec.js (jest, in-memory store, 23 tests green)
- scripts/{live-check,boot-smoke}.js prove login+convo+message persist on Base

Nothing calls mongoose.connect; hot path off mongoose. FTS (Meili) + the 7
remaining ObjectId/migration mongoose imports are Phase 2/3 (see LLM.md).
2026-07-04 03:34:18 -07:00
Hanzo AI 6fd5f38d93 refactor: /v1/ canonical paths (sweep)
Per global rule: /v1/ only, never /api/. Internal API calls + handler
comments + route registrations updated. External vendor APIs (Stripe,
KMS, etc.) left as-is. LibreChat upstream backend routes (api/server/*)
left as-is per "leave upstream-vendored paths" guideline; data-provider
SDK now points at Hanzo Cloud Gateway /v1/* canonical paths since
api.hanzo.ai is our gateway and /api/ on top of api.* is double-prefix.
2026-05-18 21:32:06 -07:00
Hanzo AI c8653b2cab chat: rename VITE_HANZO_IAM_* -> VITE_IAM_* in static SPA login
Collapse the transitional VITE_HANZO_IAM_{URL,APP,ORG} env aliases
to the canonical VITE_IAM_{URL,APP,ORG}. Aligns with the ecosystem
IAM_* convention (no HANZO_ prefix on identity vars).
2026-05-15 12:13:33 -07:00
Hanzo AI d254ca0f02 ci: add id-token: write to caller permissions
Required for hanzoai/.github/.github/workflows/docker-build.yml@main —
without it the workflow_call dies as startup_failure with no jobs
dispatched. Caller permissions are a CEILING.
2026-05-07 09:09:39 -07:00
Hanzo AI 94abdccfc5 deploy: pin :latest → semver across image refs
Replaces all ghcr.io/hanzoai/<svc>:latest pins with the latest published
semver tag for each service. Per CLAUDE.md auto-bump policy: mutable
branch tags (:latest, :main, :dev) are deprecated for cluster pins —
only immutable semver permitted.

Bulk update across services with published v* tags. Services without a
published semver remain on :latest until their release pipeline cuts a
v* tag.
2026-05-07 08:36:35 -07:00
Hanzo DevandGitHub a26e7eebb2 ci: remove upstream LibreChat CI cruft; enable arm64 in canonical (#30)
Deletes seven inherited workflows that published unused images
(chat-api, lc-dev*, lc-dev-staging*, LiteLLM ghcr_deploy/helm chart) —
none are referenced by hanzoai/universe manifests, all used QEMU-emulated
builds instead of ARC runners.

docker-publish.yml now builds linux/amd64 + linux/arm64 via the canonical
reusable. One workflow, one way, multi-arch by default.
2026-04-23 20:21:45 -07:00
Hanzo DevandGitHub 8c41dfb6a1 ci: canonical docker-build (#31)
* ci: remove upstream LibreChat CI cruft; enable arm64 in canonical

Deletes seven inherited workflows that published unused images
(chat-api, lc-dev*, lc-dev-staging*, LiteLLM ghcr_deploy/helm chart) —
none are referenced by hanzoai/universe manifests, all used QEMU-emulated
builds instead of ARC runners.

docker-publish.yml now builds linux/amd64 + linux/arm64 via the canonical
reusable. One workflow, one way, multi-arch by default.

* ci: migrate to canonical hanzoai/.github/docker-build.yml reusable
2026-04-23 19:24:34 -07:00
2459 changed files with 481721 additions and 236532 deletions
-16
View File
@@ -1,16 +0,0 @@
# depcheck configuration.
#
# ignores: dependencies that are present on purpose but never imported directly,
# so depcheck's static scan reports them as a false positive.
#
# The three @opentelemetry/* entries below are all transitive dependencies of
# @opentelemetry/sdk-node, which we DO import and drive (NodeSDK in
# packages/api/src/telemetry/sdk.ts). They are pinned at the top level to keep
# every otel package on one aligned version (mismatched otel versions break at
# runtime). None is imported by name, so depcheck flags them — expected, not
# dead. Do NOT remove them (that would unpin the otel version set). Only
# @opentelemetry/api is imported directly and stays checked.
ignores:
- "@opentelemetry/core"
- "@opentelemetry/exporter-trace-otlp-http"
- "@opentelemetry/sdk-trace-base"
-62
View File
@@ -1,62 +0,0 @@
services:
app:
build:
context: ..
dockerfile: .devcontainer/Dockerfile
# restart: always
links:
- mongodb
- meilisearch
# ports:
# - 3080:3080 # Change it to 9000:3080 to use nginx
extra_hosts: # if you are running APIs on docker you need access to, you will need to uncomment this line and next
- "host.docker.internal:host-gateway"
volumes:
# This is where VS Code should expect to find your project's source code and the value of "workspaceFolder" in .devcontainer/devcontainer.json
- ..:/workspaces:cached
# Uncomment the next line to use Docker from inside the container. See https://aka.ms/vscode-remote/samples/docker-from-docker-compose for details.
# - /var/run/docker.sock:/var/run/docker.sock
environment:
- HOST=0.0.0.0
- MONGO_URI=mongodb://mongodb:27017/LibreChat
# - OPENAI_REVERSE_PROXY=http://host.docker.internal:8070/v1
- MEILI_HOST=http://meilisearch:7700
# Runs app on the same network as the service container, allows "forwardPorts" in devcontainer.json function.
# network_mode: service:another-service
# Use "forwardPorts" in **devcontainer.json** to forward an app port locally.
# (Adding the "ports" property to this file will not forward from a Codespace.)
# Use a non-root user for all processes - See https://aka.ms/vscode-remote/containers/non-root for details.
user: vscode
# Overrides default command so things don't shut down after the process ends.
command: /bin/sh -c "while sleep 1000; do :; done"
mongodb:
container_name: chat-mongodb
expose:
- 27017
# ports:
# - 27018:27017
image: mongo
# restart: always
volumes:
- ./data-node:/data/db
command: mongod --noauth
meilisearch:
container_name: chat-meilisearch
image: getmeili/meilisearch:v1.5
# restart: always
expose:
- 7700
# Uncomment this to access meilisearch from outside docker
# ports:
# - 7700:7700 # if exposing these ports, make sure your master key is not the default value
environment:
- MEILI_NO_ANALYTICS=true
- MEILI_MASTER_KEY=5c71cf56d672d009e36070b5bc5e47b743535ae55c818ae3b735bb6ebfb4ba63
volumes:
- ./meili_data_v1.5:/meili_data
-25
View File
@@ -1,25 +0,0 @@
# Caddy reverse proxy with bearer token auth and automatic HTTPS.
# The domain is supplied via environment variable GITNEXUS_DOMAIN,
# and the auth token via API_TOKEN. Both are set in docker-compose.yml.
{$GITNEXUS_DOMAIN} {
# Health check — unauthenticated so monitoring can probe it
@health path /health
handle @health {
reverse_proxy gitnexus:4747 {
rewrite /api/info
}
}
# All other routes require bearer token
@authed {
header Authorization "Bearer {$API_TOKEN}"
}
handle @authed {
reverse_proxy gitnexus:4747
}
# Reject unauthenticated requests
respond "Unauthorized" 401
}
-45
View File
@@ -1,45 +0,0 @@
# Long-lived GitNexus image for DigitalOcean droplet deployment.
#
# This image does NOT bake in the index data. Indexes are mounted from
# the host at /indexes/<repo>/.gitnexus/ and registered at container
# startup. A fresh index only requires rsync + container restart — no
# image rebuild on every push.
FROM node:24.16.0-slim
ARG GITNEXUS_VERSION=1.5.3
# 1. Build native addons with Bookworm toolchain, then remove build tools.
# curl stays for the docker healthcheck; Caddy lives in its own container.
RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 make g++ curl \
&& npm install -g gitnexus@${GITNEXUS_VERSION} \
&& apt-get purge -y --auto-remove python3 make g++ \
&& rm -rf /var/lib/apt/lists/* /root/.npm
# 2. Upgrade libstdc++ from Trixie — @ladybugdb/core prebuilt binary needs
# GLIBCXX_3.4.32 which Bookworm (3.4.31) doesn't ship.
RUN echo "deb http://deb.debian.org/debian trixie main" > /etc/apt/sources.list.d/trixie.list \
&& apt-get update \
&& apt-get install -y -t trixie libstdc++6 \
&& rm /etc/apt/sources.list.d/trixie.list \
&& rm -rf /var/lib/apt/lists/*
# 3. Pre-install LadybugDB FTS + vector extensions so ~/.kuzu/extension/
# is baked into the image. Workaround for upstream GitNexus 1.5.3 bug.
COPY install-extensions.js /tmp/install-extensions.js
RUN node /tmp/install-extensions.js && rm -rf /tmp/install-extensions.js /tmp/lbug-ext-install
# 4. Patch lbug-adapter.js to also LOAD EXTENSION vector after FTS.
RUN LBUG_ADAPTER=/usr/local/lib/node_modules/gitnexus/dist/mcp/core/lbug-adapter.js \
&& grep -q "LOAD EXTENSION fts" "$LBUG_ADAPTER" \
&& sed -i "s|await available\[0\]\.query('LOAD EXTENSION fts');|await available[0].query('LOAD EXTENSION fts'); try { await available[0].query('LOAD EXTENSION vector'); } catch (e) { /* vector extension may not be installed */ }|g" "$LBUG_ADAPTER" \
&& grep -c "LOAD EXTENSION vector" "$LBUG_ADAPTER" \
&& echo "lbug-adapter.js patched to load vector extension"
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
EXPOSE 4747
ENTRYPOINT ["/entrypoint.sh"]
-87
View File
@@ -1,87 +0,0 @@
# GitNexus stack for the DigitalOcean droplet.
#
# Two services: the gitnexus server (bound to an internal network only)
# and a Caddy reverse proxy that handles TLS + auth.
#
# Index data lives on the host at /opt/gitnexus/indexes/ and is
# bind-mounted read-write into the gitnexus container. The deploy
# workflow rsyncs fresh indexes into that directory and restarts
# only the gitnexus container — Caddy keeps running undisturbed.
#
# Break-glass: if gitnexus is stuck unhealthy and you need to restart
# just Caddy (e.g. to push an emergency Caddyfile fix), the
# `depends_on: condition: service_healthy` would block:
# docker compose up -d caddy
# Use --no-deps to bypass the dependency check:
# docker compose up -d --no-deps caddy
name: gitnexus
# Shared logging defaults applied to both services so the droplet's
# disk doesn't fill up with unbounded json-file logs.
x-logging: &default-logging
driver: json-file
options:
max-size: '50m'
max-file: '3'
services:
gitnexus:
# Override via GITNEXUS_IMAGE in /opt/gitnexus/.env to use a fork or
# a pinned version tag like :v1.5.3 for reproducible rollbacks.
image: ${GITNEXUS_IMAGE:-ghcr.io/danny-avila/librechat-gitnexus:latest}
container_name: gitnexus
restart: unless-stopped
networks:
- gitnexus-net
volumes:
- /opt/gitnexus/indexes:/indexes
# memswap_limit equal to mem_limit disables swap for this container.
# Without it, Docker lets the process silently swap onto host disk,
# turning sub-second graph queries into multi-second ones. Hard
# OOM-kill is preferable — the container restarts via unless-stopped,
# the deploy health poll catches it, and the failure is explicit.
mem_limit: 1792m
memswap_limit: 1792m
logging: *default-logging
healthcheck:
test: ['CMD', 'curl', '-fsS', 'http://127.0.0.1:4747/api/info']
interval: 30s
timeout: 5s
retries: 3
start_period: 60s
caddy:
image: caddy:2-alpine
container_name: gitnexus-caddy
restart: unless-stopped
# service_healthy (not just service_started) ensures Caddy doesn't
# start routing traffic until gitnexus passes its initial healthcheck
# on a cold `compose up`. This only governs initial startup ordering —
# during force-recreates of gitnexus, Caddy stays up and may briefly
# return 502 while the new gitnexus container binds its port. The
# deploy workflow's health poll catches any sustained failure.
depends_on:
gitnexus:
condition: service_healthy
ports:
- '80:80'
- '443:443'
networks:
- gitnexus-net
volumes:
- /opt/gitnexus/Caddyfile:/etc/caddy/Caddyfile:ro
- caddy-data:/data
- caddy-config:/config
logging: *default-logging
environment:
GITNEXUS_DOMAIN: ${GITNEXUS_DOMAIN}
API_TOKEN: ${API_TOKEN}
networks:
gitnexus-net:
driver: bridge
volumes:
caddy-data:
caddy-config:
-48
View File
@@ -1,48 +0,0 @@
#!/bin/sh
set -e
# Cap Node heap below the container's cgroup limit (1792m in compose),
# leaving room for @ladybugdb/core's C++ heap and OS overhead. Native
# allocations happen outside V8's view, so a slim V8 budget is the only
# thing between a heavy query and a cgroup OOM-kill. Without this cap,
# gitnexus defaults to --max-old-space-size=8192 and reserves memory
# the container doesn't have.
export NODE_OPTIONS="${NODE_OPTIONS:---max-old-space-size=1280}"
# Register every index mounted under /indexes/<name>/.gitnexus/.
# This is idempotent — re-registering an existing repo updates the
# metadata pointer without touching the index data.
#
# Registration failure handling:
# - main (LibreChat) and dev (LibreChat-dev) are critical. If either
# fails to register, exit 1 so docker marks the container unhealthy
# and the deploy workflow's readiness check surfaces the error.
# - PR indexes (LibreChat-pr-*) are best-effort. A corrupt PR index
# shouldn't take the whole server down.
if [ -d /indexes ]; then
for dir in /indexes/*/; do
[ -d "$dir" ] || continue
name=$(basename "$dir")
[ -d "$dir.gitnexus" ] || continue
echo "Registering index: $name"
if ! gitnexus index "$dir" --allow-non-git; then
case "$name" in
LibreChat|LibreChat-dev)
echo "ERROR: failed to register critical index $name" >&2
exit 1
;;
*)
echo "WARN: failed to register PR index $name — skipping" >&2
;;
esac
fi
done
else
echo "WARN: /indexes directory not mounted" >&2
fi
# Bind 0.0.0.0 inside the container so Caddy (in a separate container
# on the same docker network) can reach gitnexus at gitnexus:4747.
# docker-compose.yml intentionally does NOT expose port 4747 on the
# host — only Caddy's 80/443 are published.
exec gitnexus serve --host 0.0.0.0 --port 4747
-46
View File
@@ -1,46 +0,0 @@
/**
* Pre-install LadybugDB extensions (FTS + vector) into the Docker image's
* extension cache (~/.kuzu/extension/). Without this, gitnexus serve's
* lbug-adapter calls LOAD EXTENSION fts at runtime but fails silently
* because the extension was never installed, causing all BM25 and
* semantic queries via the query() tool to return empty.
*
* Workaround for upstream GitNexus 1.5.3 bug where the CI-produced
* .gitnexus/ artifact doesn't include the extension cache.
*/
const path = require('path');
const fs = require('fs');
// @ladybugdb/core lives under the globally-installed gitnexus package.
// This path is stable across gitnexus versions because npm always nests
// transitive deps under the installed package's node_modules.
const lbugPath = '/usr/local/lib/node_modules/gitnexus/node_modules/@ladybugdb/core';
const lbug = require(lbugPath);
const tmpDir = '/tmp/lbug-ext-install';
fs.mkdirSync(tmpDir, { recursive: true });
// Open a throwaway database just to run INSTALL against. The extension
// cache persists in ~/.kuzu/extension/ regardless of which database was
// used to install it, so the throwaway db and tmpDir are deleted in the
// Dockerfile after this script finishes.
const db = new lbug.Database(path.join(tmpDir, 'db'), 0, false, false);
const conn = new lbug.Connection(db);
(async () => {
try {
await conn.query('INSTALL fts');
console.log('FTS extension installed');
} catch (err) {
console.error('FTS install failed:', err.message);
process.exit(1);
}
try {
await conn.query('INSTALL vector');
console.log('Vector extension installed');
} catch (err) {
console.error('Vector install failed:', err.message);
process.exit(1);
}
})();
+19 -40
View File
@@ -117,17 +117,6 @@ PROXY=
# Endpoint: https://api.hanzo.ai/v1
HANZO_API_KEY=
# Canonical Hanzo Cloud agents (/v1/agents). Lets signed-in users run their own
# cloud agents from chat via `/agent <name>` or the @mention picker. The chat
# backend proxies /v1/chat/agents/cloud/* to this host, forwarding the user's
# hanzo.id token server-side (never to the browser); cloud scopes to their org.
# Optional: if unset, derived from OPENAI_BASE_URL (its host, minus /v1).
# HANZO_CLOUD_URL=https://api.hanzo.ai
# A cloud-agent run is a real billable completion; these bound abuse of the proxy.
# CLOUD_AGENT_USER_MAX=30 # max cloud-agent requests per user per window
# CLOUD_AGENT_WINDOW=1 # rate-limit window, minutes
# CLOUD_AGENT_MAX_CONCURRENT=50 # process-wide in-flight ceiling to cloud (503 past it)
#===================================#
# Known Endpoints - chat.yaml #
#===================================#
@@ -474,21 +463,6 @@ ALLOW_REGISTRATION=true
ALLOW_SOCIAL_LOGIN=false
ALLOW_SOCIAL_REGISTRATION=false
ALLOW_PASSWORD_RESET=false
#=====================================================#
# Guest Chat (anonymous preview) #
#=====================================================#
# Off by default. When enabled, unauthenticated visitors get a small free
# quota on the free Zen model routed through api.hanzo.ai. After the quota is
# spent the UI surfaces the existing OpenID/hanzo.id login. Guests are scoped
# server-side to the free model only and rejected from every other route.
ALLOW_GUEST_CHAT=false
# GUEST_MESSAGE_MAX=3 # free messages per IP before login is required
# GUEST_ENDPOINT=Hanzo # custom endpoint name from librechat.yaml (api.hanzo.ai)
# GUEST_MODEL=zen3-nano # free Zen model id guests are pinned to
# GUEST_TOKEN_EXPIRY=3600000 # guest JWT lifetime in ms (default 1h)
# GUEST_TOKEN_MAX=20 # max guest sessions issued per IP per window
# GUEST_TOKEN_WINDOW=60 # guest-session issuance window in minutes
# ALLOW_ACCOUNT_DELETION=true # note: enabled by default if omitted/commented out
ALLOW_UNVERIFIED_EMAIL_LOGIN=true
@@ -836,26 +810,31 @@ OPENWEATHER_API_KEY=
# Hanzo Chat Code Interpreter API #
#====================================#
# "Run Code" routes through the Hanzo unified backend (api.hanzo.ai/v1/exec),
# which executes in a Hanzo sandbox. execute_code POSTs {BASEURL}/exec with
# X-API-Key. https://hanzo.ai/docs/chat/code-interpreter
# LIBRECHAT_CODE_BASEURL=https://api.hanzo.ai/v1
# LIBRECHAT_CODE_API_KEY=your-key # prod: KMS chat-secrets/LIBRECHAT_CODE_API_KEY
# https://hanzo.ai/docs/chat/code-interpreter
# LIBRECHAT_CODE_API_KEY=your-key
#======================#
# Web Search #
#======================#
# Web Search routes through the Hanzo unified backend ONLY — no external SaaS.
# The searxng provider + firecrawl scraper both point at api.hanzo.ai/v1/websearch
# (Hanzo metasearch + Hanzo Crawl). See librechat.yaml `webSearch`.
# Note: All of the following variable names can be customized.
# Omit values to allow user to provide them.
# For more information on configuration values, see:
# https://hanzo.ai/docs/chat/features/web_search
# SEARXNG_INSTANCE_URL=https://api.hanzo.ai/v1/websearch
# FIRECRAWL_API_URL=https://api.hanzo.ai/v1/websearch
# WEBSEARCH_API_KEY=your-key # prod: KMS chat-secrets/WEBSEARCH_API_KEY
#
# Do NOT set SERPER_API_KEY / TAVILY_API_KEY / JINA_API_KEY / COHERE_API_KEY:
# external providers are intentionally disabled (one way, unified backend).
# Search Provider (Required)
# SERPER_API_KEY=your_serper_api_key
# Scraper (Required)
# FIRECRAWL_API_KEY=your_firecrawl_api_key
# Optional: Custom Firecrawl API URL
# FIRECRAWL_API_URL=your_firecrawl_api_url
# Reranker (Required)
# JINA_API_KEY=your_jina_api_key
# or
# COHERE_API_KEY=your_cohere_api_key
#======================#
# MCP Configuration #
+3 -13
View File
@@ -5,19 +5,9 @@
OPENAI_API_KEY=sk-hanzo-your-api-key-here
OPENAI_BASE_URL=https://api.hanzo.ai/v1
# Code Interpreter ("Run Code") — Hanzo unified backend.
# LibreChat's execute_code tool POSTs {LIBRECHAT_CODE_BASEURL}/exec with header
# X-API-Key. cloud-api serves /v1/exec (-> sandboxed executor). The key is
# KMS-sourced (chat-secrets/LIBRECHAT_CODE_API_KEY). These are the ONLY vars
# LibreChat reads — the old CODE_EXECUTION_ENDPOINT/RUNTIME_API_KEY were dead.
LIBRECHAT_CODE_BASEURL=https://api.hanzo.ai/v1
LIBRECHAT_CODE_API_KEY=set-in-KMS-chat-secrets
# Web Search — Hanzo unified backend (searxng + firecrawl compat over Hanzo
# search + crawl). See librechat.yaml `webSearch`. One shared service key.
SEARXNG_INSTANCE_URL=https://api.hanzo.ai/v1/websearch
FIRECRAWL_API_URL=https://api.hanzo.ai/v1/websearch
WEBSEARCH_API_KEY=set-in-KMS-chat-secrets
# Runtime Execution (Code Interpreter)
CODE_EXECUTION_ENDPOINT=https://api.hanzo.ai/v1/execute
RUNTIME_API_KEY=${OPENAI_API_KEY}
# ====== BRANDING ======
APP_TITLE=Hanzo Chat
-9
View File
@@ -1,9 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1280" height="640" viewBox="0 0 1280 640" role="img" aria-label="chat">
<rect width="1280" height="640" fill="#0A0A0A"/>
<svg x="96" y="215" width="210" height="210" viewBox="0 0 67 67"><path d="M22.21 67V44.6369H0V67H22.21Z" fill="#fff"/><path d="M66.7038 22.3184H22.2534L0.0878906 44.6367H44.4634L66.7038 22.3184Z" fill="#fff"/><path d="M22.21 0H0V22.3184H22.21V0Z" fill="#fff"/><path d="M66.7198 0H44.5098V22.3184H66.7198V0Z" fill="#fff"/><path d="M66.7198 67V44.6369H44.5098V67H66.7198Z" fill="#fff"/></svg>
<text x="378" y="276" font-family="Inter,system-ui,-apple-system,sans-serif" font-size="78" font-weight="800" letter-spacing="-2" fill="#ffffff">chat</text>
<text x="378" y="322" font-family="Inter,system-ui,sans-serif" font-size="30" fill="#ffffff" opacity=".66">AI chat with MCP integration and multi-provider support</text>
<rect x="378" y="338" width="806" height="3" rx="1.5" fill="#ffffff" opacity=".9"/>
<text x="378" y="390" font-family="Inter,system-ui,sans-serif" font-size="24" font-weight="600" fill="#ffffff" opacity=".5">github.com/hanzoai</text>
<text x="1184" y="390" text-anchor="end" font-family="Inter,system-ui,sans-serif" font-size="24" font-weight="600" fill="#ffffff" opacity=".5">hanzo.ai</text>
</svg>

Before

Width:  |  Height:  |  Size: 1.3 KiB

-237
View File
@@ -1,237 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
CHART_PATH="${CHART_PATH:-helm/librechat/Chart.yaml}"
DEFAULT_BRANCH="${DEFAULT_BRANCH:-main}"
BASE_REF="${BASE_REF:-refs/remotes/origin/${DEFAULT_BRANCH}}"
BACKFILL_FROM_VERSION="${BACKFILL_FROM_VERSION:-1.9.0}"
PUSH_TAGS="${PUSH_TAGS:-false}"
TAG_PREFIX="${TAG_PREFIX:-chart-}"
GITHUB_SERVER_URL="${GITHUB_SERVER_URL:-https://github.com}"
DISPATCH_WORKFLOW="${DISPATCH_WORKFLOW:-}"
RELEASE_EXISTING_TAG="${RELEASE_EXISTING_TAG:-}"
SEMVER_REGEX='^(0|[1-9][0-9]*)[.](0|[1-9][0-9]*)[.](0|[1-9][0-9]*)(-[0-9A-Za-z-]+([.][0-9A-Za-z-]+)*)?([+][0-9A-Za-z-]+([.][0-9A-Za-z-]+)*)?$'
fail() {
printf '::error::%s\n' "$1" >&2
exit 1
}
git_auth_header() {
token="$(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\n')"
printf 'AUTHORIZATION: basic %s' "$token"
}
git_with_auth() {
if [ -n "${GITHUB_TOKEN:-}" ]; then
git -c "http.extraheader=$(git_auth_header)" "$@"
return
fi
git "$@"
}
dispatch_release() {
tag="$1"
if [ -z "$DISPATCH_WORKFLOW" ]; then
return
fi
if [ -z "${GITHUB_REPOSITORY:-}" ]; then
fail "GITHUB_REPOSITORY is required to dispatch ${DISPATCH_WORKFLOW}"
fi
if [[ ! "$GITHUB_REPOSITORY" =~ ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ ]]; then
fail "Unexpected repository name: ${GITHUB_REPOSITORY}"
fi
if [[ ! "$DISPATCH_WORKFLOW" =~ ^[A-Za-z0-9_.-]+[.]ya?ml$ ]]; then
fail "Unexpected workflow file: ${DISPATCH_WORKFLOW}"
fi
token="${GH_TOKEN:-${GITHUB_TOKEN:-}}"
if [ -z "$token" ]; then
fail "GH_TOKEN or GITHUB_TOKEN is required to dispatch ${DISPATCH_WORKFLOW}"
fi
command -v gh >/dev/null ||
fail "GitHub CLI is required to dispatch ${DISPATCH_WORKFLOW}"
GH_TOKEN="$token" gh workflow run "$DISPATCH_WORKFLOW" \
--repo "$GITHUB_REPOSITORY" \
--ref "$DEFAULT_BRANCH" \
-f "chart_tag=${tag}"
}
version_less_than() {
left="${1%%[-+]*}"
right="${2%%[-+]*}"
IFS=. read -r left_major left_minor left_patch <<<"$left"
IFS=. read -r right_major right_minor right_patch <<<"$right"
if (( left_major != right_major )); then
(( left_major < right_major ))
return
fi
if (( left_minor != right_minor )); then
(( left_minor < right_minor ))
return
fi
(( left_patch < right_patch ))
}
validate_chart_tag() {
tag="$1"
version="${tag#${TAG_PREFIX}}"
git check-ref-format "refs/tags/${tag}" >/dev/null ||
fail "Refusing to use invalid tag ${tag}"
if [[ "$tag" != "${TAG_PREFIX}"* || ! "$version" =~ $SEMVER_REGEX ]]; then
fail "Chart tags must use the form ${TAG_PREFIX}<semver>, for example ${TAG_PREFIX}2.0.5"
fi
}
dispatch_existing_tag() {
tag="$1"
if [ -z "$tag" ]; then
return
fi
validate_chart_tag "$tag"
if [ "$PUSH_TAGS" != "true" ]; then
printf 'Would dispatch release workflow for existing %s.\n' "$tag"
return
fi
if ! git_with_auth ls-remote --exit-code --tags origin "refs/tags/${tag}" >/dev/null 2>&1; then
fail "Remote tag ${tag} does not exist"
fi
printf 'Dispatching release workflow for existing %s.\n' "$tag"
dispatch_release "$tag"
}
chart_version_at() {
git show "${1}:${CHART_PATH}" 2>/dev/null | awk '
/^version:[[:space:]]*/ {
value = $0
sub(/^version:[[:space:]]*/, "", value)
sub(/[[:space:]]*#.*/, "", value)
gsub(/^[[:space:]"'\''"]+|[[:space:]"'\''"]+$/, "", value)
print value
exit
}
'
}
case "$PUSH_TAGS" in
true | false) ;;
*) fail "PUSH_TAGS must be true or false" ;;
esac
if [[ ! "$BACKFILL_FROM_VERSION" =~ $SEMVER_REGEX ]]; then
fail "BACKFILL_FROM_VERSION must be a valid SemVer value"
fi
git rev-parse --verify "${BASE_REF}^{commit}" >/dev/null ||
fail "Unable to resolve ${BASE_REF}; fetch ${DEFAULT_BRANCH} before running this script"
history_file="$(mktemp)"
versions_file="$(mktemp)"
seen_file="$(mktemp)"
missing_file="$(mktemp)"
cleanup() {
rm -f "$history_file" "$versions_file" "$seen_file" "$missing_file"
}
trap cleanup EXIT
git log --first-parent --reverse --format=%H "$BASE_REF" -- "$CHART_PATH" >"$history_file"
if [ ! -s "$history_file" ]; then
fail "No history found for ${CHART_PATH} on ${BASE_REF}"
fi
while IFS= read -r commit; do
version="$(chart_version_at "$commit")"
if [ -z "$version" ]; then
continue
fi
if [[ ! "$version" =~ $SEMVER_REGEX ]]; then
fail "${CHART_PATH} has invalid SemVer '${version}' at ${commit}"
fi
if version_less_than "$version" "$BACKFILL_FROM_VERSION"; then
continue
fi
if grep -Fqx "$version" "$seen_file"; then
continue
fi
printf '%s\n' "$version" >>"$seen_file"
printf '%s\t%s\n' "$version" "$commit" >>"$versions_file"
done <"$history_file"
if [ ! -s "$versions_file" ]; then
fail "No chart versions found in ${CHART_PATH}"
fi
while IFS="$(printf '\t')" read -r version commit; do
tag="${TAG_PREFIX}${version}"
validate_chart_tag "$tag"
if git rev-parse --quiet --verify "refs/tags/${tag}" >/dev/null; then
continue
fi
printf '%s\t%s\n' "$tag" "$commit" >>"$missing_file"
done <"$versions_file"
if [ ! -s "$missing_file" ]; then
printf 'All chart versions on %s already have %s tags.\n' "$BASE_REF" "$TAG_PREFIX"
dispatch_existing_tag "$RELEASE_EXISTING_TAG"
exit 0
fi
while IFS="$(printf '\t')" read -r tag commit; do
short_commit="$(git rev-parse --short "$commit")"
if [ "$PUSH_TAGS" != "true" ]; then
printf 'Would create %s at %s.\n' "$tag" "$short_commit"
continue
fi
if git_with_auth ls-remote --exit-code --tags origin "refs/tags/${tag}" >/dev/null 2>&1; then
printf 'Remote tag %s already exists; dispatching release workflow.\n' "$tag"
dispatch_release "$tag"
continue
fi
git tag "$tag" "$commit"
if git_with_auth push origin "refs/tags/${tag}"; then
printf 'Created %s at %s.\n' "$tag" "$short_commit"
dispatch_release "$tag"
continue
fi
if git_with_auth ls-remote --exit-code --tags origin "refs/tags/${tag}" >/dev/null 2>&1; then
printf 'Remote tag %s was created concurrently; dispatching release workflow.\n' "$tag"
dispatch_release "$tag"
continue
fi
fail "Failed to push ${tag}"
done <"$missing_file"
dispatch_existing_tag "$RELEASE_EXISTING_TAG"
+1 -1
View File
@@ -13,7 +13,7 @@ on:
jobs:
axe-linter:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
if: >
(github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == 'hanzoai/chat') ||
(github.event_name == 'workflow_dispatch' && github.event.inputs.run_workflow == 'true')
@@ -7,7 +7,7 @@ on:
jobs:
auto_update_price_and_context_window:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Install Dependencies
+1 -1
View File
@@ -13,7 +13,7 @@ jobs:
tests_Backend:
name: Run Backend unit tests
timeout-minutes: 60
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
env:
MONGO_URI: ${{ secrets.MONGO_URI }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+1 -1
View File
@@ -8,7 +8,7 @@ env:
jobs:
build-and-push:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
steps:
# checkout the repo
- name: 'Checkout GitHub Action'
@@ -19,7 +19,7 @@ jobs:
cache_integration_tests:
name: Integration Tests that use actual Redis Cache
timeout-minutes: 30
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
steps:
- name: Checkout repository
+1 -1
View File
@@ -19,7 +19,7 @@ permissions:
jobs:
build-and-publish:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
environment: publish # Must match npm trusted publisher config
steps:
- uses: actions/checkout@v6
+2 -2
View File
@@ -19,7 +19,7 @@ permissions:
jobs:
build:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: pnpm/action-setup@v4
@@ -32,7 +32,7 @@ jobs:
publish-npm:
needs: build
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
environment: publish # Must match npm trusted publisher config
steps:
- uses: actions/checkout@v6
+1 -1
View File
@@ -19,7 +19,7 @@ permissions:
jobs:
build-and-publish:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
environment: publish # Must match npm trusted publisher config
steps:
- uses: actions/checkout@v6
-49
View File
@@ -1,49 +0,0 @@
name: Update Test Server
on:
workflow_run:
workflows: ["Docker Dev Branch Images Build"]
types:
- completed
workflow_dispatch:
permissions:
contents: read
jobs:
deploy:
runs-on: hanzo-deploy-linux-amd64
if: |
github.repository == 'danny-avila/LibreChat' &&
(github.event_name == 'workflow_dispatch' ||
(github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.head_branch == 'dev'))
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install SSH Key
uses: shimataro/ssh-key-action@v2
with:
key: ${{ secrets.DO_SSH_PRIVATE_KEY }}
known_hosts: ${{ secrets.DO_KNOWN_HOSTS }}
- name: Run update script on DigitalOcean Droplet
env:
DO_HOST: ${{ secrets.DO_HOST }}
DO_USER: ${{ secrets.DO_USER }}
run: |
ssh ${DO_USER}@${DO_HOST} << EOF
sudo -i -u danny bash << 'EEOF'
cd ~/LibreChat && \
git fetch origin main && \
sudo npm run stop:deployed && \
sudo docker images --format "{{.Repository}}:{{.ID}}" | grep -E "lc-dev|librechat" | cut -d: -f2 | xargs -r sudo docker rmi -f || true && \
sudo npm run update:deployed && \
git checkout dev && \
git pull origin dev && \
git checkout do-deploy && \
git rebase dev && \
sudo npm run start:deployed && \
echo "Update completed. Application should be running now."
EEOF
EOF
+46
View File
@@ -0,0 +1,46 @@
name: Update Test Server
on:
workflow_run:
workflows: ["Docker Dev Branch Images Build"]
types:
- completed
workflow_dispatch:
jobs:
deploy:
runs-on: ubuntu-latest
if: |
github.repository == 'danny-avila/LibreChat' &&
(github.event_name == 'workflow_dispatch' ||
(github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.head_branch == 'dev'))
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install SSH Key
uses: shimataro/ssh-key-action@v2
with:
key: ${{ secrets.DO_SSH_PRIVATE_KEY }}
known_hosts: ${{ secrets.DO_KNOWN_HOSTS }}
- name: Run update script on DigitalOcean Droplet
env:
DO_HOST: ${{ secrets.DO_HOST }}
DO_USER: ${{ secrets.DO_USER }}
run: |
ssh -o StrictHostKeyChecking=no ${DO_USER}@${DO_HOST} << EOF
sudo -i -u danny bash << 'EEOF'
cd ~/LibreChat && \
git fetch origin main && \
sudo npm run stop:deployed && \
sudo docker images --format "{{.Repository}}:{{.ID}}" | grep -E "lc-dev|librechat" | cut -d: -f2 | xargs -r sudo docker rmi -f || true && \
sudo npm run update:deployed && \
git checkout dev && \
git pull origin dev && \
git checkout do-deploy && \
git rebase dev && \
sudo npm run start:deployed && \
echo "Update completed. Application should be running now."
EEOF
EOF
+1 -1
View File
@@ -13,7 +13,7 @@ env:
jobs:
deploy-gh-runner-aci:
runs-on: hanzo-deploy-linux-amd64
runs-on: ubuntu-latest
steps:
# checkout the repo
- name: 'Checkout GitHub Action'
-95
View File
@@ -1,95 +0,0 @@
name: Docker Dev Branch Images Build
on:
workflow_dispatch:
push:
branches:
- dev
paths:
- 'api/**'
- 'client/**'
- 'packages/**'
- 'package.json'
- 'package-lock.json'
- 'Dockerfile'
- 'Dockerfile.multi'
permissions:
contents: read
packages: write
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: hanzo-build-linux-amd64
timeout-minutes: 130
strategy:
matrix:
include:
- target: api-build
file: Dockerfile.multi
image_name: lc-dev-api
- target: node
file: Dockerfile
image_name: lc-dev
steps:
# Check out the repository
- name: Checkout
uses: actions/checkout@v4
# Set up QEMU
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
# Set up Docker Buildx
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# Log in to GitHub Container Registry
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Login to Docker Hub
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# Prepare the environment
- name: Prepare environment
run: |
cp .env.example .env
- name: Compute build metadata
run: |
echo "BUILD_COMMIT=${{ github.sha }}" >> $GITHUB_ENV
echo "BUILD_BRANCH=${{ github.ref_name }}" >> $GITHUB_ENV
echo "BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_ENV
# Build and push Docker images for each target
- name: Build and push Docker images
uses: docker/build-push-action@v5
with:
context: .
file: ${{ matrix.file }}
push: true
tags: |
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:${{ github.sha }}
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:latest
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:${{ github.sha }}
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:latest
platforms: linux/amd64,linux/arm64
target: ${{ matrix.target }}
build-args: |
BUILD_COMMIT=${{ env.BUILD_COMMIT }}
BUILD_BRANCH=${{ env.BUILD_BRANCH }}
BUILD_DATE=${{ env.BUILD_DATE }}
-91
View File
@@ -1,91 +0,0 @@
name: Docker Dev Images Build
on:
workflow_dispatch:
push:
branches:
- main
paths:
- 'api/**'
- 'client/**'
- 'packages/**'
- 'package.json'
- 'package-lock.json'
- 'Dockerfile'
- 'Dockerfile.multi'
permissions:
contents: read
packages: write
jobs:
build:
runs-on: hanzo-build-linux-amd64
timeout-minutes: 130
strategy:
matrix:
include:
- target: api-build
file: Dockerfile.multi
image_name: librechat-dev-api
- target: node
file: Dockerfile
image_name: librechat-dev
steps:
# Check out the repository
- name: Checkout
uses: actions/checkout@v4
# Set up QEMU
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
# Set up Docker Buildx
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# Log in to GitHub Container Registry
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Login to Docker Hub
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# Prepare the environment
- name: Prepare environment
run: |
cp .env.example .env
- name: Compute build metadata
run: |
echo "BUILD_COMMIT=${{ github.sha }}" >> $GITHUB_ENV
echo "BUILD_BRANCH=${{ github.ref_name }}" >> $GITHUB_ENV
echo "BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_ENV
# Build and push Docker images for each target
- name: Build and push Docker images
uses: docker/build-push-action@v5
with:
context: .
file: ${{ matrix.file }}
push: true
tags: |
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:${{ github.sha }}
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:latest
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:${{ github.sha }}
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:latest
platforms: linux/amd64,linux/arm64
target: ${{ matrix.target }}
build-args: |
BUILD_COMMIT=${{ env.BUILD_COMMIT }}
BUILD_BRANCH=${{ env.BUILD_BRANCH }}
BUILD_DATE=${{ env.BUILD_DATE }}
-79
View File
@@ -1,79 +0,0 @@
name: Docker Dev Staging Images Build
on:
workflow_dispatch:
permissions:
contents: read
packages: write
jobs:
build:
runs-on: hanzo-build-linux-amd64
strategy:
matrix:
include:
- target: api-build
file: Dockerfile.multi
image_name: lc-dev-staging-api
- target: node
file: Dockerfile
image_name: lc-dev-staging
steps:
# Check out the repository
- name: Checkout
uses: actions/checkout@v4
# Set up QEMU
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
# Set up Docker Buildx
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# Log in to GitHub Container Registry
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Login to Docker Hub
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# Prepare the environment
- name: Prepare environment
run: |
cp .env.example .env
- name: Compute build metadata
run: |
echo "BUILD_COMMIT=${{ github.sha }}" >> $GITHUB_ENV
echo "BUILD_BRANCH=${{ github.ref_name }}" >> $GITHUB_ENV
echo "BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_ENV
# Build and push Docker images for each target
- name: Build and push Docker images
uses: docker/build-push-action@v5
with:
context: .
file: ${{ matrix.file }}
push: true
tags: |
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:${{ github.sha }}
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:latest
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:${{ github.sha }}
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:latest
platforms: linux/amd64,linux/arm64
target: ${{ matrix.target }}
build-args: |
BUILD_COMMIT=${{ env.BUILD_COMMIT }}
BUILD_BRANCH=${{ env.BUILD_BRANCH }}
BUILD_DATE=${{ env.BUILD_DATE }}
+42 -51
View File
@@ -1,14 +1,4 @@
# Hanzo Chat — image build (amd64, self-hosted hanzoai pool).
#
# chat is a PUBLIC repo, so it cannot reuse the private hanzoai/.github
# reusable workflow (GitHub blocks public->private workflow reuse).
# This is therefore self-contained.
#
# Builds ghcr.io/hanzoai/chat on:
# - push to main -> immutable sha-<sha7> tag
# - v* tags -> semver image tag (e.g. v0.7.10 -> 0.7.10)
# No :latest, no floating tags (semver-only policy). amd64 only
# (DOKS is all amd64 — no arm64 runners).
# Deployment managed by hanzoai/universe — this workflow only builds and pushes images
name: Docker Release
on:
@@ -18,54 +8,55 @@ on:
workflow_dispatch:
concurrency:
group: docker-release-${{ github.ref }}
group: docker-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false
permissions:
contents: read
packages: write
id-token: write
jobs:
build-amd64:
# hanzoai org pool — ARC routes by runnerScaleSetName, not labels.
runs-on: hanzo-build-linux-amd64
docker:
uses: hanzoai/.github/.github/workflows/docker-build.yml@main
with:
image: ghcr.io/hanzoai/chat
platforms: linux/amd64,linux/arm64
secrets: inherit
dispatch-to-universe:
needs: docker
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver: docker-container
driver-opts: network=host
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
- name: Compute image URI
id: meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/hanzoai/chat
tags: |
type=semver,pattern={{version}}
type=sha,prefix=sha-,format=short
flavor: |
latest=false
run: |
IMAGE="ghcr.io/hanzoai/chat"
if [[ "$GITHUB_REF" == refs/tags/v* ]]; then
TAG="${GITHUB_REF#refs/tags/v}"
elif [[ "$GITHUB_REF" == refs/heads/* ]]; then
TAG="${GITHUB_REF#refs/heads/}"
else
TAG="${GITHUB_SHA}"
fi
echo "image=${IMAGE}:${TAG}" >> "$GITHUB_OUTPUT"
- name: Build and push amd64 image
uses: docker/build-push-action@v6
- name: Dispatch image update to universe
uses: peter-evans/repository-dispatch@v3
with:
context: .
file: ./Dockerfile
platforms: linux/amd64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=ghcr.io/hanzoai/chat-cache:amd64
cache-to: type=registry,ref=ghcr.io/hanzoai/chat-cache:amd64,mode=max
provenance: false
token: ${{ secrets.UNIVERSE_DISPATCH_TOKEN }}
repository: hanzoai/universe
event-type: image-update
client-payload: |
{
"service": "chat",
"image": "${{ steps.meta.outputs.image }}",
"sha": "${{ github.sha }}",
"repo": "${{ github.repository }}"
}
- name: Summary
run: |
echo "### Deploy dispatched to universe" >> "$GITHUB_STEP_SUMMARY"
echo "- **Image:** ${{ steps.meta.outputs.image }}" >> "$GITHUB_STEP_SUMMARY"
echo "- **Flow:** image-receiver -> staging -> E2E -> production" >> "$GITHUB_STEP_SUMMARY"
-36
View File
@@ -1,36 +0,0 @@
name: Docker Build Smoke Tests
on:
workflow_dispatch:
pull_request:
paths:
- '.github/workflows/docker-smoke.yml'
- '.dockerignore'
- 'Dockerfile.multi'
- 'package.json'
- 'package-lock.json'
- 'packages/client/**'
- 'packages/data-provider/**'
permissions:
contents: read
jobs:
client-package-target:
name: Build Docker client package target
runs-on: hanzo-build-linux-amd64
timeout-minutes: 25
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build client package target
uses: docker/build-push-action@v5
with:
context: .
file: Dockerfile.multi
platforms: linux/amd64
push: false
target: client-package-build
-74
View File
@@ -1,74 +0,0 @@
name: E2E Tests
on:
pull_request:
branches: [main]
paths:
- 'client/**'
- 'api/**'
- 'packages/**'
- 'e2e/**'
concurrency:
group: e2e-${{ github.ref }}
cancel-in-progress: true
jobs:
e2e:
name: Playwright E2E
if: github.event.pull_request.head.repo.full_name == 'hanzoai/chat'
timeout-minutes: 30
runs-on: hanzo-build-linux-amd64
env:
CI: true
# pnpm run frontend builds packages/api (rollup, ~5 GiB peak) + the client;
# the default ~2 GiB heap OOMs. Match the review workflows' heap knob.
NODE_OPTIONS: '--max-old-space-size=${{ secrets.NODE_MAX_OLD_SPACE_SIZE || 6144 }}'
NODE_ENV: CI
SEARCH: false
MONGO_URI: mongodb://localhost:27017/librechat-test
JWT_SECRET: e2e-test-jwt-secret
JWT_REFRESH_SECRET: e2e-test-jwt-refresh-secret
CREDS_KEY: e2e-test-creds-key-32chars00000000
CREDS_IV: e2e-test-creds-iv-16c
DOMAIN_CLIENT: http://localhost:3080
DOMAIN_SERVER: http://localhost:3080
ALLOW_REGISTRATION: true
TITLE_CONVO: false
services:
mongodb:
image: mongo:7
ports:
- 27017:27017
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build client
run: pnpm run frontend
- name: Install Playwright
run: |
npx playwright install --with-deps chromium
- name: Run E2E tests
run: pnpm run e2e:ci
- name: Upload report
uses: actions/upload-artifact@v4
if: always()
with:
name: playwright-report
path: e2e/playwright-report/
retention-days: 14
+1 -1
View File
@@ -14,7 +14,7 @@ on:
jobs:
eslint_checks:
name: Run ESLint Linting
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
tests_frontend_ubuntu:
name: Run frontend unit tests on Ubuntu
timeout-minutes: 60
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
env:
NODE_OPTIONS: '--max-old-space-size=${{ secrets.NODE_MAX_OLD_SPACE_SIZE || 6144 }}'
steps:
+1 -1
View File
@@ -9,7 +9,7 @@ on:
jobs:
generate:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: supabase/embeddings-generator@v0.0.6
-91
View File
@@ -1,91 +0,0 @@
# Removes a PR's GitNexus index from the droplet when the PR is closed
# (merged or not). The deploy workflow also prunes stale folders as a
# safety net, but this gives us immediate cleanup without waiting for
# the next deploy trigger.
name: GitNexus Cleanup PR
on:
pull_request:
types: [closed]
permissions:
contents: read
actions: read
concurrency:
group: gitnexus-cleanup-pr-${{ github.event.pull_request.number }}
cancel-in-progress: false
jobs:
cleanup:
# Skip fork PRs entirely. GitHub withholds repository secrets from
# pull_request events originating on forks, so an SSH deploy job run
# from a fork close would fail noisily. The deploy workflow's stale-
# folder pruning step catches any fork-contributor indexes that
# actually made it onto the droplet.
if: github.event.pull_request.head.repo.full_name == github.repository
runs-on: hanzo-build-linux-amd64
timeout-minutes: 5
steps:
# Skip the SSH round-trip entirely when no index artifact was ever
# built for this PR (docs-only PRs, paths-ignored PRs, PRs closed
# before indexing finished, etc). Eliminates ~95% of no-op SSH
# sessions on a busy repo.
- name: Check for index artifact
id: check
uses: actions/github-script@v7
with:
script: |
const { data } = await github.rest.actions.listArtifactsForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
name: `gitnexus-index-pr-${context.payload.pull_request.number}`,
per_page: 1,
});
const hasArtifact = data.total_count > 0;
core.info(`Artifact exists: ${hasArtifact}`);
core.setOutput('has_artifact', hasArtifact ? 'true' : 'false');
- name: Setup SSH
if: steps.check.outputs.has_artifact == 'true'
env:
SSH_KEY: ${{ secrets.GITNEXUS_DO_SSH_KEY }}
KNOWN_HOST: ${{ secrets.GITNEXUS_DO_KNOWN_HOST }}
run: |
set -e
mkdir -p ~/.ssh
chmod 700 ~/.ssh
printf '%s\n' "$SSH_KEY" > ~/.ssh/deploy_key
chmod 600 ~/.ssh/deploy_key
if [ -z "$KNOWN_HOST" ]; then
echo "::error::GITNEXUS_DO_KNOWN_HOST secret is empty"
exit 1
fi
printf '%s\n' "$KNOWN_HOST" > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
- name: Remove PR index from droplet
if: steps.check.outputs.has_artifact == 'true'
env:
SSH_USER: ${{ secrets.GITNEXUS_DO_USER }}
SSH_HOST: ${{ secrets.GITNEXUS_DO_HOST }}
PR_NUM: ${{ github.event.pull_request.number }}
run: |
ssh -i ~/.ssh/deploy_key "$SSH_USER@$SSH_HOST" PR_NUM="$PR_NUM" bash <<'REMOTE'
set -e
TARGET="/opt/gitnexus/indexes/LibreChat-pr-$PR_NUM"
if [ -d "$TARGET" ]; then
echo "Removing $TARGET"
rm -rf "$TARGET"
cd /opt/gitnexus
docker compose up -d --force-recreate gitnexus
echo "GitNexus restarted without PR #$PR_NUM"
else
echo "No index to clean up for PR #$PR_NUM (artifact existed but droplet folder did not)"
fi
REMOTE
- name: Cleanup SSH key
if: always()
run: rm -f ~/.ssh/deploy_key
-600
View File
@@ -1,600 +0,0 @@
# Deploys GitNexus indexes to a droplet via SSH + rsync.
#
# Architecture:
# GitHub Actions (deploy)
# 1. Resolves latest successful index runs for main, dev, and every
# open PR that already has an index artifact (contributor-gated
# upstream by the index workflow's author_association check)
# 2. Downloads each matching .gitnexus/ artifact
# 3. Rsyncs them into /opt/gitnexus/indexes/<name>/ on the droplet
# 4. Removes any stale folders on the droplet for PRs that closed
# (even though gitnexus-cleanup-pr.yml also handles that path,
# this is a safety net in case the close event was missed)
# 5. Pulls latest image, force-recreates gitnexus, reloads Caddy,
# and polls docker health until the container reports healthy
# The caddy container is untouched — no TLS churn.
#
# First-time droplet bootstrap (run once, manually):
# 1. Create 2GB+ Ubuntu 24.04 droplet, add SSH key
# 2. Point DNS A record for your subdomain at the droplet IP
# 3. SSH in and run:
# curl -fsSL https://get.docker.com | sh
# systemctl enable --now docker
# mkdir -p /opt/gitnexus/indexes
# useradd -m -s /bin/bash deploy
# usermod -aG docker deploy
# mkdir -p /home/deploy/.ssh
# # Add deploy pubkey to /home/deploy/.ssh/authorized_keys
# chown -R deploy:deploy /home/deploy/.ssh /opt/gitnexus
# chmod 700 /home/deploy/.ssh
# ufw allow 22,80,443/tcp
# ufw --force enable
# 4. Copy .do/gitnexus/docker-compose.yml and Caddyfile into /opt/gitnexus/
# 5. Create /opt/gitnexus/.env with: GITNEXUS_DOMAIN=... and API_TOKEN=...
# 6. cd /opt/gitnexus && docker compose up -d
#
# Then capture the droplet's SSH host key from your workstation and
# save it as the GITNEXUS_DO_KNOWN_HOST secret (below) so CI can pin it:
# ssh-keyscan -H gitnexus.yourdomain.com
#
# GHCR image: the workflow runs `docker login ghcr.io` on the droplet
# on every deploy using GITHUB_TOKEN, so the package can stay private.
# If you'd rather not have CI manage droplet auth, make the package
# public under repo Settings -> Packages.
#
# Required GitHub secrets:
# GITNEXUS_DO_HOST — droplet IP or hostname
# GITNEXUS_DO_USER — SSH user (e.g. "deploy")
# GITNEXUS_DO_SSH_KEY — private key matching the authorized pubkey
# GITNEXUS_DO_KNOWN_HOST — output of `ssh-keyscan -H <host>` pinning the
# droplet's host keys (prevents MITM/TOFU risk)
name: GitNexus Deploy
on:
workflow_run:
workflows: ['GitNexus Index']
types: [completed]
workflow_dispatch:
inputs:
pr_number:
description: 'Optional PR number to post completion comment on (set by bot-triggered dispatches from gitnexus-index.yml)'
type: string
default: ''
permissions:
actions: read
contents: read
pull-requests: write # post completion comments on served PR indexes
# Global serialization. Earlier versions used per-ref concurrency with
# cancel-in-progress so rapid pushes to the same ref coalesced but deploys
# targeting different refs ran in parallel. That had a data race: the
# prune-stale-indexes step computes its active_names up front, so if
# deploy A is rsyncing /opt/gitnexus/indexes/LibreChat-pr-12580 while
# deploy B (started slightly later with a different ref) prunes, B can
# rm -rf a folder A is still uploading into.
#
# All deploys now queue behind a single group. cancel-in-progress is
# false so a running rsync/docker-compose restart never gets killed
# mid-operation (which would leave the droplet in a partial state).
# The 20-minute job timeout bounds total queue depth.
concurrency:
group: gitnexus-deploy
cancel-in-progress: false
env:
GITNEXUS_VERSION: '1.5.3'
IMAGE_NAME: ghcr.io/${{ github.repository_owner }}/librechat-gitnexus
jobs:
# Rebuilds the long-lived image only when Dockerfile/entrypoint/extensions
# change. Skipped on every other run, so index-only deploys are fast.
build-image:
if: |
github.event_name == 'workflow_dispatch' ||
github.event.workflow_run.conclusion == 'success'
runs-on: hanzo-build-linux-amd64
timeout-minutes: 20
permissions:
contents: read
packages: write # push image to GHCR
outputs:
image_tag: ${{ steps.tag.outputs.value }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 2
- name: Detect image changes
id: changes
run: |
# Default to rebuild when we can't cleanly diff (first commit,
# workflow_run from a PR branch where HEAD isn't the trigger, etc).
# Rebuild on miss > skip when we should have rebuilt.
if git rev-parse --verify HEAD~1 >/dev/null 2>&1 && \
git diff --quiet HEAD~1 HEAD -- .do/gitnexus/Dockerfile .do/gitnexus/entrypoint.sh .do/gitnexus/install-extensions.js; then
echo "changed=false" >> "$GITHUB_OUTPUT"
else
echo "changed=true" >> "$GITHUB_OUTPUT"
fi
- name: Compute image tag
id: tag
run: echo "value=v${{ env.GITNEXUS_VERSION }}" >> "$GITHUB_OUTPUT"
- name: Set up Docker Buildx
if: steps.changes.outputs.changed == 'true' || github.event_name == 'workflow_dispatch'
uses: docker/setup-buildx-action@v3
- name: Log in to GHCR
if: steps.changes.outputs.changed == 'true' || github.event_name == 'workflow_dispatch'
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push image
if: steps.changes.outputs.changed == 'true' || github.event_name == 'workflow_dispatch'
uses: docker/build-push-action@v5
with:
context: .do/gitnexus
file: .do/gitnexus/Dockerfile
push: true
tags: |
${{ env.IMAGE_NAME }}:latest
${{ env.IMAGE_NAME }}:${{ steps.tag.outputs.value }}
build-args: |
GITNEXUS_VERSION=${{ env.GITNEXUS_VERSION }}
cache-from: type=gha
cache-to: type=gha,mode=max
deploy:
needs: build-image
runs-on: hanzo-deploy-linux-amd64
timeout-minutes: 20
permissions:
actions: read
contents: read
pull-requests: write # post deploy-complete comments on served PR indexes
steps:
- name: Checkout deploy config
uses: actions/checkout@v4
with:
sparse-checkout: .do/gitnexus
fetch-depth: 1
# Resolve every index to serve. All resolutions go through
# listArtifactsForRepo keyed by the expected artifact name, so a
# run's branch or event type doesn't matter — we always pick the
# freshest artifact that actually exists.
#
# Why this matters: a /gitnexus index command dispatches
# gitnexus-index.yml with ref=main and an input pr_number, which
# produces a run whose head_branch is "main" but whose artifact
# is gitnexus-index-pr-<N>. listWorkflowRuns(branch='main') would
# happily return that run, and we'd then try to download a
# nonexistent gitnexus-index-main artifact from it. Querying by
# artifact name directly avoids the whole mess.
- name: Resolve indexes to serve
id: resolve
uses: actions/github-script@v7
with:
script: |
const serve = []; // [{ name, artifactName, runId }]
// Helper — pick the newest non-expired artifact matching a name.
const latestArtifact = async (artifactName) => {
const { data } = await github.rest.actions.listArtifactsForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
name: artifactName,
per_page: 10,
});
return data.artifacts
.filter((a) => !a.expired)
.sort((a, b) => new Date(b.created_at) - new Date(a.created_at))[0];
};
// --- main and dev branches ---
for (const [branch, name] of [
['main', 'LibreChat'],
['dev', 'LibreChat-dev'],
]) {
const artifactName = `gitnexus-index-${branch}`;
const fresh = await latestArtifact(artifactName);
if (!fresh) {
core.warning(`No artifact found for ${branch} (expected ${artifactName})`);
continue;
}
serve.push({
name,
artifactName,
runId: fresh.workflow_run.id,
});
core.info(`${branch}: run ${fresh.workflow_run.id} -> ${name}`);
}
// --- open PRs with at least one successful index run ---
// github.paginate handles the 100-per-page ceiling automatically
// so the resolution works on repos with 200+ concurrent open PRs.
const openPrs = await github.paginate(github.rest.pulls.list, {
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
per_page: 100,
});
core.info(`Found ${openPrs.length} open PRs`);
// Parallelize artifact lookups in fixed-size batches so the
// resolve step runs in seconds instead of minutes on big repos,
// without burning the GitHub API rate limit all at once.
const BATCH_SIZE = 10;
const prMatches = [];
for (let i = 0; i < openPrs.length; i += BATCH_SIZE) {
const batch = openPrs.slice(i, i + BATCH_SIZE);
const results = await Promise.all(
batch.map(async (pr) => {
const artifactName = `gitnexus-index-pr-${pr.number}`;
const fresh = await latestArtifact(artifactName);
return fresh ? { pr, artifactName, fresh } : null;
}),
);
for (const hit of results) {
if (hit) prMatches.push(hit);
}
}
// Cap to the N most recent PR indexes by artifact creation time.
// On a 10GB droplet each index is ~130MB; 3 PRs + main + dev ≈
// 650MB of index data, leaving headroom for the ~700MB Docker image
// and OS. Older PR indexes are evicted by the prune step.
const MAX_PR_INDEXES = 3;
prMatches.sort(
(a, b) => new Date(b.fresh.created_at) - new Date(a.fresh.created_at),
);
const keptPrs = prMatches.slice(0, MAX_PR_INDEXES);
const evictedPrs = prMatches.slice(MAX_PR_INDEXES);
for (const { pr, artifactName, fresh } of keptPrs) {
serve.push({
name: `LibreChat-pr-${pr.number}`,
artifactName,
runId: fresh.workflow_run.id,
});
core.info(`PR #${pr.number}: run ${fresh.workflow_run.id} -> LibreChat-pr-${pr.number}`);
}
if (evictedPrs.length) {
core.info(
`Evicted ${evictedPrs.length} older PR indexes (cap=${MAX_PR_INDEXES}): ` +
evictedPrs.map((e) => `#${e.pr.number}`).join(', '),
);
}
core.info(`Serving ${keptPrs.length} PR indexes out of ${prMatches.length} with artifacts (${openPrs.length} open PRs total)`);
if (!serve.length) {
core.setFailed('No indexes to serve');
return;
}
core.setOutput('matrix', JSON.stringify(serve));
core.setOutput('active_names', serve.map((s) => s.name).join(','));
- name: Download each index artifact
env:
MATRIX: ${{ steps.resolve.outputs.matrix }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -e
mkdir -p staging
# main/dev artifact download failures are fatal — a missing
# main/dev index is a real deploy failure. PR artifact failures
# are soft — a PR artifact deleted mid-deploy shouldn't abort
# the whole deploy and take main/dev down with it.
echo "$MATRIX" | jq -c '.[]' | while read -r entry; do
name=$(echo "$entry" | jq -r '.name')
artifact=$(echo "$entry" | jq -r '.artifactName')
runId=$(echo "$entry" | jq -r '.runId')
target="staging/${name}/.gitnexus"
echo "Downloading $artifact from run $runId -> $target"
mkdir -p "$target"
if ! gh run download "$runId" \
--repo "${{ github.repository }}" \
--name "$artifact" \
--dir "$target"; then
case "$name" in
LibreChat|LibreChat-dev)
echo "::error::Failed to download critical artifact $artifact"
exit 1
;;
*)
# The name stays in active_names so the prune step
# won't remove the droplet's existing copy. The old
# index keeps being served instead of being wiped to
# nothing — stale beats empty — but observability
# requires an explicit notice since this path is
# invisible in the happy-path deploy log.
echo "::warning::Failed to download PR artifact $artifact — skipping fresh sync; previous index (if any) will continue being served from the droplet"
rm -rf "staging/${name}"
;;
esac
fi
done
echo ""
echo "Staged for rsync:"
du -sh staging/*/.gitnexus/ 2>/dev/null || echo "(none)"
- name: Setup SSH
env:
SSH_KEY: ${{ secrets.GITNEXUS_DO_SSH_KEY }}
KNOWN_HOST: ${{ secrets.GITNEXUS_DO_KNOWN_HOST }}
run: |
set -e
mkdir -p ~/.ssh
chmod 700 ~/.ssh
printf '%s\n' "$SSH_KEY" > ~/.ssh/deploy_key
chmod 600 ~/.ssh/deploy_key
# Pin the droplet's SSH host key from a repository secret instead
# of trusting whatever ssh-keyscan returns at deploy time. The
# secret is populated from `ssh-keyscan -H <host>` at bootstrap.
if [ -z "$KNOWN_HOST" ]; then
echo "::error::GITNEXUS_DO_KNOWN_HOST secret is empty. Run ssh-keyscan -H <host> and paste the output as this secret."
exit 1
fi
printf '%s\n' "$KNOWN_HOST" > ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
- name: Authenticate droplet with GHCR
# GHCR packages pushed by GITHUB_TOKEN start private. The droplet
# pulls the image on every deploy, so we re-authenticate it here
# using the same short-lived token. If the package is public, this
# step is redundant but harmless.
#
# The token MUST travel through SSH stdin (not as a command arg)
# so it's never visible in the droplet's process table via
# /proc/<pid>/cmdline. `printf '%s'` is preferred over `echo`
# so the exact byte sequence sent is explicit — docker login
# tolerates a trailing newline but `printf` makes the intent
# obvious and portable across shells.
env:
SSH_USER: ${{ secrets.GITNEXUS_DO_USER }}
SSH_HOST: ${{ secrets.GITNEXUS_DO_HOST }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_ACTOR: ${{ github.actor }}
run: |
printf '%s' "$GH_TOKEN" | ssh -i ~/.ssh/deploy_key "$SSH_USER@$SSH_HOST" \
"docker login ghcr.io -u '$GH_ACTOR' --password-stdin"
- name: Upload config files
env:
SSH_USER: ${{ secrets.GITNEXUS_DO_USER }}
SSH_HOST: ${{ secrets.GITNEXUS_DO_HOST }}
run: |
rsync -az -e "ssh -i ~/.ssh/deploy_key" \
.do/gitnexus/docker-compose.yml \
.do/gitnexus/Caddyfile \
"$SSH_USER@$SSH_HOST:/opt/gitnexus/"
- name: Prune stale indexes then sync fresh ones
env:
SSH_USER: ${{ secrets.GITNEXUS_DO_USER }}
SSH_HOST: ${{ secrets.GITNEXUS_DO_HOST }}
ACTIVE_NAMES: ${{ steps.resolve.outputs.active_names }}
run: |
set -e
# ── Step 1: prune FIRST ────────────────────────────────
# Remove any folders on the droplet that aren't in the active set.
# This frees disk BEFORE rsyncing new data, which matters on a
# 10GB disk where each index is ~130MB.
echo "Pruning stale indexes (keeping: $ACTIVE_NAMES)"
ssh -i ~/.ssh/deploy_key "$SSH_USER@$SSH_HOST" \
ACTIVE_NAMES="$ACTIVE_NAMES" bash <<'REMOTE'
set -e
cd /opt/gitnexus/indexes || exit 0
shopt -s nullglob
IFS=',' read -ra ACTIVE <<< "$ACTIVE_NAMES"
for dir in */; do
dir="${dir%/}"
keep=false
for a in "${ACTIVE[@]}"; do
if [ "$dir" = "$a" ]; then keep=true; break; fi
done
if [ "$keep" = false ]; then
echo "Removing stale index: $dir"
rm -rf "$dir"
fi
done
echo "Disk after prune:"
df -h / | tail -1
REMOTE
# ── Step 2: rsync-then-swap ─────────────────────────────
# Upload each index to a temp directory, then atomically swap
# it into place. If rsync fails, the old index survives intact
# and the partial temp dir is cleaned up — no production data
# is lost. The brief period where both old + new exist costs
# ~130MB of extra disk, but the prune step already freed
# space from evicted PR indexes so this fits on a 10GB disk.
for dir in staging/*/; do
[ -d "$dir" ] || continue
name=$(basename "$dir")
echo "Syncing $name (rsync-then-swap)"
ssh -i ~/.ssh/deploy_key "$SSH_USER@$SSH_HOST" \
"mkdir -p /opt/gitnexus/indexes/${name}.new"
if rsync -az -e "ssh -i ~/.ssh/deploy_key" \
"$dir" \
"$SSH_USER@$SSH_HOST:/opt/gitnexus/indexes/${name}.new/"; then
# Swap: remove old, rename new into place
ssh -i ~/.ssh/deploy_key "$SSH_USER@$SSH_HOST" \
"rm -rf /opt/gitnexus/indexes/$name && mv /opt/gitnexus/indexes/${name}.new /opt/gitnexus/indexes/$name"
echo " $name swapped successfully"
else
# Clean up the partial temp dir
ssh -i ~/.ssh/deploy_key "$SSH_USER@$SSH_HOST" \
"rm -rf /opt/gitnexus/indexes/${name}.new"
# main/dev are critical — abort the deploy so the failure
# is visible and the container isn't restarted with stale
# or missing data. PR indexes are best-effort.
case "$name" in
LibreChat|LibreChat-dev)
echo "::error::rsync failed for critical index $name — aborting deploy"
exit 1
;;
*)
echo "::warning::rsync failed for PR index $name — keeping previous index"
;;
esac
fi
done
- name: Pull image, restart gitnexus, reload Caddy, wait for healthy
env:
SSH_USER: ${{ secrets.GITNEXUS_DO_USER }}
SSH_HOST: ${{ secrets.GITNEXUS_DO_HOST }}
run: |
ssh -i ~/.ssh/deploy_key "$SSH_USER@$SSH_HOST" bash <<'REMOTE'
set -e
cd /opt/gitnexus
# ── Disk cleanup ──────────────────────────────────────
# Docker accumulates old image layers, dangling images, and
# build cache across deploys. On a 60GB droplet with a 700MB+
# gitnexus image, this fills the disk after ~40 deploys.
# Prune everything not used by currently-running containers
# BEFORE pulling the new image so the extract has room.
echo "Disk before cleanup:"
df -h / | tail -1
# Omit --volumes: Caddy's caddy-data and caddy-config volumes
# hold TLS certificates and ACME state. If Caddy happens to be
# stopped when this runs (the workflow handles that case later),
# --volumes would wipe them, forcing Let's Encrypt re-issuance
# and risking rate-limit lockout (5 certs/domain/week).
docker system prune -af 2>/dev/null || true
echo "Disk after cleanup:"
df -h / | tail -1
# Fail fast if disk is critically low even after prune
AVAIL_MB=$(df --output=avail -m / | tail -1 | tr -d ' ')
if [ "$AVAIL_MB" -lt 2048 ]; then
echo "::error::Disk critically low (${AVAIL_MB}MB free). Aborting deploy."
exit 1
fi
docker compose pull gitnexus
docker compose up -d --force-recreate gitnexus
# Reload Caddy in-place so a changed Caddyfile takes effect
# without losing TLS certs or restarting connections. If caddy
# isn't running yet (first-time bootstrap), bring it up.
if docker compose ps --status running caddy 2>/dev/null | grep -q caddy; then
echo "Reloading Caddy config"
docker compose exec -T caddy caddy reload --config /etc/caddy/Caddyfile || {
echo "Caddy reload failed — forcing restart"
docker compose up -d --force-recreate caddy
}
else
echo "Caddy not running — starting"
docker compose up -d caddy
fi
# Poll gitnexus health until ready or timeout. Docker's own
# unhealthy detection takes up to 150s (start_period 60s +
# retries 3 * interval 30s), so the poll ceiling must clear
# that to avoid false negatives when gitnexus legitimately
# takes ~2.5 min to warm up.
# Max wait = 36 sleeps * 5s = 180s (final iteration exits
# before its sleep on failure, so 37 iterations is the
# correct upper bound for a true 180s ceiling).
echo "Waiting for gitnexus to report healthy..."
for i in $(seq 1 37); do
STATUS=$(docker inspect --format='{{.State.Health.Status}}' gitnexus 2>/dev/null || echo unknown)
echo "[$i/37] gitnexus health: $STATUS"
if [ "$STATUS" = "healthy" ]; then
echo "gitnexus is healthy"
break
fi
if [ "$i" -eq 37 ]; then
echo "ERROR: gitnexus failed to become healthy after 180s"
docker compose ps
docker compose logs --tail 80 gitnexus
exit 1
fi
sleep 5
done
docker compose ps
echo "--- Caddy logs (last 20 lines) ---"
docker compose logs --tail 20 caddy || true
echo "--- GitNexus logs (last 30 lines) ---"
docker compose logs --tail 30 gitnexus || true
REMOTE
# When the deploy was triggered by a PR command path, post a
# terminal status comment on that one PR only. Two sub-cases:
#
# 1. workflow_run trigger: the PR's native auto-index run fired
# workflow_run, so github.event.workflow_run.id is the trigger.
# Find the matching PR via the matrix entry whose runId matches.
#
# 2. workflow_dispatch trigger with inputs.pr_number set: the
# index workflow's bot-fallback path dispatched us directly
# because workflow_run is suppressed for GITHUB_TOKEN triggers.
# Use inputs.pr_number as the comment target.
#
# Broadcast-commenting on every active PR would be noise — only the
# PR that asked for a fresh index gets a reply.
- name: Comment on PR — deploy complete
if: always()
uses: actions/github-script@v7
env:
MATRIX: ${{ steps.resolve.outputs.matrix }}
TRIGGER_RUN_ID: ${{ github.event.workflow_run.id }}
DISPATCH_PR_NUMBER: ${{ github.event.inputs.pr_number }}
DEPLOY_STATUS: ${{ job.status }}
with:
script: |
let prNum = null;
// Case 1: dispatched directly with pr_number (bot-fallback path)
if (process.env.DISPATCH_PR_NUMBER && process.env.DISPATCH_PR_NUMBER !== '') {
prNum = parseInt(process.env.DISPATCH_PR_NUMBER, 10);
}
// Case 2: workflow_run trigger from a PR index run
else if (context.eventName === 'workflow_run') {
const matrix = JSON.parse(process.env.MATRIX || '[]');
const triggerRunId = Number(process.env.TRIGGER_RUN_ID);
const match = matrix.find(
(m) => m.runId === triggerRunId && m.name.startsWith('LibreChat-pr-'),
);
if (match) {
prNum = parseInt(match.name.replace('LibreChat-pr-', ''), 10);
}
}
if (!prNum) {
core.info('No PR to comment on (trigger was not a PR-scoped index); skipping.');
return;
}
const deployUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
const ok = process.env.DEPLOY_STATUS === 'success';
const body = [
`### GitNexus: ${ok ? '🚀 deployed' : '❌ deploy failed'}`,
'',
ok
? `The \`LibreChat-pr-${prNum}\` index is now live on the MCP server.`
: `The deploy failed — the previous index (if any) continues to be served.`,
`[Deploy run](${deployUrl})`,
].join('\n');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNum,
body,
});
- name: Cleanup SSH key
if: always()
run: rm -f ~/.ssh/deploy_key
-310
View File
@@ -1,310 +0,0 @@
name: GitNexus Index
on:
push:
branches: [main, dev]
paths-ignore: ['**.md', 'docs/**', 'LICENSE', '.github/**']
pull_request:
branches: [main, dev]
paths-ignore: ['**.md', 'docs/**', 'LICENSE', '.github/**']
workflow_dispatch:
inputs:
embeddings:
description: 'Enable embedding generation (slow, increases index size)'
type: boolean
default: false
force:
description: 'Force full re-index'
type: boolean
default: false
# When invoked from the /gitnexus index PR command, the command
# workflow fills these so the index is built from the PR's head
# ref and uploaded under the PR-numbered artifact name.
pr_number:
description: 'PR number to index (set by /gitnexus command)'
type: string
default: ''
pr_ref:
description: 'Optional PR head ref to check out; defaults to refs/pull/<pr_number>/head when pr_number is set'
type: string
default: ''
deploy_after:
description: 'Dispatch GitNexus Deploy after a successful index run'
type: boolean
default: false
permissions:
contents: read
concurrency:
# When triggered by the /gitnexus command, group by PR number so rapid
# re-runs coalesce. Otherwise group by git ref as before.
group: gitnexus-${{ inputs.pr_number != '' && format('pr-{0}', inputs.pr_number) || github.ref }}
cancel-in-progress: true
env:
GITNEXUS_VERSION: '1.5.3'
jobs:
index:
permissions:
contents: read
pull-requests: read # read changed files to decide whether embeddings are needed
# Push + dispatch run unconditionally. Native pull_request events
# are restricted to PRs authored by danny-avila only — this keeps
# automatic CI spend low on a repo with 200+ open PRs.
#
# Other contributors' PRs can still be indexed on demand:
# - /gitnexus index (PR comment command, contributor-gated)
# - workflow_dispatch (manual dispatch from Actions UI)
# Both bypass this filter because they arrive as workflow_dispatch,
# not pull_request.
if: |
github.event_name != 'pull_request' ||
github.event.pull_request.user.login == 'danny-avila'
runs-on: hanzo-build-linux-amd64
timeout-minutes: 25
steps:
- name: Validate dispatch inputs
if: github.event_name == 'workflow_dispatch'
env:
PR_NUMBER: ${{ inputs.pr_number }}
PR_REF: ${{ inputs.pr_ref }}
run: |
set -euo pipefail
if [ -n "$PR_NUMBER" ]; then
if [[ ! "$PR_NUMBER" =~ ^[0-9]+$ ]]; then
echo "::error::pr_number must be numeric"
exit 1
fi
EXPECTED_REF="refs/pull/${PR_NUMBER}/head"
if [ -n "$PR_REF" ] && [ "$PR_REF" != "$EXPECTED_REF" ]; then
echo "::error::pr_ref must match ${EXPECTED_REF}"
exit 1
fi
elif [ -n "$PR_REF" ]; then
echo "::error::pr_ref requires pr_number"
exit 1
fi
- name: Resolve GitNexus flags
id: flags
env:
EVENT_NAME: ${{ github.event_name }}
ENABLE_EMBEDDINGS_INPUT: ${{ inputs.embeddings }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUM: ${{ github.event.pull_request.number }}
run: |
set -euo pipefail
# Decide whether to generate embeddings. Rules:
# push (main/dev) -> always embed
# pull_request -> embed ONLY when the PR changes files
# under paths that also trigger backend
# or frontend unit tests (api/, client/,
# packages/). Docs/config-only PRs skip
# embeddings to save ~3-5 min of CI.
# workflow_dispatch -> respect the explicit `embeddings` input
# (default false). This also covers the
# /gitnexus index [embeddings] command.
ENABLE_EMBEDDINGS=false
case "$EVENT_NAME" in
workflow_dispatch)
[ "$ENABLE_EMBEDDINGS_INPUT" = "true" ] && ENABLE_EMBEDDINGS=true
;;
push)
ENABLE_EMBEDDINGS=true
;;
pull_request)
CHANGED=$(gh api "repos/${{ github.repository }}/pulls/$PR_NUM/files" \
--paginate --jq '.[].filename' 2>/dev/null || echo "")
if printf '%s\n' "$CHANGED" | grep -qE '^(api/|client/|packages/)'; then
echo "PR #$PR_NUM touches unit-test paths (api|client|packages) — enabling embeddings"
ENABLE_EMBEDDINGS=true
else
echo "PR #$PR_NUM does not touch unit-test paths — graph-only index"
fi
;;
esac
if [ "$ENABLE_EMBEDDINGS" = "true" ]; then
echo "enable_embeddings=true" >> "$GITHUB_OUTPUT"
else
echo "enable_embeddings=false" >> "$GITHUB_OUTPUT"
fi
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '24.16.0'
- name: Install GitNexus CLI
working-directory: ${{ runner.temp }}
env:
NPM_CONFIG_AUDIT: false
NPM_CONFIG_CACHE: ${{ runner.temp }}/gitnexus-npm-cache
NPM_CONFIG_FUND: false
NPM_CONFIG_GLOBALCONFIG: ${{ runner.temp }}/gitnexus-cli/global-npmrc
NPM_CONFIG_REGISTRY: https://registry.npmjs.org/
NPM_CONFIG_USERCONFIG: ${{ runner.temp }}/gitnexus-cli/.npmrc
run: |
set -euo pipefail
mkdir -p "$RUNNER_TEMP/gitnexus-cli" "$RUNNER_TEMP/gitnexus-npm-cache"
: > "$RUNNER_TEMP/gitnexus-cli/global-npmrc"
printf '%s\n' \
'registry=https://registry.npmjs.org/' \
'audit=false' \
'fund=false' \
> "$RUNNER_TEMP/gitnexus-cli/.npmrc"
# Keep GitNexus' native DB dependency deterministic in fresh CI installs.
npm install \
--prefix "$RUNNER_TEMP/gitnexus-cli" \
--no-save \
--no-package-lock \
"gitnexus@${{ env.GITNEXUS_VERSION }}" \
"@ladybugdb/core@0.15.2"
test -x "$RUNNER_TEMP/gitnexus-cli/node_modules/.bin/gitnexus"
- name: Checkout repository
uses: actions/checkout@v4
with:
# When the /gitnexus command dispatches us with a pr_ref, it's
# a refs/pull/<N>/head ref that GitHub mirrors into the base
# repo for every PR, so checkout works for fork PRs too. When
# pr_ref is empty (native push/pull_request), fall back to the
# default ref actions/checkout would use.
ref: ${{ inputs.pr_ref || (inputs.pr_number != '' && format('refs/pull/{0}/head', inputs.pr_number) || '') }}
fetch-depth: 1
persist-credentials: false
- name: Run GitNexus Analyze
working-directory: ${{ runner.temp }}
env:
ENABLE_EMBEDDINGS: ${{ steps.flags.outputs.enable_embeddings }}
FORCE: ${{ inputs.force }}
GITNEXUS_BIN: ${{ runner.temp }}/gitnexus-cli/node_modules/.bin/gitnexus
NPM_CONFIG_AUDIT: false
NPM_CONFIG_CACHE: ${{ runner.temp }}/gitnexus-npm-cache
NPM_CONFIG_FUND: false
NPM_CONFIG_GLOBALCONFIG: ${{ runner.temp }}/gitnexus-cli/global-npmrc
NPM_CONFIG_REGISTRY: https://registry.npmjs.org/
NPM_CONFIG_USERCONFIG: ${{ runner.temp }}/gitnexus-cli/.npmrc
run: |
set -euo pipefail
FLAGS=(--skip-agents-md --verbose)
if [ "$ENABLE_EMBEDDINGS" = "true" ]; then
FLAGS+=(--embeddings)
fi
if [ "$FORCE" = "true" ]; then
FLAGS+=(--force)
fi
"$GITNEXUS_BIN" analyze "$GITHUB_WORKSPACE" "${FLAGS[@]}"
- name: Verify index
run: |
if [ ! -d ".gitnexus" ] || [ ! -f ".gitnexus/meta.json" ]; then
echo "::error::GitNexus index was not created"
exit 1
fi
echo "::group::Index metadata"
cat .gitnexus/meta.json
echo ""
echo "::endgroup::"
- name: Upload GitNexus index
uses: actions/upload-artifact@v4
with:
# Artifact naming order of precedence:
# 1. /gitnexus command dispatch: inputs.pr_number -> pr-<N>
# 2. Native pull_request event: github.event.pull_request.number
# 3. Push or manual dispatch without pr_number: github.ref_name
name: >-
gitnexus-index-${{
inputs.pr_number != ''
&& format('pr-{0}', inputs.pr_number)
|| (github.event_name == 'pull_request'
&& format('pr-{0}', github.event.pull_request.number)
|| github.ref_name)
}}
path: .gitnexus/
include-hidden-files: true
retention-days: 30
post-index:
needs: index
if: |
always() &&
(inputs.pr_number != '' ||
inputs.deploy_after)
runs-on: hanzo-build-linux-amd64
timeout-minutes: 5
permissions:
contents: read
actions: write # dispatch gitnexus-deploy.yml when deploy_after is set
pull-requests: write # post completion comments for /gitnexus command runs
steps:
# GitHub suppresses workflow_run events for workflow runs triggered
# by GITHUB_TOKEN (to prevent recursive chaining). Command-triggered
# index runs opt into a deploy by setting deploy_after=true.
- name: Trigger deploy workflow after command-triggered runs
if: inputs.deploy_after && needs.index.result == 'success'
uses: actions/github-script@v7
env:
PR_NUMBER: ${{ inputs.pr_number }}
with:
script: |
core.info('deploy_after=true; dispatching gitnexus-deploy.yml manually.');
// Pass pr_number through so the deploy workflow knows which
// PR to post its completion comment on (for /gitnexus
// command runs this will be set; for other bot dispatches
// it's empty and the deploy step falls back to matrix match).
await github.rest.actions.createWorkflowDispatch({
owner: context.repo.owner,
repo: context.repo.repo,
workflow_id: 'gitnexus-deploy.yml',
ref: 'main',
inputs: {
pr_number: process.env.PR_NUMBER || '',
},
});
# Reply on the PR when the /gitnexus command path runs so the
# requester knows the index step finished. This fires when
# inputs.pr_number is set and reports the index job result. A
# separate comment posts from the deploy workflow when the live
# server has the fresh index.
- name: Comment on PR — index complete
if: inputs.pr_number != ''
uses: actions/github-script@v7
env:
EMBEDDINGS_INPUT: ${{ inputs.embeddings }}
INDEX_RESULT: ${{ needs.index.result }}
PR_NUMBER: ${{ inputs.pr_number }}
with:
script: |
const indexSucceeded = process.env.INDEX_RESULT === 'success';
const outcome = indexSucceeded ? '✅ indexed' : '❌ index failed';
const prNum = parseInt(process.env.PR_NUMBER || '', 10);
if (!Number.isSafeInteger(prNum)) {
core.setFailed(`Invalid PR number: ${process.env.PR_NUMBER}`);
return;
}
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
const embeddingsFlag = process.env.EMBEDDINGS_INPUT === 'true' ? 'with embeddings' : 'graph-only';
const body = [
`### GitNexus: ${outcome}`,
``,
`PR #${prNum} was indexed ${embeddingsFlag}.`,
`[Index run](${runUrl})`,
'',
indexSucceeded
? '⏳ Waiting for deploy to serve the fresh index…'
: '_Index run failed — the previous index (if any) continues to be served._',
].join('\n');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNum,
body,
});
-141
View File
@@ -1,141 +0,0 @@
# Responds to `/gitnexus index` comments on pull requests.
#
# Gated to the same author_association roles (OWNER, MEMBER, COLLABORATOR)
# as the automatic PR index trigger, but applied to the COMMENTER, not
# the PR author. This intentionally lets a contributor index a PR from
# a non-contributor / first-time fork author — the contributor takes
# responsibility for the trust boundary by typing the command.
#
# When a matching comment lands on a PR, this workflow dispatches
# `gitnexus-index.yml` with the PR number and the `refs/pull/<N>/head`
# ref so indexing works for fork PRs too (GitHub mirrors every PR's
# head ref into the base repo regardless of which fork it originated
# from, so actions/checkout can always resolve it).
#
# Use cases:
# - Re-index a PR after a rebase without pushing a new commit
# - Index a docs-only PR that was skipped by paths-ignore
# - Index a non-contributor (fork) PR that the auto-trigger skipped
# - Re-run a failed index
#
# Supported commands:
# /gitnexus index — index the PR with embeddings (default)
# /gitnexus index embeddings — explicit form of the above; same effect
# /gitnexus index fast — graph-only index (skip embeddings), for
# a quick re-index without waiting ~5 min
# of embedding generation
name: GitNexus PR Command
on:
issue_comment:
types: [created]
permissions:
contents: read
pull-requests: write
actions: write # needed to dispatch gitnexus-index.yml
concurrency:
group: gitnexus-pr-command-${{ github.event.issue.number }}
cancel-in-progress: false
jobs:
dispatch:
# Only run for PR comments that start with /gitnexus from trusted
# commenters. Intentionally checks the COMMENTER's association so a
# contributor can index a non-contributor's PR on demand.
if: |
github.event.issue.pull_request != null &&
startsWith(github.event.comment.body, '/gitnexus') &&
(github.event.comment.author_association == 'OWNER' ||
github.event.comment.author_association == 'MEMBER' ||
github.event.comment.author_association == 'COLLABORATOR')
runs-on: hanzo-build-linux-amd64
timeout-minutes: 5
steps:
- name: Parse command and resolve PR head ref
id: parse
uses: actions/github-script@v7
with:
script: |
const body = context.payload.comment.body.trim();
const match = body.match(/^\/gitnexus\s+(\w+)(?:\s+(\w+))?/);
if (!match) {
core.setFailed(`Unrecognized command: ${body}. Try: /gitnexus index [fast]`);
return;
}
const [, subcommand, modifier] = match;
if (subcommand !== 'index') {
core.setFailed(`Unknown subcommand: ${subcommand}. Only 'index' is supported.`);
return;
}
// Default to embeddings on — a contributor typing the command
// has already decided they want a full re-index. The `fast`
// modifier is the explicit opt-out for graph-only runs.
// `embeddings` is accepted as a no-op alias for backwards
// compat with the previous command form.
let embeddings = 'true';
if (modifier === 'fast' || modifier === 'graph-only' || modifier === 'no-embeddings') {
embeddings = 'false';
}
// Use refs/pull/<N>/head instead of the raw head SHA. GitHub
// mirrors every PR's head into the base repo as this ref, so
// actions/checkout can always resolve it — even for PRs from
// forks whose raw SHAs don't exist in the base repo.
const prNum = context.payload.issue.number;
core.setOutput('pr_number', String(prNum));
core.setOutput('pr_ref', `refs/pull/${prNum}/head`);
core.setOutput('embeddings', embeddings);
core.info(
`Dispatching index for PR #${prNum} at refs/pull/${prNum}/head (embeddings=${embeddings}, modifier=${modifier || '(none)'})`,
);
- name: Dispatch gitnexus-index workflow
uses: actions/github-script@v7
env:
EMBEDDINGS: ${{ steps.parse.outputs.embeddings }}
PR_NUMBER: ${{ steps.parse.outputs.pr_number }}
PR_REF: ${{ steps.parse.outputs.pr_ref }}
with:
script: |
const prNumber = process.env.PR_NUMBER || '';
const prRef = process.env.PR_REF || '';
const embeddings = process.env.EMBEDDINGS || 'false';
if (!/^[0-9]+$/.test(prNumber)) {
core.setFailed(`Invalid PR number: ${prNumber}`);
return;
}
if (prRef !== `refs/pull/${prNumber}/head`) {
core.setFailed(`Invalid PR ref: ${prRef}`);
return;
}
if (!['true', 'false'].includes(embeddings)) {
core.setFailed(`Invalid embeddings value: ${embeddings}`);
return;
}
await github.rest.actions.createWorkflowDispatch({
owner: context.repo.owner,
repo: context.repo.repo,
workflow_id: 'gitnexus-index.yml',
ref: 'main',
inputs: {
pr_number: prNumber,
pr_ref: prRef,
embeddings,
force: 'false',
deploy_after: 'true',
},
});
- name: React to the comment
uses: actions/github-script@v7
with:
script: |
await github.rest.reactions.createForIssueComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: context.payload.comment.id,
content: 'rocket',
});
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
permissions:
contents: write
packages: write
runs-on: hanzo-deploy-linux-amd64
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
+23 -24
View File
@@ -16,7 +16,7 @@ on:
jobs:
detect-unused-i18n-keys:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
@@ -119,31 +119,30 @@ jobs:
echo "unused_keys=[]" >> $GITHUB_ENV
fi
# Post via github-script (Octokit + the built-in token) rather than the gh
# CLI: gh is not installed on the self-hosted ARC runners, so `gh api` here
# exits 127. The job already grants `pull-requests: write`.
- name: Post verified comment on PR
if: env.unused_keys != '[]'
uses: actions/github-script@v7
with:
script: |
const keys = JSON.parse(process.env.unused_keys || '[]').filter(Boolean);
const list = keys.map((k) => `- [ ] \`${k}\``).join('\n');
const body = [
'### 🚨 Unused i18next Keys Detected',
'',
'The following translation keys are defined in `translation.json` but are **not used** in the codebase:',
'',
list,
'',
'⚠️ **Please remove these unused keys to keep the translation files clean.**',
].join('\n');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
run: |
PR_NUMBER=$(jq --raw-output .pull_request.number "$GITHUB_EVENT_PATH")
# Format the unused keys list as checkboxes for easy manual checking.
FILTERED_KEYS=$(echo "$unused_keys" | jq -r '.[]' | grep -v '^\s*$' | sed 's/^/- [ ] `/;s/$/`/' )
COMMENT_BODY=$(cat <<EOF
### 🚨 Unused i18next Keys Detected
The following translation keys are defined in \`translation.json\` but are **not used** in the codebase:
$FILTERED_KEYS
⚠️ **Please remove these unused keys to keep the translation files clean.**
EOF
)
gh api "repos/${{ github.repository }}/issues/${PR_NUMBER}/comments" \
-f body="$COMMENT_BODY" \
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Fail workflow if unused keys found
if: env.unused_keys != '[]'
+1 -1
View File
@@ -7,7 +7,7 @@ on:
jobs:
add-mlops-label:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
steps:
- name: Check if ML Ops Team is selected
uses: actions-ecosystem/action-add-labels@v1
@@ -13,7 +13,7 @@ on:
jobs:
run-llm-translation-tests:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
timeout-minutes: 90
steps:
+2 -2
View File
@@ -9,7 +9,7 @@ on:
jobs:
sync-translations:
name: Sync Translation Keys with Locize
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@v6
@@ -36,7 +36,7 @@ jobs:
create-pull-request:
name: Create Translation PR on Version Published
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
needs: sync-translations
permissions:
contents: write
-92
View File
@@ -1,92 +0,0 @@
name: Docker Compose Build Latest Main Image Tag (Manual Dispatch)
on:
workflow_dispatch:
permissions:
contents: read
packages: write
jobs:
build:
runs-on: hanzo-build-linux-amd64
strategy:
matrix:
include:
- target: api-build
file: Dockerfile.multi
image_name: librechat-api
- target: node
file: Dockerfile
image_name: librechat
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: main
fetch-depth: 0
- name: Fetch tags and set the latest tag
run: |
set -euo pipefail
git fetch --tags --force
LATEST_TAG=$(git tag --list 'v[0-9]*' --sort=-v:refname | grep -E '^v[0-9]+[.][0-9]+[.][0-9]+$' | head -n 1)
if [ -z "$LATEST_TAG" ]; then
echo "::error::No stable v<semver> tag found"
exit 1
fi
printf 'LATEST_TAG=%s\n' "$LATEST_TAG" >> "$GITHUB_ENV"
- name: Compute build metadata
run: |
printf 'BUILD_COMMIT=%s\n' "$(git rev-parse HEAD)" >> "$GITHUB_ENV"
printf 'BUILD_BRANCH=main\n' >> "$GITHUB_ENV"
printf 'BUILD_DATE=%s\n' "$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> "$GITHUB_ENV"
# Set up QEMU
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
# Set up Docker Buildx
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# Log in to GitHub Container Registry
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Login to Docker Hub
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# Prepare the environment
- name: Prepare environment
run: |
cp .env.example .env
# Build and push Docker images for each target
- name: Build and push Docker images
uses: docker/build-push-action@v5
with:
context: .
file: ${{ matrix.file }}
push: true
tags: |
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:${{ env.LATEST_TAG }}
ghcr.io/${{ github.repository_owner }}/${{ matrix.image_name }}:latest
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:${{ env.LATEST_TAG }}
${{ secrets.DOCKERHUB_USERNAME }}/${{ matrix.image_name }}:latest
platforms: linux/amd64,linux/arm64
target: ${{ matrix.target }}
build-args: |
BUILD_COMMIT=${{ env.BUILD_COMMIT }}
BUILD_BRANCH=${{ env.BUILD_BRANCH }}
BUILD_DATE=${{ env.BUILD_DATE }}
+1 -1
View File
@@ -5,7 +5,7 @@ on:
jobs:
publish-dev-release:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
steps:
- name: Checkout code
-21
View File
@@ -1,21 +0,0 @@
name: PR Preview
on:
pull_request:
types: [opened, synchronize, reopened, closed]
permissions:
contents: read
packages: write
pull-requests: write
jobs:
preview:
uses: hanzoai/.github/.github/workflows/pr-preview.yml@main
with:
service: chat
image: ghcr.io/hanzoai/chat
port: "3080"
health-path: /health
e2e-dir: e2e
secrets: inherit
+1 -1
View File
@@ -7,7 +7,7 @@ on:
jobs:
stale:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v10
with:
@@ -1,85 +0,0 @@
name: Sync Helm Chart Tags
on:
push:
branches:
- main
workflow_dispatch:
inputs:
release_existing_tag:
description: "Existing chart-* tag to dispatch if tag creation succeeded but release dispatch failed"
required: false
type: string
permissions:
contents: read
concurrency:
group: sync-helm-chart-tags
cancel-in-progress: false
jobs:
noop:
name: Ignore non-main push
if: github.event_name == 'push' && github.ref != 'refs/heads/main'
runs-on: hanzo-build-linux-amd64
timeout-minutes: 1
steps:
- name: Skip tag sync
run: echo "Helm chart tag sync only runs on main pushes or manual dispatch."
sync:
name: Sync chart tags
if: github.event_name == 'workflow_dispatch' || github.ref == 'refs/heads/main'
runs-on: hanzo-build-linux-amd64
timeout-minutes: 10
permissions:
actions: write
contents: write
env:
BASE_REF: refs/remotes/origin/main
BACKFILL_FROM_VERSION: 1.9.0
CHART_PATH: helm/librechat/Chart.yaml
DEFAULT_BRANCH: main
DISPATCH_WORKFLOW: helmcharts.yml
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
RELEASE_EXISTING_TAG: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.release_existing_tag || '' }}
REPO_DIR: /tmp/librechat-sync
TAG_PREFIX: chart-
steps:
- name: Fetch main and tags
shell: bash
run: |
set -euo pipefail
if [[ ! "$GITHUB_REPOSITORY" =~ ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ ]]; then
echo "::error::Unexpected repository name: $GITHUB_REPOSITORY"
exit 1
fi
if [[ "$GITHUB_SERVER_URL" != "https://github.com" ]]; then
echo "::error::Unexpected GitHub server URL: $GITHUB_SERVER_URL"
exit 1
fi
rm -rf "$REPO_DIR"
git init "$REPO_DIR"
cd "$REPO_DIR"
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
AUTH_HEADER="$(printf 'x-access-token:%s' "$GITHUB_TOKEN" | base64 | tr -d '\n')"
git -c "http.extraheader=AUTHORIZATION: basic ${AUTH_HEADER}" \
fetch --prune --force --tags origin \
"+refs/heads/${DEFAULT_BRANCH}:refs/remotes/origin/${DEFAULT_BRANCH}"
git checkout --detach "$BASE_REF"
- name: Create missing chart tags
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PUSH_TAGS: "true"
run: |
set -euo pipefail
cd "$REPO_DIR"
.github/scripts/sync-helm-chart-tags.sh
-116
View File
@@ -1,116 +0,0 @@
name: Docker Images Build on Tag
on:
push:
tags:
- 'v*'
permissions:
contents: read
packages: write
jobs:
build:
runs-on: hanzo-build-linux-amd64
strategy:
matrix:
include:
- target: api-build
file: Dockerfile.multi
image_name: librechat-api
- target: node
file: Dockerfile
image_name: librechat
steps:
# Check out the repository
- name: Checkout
uses: actions/checkout@v4
- name: Validate release tag
id: release-tag
env:
REF_NAME: ${{ github.ref_name }}
run: |
set -euo pipefail
TAG_REGEX='^v[0-9]+[.][0-9]+[.][0-9]+(-rc[0-9]+)?$'
STABLE_TAG_REGEX='^v[0-9]+[.][0-9]+[.][0-9]+$'
if [[ ! "$REF_NAME" =~ $TAG_REGEX ]]; then
echo "::error::Docker release tags must use v<semver> or v<semver>-rcN, for example v0.8.5 or v0.8.5-rc1"
exit 1
fi
printf 'image_tag=%s\n' "$REF_NAME" >> "$GITHUB_OUTPUT"
if [[ "$REF_NAME" =~ $STABLE_TAG_REGEX ]]; then
echo "is_stable=true" >> "$GITHUB_OUTPUT"
else
echo "is_stable=false" >> "$GITHUB_OUTPUT"
fi
# Set up QEMU
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
# Set up Docker Buildx
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# Log in to GitHub Container Registry
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Login to Docker Hub
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# Prepare the environment
- name: Prepare environment
run: |
cp .env.example .env
- name: Compute build metadata
run: |
echo "BUILD_COMMIT=${{ github.sha }}" >> $GITHUB_ENV
echo "BUILD_BRANCH=${{ github.ref_name }}" >> $GITHUB_ENV
echo "BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_ENV
- name: Resolve image tags
id: image-tags
env:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
IMAGE_NAME: ${{ matrix.image_name }}
IMAGE_TAG: ${{ steps.release-tag.outputs.image_tag }}
IS_STABLE: ${{ steps.release-tag.outputs.is_stable }}
run: |
set -euo pipefail
{
echo 'tags<<EOF'
printf 'ghcr.io/%s/%s:%s\n' "$GITHUB_REPOSITORY_OWNER" "$IMAGE_NAME" "$IMAGE_TAG"
printf '%s/%s:%s\n' "$DOCKERHUB_USERNAME" "$IMAGE_NAME" "$IMAGE_TAG"
if [ "$IS_STABLE" = "true" ]; then
printf 'ghcr.io/%s/%s:latest\n' "$GITHUB_REPOSITORY_OWNER" "$IMAGE_NAME"
printf '%s/%s:latest\n' "$DOCKERHUB_USERNAME" "$IMAGE_NAME"
fi
echo 'EOF'
} >> "$GITHUB_OUTPUT"
# Build and push Docker images for each target
- name: Build and push Docker images
uses: docker/build-push-action@v5
with:
context: .
file: ${{ matrix.file }}
push: true
tags: ${{ steps.image-tags.outputs.tags }}
platforms: linux/amd64,linux/arm64
target: ${{ matrix.target }}
build-args: |
BUILD_COMMIT=${{ env.BUILD_COMMIT }}
BUILD_BRANCH=${{ env.BUILD_BRANCH }}
BUILD_DATE=${{ env.BUILD_DATE }}
+57
View File
@@ -0,0 +1,57 @@
name: LiteLLM Linting
on:
pull_request:
branches: [ main ]
jobs:
lint:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.12'
- name: Install Poetry
uses: snok/install-poetry@v1
- name: Install dependencies
run: |
pip install openai==1.81.0
poetry install --with dev
pip install openai==1.81.0
- name: Run Black formatting
run: |
cd litellm
poetry run black .
cd ..
- name: Run Ruff linting
run: |
cd litellm
poetry run ruff check .
cd ..
- name: Run MyPy type checking
run: |
cd litellm
poetry run mypy . --ignore-missing-imports
cd ..
- name: Check for circular imports
run: |
cd litellm
poetry run python ../tests/documentation_tests/test_circular_imports.py
cd ..
- name: Check import safety
run: |
poetry run python -c "from litellm import *" || (echo '🚨 import failed, this means you introduced unprotected imports! 🚨'; exit 1)
+2 -18
View File
@@ -8,13 +8,7 @@ on:
jobs:
test:
runs-on: hanzo-build-linux-amd64
# The packages/api rollup build peaks ~5 GiB RSS and OOMs at Node's default
# ~2 GiB old-space heap. Match the heap the review workflows already use
# (well under the 8 GiB runner limit). One knob, same expression everywhere.
env:
NODE_OPTIONS: '--max-old-space-size=${{ secrets.NODE_MAX_OLD_SPACE_SIZE || 6144 }}'
runs-on: ubuntu-latest
services:
mongodb:
@@ -39,7 +33,7 @@ jobs:
- name: Setup pnpm
uses: pnpm/action-setup@v4
- name: Get pnpm store directory
id: pnpm-cache
shell: bash
@@ -72,15 +66,5 @@ jobs:
MEILI_HOST: http://localhost:7700
MEILI_MASTER_KEY: test_master_key
# Gate against tests-without-implementation drift in the MCP connection/
# transport layer (proxy, response-size caps, WS SSRF, allowedAddresses).
# These suites live in packages/api and were never run by this workflow —
# which is how the connection.ts port drifted from its byte-identical tests.
# A future half-port now fails here loudly.
- name: Run MCP connection/transport tests
run: cd packages/api && pnpm run test:transport
env:
NODE_ENV: test
- name: Run client tests
run: pnpm run test:client
+28 -28
View File
@@ -12,7 +12,7 @@ on:
jobs:
detect-unused-packages:
runs-on: hanzo-build-linux-amd64
runs-on: ubuntu-latest
permissions:
pull-requests: write
@@ -248,35 +248,35 @@ jobs:
cd ..
fi
# Post via github-script (Octokit + the built-in token) rather than the gh
# CLI: gh is not installed on the self-hosted ARC runners, so `gh api` here
# exits 127. The job already grants `pull-requests: write`.
- name: Post comment on PR if unused dependencies are found
if: env.ROOT_UNUSED != '' || env.CLIENT_UNUSED != '' || env.API_UNUSED != ''
uses: actions/github-script@v7
with:
script: |
const section = (title, val) => {
const items = (val || '').split('\n').map((s) => s.trim()).filter(Boolean);
if (!items.length) return '';
return `#### 📂 ${title}\n\n${items.map((i) => `- \`${i}\``).join('\n')}\n`;
};
const body = [
'### 🚨 Unused NPM Packages Detected',
'',
'The following **unused dependencies** were found:',
'',
section('Root `package.json`', process.env.ROOT_UNUSED),
section('Client `client/package.json`', process.env.CLIENT_UNUSED),
section('API `api/package.json`', process.env.API_UNUSED),
'⚠️ **Please remove these unused dependencies to keep your project clean.**',
].filter(Boolean).join('\n');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
run: |
PR_NUMBER=$(jq --raw-output .pull_request.number "$GITHUB_EVENT_PATH")
ROOT_LIST=$(echo "$ROOT_UNUSED" | awk '{print "- `" $0 "`"}')
CLIENT_LIST=$(echo "$CLIENT_UNUSED" | awk '{print "- `" $0 "`"}')
API_LIST=$(echo "$API_UNUSED" | awk '{print "- `" $0 "`"}')
COMMENT_BODY=$(cat <<EOF
### 🚨 Unused NPM Packages Detected
The following **unused dependencies** were found:
$(if [[ ! -z "$ROOT_UNUSED" ]]; then echo "#### 📂 Root \`package.json\`"; echo ""; echo "$ROOT_LIST"; echo ""; fi)
$(if [[ ! -z "$CLIENT_UNUSED" ]]; then echo "#### 📂 Client \`client/package.json\`"; echo ""; echo "$CLIENT_LIST"; echo ""; fi)
$(if [[ ! -z "$API_UNUSED" ]]; then echo "#### 📂 API \`api/package.json\`"; echo ""; echo "$API_LIST"; echo ""; fi)
⚠️ **Please remove these unused dependencies to keep your project clean.**
EOF
)
gh api "repos/${{ github.repository }}/issues/${PR_NUMBER}/comments" \
-f body="$COMMENT_BODY" \
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Fail workflow if unused dependencies found
if: env.ROOT_UNUSED != '' || env.CLIENT_UNUSED != '' || env.API_UNUSED != ''
+162 -132
View File
@@ -1,154 +1,184 @@
### node etc ###
# Logs
data-node
meili_data*
data/
logs
*.log
# Runtime data
pids
*.pid
*.seed
.git
# CI/CD data
test-image*
dump.rdb
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# translation services
config/translations/stores/*
client/src/localization/languages/*_missing_keys.json
# Turborepo
.turbo
# Compiled Dirs (http://nodejs.org/api/addons.html)
build/
dist/
public/main.js
public/main.js.map
public/main.js.LICENSE.txt
client/public/images/
client/public/main.js
client/public/main.js.map
client/public/main.js.LICENSE.txt
# Azure Blob Storage Emulator (Azurite)
__azurite**
__blobstorage__/**/*
!.env.hanzo-cloud
!**/.env.example
!**/.env.test.example
!/client/src/@types/i18next.d.ts
!client/src/components/Nav/SettingsTabs/Data/
!memory/agents/README.md
!memory/sessions/README.md
# Dependency directorys
# Deployed apps should consider commenting these lines out:
# see https://npmjs.org/doc/faq.html#Should-I-check-my-node_modules-folder-into-git
node_modules/
meili_data/
api/node_modules/
client/node_modules/
bower_components/
*.d.ts
!vite-env.d.ts
.aider*
.claude-flow/
.claude/settings.local.json
# AI
.clineignore
.cursor
.DS_Store
.env.hanzo
.env*
.aider*
# Floobits
.floo
.floobit
.floo
.flooignore
.git
.grunt
.hive-mind/
.idea
.idx
.mcp.json
#config file
librechat.yaml
librechat.yml
# Environment
.npmrc
.swarm/
.turbo
.env*
my.secrets
!**/.env.example
!**/.env.test.example
cache.json
api/data/
owner.yml
archive
.vscode/settings.json
*.cert
src/style - official.css
/e2e/specs/.test-results/
/e2e/playwright-report/
/playwright/.cache/
.DS_Store
*.code-workspace
*.d.ts
.idx
monospace.json
.idea
*.iml
*.pem
config.local.ts
**/storageState.json
junit.xml
**/.venv/
**/venv/
# docker override file
docker-compose.override.yaml
docker-compose.override.yml
# meilisearch
meilisearch
meilisearch.exe
data.ms/*
auth.json
/packages/ux-shared/
/images
!client/src/components/Nav/SettingsTabs/Data/
# User uploads
uploads/
# owner
release/
# Helm
helm/librechat/Chart.lock
helm/**/charts/
helm/**/.values.yaml
!/client/src/@types/i18next.d.ts
# SAML Idp cert
*.cert
# AI Assistants
/.claude/
/.cursor/
/.copilot/
/.aider/
/.openai/
/.tabnine/
/.codeium
*.local.md
# Removed Windows wrapper files per user request
hive-mind-prompt-*.txt
# Claude Flow generated files
.claude/settings.local.json
.mcp.json
claude-flow.config.json
.swarm/
.hive-mind/
.claude-flow/
memory/
coordination/
memory/claude-flow-data.json
memory/sessions/*
!memory/sessions/README.md
memory/agents/*
!memory/agents/README.md
coordination/memory_bank/*
coordination/subtasks/*
coordination/orchestration/*
*.db
*.db-journal
*.db-wal
*.iml
*.local.md
*.log
*.pem
*.pid
*.seed
*.sqlite
*.sqlite-journal
*.sqlite-wal
**/.venv/
**/dist/
**/storageState.json
**/venv/
/.aider/
/.claude/
/.codeium
/.copilot/
/.cursor/
/.openai/
/.tabnine/
/e2e/playwright-report/
/e2e/specs/.test-results/
/images
/packages/ux-shared/
/playwright/.cache/
# AI
# AI Assistants
# Azure Blob Storage Emulator (Azurite)
# CI/CD data
# Claude Flow generated files
# Compiled Dirs (http://nodejs.org/api/addons.html)
# Coverage directory used by tools like istanbul
# Dependency directorys
# Deployed apps should consider commenting these lines out:
# Directory for instrumented libs generated by jscoverage/JSCover
# docker override file
# Environment
# Floobits
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
# Hanzo-specific
# Helm
# Logs
# meilisearch
# owner
# Removed Windows wrapper files per user request
# Runtime data
# SAML Idp cert
# see https://npmjs.org/doc/faq.html#Should-I-check-my-node_modules-folder-into-git
# translation services
# Turborepo
# User uploads
### node etc ###
#config file
api/data/
api/node_modules/
archive
auth.json
bower_components/
build/
cache.json
claude-flow
claude-flow.config.json
CLAUDE.md
client/node_modules/
client/public/images/
client/public/main.js
client/public/main.js.LICENSE.txt
client/public/main.js.map
client/src/localization/languages/*_missing_keys.json
config.local.ts
config/translations/stores/*
coordination/
coordination/memory_bank/*
coordination/orchestration/*
coordination/subtasks/*
coverage
data-node
data.ms/*
data/
dist/
docker-compose.override.yaml
docker-compose.override.yml
dump.rdb
helm/**/.values.yaml
helm/**/charts/
helm/librechat/Chart.lock
# Removed Windows wrapper files per user request
hive-mind-prompt-*.txt
junit.xml
lib-cov
librechat.yaml
librechat.yml
logs
meili_data*
meili_data/
meilisearch
meilisearch.exe
memory/
memory/agents/*
memory/claude-flow-data.json
memory/sessions/*
monospace.json
my.secrets
node_modules/
owner.yml
pids
public/main.js
public/main.js.LICENSE.txt
public/main.js.map
release/
src/style - official.css
test-image*
tests/**/*.jpg
# Hanzo-specific
CLAUDE.md
.env.hanzo
!.env.hanzo-cloud
tests/**/*.wav
tests/**/*.mp3
tests/**/*.png
tests/**/*.wav
uploads/
tests/**/*.jpg
-1
View File
@@ -1 +0,0 @@
24.16.0
+2 -13
View File
@@ -1,9 +1,7 @@
# v0.8.3-rc1
# Base node image — Hanzo Node 24 (Alpine) base: node:sqlite (DatabaseSync) is
# built in and native better-sqlite3 compiles (build-base + python3 + g++ baked
# in), so the CHAT_STORE_SQLITE document store runs. Node 20 lacked node:sqlite.
FROM ghcr.io/hanzoai/nodejs:v24.18.0 AS node
# Base node image
FROM node:20-alpine AS node
# Install jemalloc
RUN apk add --no-cache jemalloc
@@ -44,15 +42,6 @@ RUN \
COPY --chown=node:node . .
# Bake the Umami analytics website id into the Vite build so the index.html
# %VITE_ANALYTICS_SITE_ID% placeholder is substituted at build time. Vite's HTML
# env replacement (loadEnv, envPrefix includes VITE_) picks this up from process.env.
# Without it the tracker POSTs a literal "%VITE_ANALYTICS_SITE_ID%" as the website
# id and analytics.hanzo.ai/api/send answers 400. Public site identifier (it appears
# in the served HTML), safe to default here; override with --build-arg if needed.
ARG VITE_ANALYTICS_SITE_ID=2f72b944-f1f8-4d2d-8f6c-26063bde0d1a
ENV VITE_ANALYTICS_SITE_ID=$VITE_ANALYTICS_SITE_ID
RUN \
# React client build with configurable memory
NODE_OPTIONS="--max-old-space-size=${NODE_MAX_OLD_SPACE_SIZE}" pnpm run frontend; \
+2 -2
View File
@@ -5,7 +5,7 @@
ARG NODE_MAX_OLD_SPACE_SIZE=6144
# Base for all builds
FROM ghcr.io/hanzoai/nodejs:v24.18.0 AS base-min
FROM node:20-alpine AS base-min
# Install jemalloc
RUN apk add --no-cache jemalloc
# Set environment variable to use jemalloc
@@ -16,7 +16,7 @@ RUN corepack enable && corepack prepare pnpm@latest --activate
WORKDIR /app
RUN apk --no-cache add curl
COPY package.json pnpm-lock.yaml pnpm-workspace.yaml .npmrc ./
COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./
COPY packages/data-provider/package.json ./packages/data-provider/
COPY packages/api/package.json ./packages/api/
COPY packages/data-schemas/package.json ./packages/data-schemas/
+1 -1
View File
@@ -6,7 +6,7 @@
# Build: docker build -f Dockerfile.static -t ghcr.io/hanzoai/chat:latest .
# Run: docker run -p 3080:3080 ghcr.io/hanzoai/chat:latest
FROM ghcr.io/hanzoai/nodejs:v24.18.0
FROM node:22-alpine
RUN npm install -g serve@14
-1
View File
@@ -5,7 +5,6 @@ Portions of this software are licensed as follows:
---
MIT License
Copyright (c) 2023 LibreChat (Danny Avila)
Copyright (c) 2023 Hanzo AI Inc
Originally based on LibreChat by Berri AI.
+119 -209
View File
@@ -46,7 +46,8 @@ npm run format # Prettier
```
api/ # Express backend (port 3080)
server/ # Entry point, routes, controllers, middleware
models/ # Mongoose models (MongoDB)
db/base/ # Hanzo Base data-layer adapter (replaces MongoDB) — see below
models/ # Model method wrappers (persist to Base via the adapter)
client/ # React frontend (Vite)
src/components/ # UI components
src/routes/ # Client-side routing
@@ -54,12 +55,127 @@ client/ # React frontend (Vite)
packages/
data-provider/ # Shared data layer (librechat-data-provider)
data-schemas/ # Validation schemas
api/ # API client package (@hanzochat/api)
api/ # API client package (@librechat/api)
client/ # Shared client components
agents/ # Agent definitions
mcp/ # MCP server integration
```
## Data Layer — Hanzo Base (MongoDB dropped)
Hanzo Chat does **not** use MongoDB. Persistence runs on **Hanzo Base** (the Go
PocketBase-lineage server; SQLite embedded for dev, Postgres for prod). The
"drop mongo completely" directive (#48) is realised by a thin adapter under
`api/db/base/` that presents a Mongoose-Model-compatible surface backed by the
`@hanzo/base` JS client. There is **one** adapter; every model comes along.
### How it works (one adapter, all models)
`@librechat/data-schemas` builds its schemas with real `mongoose` purely as a
schema DSL, then asks a mongoose instance to turn them into models
(`createModels`/`createMethods`). We hand it a **facade** instead of real
mongoose:
- `api/db/base/mongoose.js` — a `mongoose`-compatible facade. Delegates
schema/ObjectId/type concerns to real mongoose (never connected) but
overrides `model()`/`models`/`connect`/`connection`. `model(name, schema)`
returns a Mongoose-style **constructor** (statics + `new Model(data)` docs).
- `api/db/base/model.js``BaseModel`: the Mongoose Model static surface
(`find/findOne/findOneAndUpdate/create/insertMany/updateOne/updateMany/
deleteOne/deleteMany/countDocuments/bulkWrite/aggregate/distinct/exists`),
a chainable/thenable `Query` (`select/sort/limit/skip/lean/populate` +
`find(A).deleteMany(B)`), and lightweight documents (`toObject/save`).
- `api/db/base/query.js` — a correct in-JS Mongo query/update engine
(`$or/$and/$in/$nin/$gt../$exists/$regex/$elemMatch`, update ops
`$set/$unset/$inc/$push/$addToSet/$pull/$setOnInsert`, projection, sort).
This is the correctness backstop — every predicate is evaluated here.
- `api/db/base/store.js` — the Base backend via `@hanzo/base` `BaseClient`.
Each model → one Base collection whose records hold the **full document as a
JSON `data` field** plus promoted scalar columns (from indexed/unique schema
paths) used only for **filter pushdown** (a guaranteed superset; the JS
engine then narrows). `@hanzo/base` is ESM-only → loaded via dynamic
`import()` from this CommonJS code.
- `api/db/base/schema.js` — introspects a Mongoose schema for defaults,
timestamps, and promotable columns.
- `api/db/base/index.js``connectDb()` inits the Base client, health-checks,
and provisions every registered model's collection (idempotent).
### Wiring (all app code off mongoose)
`api/db/{connect,models,index}.js`, `api/models/index.js`, and every remaining
app file (`Agent`, `inviteUser`, `PermissionService`, `PermissionsController`,
`initializeMCPs`, `migration`) import the facade (`~/db/base`) — not `mongoose`.
The facade itself carries **no `mongoose` runtime dependency** (own ObjectId,
Schema.Types shim, no-op session/connection). `grep "require('mongoose')"` over
non-test `api/` is **zero**; **nothing calls `mongoose.connect`**. `_id` is a
real 24-hex ObjectId string stored inside the document; conversations/messages
remain keyed by their `conversationId`/`messageId` UUIDs.
### Search (FTS on Base/SQLite)
MeiliSearch is dropped. `BaseModel.meiliSearch(query, {filter})` runs a
user-scoped, case-insensitive substring search over content fields flagged
`meiliIndex` (title, text), returning Meili-shaped `{ hits }`. `indexSync.js`
is a no-op (no external index); `/search/enable` reports availability from Base
(set `SEARCH=false` to disable). A future optimisation is a SQLite FTS5 index
instead of the per-user scan.
### Security invariants (do not regress — from blue+red review)
- **Prototype pollution:** `query.js` path helpers reject `__proto__` /
`constructor` / `prototype` segments (attacker update keys flow in via
`saveConvo` / conversation import). Keep the guard.
- **`select:false`:** secrets (password, totpSecret, backupCodes, keyHash) are
excluded from reads by default; only `+field` returns them. `save()` merges
onto the stored record so a projected load never erases them.
- **Pushdown = SUPERSET:** `store.js buildFilter` must only ever broaden; the JS
matcher narrows. Never push a predicate that could wrongly exclude — Date
columns and non-finite numbers are intentionally NOT pushed (Base date-column
normalization vs ISO would silently drop records).
- **DSL injection:** `quote()` escapes `\` and `'`; verified a crafted
`conversationId` cannot inject `||`/operators.
### Uniqueness / concurrency
Collections carry a Base **UNIQUE** index on always-present unique keys (`_id`,
`conversationId`, `messageId`, `email`); optional/sparse unique fields
(`googleId`, `openidId`, …) stay plain-indexed to avoid null conflicts, with
logical uniqueness enforced by the adapter upsert. The unique index is the
race-safety net (verified: a duplicate `conversationId` insert is rejected).
### Env / running
```
HANZO_BASE_URL=http://… # Hanzo Base instance (replaces MONGO_URI)
HANZO_BASE_TOKEN=… # Base superuser/service token (IAM in prod)
```
Local Base for dev: build a pure-SQLite launcher from `~/work/hanzo/base`
(`base.New()` without the IAM platform plugin), `serve --http=127.0.0.1:8090`;
the first serve prints a 30-min superuser installer token. API prefix is `/v1`.
### Tests
- `api/db/base/query.spec.js`, `api/db/base/model.spec.js` — jest, in-memory
store, no Base/Mongo needed (CI). `cd api && npx jest db/base`.
- `api/db/base/scripts/live-check.js` — real data-schemas factories vs a live
Base: createUser + balance grant + findUser + bcrypt + saveConvo + saveMessage,
verified via raw Base REST. Env-gated (`HANZO_BASE_URL`/`HANZO_BASE_TOKEN`).
- `api/db/base/scripts/boot-smoke.js` — drives the wired `~/db` + `~/models`
boot path (`connectDb` + `seedDatabase`) against live Base.
### Phased status
- **Phase 1 (done):** adapter + core hot path verified on Base (User, Balance,
Conversation, Message, Agent, plus Role/Session/Token/Category via the same
adapter — 29 collections provisioned). Nothing connects to Mongo.
- **Phase 2 (done):** Base/SQLite full-text search replaces MeiliSearch;
`indexSync` no-op; dead Meili helpers removed. Live-verified real hits.
- **Phase 3 (done):** all app code off `mongoose` (self-contained facade);
Base UNIQUE indexes on natural keys (race-safety, live-verified);
`select:false` secrets honored; all Date fields hydrated.
- **Remaining tail (test migration):** ~28 legacy `*.spec.js` still build models
on real `mongoose` + `mongodb-memory-server` (e.g. `api/models/*.spec.js`).
They're incompatible with the port (they expect mongoose-backed models) and
must be migrated to the adapter (in-memory store harness) — that is what gates
dropping the `mongoose`/`mongodb-memory-server` devDeps and a literal
`grep -riE 'mongoose|mongodb' api/` = 0. The adapter's own suites
(`api/db/base/*.spec.js`) are the migration template.
- **Deploy:** build image via CI → `registry.hanzo.ai`, deploy to DOKS with a
Base instance + IAM service token (`HANZO_BASE_URL`/`HANZO_BASE_TOKEN`), then
Playwright-verify login+chat+agent+search in the browser. Gated on cluster access.
## Configuration
- `librechat.yaml` (or ConfigMap `chat-config` -> `/app/librechat.yaml`)
@@ -90,216 +206,10 @@ CREDS_KEY= CREDS_IV= # Credential encryption
- `uv` bundled for MCP server support
- `dompurify` must be in `client/package.json` (externalized by bundler)
## Guest Chat (anonymous preview)
Off by default (`ALLOW_GUEST_CHAT=false`). When enabled, the landing IS the chat
composer (ChatGPT-style): an unauthenticated visitor renders the real chat view —
composer, starter cards, model picker — WITHOUT logging in, scoped to the free
Zen model (`GUEST_MODEL`, default `zen3-nano`) via the `Hanzo` custom endpoint
(`api.hanzo.ai`). Prod uses `GUEST_MESSAGE_MAX=2`. Exhausting the quota returns
`402 {type:'GUEST_LIMIT'}` and the client opens the existing OpenID/hanzo.id login.
Client render path (guest === chat, not a marketing gate):
- `AuthContext` auto-acquires a guest token when `startupConfig.allowGuestChat`
is true (`silentRefresh` fallback + a dedicated effect closing the config race);
sets `isGuest=true`, `isAuthenticated=false`. `useAuthRedirect` keeps guests on
the chat surface (only truly anonymous, non-guest, non-guest-enabled users go to
`/login`). `Root` shows the chat shell for `isAuthenticated || isGuest`.
- `ChatRoute` renders `ChatView` for `canChat = isAuthenticated || isGuest`; the
`/v1/chat/models` + `/v1/chat/endpoints` queries run for guests (both routes use
`requireGuestOrJwtAuth` and return the guest-scoped single-model config), and the
roles gate treats a guest as loaded (guests have no agent access). Files:
`client/src/routes/{ChatRoute,useAuthRedirect}.tsx`, `hooks/useGuestAuth.ts`,
`hooks/AuthContext.tsx`, `components/Auth/GuestLimitDialog.tsx`.
Security model (fail-closed, server-enforced):
- `POST /v1/chat/auth/guest` issues a short-lived guest JWT (`{guest:true}`,
per-token random id) signed with `JWT_SECRET`. Rate-limited per IP
(`guestTokenLimiter`, `GUEST_TOKEN_MAX`/`GUEST_TOKEN_WINDOW`) so tokens can't be
spam-minted.
- `requireGuestOrJwtAuth` (chat-completion route ONLY) accepts guest tokens;
the standard `jwt` strategy rejects them everywhere else (no DB user), so
every other route stays closed. `enforceGuestScope` pins endpoint+model and
strips agents/tools/files/spec/preset. Guests always use the shared, capped
`HANZO_API_KEY` (per-user `hk-` billing is skipped for `guest` principals).
- `guestMessageLimiter` enforces the quota against the REAL client IP
(`utils/guestClientIp` → Cloudflare `CF-Connecting-IP`, falls back to `req.ip`),
NOT the token — clearing cookies / incognito / minting a fresh token does NOT
reset it. Backed by the shared Redis `limiterCache` so it holds across replicas.
`USE_REDIS=true` is MANDATORY (a memory store would let a visitor round-robin
pods to multiply their quota).
- Key files: `api/server/services/guestConfig.js`,
`api/server/controllers/auth/GuestController.js`,
`api/server/middleware/{requireGuestOrJwtAuth,enforceGuestScope}.js`,
`api/server/middleware/limiters/{guestLimiters,guestMessageLimiter}.js`,
`api/server/utils/guestClientIp.js`,
router wiring in `api/server/routes/agents/index.js`.
- Env: `ALLOW_GUEST_CHAT`, `GUEST_MESSAGE_MAX`, `GUEST_ENDPOINT`, `GUEST_MODEL`,
`GUEST_TOKEN_EXPIRY`, `GUEST_TOKEN_MAX`, `GUEST_TOKEN_WINDOW`. Requires
`HANZO_API_KEY` (the free publishable gateway key) and `USE_REDIS` for the
shared per-IP quota across replicas.
## Cloud Agents (canonical /v1/agents)
Chat can RUN a user's canonical Hanzo Cloud agents (cloud `/v1/agents`, the ONE
production agent registry) from the thread — alongside the LibreChat-legacy local
agent builder, which is untouched.
- Two surfaces, ONE run path: the `/agent <name> [prompt]` slash command and the
@mention picker (cloud agents appear as a `cloudAgent` type). Both funnel
through `useRunCloudAgent``POST /v1/chat/agents/cloud/:name/run`. The @mention /
`/agent` picker arms `/agent <name> ` in the composer; submit is intercepted in
`ChatForm` (`parseAgentCommand`) and dispatched to the run path.
- Server proxy + auth (token never reaches the browser): the chat backend reads
the user's hanzo.id token from the server-side session
(`req.session.openidTokens.idToken`, then `accessToken`, then the httpOnly
cookies) and forwards it as `Authorization: Bearer` to cloud. Cloud's
`SanitizeIdentity` (HIP-0026) validates it and pins `X-Org-Id` from the `owner`
claim, so a user only ever reaches their OWN org's agents — chat never asserts
an org. `requireJwtAuth` gates the proxy (guests rejected); missing token →
honest 401, never a service-token fallback (fail-secure). Agent name is
validated against cloud's handle grammar (traversal/SSRF guard); it is NOT an
open proxy (three fixed endpoints). Principal guard: the on-behalf-of decision
keys off the VALIDATED principal (`req.user.provider==='openid'`), not the
mutable `token_provider` refresh-strategy cookie, so a local user (who never
carries a hanzo.id token) can't run under a stale OpenID session. EVERY
forwarded token — id_token preferred, access_token fallback, from session or
the httpOnly cookie — must pass `isForwardableToken`: a decodable JWT whose
`sub` EQUALS `req.user.openidId` (binding MANDATORY — absent `openidId`/`sub`
or a mismatch ⇒ no forward, no fail-open) and is unexpired. Decode-only is
sound because cloud does the authoritative JWKS validation over the SAME claims,
so it runs as exactly that `sub`; the gate only ever removes a token. An
unbindable/expired/foreign token yields an honest 401, never a fabricated or
wrong-principal run.
- id_token persistence is DECOUPLED from the refresh strategy: OIDC login ALWAYS
persists the on-behalf-of BEARER (id_token + access_token) to
`req.session.openidTokens` (server-side only), regardless of
`OPENID_REUSE_TOKENS`. It does NOT persist the OIDC refresh credential in the
decoupled default — the session refreshes via the local JWT cookie, so
`session.openidTokens.refreshToken` is written ONLY in REUSE mode (where
`refreshController`/`logoutController` read it). That keeps login, refresh AND
logout on the local-JWT path byte-identical to a non-OpenID login; that flag
SOLELY gates whether `/v1/chat/auth/refresh` performs the OIDC refresh-grant.
The ~1h id_token is used while valid; durable refresh (hanzo.id/Casdoor OIDC
refresh or an RFC-8693 token-exchange from the chat session) is a tracked
FOLLOW-UP — the login-breaking refresh-grant is NOT enabled here.
- Abuse limits (a run is a real billable completion): a per-user rate limiter
(`cloudAgentLimiter`, `CLOUD_AGENT_USER_MAX`/`CLOUD_AGENT_WINDOW`) guards the
whole `/cloud` router; the client caps input by UTF-8 **bytes** (128 KiB), caps
the buffered response (4 MiB → 502), and sheds load past a process-wide in-flight
ceiling (`CLOUD_AGENT_MAX_CONCURRENT`, 503).
- Key files: backend `api/server/services/CloudAgentsClient.js`,
`api/server/routes/agents/cloud.js` (mounted `/cloud` in
`api/server/routes/agents/index.js`); data layer
`packages/data-provider/src/{types/cloudAgents.ts,api-endpoints.ts,data-service.ts}`;
client `client/src/hooks/Agents/useRunCloudAgent.ts`,
`client/src/utils/agentCommand.ts`,
`client/src/components/Chat/Input/AgentsCommand.tsx`, and the @mention wiring in
`client/src/hooks/Input/useMentions.ts` + `Mention.tsx`.
- Env: `HANZO_CLOUD_URL` (optional; falls back to the `OPENAI_BASE_URL` host).
- Convergence path (later): chat's LibreChat-legacy `/v1/chat/agents` CRUD should
converge onto cloud `/v1/agents`; this step only ADDS cloud-agent RUN.
## Unified cloud architecture (2026-07) — investigate-before-ripping map
hanzo.chat is the **chat view** of the Hanzo AI cloud (sibling to hanzo.app =
builder, console = admin). This section is the honest map of what is ALREADY
unified onto the Go backend (`api.hanzo.ai/v1`) vs the one real seam that is not.
Verified by full call-graph + route-table trace; do NOT rip blind.
### What already routes through the Go backend `api.hanzo.ai/v1` (no shadow LLM)
- **Chat completions**: client `useSSE``POST /v1/chat/agents/chat/Hanzo` (all
chat, incl. plain-model, goes through the agents framework) → custom-endpoint
resolver (`packages/api/src/endpoints/custom/initialize.ts`) reads
`HANZO_API_KEY` + literal `baseURL https://api.hanzo.ai/v1` from the loaded
config → LangChain OpenAI client → **`POST https://api.hanzo.ai/v1/chat/completions`**
(SSE stream, resumable via `GenerationJobManager`). Per-user `hk-` key +
per-org Commerce debit; fail-closed 402. THIS is the one inference path.
- **Code interpreter** → `LIBRECHAT_CODE_BASEURL` = cloud `/v1/exec`.
- **Web search** → `webSearch` block (searxng+firecrawl contracts) = cloud
`/v1/websearch`.
- **Cloud agents** → `POST /v1/chat/agents/cloud/:name/run` server-proxies to cloud
`/v1/agents` with the user's hanzo.id bearer (see "Cloud Agents" section).
- **Model list**: curated **zen-only** (`fetch:false`) in the loaded config —
NO raw upstream names (brand policy). Authoritative prod list lives in the
`chat-config` ConfigMap (`universe infra/k8s/chat/configmap.yaml`); repo
`librechat.yaml` mirrors it (one way).
### DEAD residue — do NOT treat as a live backend
- `config.yaml` (LiteLLM `model_list`/`litellm_params`), `docker/Dockerfile.{simple,dev,custom_ui}`
(`CMD litellm`), `deploy/migrations/*` (`LiteLLM_*` Prisma tables),
`CONTRIBUTING.md` (upstream LiteLLM's), `scripts/cleanup-{litellm,hanzo-chat}.sh`:
all **unreferenced** by any compose/k8s/helm/Dockerfile.multi. Prod runs
`node server/index.js` and hits `api.hanzo.ai/v1` directly — NO local litellm
sidecar. This is upstream merge residue; safe to delete in a dedicated sweep.
### The ONE real parallel store (FLAG — needs a Go-backend home)
LibreChat's Express backend owns, in **MongoDB** (`HanzoChat` DB), all of:
`convos`, `messages`, `presets`, `prompts`/`promptGroups`, `users`, `balances`/
`transactions`, `files`, `sessions` (refresh-token hashes), plus agents/assistants/
memory/RBAC. Schemas: `packages/data-schemas/src/schema/*`. This is the shadow
store that is NOT on the Go backend.
- The Go backend (`hanzoai/ai`, mounted at bare `/v1/*` in cloud) DOES have a
persistence home, but under **casibase names** (`/v1/get-chats`, `/v1/get-chat`,
`/v1/add-chat`, `/v1/get-messages`, `/v1/add-message`, `/v1/get-usages`) — a
different schema/shape than LibreChat's Mongo.
- The canonical OpenAPI repo has `chat/openapi.yaml` describing the INTENDED
LibreChat-shaped REST surface (`/v1/chat/convos`, `/v1/chat/messages`,
`/v1/chat/presets`, `/v1/chat/balance`, `/v1/chat/auth/*`) — but the Go binary
**does not implement it yet**, and `ai/openapi.yaml` under-documents the real
casibase routes.
- To truly kill the parallel store WITHOUT breaking live chat: the Go backend
(or Base) must implement `chat/openapi.yaml` (conversations/messages/presets),
then repoint chat's data layer at it behind a flag and dual-write during
cutover. Until then Mongo stays (ripping it = data loss + dead chat).
**Coordinate with the openapi agent** (canonical spec + SDK regen).
### IAM-native auth (HIP-0111) — federated to hanzo.id, LIVE
- **Prod (backend-proxied)**: LibreChat passport `openid-client` strategy,
OIDC **discovery** from `${OPENID_ISSUER}` = `https://hanzo.id`
(`/.well-known/openid-configuration`; discovery fetched via in-cluster
`iam.hanzo.svc` to dodge the CF hairpin), client_id **`hanzo-chat`**, callback
`/oauth/openid/callback`. Local email/password is OFF in prod
(`ALLOW_EMAIL_LOGIN=false`, `ALLOW_REGISTRATION=false`); social OIDC only.
Files: `api/strategies/openidStrategy.js`, `api/server/socialLogins.js`,
`api/server/routes/oauth.js`. This IS IAM-native (federated), just not the
console `@hanzo/iam-js-sdk` shape.
- **Static/IAM SPA mode** (`Dockerfile.static`, not the live prod deploy): browser
`@hanzo/iam` `BrowserIamSdk` PKCE straight to hanzo.id
(`client/src/utils/iam.ts`, `OAuthCallback.tsx`). ⚠️ INCONSISTENCY: uses
client_id **`app-chat`** while prod uses `hanzo-chat` — align to `hanzo-chat`.
`@hanzo/iam` is pinned `^0.4.0` (HIP-0111 wants ≥0.11.0); this path is dormant.
### One brand system, but pick the RIGHT one for a Tailwind app
`@hanzo/ui` and `@hanzo/gui` are **two different, non-overlapping** design systems:
- **`@hanzo/ui`** = shadcn/ui + Tailwind + Radix (multi-framework). chat is
Vite + React 18 + Tailwind, so THIS is the correct shared lib. Already used:
`client/src/components/Nav/HanzoHeader.tsx` mounts `@hanzo/ui/navigation`
`HanzoHeader` for cross-app chrome. Monochrome rebrand already done (grey ramp,
H mark, favicon = hanzo.app set).
- **`@hanzo/gui`** = a **Tamagui** fork (Next.js 15 / React 19, RN-web) — console's
stack. Forcing it into the Vite/React18 LibreChat client = a ground-up rewrite
of a live product; NOT done. Unify by widening `@hanzo/ui` adoption + matching
console's monochrome tokens, NOT by swapping component frameworks.
### Config filename caveat
`loadCustomConfig.js` defaults to **`chat.yaml`** (`CONFIG_PATH || <root>/chat.yaml`).
Prod sets `CONFIG_PATH=/app/chat.yaml` (ConfigMap mount). Repo ships
`librechat.yaml` (reference); a deploy that doesn't set `CONFIG_PATH` to it (or
provide `chat.yaml`) falls back to the built-in `openAI` endpoint. `OPENAI_BASE_URL`
in `compose.prod.yml` is inert here (built-in openAI reads `OPENAI_REVERSE_PROXY`).
## Internal Package Names
These are kept as-is from upstream (npm deps, not worth renaming):
- `@hanzochat/api`, `@librechat/client`, `@librechat/data-schemas`, `librechat-data-provider`, `@librechat/agents`
- `@librechat/api`, `@librechat/client`, `@librechat/data-schemas`, `librechat-data-provider`, `@librechat/agents`
- Functions: `extractLibreChatParams`, `importLibreChatConvo`
- Type names: `LibreChatKeys`, `LibreChatParams`
- Config filename: `librechat.yaml` (upstream convention)
-6
View File
@@ -1,6 +0,0 @@
chat
Copyright (c) 2023 Hanzo AI Inc.
This product includes software from LibreChat (https://github.com/danny-avila/LibreChat), licensed under MIT:
Copyright (c) 2023 LibreChat (Danny Avila)
-2
View File
@@ -1,5 +1,3 @@
<p align="center"><img src=".github/hero.svg" alt="chat" width="880"></p>
# Hanzo AI Chat
AI-powered chat platform with enterprise features, using Hanzo's cloud API or local deployment.
-227
View File
@@ -1,227 +0,0 @@
<!-- Last synced with README.md: 2026-03-28 (cae3888) -->
<p align="center">
<a href="https://librechat.ai">
<img src="client/public/assets/logo.svg" height="256">
</a>
<h1 align="center">
<a href="https://librechat.ai">LibreChat</a>
</h1>
</p>
<p align="center">
<a href="README.md">English</a> ·
<strong>中文</strong>
</p>
<p align="center">
<a href="https://discord.librechat.ai">
<img
src="https://img.shields.io/discord/1086345563026489514?label=&logo=discord&style=for-the-badge&logoWidth=20&logoColor=white&labelColor=000000&color=blueviolet">
</a>
<a href="https://www.youtube.com/@LibreChat">
<img
src="https://img.shields.io/badge/YOUTUBE-red.svg?style=for-the-badge&logo=youtube&logoColor=white&labelColor=000000&logoWidth=20">
</a>
<a href="https://docs.librechat.ai">
<img
src="https://img.shields.io/badge/DOCS-blue.svg?style=for-the-badge&logo=read-the-docs&logoColor=white&labelColor=000000&logoWidth=20">
</a>
<a aria-label="Sponsors" href="https://github.com/sponsors/danny-avila">
<img
src="https://img.shields.io/badge/SPONSORS-brightgreen.svg?style=for-the-badge&logo=github-sponsors&logoColor=white&labelColor=000000&logoWidth=20">
</a>
</p>
<p align="center">
<a href="https://railway.com/deploy/librechat-official?referralCode=HI9hWz&utm_medium=integration&utm_source=readme&utm_campaign=librechat">
<img src="https://railway.com/button.svg" alt="Deploy on Railway" height="30">
</a>
<a href="https://zeabur.com/templates/0X2ZY8">
<img src="https://zeabur.com/button.svg" alt="Deploy on Zeabur" height="30"/>
</a>
<a href="https://template.cloud.sealos.io/deploy?templateName=librechat">
<img src="https://raw.githubusercontent.com/labring-actions/templates/main/Deploy-on-Sealos.svg" alt="Deploy on Sealos" height="30">
</a>
</p>
<p align="center">
<a href="https://www.librechat.ai/docs/translation">
<img
src="https://img.shields.io/badge/dynamic/json.svg?style=for-the-badge&color=2096F3&label=locize&query=%24.translatedPercentage&url=https://api.locize.app/badgedata/4cb2598b-ed4d-469c-9b04-2ed531a8cb45&suffix=%+translated"
alt="翻译进度">
</a>
</p>
# ✨ 功能
- 🖥️ **UI 与体验**:受 ChatGPT 启发,并具备更强的设计与功能。
- 🤖 **AI 模型选择**
- Anthropic (Claude), AWS Bedrock, OpenAI, Azure OpenAI, Google, Vertex AI, OpenAI Responses API (包含 Azure)
- [自定义端点 (Custom Endpoints)](https://www.librechat.ai/docs/quick_start/custom_endpoints)LibreChat 支持任何兼容 OpenAI 规范的 API,无需代理。
- 兼容[本地与远程 AI 服务商](https://www.librechat.ai/docs/configuration/librechat_yaml/ai_endpoints)
- Ollama, groq, Cohere, Mistral AI, Apple MLX, koboldcpp, together.ai,
- OpenRouter, Helicone, Perplexity, ShuttleAI, Deepseek, Qwen 等。
- 🔧 **[代码解释器 (Code Interpreter) API](https://www.librechat.ai/docs/features/code_interpreter)**
- 安全的沙箱执行环境,支持 Python, Node.js (JS/TS), Go, C/C++, Java, PHP, Rust 和 Fortran。
- 无缝文件处理:直接上传、处理并下载文件。
- 隐私无忧:完全隔离且安全的执行环境。
- 🔦 **智能体与工具集成**
- **[LibreChat 智能体 (Agents)](https://www.librechat.ai/docs/features/agents)**
- 无代码定制助手:无需编程即可构建专业化的 AI 驱动助手。
- 智能体市场:发现并部署社区构建的智能体。
- 协作共享:与特定用户和群组共享智能体。
- 灵活且可扩展:支持 MCP 服务器、工具、文件搜索、代码执行等。
- 兼容自定义端点、OpenAI, Azure, Anthropic, AWS Bedrock, Google, Vertex AI, Responses API 等。
- [支持模型上下文协议 (MCP)](https://modelcontextprotocol.io/clients#librechat) 用于工具调用。
- 🔍 **网页搜索**
- 搜索互联网并检索相关信息以增强 AI 上下文。
- 结合搜索提供商、内容爬虫和结果重排序,确保最佳检索效果。
- **可定制 Jina 重排序**:配置自定义 Jina API URL 用于重排序服务。
- **[了解更多 →](https://www.librechat.ai/docs/features/web_search)**
- 🪄 **支持代码 Artifacts 的生成式 UI**
- [代码 Artifacts](https://youtu.be/GfTj7O4gmd0?si=WJbdnemZpJzBrJo3) 允许在对话中直接创建 React 组件、HTML 页面和 Mermaid 图表。
- 🎨 **图像生成与编辑**
- 使用 [GPT-Image-1](https://www.librechat.ai/docs/features/image_gen#1--openai-image-tools-recommended) 进行文生图与图生图。
- 支持 [DALL-E (3/2)](https://www.librechat.ai/docs/features/image_gen#2--dalle-legacy), [Stable Diffusion](https://www.librechat.ai/docs/features/image_gen#3--stable-diffusion-local), [Flux](https://www.librechat.ai/docs/features/image_gen#4--flux) 或任何 [MCP 服务器](https://www.librechat.ai/docs/features/image_gen#5--model-context-protocol-mcp)。
- 根据提示词生成惊艳的视觉效果,或通过指令精修现有图像。
- 💾 **预设与上下文管理**
- 创建、保存并分享自定义预设。
- 在对话中随时切换 AI 端点和预设。
- 编辑、重新提交并通过对话分支继续消息。
- 创建并与特定用户和群组共享提示词。
- [消息与对话分叉 (Fork)](https://www.librechat.ai/docs/features/fork) 以实现高级上下文控制。
- 💬 **多模态与文件交互**
- 使用 Claude 3, GPT-4.5, GPT-4o, o1, Llama-Vision 和 Gemini 上传并分析图像 📸。
- 支持通过自定义端点、OpenAI, Azure, Anthropic, AWS Bedrock 和 Google 进行文件对话 🗃️。
- 🌎 **多语言 UI**
- English, 中文 (简体), 中文 (繁體), العربية, Deutsch, Español, Français, Italiano
- Polski, Português (PT), Português (BR), Русский, 日本語, Svenska, 한국어, Tiếng Việt
- Türkçe, Nederlands, עברית, Català, Čeština, Dansk, Eesti, فارسی
- Suomi, Magyar, Հայերեն, Bahasa Indonesia, ქართული, Latviešu, ไทย, ئۇيغۇرچە
- 🧠 **推理 UI**
- 针对 DeepSeek-R1 等思维链/推理 AI 模型的动态推理 UI。
- 🎨 **可定制界面**
- 可定制的下拉菜单和界面,同时适配高级用户和初学者。
- 🌊 **[可恢复流 (Resumable Streams)](https://www.librechat.ai/docs/features/resumable_streams)**
- 永不丢失响应:AI 响应在连接中断后自动重连并继续。
- 多标签页与多设备同步:在多个标签页打开同一对话,或在另一设备上继续。
- 生产级可靠性:支持从单机部署到基于 Redis 的水平扩展。
- 🗣️ **语音与音频**
- 通过语音转文字和文字转语音实现免提对话。
- 自动发送并播放音频。
- 支持 OpenAI, Azure OpenAI 和 Elevenlabs。
- 📥 **导入与导出对话**
- 从 LibreChat, ChatGPT, Chatbot UI 导入对话。
- 将对话导出为截图、Markdown、文本、JSON。
- 🔍 **搜索与发现**
- 搜索所有消息和对话。
- 👥 **多用户与安全访问**
- 支持 OAuth2, LDAP 和电子邮件登录的多用户安全认证。
- 内置审核系统和 Token 消耗管理工具。
- ⚙️ **配置与部署**
- 支持代理、反向代理、Docker 及多种部署选项。
- 可完全本地运行或部署在云端。
- 📖 **开源与社区**
- 完全开源且在公众监督下开发。
- 社区驱动的开发、支持与反馈。
[查看我们的文档了解更多功能详情](https://docs.librechat.ai/) 📚
## 🪶 LibreChat:全方位的 AI 对话平台
LibreChat 是一个自托管的 AI 对话平台,在一个注重隐私的统一界面中整合了所有主流 AI 服务商。
除了对话功能外,LibreChat 还提供 AI 智能体、模型上下文协议 (MCP) 支持、Artifacts、代码解释器、自定义操作、对话搜索,以及企业级多用户认证。
开源、活跃开发中,专为重视 AI 基础设施自主可控的用户而构建。
---
## 🌐 资源
**GitHub 仓库:**
- **RAG API:** [github.com/danny-avila/rag_api](https://github.com/danny-avila/rag_api)
- **网站:** [github.com/LibreChat-AI/librechat.ai](https://github.com/LibreChat-AI/librechat.ai)
**其他:**
- **官方网站:** [librechat.ai](https://librechat.ai)
- **帮助文档:** [librechat.ai/docs](https://librechat.ai/docs)
- **博客:** [librechat.ai/blog](https://librechat.ai/blog)
---
## 📝 更新日志
访问发布页面和更新日志以了解最新动态:
- [发布页面 (Releases)](https://github.com/danny-avila/LibreChat/releases)
- [更新日志 (Changelog)](https://www.librechat.ai/changelog)
**⚠️ 在更新前请务必查看[更新日志](https://www.librechat.ai/changelog)以了解破坏性更改。**
---
## ⭐ Star 历史
<p align="center">
<a href="https://star-history.com/#danny-avila/LibreChat&Date">
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=danny-avila/LibreChat&type=Date&theme=dark" onerror="this.src='https://api.star-history.com/svg?repos=danny-avila/LibreChat&type=Date'" />
</a>
</p>
<p align="center">
<a href="https://trendshift.io/repositories/4685" target="_blank" style="padding: 10px;">
<img src="https://trendshift.io/api/badge/repositories/4685" alt="danny-avila%2FLibreChat | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/>
</a>
<a href="https://runacap.com/ross-index/q1-24/" target="_blank" rel="noopener" style="margin-left: 20px;">
<img style="width: 260px; height: 56px" src="https://runacap.com/wp-content/uploads/2024/04/ROSS_badge_white_Q1_2024.svg" alt="ROSS Index - 2024年第一季度增长最快的开源初创公司 | Runa Capital" width="260" height="56"/>
</a>
</p>
---
## ✨ 贡献
欢迎任何形式的贡献、建议、错误报告和修复!
对于新功能、组件或扩展,请在发送 PR 前开启 issue 进行讨论。
如果您想帮助我们将 LibreChat 翻译成您的母语,我们非常欢迎!改进翻译不仅能让全球用户更轻松地使用 LibreChat,还能提升整体用户体验。请查看我们的[翻译指南](https://www.librechat.ai/docs/translation)。
---
## 💖 感谢所有贡献者
<a href="https://github.com/danny-avila/LibreChat/graphs/contributors">
<img src="https://contrib.rocks/image?repo=danny-avila/LibreChat" />
</a>
---
## 🎉 特别鸣谢
感谢 [Locize](https://locize.com) 提供的翻译管理工具,支持 LibreChat 的多语言功能。
<p align="center">
<a href="https://locize.com" target="_blank" rel="noopener noreferrer">
<img src="https://github.com/user-attachments/assets/d6b70894-6064-475e-bb65-92a9e23e0077" alt="Locize Logo" height="50">
</a>
</p>
+1 -1
View File
@@ -8,7 +8,7 @@ const {
encodeAndFormatAudios,
encodeAndFormatVideos,
encodeAndFormatDocuments,
} = require('@hanzochat/api');
} = require('@librechat/api');
const {
Constants,
ErrorTypes,
+1 -1
View File
@@ -4,7 +4,7 @@ const { Ollama } = require('ollama');
const { sleep } = require('@librechat/agents');
const { logger } = require('@librechat/data-schemas');
const { Constants } = require('librechat-data-provider');
const { resolveHeaders, deriveBaseURL } = require('@hanzochat/api');
const { resolveHeaders, deriveBaseURL } = require('@librechat/api');
const ollamaPayloadSchema = z.object({
mirostat: z.number().optional(),
+6 -6
View File
@@ -49,7 +49,7 @@ Artifacts are for substantial, self-contained content that users might modify or
4. Add a \`type\` attribute to specify the type of content the artifact represents. Assign one of the following values to the \`type\` attribute:
- HTML: "text/html"
- The user interface can render single file HTML pages placed within the artifact tags. HTML, JS, and CSS should be in a single file when using the \`text/html\` type.
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/v1/chat/placeholder/400/320" alt="placeholder" />\`
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/api/placeholder/400/320" alt="placeholder" />\`
- The only place external scripts can be imported from is https://cdnjs.cloudflare.com
- Mermaid Diagrams: "application/vnd.mermaid"
- The user interface will render Mermaid diagrams placed within the artifact tags.
@@ -63,7 +63,7 @@ Artifacts are for substantial, self-contained content that users might modify or
- The assistant can use prebuilt components from the \`shadcn/ui\` library after it is imported: \`import { Alert, AlertDescription, AlertTitle, AlertDialog, AlertDialogAction } from '/components/ui/alert';\`. If using components from the shadcn/ui library, the assistant mentions this to the user and offers to help them install the components if necessary.
- Components MUST be imported from \`/components/ui/name\` and NOT from \`/components/name\` or \`@/components/ui/name\`.
- NO OTHER LIBRARIES (e.g. zod, hookform) ARE INSTALLED OR ABLE TO BE IMPORTED.
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/v1/chat/placeholder/400/320" alt="placeholder" />\`
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/api/placeholder/400/320" alt="placeholder" />\`
- If you are unable to follow the above requirements for any reason, don't use artifacts and use regular code blocks instead, which will not attempt to render the component.
5. Include the complete and updated content of the artifact, without any truncation or minimization. Don't use "// rest of the code remains the same...".
6. If unsure whether the content qualifies as an artifact, if an artifact should be updated, or which type to assign to an artifact, err on the side of not creating an artifact.
@@ -162,7 +162,7 @@ Artifacts are for substantial, self-contained content that users might modify or
4. Add a \`type\` attribute to specify the type of content the artifact represents. Assign one of the following values to the \`type\` attribute:
- HTML: "text/html"
- The user interface can render single file HTML pages placed within the artifact tags. HTML, JS, and CSS should be in a single file when using the \`text/html\` type.
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/v1/chat/placeholder/400/320" alt="placeholder" />\`
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/api/placeholder/400/320" alt="placeholder" />\`
- The only place external scripts can be imported from is https://cdnjs.cloudflare.com
- SVG: "image/svg+xml"
- The user interface will render the Scalable Vector Graphics (SVG) image within the artifact tags.
@@ -186,7 +186,7 @@ Artifacts are for substantial, self-contained content that users might modify or
- The assistant can use prebuilt components from the \`shadcn/ui\` library after it is imported: \`import { Alert, AlertDescription, AlertTitle, AlertDialog, AlertDialogAction } from '/components/ui/alert';\`. If using components from the shadcn/ui library, the assistant mentions this to the user and offers to help them install the components if necessary.
- Components MUST be imported from \`/components/ui/name\` and NOT from \`/components/name\` or \`@/components/ui/name\`.
- NO OTHER LIBRARIES (e.g. zod, hookform) ARE INSTALLED OR ABLE TO BE IMPORTED.
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/v1/chat/placeholder/400/320" alt="placeholder" />\`
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/api/placeholder/400/320" alt="placeholder" />\`
- When iterating on code, ensure that the code is complete and functional without any snippets, placeholders, or ellipses.
- If you are unable to follow the above requirements for any reason, don't use artifacts and use regular code blocks instead, which will not attempt to render the component.
5. Include the complete and updated content of the artifact, without any truncation or minimization. Don't use "// rest of the code remains the same...".
@@ -367,7 +367,7 @@ Artifacts are for substantial, self-contained content that users might modify or
4. Add a \`type\` attribute to specify the type of content the artifact represents. Assign one of the following values to the \`type\` attribute:
- HTML: "text/html"
- The user interface can render single file HTML pages placed within the artifact tags. HTML, JS, and CSS should be in a single file when using the \`text/html\` type.
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/v1/chat/placeholder/400/320" alt="placeholder" />\`
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/api/placeholder/400/320" alt="placeholder" />\`
- The only place external scripts can be imported from is https://cdnjs.cloudflare.com
- SVG: "image/svg+xml"
- The user interface will render the Scalable Vector Graphics (SVG) image within the artifact tags.
@@ -391,7 +391,7 @@ Artifacts are for substantial, self-contained content that users might modify or
- The assistant can use prebuilt components from the \`shadcn/ui\` library after it is imported: \`import { Alert, AlertDescription, AlertTitle, AlertDialog, AlertDialogAction } from '/components/ui/alert';\`. If using components from the shadcn/ui library, the assistant mentions this to the user and offers to help them install the components if necessary.
- Components MUST be imported from \`/components/ui/name\` and NOT from \`/components/name\` or \`@/components/ui/name\`.
- NO OTHER LIBRARIES (e.g. zod, hookform) ARE INSTALLED OR ABLE TO BE IMPORTED.
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/v1/chat/placeholder/400/320" alt="placeholder" />\`
- Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so \`<img src="/api/placeholder/400/320" alt="placeholder" />\`
- When iterating on code, ensure that the code is complete and functional without any snippets, placeholders, or ellipses.
- If you are unable to follow the above requirements for any reason, don't use artifacts and use regular code blocks instead, which will not attempt to render the component.
5. Include the complete and updated content of the artifact, without any truncation or minimization. Don't use "// rest of the code remains the same...".
@@ -1,6 +1,6 @@
const axios = require('axios');
const { logger } = require('@librechat/data-schemas');
const { isEnabled, generateShortLivedToken } = require('@hanzochat/api');
const { isEnabled, generateShortLivedToken } = require('@librechat/api');
const footer = `Use the context as your learned knowledge to better answer the user.
+1 -1
View File
@@ -1,4 +1,4 @@
const { getModelMaxTokens } = require('@hanzochat/api');
const { getModelMaxTokens } = require('@librechat/api');
const BaseClient = require('../BaseClient');
class FakeClient extends BaseClient {
+9 -19
View File
@@ -4,7 +4,7 @@ const { v4: uuidv4 } = require('uuid');
const { ProxyAgent, fetch } = require('undici');
const { Tool } = require('@langchain/core/tools');
const { logger } = require('@librechat/data-schemas');
const { getImageBasename, extractBaseURL } = require('@hanzochat/api');
const { getImageBasename, extractBaseURL } = require('@librechat/api');
const { FileContext, ContentTypes } = require('librechat-data-provider');
const dalle3JsonSchema = {
@@ -143,26 +143,16 @@ class DALLE3 extends Tool {
throw new Error('Missing required field: prompt');
}
// Model is configurable (DALLE3_MODEL) so this tool drives the Hanzo image
// family (e.g. zen3-image) through the same OpenAI /images/generations shape.
// `quality` and `style` are DALL-E-3-only knobs — send them ONLY for a dall-e
// model so a non-DALL-E backend never sees a parameter it may reject.
const model = process.env.DALLE3_MODEL || 'dall-e-3';
const isDallE = model.startsWith('dall-e');
const genParams = {
model,
size,
prompt: this.replaceUnwantedChars(prompt),
n: 1,
};
if (isDallE) {
genParams.quality = quality;
genParams.style = style;
}
let resp;
try {
resp = await this.openai.images.generate(genParams);
resp = await this.openai.images.generate({
model: 'dall-e-3',
quality,
style,
size,
prompt: this.replaceUnwantedChars(prompt),
n: 1,
});
} catch (error) {
logger.error('[DALL-E-3] Problem generating the image:', error);
return this
@@ -17,7 +17,7 @@ const {
loadServiceKey,
getBalanceConfig,
getTransactionsConfig,
} = require('@hanzochat/api');
} = require('@librechat/api');
const { getStrategyFunctions } = require('~/server/services/Files/strategies');
const { spendTokens } = require('~/models/spendTokens');
const { getFiles } = require('~/models/File');
@@ -7,7 +7,7 @@ const { tool } = require('@langchain/core/tools');
const { logger } = require('@librechat/data-schemas');
const { HttpsProxyAgent } = require('https-proxy-agent');
const { ContentTypes, EImageOutputType } = require('librechat-data-provider');
const { logAxiosError, oaiToolkit, extractBaseURL } = require('@hanzochat/api');
const { logAxiosError, oaiToolkit, extractBaseURL } = require('@librechat/api');
const { getStrategyFunctions } = require('~/server/services/Files/strategies');
const { getFiles } = require('~/models');
@@ -7,7 +7,7 @@ const { v4: uuidv4 } = require('uuid');
const { Tool } = require('@langchain/core/tools');
const { logger } = require('@librechat/data-schemas');
const { FileContext, ContentTypes } = require('librechat-data-provider');
const { getBasePath } = require('@hanzochat/api');
const { getBasePath } = require('@librechat/api');
const paths = require('~/config/paths');
const stableDiffusionJsonSchema = {
@@ -1,399 +0,0 @@
/**
* Regression tests for image tool agent mode — verifies that invoke() returns
* a ToolMessage with base64 in artifact.content rather than serialized into content.
*
* Root cause: DALLE3/FluxAPI/StableDiffusion extend LangChain's Tool but did not
* set responseFormat = 'content_and_artifact'. LangChain's invoke() would then
* JSON.stringify the entire [content, artifact] tuple into ToolMessage.content,
* dumping base64 into token counting and causing context exhaustion.
*/
const axios = require('axios');
const OpenAI = require('openai');
const undici = require('undici');
const fetch = require('node-fetch');
const { ContentTypes } = require('librechat-data-provider');
const { ToolMessage } = require('@librechat/agents/langchain/messages');
const StableDiffusionAPI = require('../StableDiffusion');
const FluxAPI = require('../FluxAPI');
const DALLE3 = require('../DALLE3');
jest.mock('axios');
jest.mock('openai');
jest.mock('node-fetch');
jest.mock('undici', () => ({
ProxyAgent: jest.fn(),
fetch: jest.fn(),
}));
jest.mock('@librechat/data-schemas', () => ({
logger: { info: jest.fn(), warn: jest.fn(), debug: jest.fn(), error: jest.fn() },
}));
jest.mock('path', () => ({
resolve: jest.fn(),
join: jest.fn().mockReturnValue('/mock/path'),
relative: jest.fn().mockReturnValue('relative/path'),
extname: jest.fn().mockReturnValue('.png'),
}));
jest.mock('fs', () => ({
existsSync: jest.fn().mockReturnValue(true),
mkdirSync: jest.fn(),
promises: { writeFile: jest.fn(), readFile: jest.fn(), unlink: jest.fn() },
}));
const FAKE_BASE64 = 'aGVsbG8=';
const makeToolCall = (name, args) => ({
id: 'call_test_123',
name,
args,
type: 'tool_call',
});
describe('image tools - agent mode ToolMessage format', () => {
const ENV_KEYS = ['DALLE_API_KEY', 'FLUX_API_KEY', 'SD_WEBUI_URL', 'PROXY'];
let savedEnv = {};
beforeEach(() => {
jest.clearAllMocks();
for (const key of ENV_KEYS) {
savedEnv[key] = process.env[key];
}
process.env.DALLE_API_KEY = 'test-dalle-key';
process.env.FLUX_API_KEY = 'test-flux-key';
process.env.SD_WEBUI_URL = 'http://localhost:7860';
delete process.env.PROXY;
});
afterEach(() => {
for (const key of ENV_KEYS) {
if (savedEnv[key] === undefined) {
delete process.env[key];
} else {
process.env[key] = savedEnv[key];
}
}
savedEnv = {};
});
describe('DALLE3', () => {
beforeEach(() => {
OpenAI.mockImplementation(() => ({
images: {
generate: jest.fn().mockResolvedValue({
data: [{ url: 'https://example.com/image.png' }],
}),
},
}));
undici.fetch.mockResolvedValue({
arrayBuffer: () => Promise.resolve(Buffer.from(FAKE_BASE64, 'base64')),
});
});
it('sets responseFormat to content_and_artifact when isAgent is true', () => {
const dalle = new DALLE3({ isAgent: true });
expect(dalle.responseFormat).toBe('content_and_artifact');
});
it('does not set responseFormat when isAgent is false', () => {
const dalle = new DALLE3({ isAgent: false, processFileURL: jest.fn() });
expect(dalle.responseFormat).not.toBe('content_and_artifact');
});
it('keeps tenant context without retaining the request object', () => {
const req = {
user: { id: 'user-1', tenantId: 'tenant-a' },
body: { conversationId: 'convo-1', isTemporary: 'true' },
config: { interfaceConfig: { retentionMode: 'all' } },
socket: {},
};
const dalle = new DALLE3({ isAgent: false, processFileURL: jest.fn(), req });
expect(dalle.tenantId).toBe('tenant-a');
expect(dalle.req).toBeUndefined();
expect(dalle.retentionRequest).toEqual({
user: { id: 'user-1', tenantId: 'tenant-a' },
body: { conversationId: 'convo-1', isTemporary: 'true' },
config: { interfaceConfig: { retentionMode: 'all' } },
});
});
it('invoke() returns ToolMessage with base64 in artifact, not serialized in content', async () => {
const dalle = new DALLE3({ isAgent: true });
const result = await dalle.invoke(
makeToolCall('dalle', {
prompt: 'a box',
quality: 'standard',
size: '1024x1024',
style: 'vivid',
}),
);
expect(result).toBeInstanceOf(ToolMessage);
const contentStr =
typeof result.content === 'string' ? result.content : JSON.stringify(result.content);
expect(contentStr).not.toContain(FAKE_BASE64);
expect(result.artifact).toBeDefined();
const artifactContent = result.artifact?.content;
expect(Array.isArray(artifactContent)).toBe(true);
expect(artifactContent[0].type).toBe(ContentTypes.IMAGE_URL);
expect(artifactContent[0].image_url.url).toContain('base64');
});
it('invoke() returns ToolMessage with error string in content when API fails', async () => {
OpenAI.mockImplementation(() => ({
images: { generate: jest.fn().mockRejectedValue(new Error('API error')) },
}));
const dalle = new DALLE3({ isAgent: true });
const result = await dalle.invoke(
makeToolCall('dalle', {
prompt: 'a box',
quality: 'standard',
size: '1024x1024',
style: 'vivid',
}),
);
expect(result).toBeInstanceOf(ToolMessage);
const contentStr =
typeof result.content === 'string' ? result.content : JSON.stringify(result.content);
expect(contentStr).toContain('Something went wrong');
expect(result.artifact).toBeDefined();
});
});
describe('FluxAPI', () => {
beforeEach(() => {
jest.useFakeTimers();
axios.post.mockResolvedValue({ data: { id: 'task-123' } });
axios.get.mockResolvedValue({
data: { status: 'Ready', result: { sample: 'https://example.com/image.png' } },
});
fetch.mockResolvedValue({
arrayBuffer: () => Promise.resolve(Buffer.from(FAKE_BASE64, 'base64')),
});
});
afterEach(() => {
jest.useRealTimers();
});
it('sets responseFormat to content_and_artifact when isAgent is true', () => {
const flux = new FluxAPI({ isAgent: true });
expect(flux.responseFormat).toBe('content_and_artifact');
});
it('does not set responseFormat when isAgent is false', () => {
const flux = new FluxAPI({ isAgent: false, processFileURL: jest.fn() });
expect(flux.responseFormat).not.toBe('content_and_artifact');
});
it('keeps tenant context without retaining the request object', () => {
const req = {
user: { id: 'user-1', tenantId: 'tenant-a' },
body: { conversationId: 'convo-1', isTemporary: 'true' },
config: { interfaceConfig: { retentionMode: 'all' } },
socket: {},
};
const flux = new FluxAPI({ isAgent: false, processFileURL: jest.fn(), req });
expect(flux.tenantId).toBe('tenant-a');
expect(flux.req).toBeUndefined();
expect(flux.retentionRequest).toEqual({
user: { id: 'user-1', tenantId: 'tenant-a' },
body: { conversationId: 'convo-1', isTemporary: 'true' },
config: { interfaceConfig: { retentionMode: 'all' } },
});
});
it('passes minimal retention context when saving generated images', async () => {
const processFileURL = jest.fn().mockResolvedValue({ filepath: '/images/generated.png' });
const req = {
user: { id: 'user-1', tenantId: 'tenant-a' },
body: { conversationId: 'convo-1', isTemporary: 'true' },
config: { interfaceConfig: { retentionMode: 'all' } },
socket: {},
};
const flux = new FluxAPI({
isAgent: false,
processFileURL,
req,
userId: 'user-1',
fileStrategy: 'local',
});
const invokePromise = flux.invoke(
makeToolCall('flux', { prompt: 'a box', endpoint: '/v1/flux-dev' }),
);
await jest.runAllTimersAsync();
await invokePromise;
expect(processFileURL).toHaveBeenCalledWith(
expect.objectContaining({
req: {
user: { id: 'user-1', tenantId: 'tenant-a' },
body: { conversationId: 'convo-1', isTemporary: 'true' },
config: { interfaceConfig: { retentionMode: 'all' } },
},
}),
);
});
it('passes minimal retention context when saving finetuned generated images', async () => {
const processFileURL = jest.fn().mockResolvedValue({ filepath: '/images/generated.png' });
const req = {
user: { id: 'user-1', tenantId: 'tenant-a' },
body: { conversationId: 'convo-1', isTemporary: 'true' },
config: { interfaceConfig: { retentionMode: 'all' } },
socket: {},
};
const flux = new FluxAPI({
isAgent: false,
processFileURL,
req,
userId: 'user-1',
fileStrategy: 'local',
});
const invokePromise = flux.invoke(
makeToolCall('flux', {
action: 'generate_finetuned',
prompt: 'a box',
finetune_id: 'ft-abc123',
endpoint: '/v1/flux-pro-finetuned',
}),
);
await jest.runAllTimersAsync();
await invokePromise;
expect(processFileURL).toHaveBeenCalledWith(
expect.objectContaining({
req: {
user: { id: 'user-1', tenantId: 'tenant-a' },
body: { conversationId: 'convo-1', isTemporary: 'true' },
config: { interfaceConfig: { retentionMode: 'all' } },
},
}),
);
});
it('invoke() returns ToolMessage with base64 in artifact, not serialized in content', async () => {
const flux = new FluxAPI({ isAgent: true });
const invokePromise = flux.invoke(
makeToolCall('flux', { prompt: 'a box', endpoint: '/v1/flux-dev' }),
);
await jest.runAllTimersAsync();
const result = await invokePromise;
expect(result).toBeInstanceOf(ToolMessage);
const contentStr =
typeof result.content === 'string' ? result.content : JSON.stringify(result.content);
expect(contentStr).not.toContain(FAKE_BASE64);
expect(result.artifact).toBeDefined();
const artifactContent = result.artifact?.content;
expect(Array.isArray(artifactContent)).toBe(true);
expect(artifactContent[0].type).toBe(ContentTypes.IMAGE_URL);
expect(artifactContent[0].image_url.url).toContain('base64');
});
it('invoke() returns ToolMessage with base64 in artifact for generate_finetuned action', async () => {
const flux = new FluxAPI({ isAgent: true });
const invokePromise = flux.invoke(
makeToolCall('flux', {
action: 'generate_finetuned',
prompt: 'a box',
finetune_id: 'ft-abc123',
endpoint: '/v1/flux-pro-finetuned',
}),
);
await jest.runAllTimersAsync();
const result = await invokePromise;
expect(result).toBeInstanceOf(ToolMessage);
const contentStr =
typeof result.content === 'string' ? result.content : JSON.stringify(result.content);
expect(contentStr).not.toContain(FAKE_BASE64);
expect(result.artifact).toBeDefined();
const artifactContent = result.artifact?.content;
expect(Array.isArray(artifactContent)).toBe(true);
expect(artifactContent[0].type).toBe(ContentTypes.IMAGE_URL);
expect(artifactContent[0].image_url.url).toContain('base64');
});
it('invoke() returns ToolMessage with error string in content when task submission fails', async () => {
axios.post.mockRejectedValue(new Error('Network error'));
const flux = new FluxAPI({ isAgent: true });
const invokePromise = flux.invoke(
makeToolCall('flux', { prompt: 'a box', endpoint: '/v1/flux-dev' }),
);
await jest.runAllTimersAsync();
const result = await invokePromise;
expect(result).toBeInstanceOf(ToolMessage);
const contentStr =
typeof result.content === 'string' ? result.content : JSON.stringify(result.content);
expect(contentStr).toContain('Something went wrong');
expect(result.artifact).toBeDefined();
});
});
describe('StableDiffusion', () => {
beforeEach(() => {
axios.post.mockResolvedValue({
data: {
images: [FAKE_BASE64],
info: JSON.stringify({ height: 1024, width: 1024, seed: 42, infotexts: [] }),
},
});
});
it('sets responseFormat to content_and_artifact when isAgent is true', () => {
const sd = new StableDiffusionAPI({ isAgent: true, override: true });
expect(sd.responseFormat).toBe('content_and_artifact');
});
it('does not set responseFormat when isAgent is false', () => {
const sd = new StableDiffusionAPI({
isAgent: false,
override: true,
uploadImageBuffer: jest.fn(),
});
expect(sd.responseFormat).not.toBe('content_and_artifact');
});
it('invoke() returns ToolMessage with base64 in artifact, not serialized in content', async () => {
const sd = new StableDiffusionAPI({ isAgent: true, override: true, userId: 'user-1' });
const result = await sd.invoke(
makeToolCall('stable-diffusion', { prompt: 'a box', negative_prompt: '' }),
);
expect(result).toBeInstanceOf(ToolMessage);
const contentStr =
typeof result.content === 'string' ? result.content : JSON.stringify(result.content);
expect(contentStr).not.toContain(FAKE_BASE64);
expect(result.artifact).toBeDefined();
const artifactContent = result.artifact?.content;
expect(Array.isArray(artifactContent)).toBe(true);
expect(artifactContent[0].type).toBe(ContentTypes.IMAGE_URL);
expect(artifactContent[0].image_url.url).toContain('base64');
});
it('invoke() returns ToolMessage with error string in content when API fails', async () => {
axios.post.mockRejectedValue(new Error('Connection refused'));
const sd = new StableDiffusionAPI({ isAgent: true, override: true, userId: 'user-1' });
const result = await sd.invoke(
makeToolCall('stable-diffusion', { prompt: 'a box', negative_prompt: '' }),
);
expect(result).toBeInstanceOf(ToolMessage);
const contentStr =
typeof result.content === 'string' ? result.content : JSON.stringify(result.content);
expect(contentStr).toContain('Error making API request');
});
});
});
+1 -1
View File
@@ -1,7 +1,7 @@
const axios = require('axios');
const { tool } = require('@langchain/core/tools');
const { logger } = require('@librechat/data-schemas');
const { generateShortLivedToken } = require('@hanzochat/api');
const { generateShortLivedToken } = require('@librechat/api');
const { Tools, EToolResources } = require('librechat-data-provider');
const { filterFilesByAgentAccess } = require('~/server/services/Files/permissions');
const { getFiles } = require('~/models');
+2 -27
View File
@@ -12,9 +12,7 @@ const {
loadWebSearchAuth,
buildImageToolContext,
buildWebSearchContext,
resolveHanzoCloudKey,
isHanzoPerUserKeyEnabled,
} = require('@hanzochat/api');
} = require('@librechat/api');
const { getMCPServersRegistry } = require('~/config');
const {
Tools,
@@ -234,30 +232,7 @@ const loadTools = async ({
const requestedTools = {};
if (functions === true) {
// dalle (image generation) drives the Hanzo image family via a PER-USER hk-
// key so generation is metered to the signed-in user — mirroring the chat
// per-user key path in custom/initialize.ts. A guest keeps the shared env
// key; an authenticated user whose key cannot be resolved FAILS CLOSED
// (throws) rather than silently billing the shared org. (Custom constructors
// take precedence over the generic toolConstructors path in the loop below.)
customConstructors.dalle = async () => {
const authFields = getAuthFields('dalle');
const authValues = await loadAuthValues({ userId: user, authFields });
const billingUser = options.req?.user;
const isAuthenticatedUser = Boolean(
billingUser && !billingUser.guest && billingUser.email,
);
if (isHanzoPerUserKeyEnabled() && isAuthenticatedUser) {
const perUserKey = await resolveHanzoCloudKey(billingUser);
if (!perUserKey) {
throw new Error(
'Your Hanzo Cloud account is not linked for billing yet. Please sign out and back in, then claim your starter credit at https://billing.hanzo.ai',
);
}
authValues.DALLE3_API_KEY = perUserKey;
}
return new DALLE3({ ...imageGenOptions, ...authValues, userId: user });
};
toolConstructors.dalle = DALLE3;
}
/** @type {ImageGenOptions} */
+1 -1
View File
@@ -1,5 +1,5 @@
const { logger } = require('@librechat/data-schemas');
const { isEnabled, math } = require('@hanzochat/api');
const { isEnabled, math } = require('@librechat/api');
const { ViolationTypes } = require('librechat-data-provider');
const { deleteAllUserSessions } = require('~/models');
const { removePorts } = require('~/server/utils');
+1 -1
View File
@@ -1,4 +1,4 @@
const { isEnabled } = require('@hanzochat/api');
const { isEnabled } = require('@librechat/api');
const { Time, CacheKeys } = require('librechat-data-provider');
const getLogStores = require('./getLogStores');
+1 -1
View File
@@ -7,7 +7,7 @@ const {
sessionCache,
standardCache,
violationCache,
} = require('@hanzochat/api');
} = require('@librechat/api');
const namespaces = {
[ViolationTypes.GENERAL]: new Keyv({ store: logFile, namespace: 'violations' }),
+1 -1
View File
@@ -1,4 +1,4 @@
const { isEnabled } = require('@hanzochat/api');
const { isEnabled } = require('@librechat/api');
const { ViolationTypes } = require('librechat-data-provider');
const getLogStores = require('./getLogStores');
const banViolation = require('./banViolation');
-72
View File
@@ -1,72 +0,0 @@
const fs = require('fs');
const ORIGINAL_ENV = process.env;
const mockDataSchemas = () => {
jest.doMock('@librechat/data-schemas', () => ({
getTenantId: jest.fn(),
getUserId: jest.fn(),
getRequestId: jest.fn(),
SYSTEM_TENANT_ID: 'system',
}));
};
const mockReadOnlyDockerLogDir = () => {
const originalExistsSync = fs.existsSync;
const originalMkdirSync = fs.mkdirSync;
jest.spyOn(process, 'cwd').mockReturnValue('/app');
jest
.spyOn(fs, 'existsSync')
.mockImplementation((target) =>
target === '/app/logs' ? false : originalExistsSync.call(fs, target),
);
return jest.spyOn(fs, 'mkdirSync').mockImplementation((target, options) => {
if (target === '/app/logs') {
throw new Error('Attempted to create Docker log directory');
}
return originalMkdirSync.call(fs, target, options);
});
};
const prepareLoggerWithoutFileLogging = () => {
jest.resetModules();
jest.clearAllMocks();
mockDataSchemas();
process.env = {
...ORIGINAL_ENV,
DEBUG_LOGGING: 'true',
LOG_TO_FILE: 'false',
};
return mockReadOnlyDockerLogDir();
};
describe('LOG_TO_FILE', () => {
afterEach(() => {
process.env = ORIGINAL_ENV;
jest.restoreAllMocks();
});
it('does not create the API log directory when winston file logging is disabled', () => {
const mkdirSyncSpy = prepareLoggerWithoutFileLogging();
expect(() => require('../winston')).not.toThrow();
const winston = require('winston');
expect(winston.transports.DailyRotateFile).not.toHaveBeenCalled();
expect(mkdirSyncSpy).not.toHaveBeenCalledWith('/app/logs', expect.anything());
});
it('does not create the API log directory when Meili file logging is disabled', () => {
const mkdirSyncSpy = prepareLoggerWithoutFileLogging();
expect(() => require('../meiliLogger')).not.toThrow();
const winston = require('winston');
expect(winston.transports.DailyRotateFile).not.toHaveBeenCalled();
expect(mkdirSyncSpy).not.toHaveBeenCalledWith('/app/logs', expect.anything());
});
});
-430
View File
@@ -1,430 +0,0 @@
jest.unmock('winston');
const { formatConsoleMeta, redactMessage, redactFormat, debugTraverse } =
jest.requireActual('../parsers');
const SPLAT_SYMBOL = Symbol.for('splat');
describe('formatConsoleMeta', () => {
it('returns empty string when there is no user metadata', () => {
expect(
formatConsoleMeta({
level: 'error',
message: 'oops',
timestamp: '2026-04-18 02:25:22',
}),
).toBe('');
});
it('serializes user-supplied metadata keys', () => {
const meta = formatConsoleMeta({
level: 'error',
message: '[agents:summarize] Summarization LLM call failed',
timestamp: '2026-04-18 02:25:22',
provider: 'azureOpenAI',
model: 'gpt-5.4-mini',
messagesToRefineCount: 42,
});
expect(meta).toContain('"provider":"azureOpenAI"');
expect(meta).toContain('"model":"gpt-5.4-mini"');
expect(meta).toContain('"messagesToRefineCount":42');
});
it('omits the system tenant sentinel from metadata trailers', () => {
const meta = formatConsoleMeta({
level: 'warn',
message: 'system task',
timestamp: 'ts',
tenantId: '__SYSTEM__',
userId: 'user-1',
});
expect(meta).toBe('{"userId":"user-1"}');
});
it('ignores reserved winston keys but preserves legitimate fields like _id', () => {
const meta = formatConsoleMeta({
level: 'error',
message: 'boom',
timestamp: 'ts',
splat: [1, 2],
_id: '507f191e810c19729de860ea',
userField: 'keep',
});
expect(meta).toContain('"_id":"507f191e810c19729de860ea"');
expect(meta).toContain('"userField":"keep"');
expect(meta).not.toContain('"splat"');
});
it('drops numeric-index-like keys (splat artifacts from primitive args)', () => {
const meta = formatConsoleMeta({
level: 'warn',
message: 'Unhandled step:',
timestamp: 'ts',
0: 'f',
1: 'o',
2: 'o',
realField: 'real',
});
expect(meta).toBe('{"realField":"real"}');
});
it('drops empty, null, undefined, function, and symbol values', () => {
const meta = formatConsoleMeta({
level: 'warn',
message: 'noise',
timestamp: 'ts',
empty: '',
nullish: null,
undef: undefined,
fn: () => 1,
sym: Symbol('x'),
kept: 'yes',
});
expect(meta).toBe('{"kept":"yes"}');
});
it('truncates very long string values to avoid console spam', () => {
const longString = 'x'.repeat(5000);
const meta = formatConsoleMeta({
level: 'error',
message: 'long',
timestamp: 'ts',
errorStack: longString,
});
expect(meta.length).toBeLessThan(longString.length);
expect(meta).toContain('...');
});
it('preserves non-circular fields when one value is circular', () => {
const circular = {};
circular.self = circular;
const meta = formatConsoleMeta({
level: 'error',
message: 'circular',
timestamp: 'ts',
provider: 'openai',
model: 'gpt-5.4-mini',
circular,
});
expect(meta).toContain('"provider":"openai"');
expect(meta).toContain('"model":"gpt-5.4-mini"');
expect(meta).toContain('[Circular]');
});
it('falls back to per-field serialization when a value toJSON throws', () => {
const meta = formatConsoleMeta({
level: 'error',
message: 'crash',
timestamp: 'ts',
provider: 'azure',
model: 'gpt-5.4-mini',
broken: {
toJSON() {
throw new Error('nope');
},
},
});
expect(meta).toContain('"provider":"azure"');
expect(meta).toContain('"model":"gpt-5.4-mini"');
expect(meta).toContain('[Unserializable]');
});
it('redacts sensitive strings nested inside metadata objects', () => {
const meta = formatConsoleMeta({
level: 'error',
message: 'nested leak',
timestamp: 'ts',
config: {
headers: {
authorization: 'Bearer eyJhbGciOi.nestedTokenValue',
},
query: 'https://example.com/?key=AIzaNested',
},
openaiKey: 'sk-outerKey123',
});
expect(meta).not.toContain('eyJhbGciOi.nestedTokenValue');
expect(meta).not.toContain('AIzaNested');
expect(meta).not.toContain('sk-outerKey123');
expect(meta).toContain('Bearer [REDACTED]');
expect(meta).toContain('key=[REDACTED]');
expect(meta).toContain('sk-[REDACTED]');
});
it('redacts the Azure-style mixed-case Api-Key header', () => {
const meta = formatConsoleMeta({
level: 'error',
message: 'azure call',
timestamp: 'ts',
headers: 'Api-Key: 0123456789abcdef',
});
expect(meta).not.toContain('0123456789abcdef');
expect(meta).toContain('Api-Key: [REDACTED]');
});
it('redacts sensitive patterns inside string metadata values', () => {
const meta = formatConsoleMeta({
level: 'error',
message: 'leak test',
timestamp: 'ts',
openaiKey: 'sk-abc123def456',
auth: 'Bearer eyJhbGciOi...tokenvalue',
google: 'https://example.com/?key=AIzaSyXX',
});
expect(meta).not.toContain('sk-abc123def456');
expect(meta).not.toContain('eyJhbGciOi...tokenvalue');
expect(meta).not.toContain('AIzaSyXX');
expect(meta).toContain('sk-[REDACTED]');
expect(meta).toContain('Bearer [REDACTED]');
expect(meta).toContain('key=[REDACTED]');
});
it('redacts multiple occurrences of the same pattern in one value', () => {
const meta = formatConsoleMeta({
level: 'error',
message: 'two keys',
timestamp: 'ts',
combined: 'first sk-aaa and then sk-bbb',
});
expect(meta).not.toContain('sk-aaa');
expect(meta).not.toContain('sk-bbb');
expect(meta.match(/sk-\[REDACTED\]/g)?.length).toBe(2);
});
});
describe('redactMessage', () => {
it('redacts sk- keys that are not at line start (inside JSON-like text)', () => {
const input = '{"apiKey":"sk-abc123"}';
expect(redactMessage(input)).toBe('{"apiKey":"sk-[REDACTED]"}');
});
it('redacts all sk- occurrences in a single pass', () => {
const input = 'sk-one sk-two sk-three';
expect(redactMessage(input)).toBe('sk-[REDACTED] sk-[REDACTED] sk-[REDACTED]');
});
it('trims redacted output when trimLength is provided', () => {
const input = 'Bearer supersecretvalue';
expect(redactMessage(input, 10)).toBe('Bearer [RE...');
});
it('returns empty string for falsy input', () => {
expect(redactMessage('')).toBe('');
expect(redactMessage(undefined)).toBe('');
});
it('does not redact ordinary words that contain "sk-" inside them', () => {
expect(redactMessage('task-runner failed')).toBe('task-runner failed');
expect(redactMessage('mask-value computed')).toBe('mask-value computed');
expect(redactMessage('desk-lamp is on')).toBe('desk-lamp is on');
});
it('does not redact words that contain "key=" inside them', () => {
expect(redactMessage('monkey=10 bananas')).toBe('monkey=10 bananas');
});
it('still redacts standalone sk- keys at word boundaries', () => {
expect(redactMessage('token: sk-abc123def')).toBe('token: sk-[REDACTED]');
expect(redactMessage('"sk-abc123def"')).toBe('"sk-[REDACTED]"');
});
});
describe('redactFormat', () => {
const runFormat = (info) => redactFormat().transform(info) || info;
it('redacts info.message for error level before any colorize step runs', () => {
const info = runFormat({ level: 'error', message: 'Bearer secretvalue' });
expect(info.message).toBe('Bearer [REDACTED]');
});
it('redacts info.message for warn level too (avoids ANSI boundary issues later)', () => {
const info = runFormat({ level: 'warn', message: 'apiKey=sk-abc123def' });
expect(info.message).toContain('sk-[REDACTED]');
});
it('leaves info.message untouched for info and debug levels', () => {
const infoInfo = runFormat({ level: 'info', message: 'Bearer looksSensitive' });
expect(infoInfo.message).toBe('Bearer looksSensitive');
const infoDebug = runFormat({ level: 'debug', message: 'Bearer looksSensitive' });
expect(infoDebug.message).toBe('Bearer looksSensitive');
});
});
describe('debugTraverse', () => {
const runFormatter = (info) => {
const transformed = debugTraverse.transform(info);
const MESSAGE = Symbol.for('message');
if (transformed && typeof transformed === 'object') {
return transformed[MESSAGE] ?? String(transformed);
}
return String(transformed);
};
const buildInfo = (level, meta) => {
const info = {
level,
message: 'test',
timestamp: 'ts',
...meta,
};
info[SPLAT_SYMBOL] = [meta];
return info;
};
it('redacts sensitive strings in metadata for error level', () => {
const out = runFormatter(buildInfo('error', { auth: 'Bearer eyJabc123', openai: 'sk-abc123' }));
expect(out).not.toContain('eyJabc123');
expect(out).not.toContain('sk-abc123');
expect(out).toContain('Bearer [REDACTED]');
expect(out).toContain('sk-[REDACTED]');
});
it('redacts sensitive strings in metadata for warn level', () => {
const out = runFormatter(buildInfo('warn', { header: 'Bearer supersecrettoken' }));
expect(out).not.toContain('supersecrettoken');
expect(out).toContain('Bearer [REDACTED]');
});
it('preserves debug-level metadata unmodified (existing behavior)', () => {
const out = runFormatter(buildInfo('debug', { someField: 'not-sensitive' }));
expect(out).toContain('not-sensitive');
});
it('prefers structured metadata over a consumed printf arg in SPLAT[0]', () => {
const info = {
level: 'warn',
message: 'failed for tenant-7',
timestamp: 'ts',
provider: 'openai',
[SPLAT_SYMBOL]: ['tenant-7', { provider: 'openai' }],
};
const out = runFormatter(info);
expect(out).toContain('openai');
const tenantMatches = out.match(/tenant-7/g) ?? [];
expect(tenantMatches.length).toBeLessThanOrEqual(1);
});
it('does not duplicate a consumed %s arg when there is no structured metadata', () => {
const info = {
level: 'warn',
message: 'failed for tenant-7',
timestamp: 'ts',
[SPLAT_SYMBOL]: ['tenant-7'],
};
const out = runFormatter(info);
const tenantMatches = out.match(/tenant-7/g) ?? [];
expect(tenantMatches.length).toBe(1);
});
it('appends request context metadata for non-debug lines', () => {
const out = runFormatter(
buildInfo('info', {
tenantId: 'tenant-1',
userId: 'user-1',
requestId: 'req-1',
}),
);
expect(out).toContain('"tenantId":"tenant-1"');
expect(out).toContain('"userId":"user-1"');
expect(out).toContain('"requestId":"req-1"');
});
it('does not append the system tenant sentinel as tenantId', () => {
const out = runFormatter(
buildInfo('info', {
tenantId: '__SYSTEM__',
userId: 'user-1',
requestId: 'req-1',
}),
);
expect(out).not.toContain('__SYSTEM__');
expect(out).not.toContain('"tenantId"');
expect(out).toContain('"userId":"user-1"');
expect(out).toContain('"requestId":"req-1"');
});
it('omits the system tenant sentinel from debug object metadata', () => {
const out = runFormatter(
buildInfo('debug', {
tenantId: '__SYSTEM__',
userId: 'user-1',
}),
);
expect(out).not.toContain('__SYSTEM__');
expect(out).not.toMatch(/tenantId:/);
expect(out).toContain('userId');
});
it('appends request context metadata for debug lines without object metadata', () => {
const info = {
level: 'debug',
message: 'prefix:',
timestamp: 'ts',
tenantId: 'tenant-1',
userId: 'user-1',
requestId: 'req-1',
[SPLAT_SYMBOL]: ['detailValueXYZ'],
};
const out = runFormatter(info);
expect(out).toContain('detailValueXYZ');
expect(out).toContain('"tenantId":"tenant-1"');
expect(out).toContain('"userId":"user-1"');
expect(out).toContain('"requestId":"req-1"');
});
it('omits numeric splat-artifact keys from the traversed output', () => {
const info = {
level: 'error',
message: 'boom',
timestamp: 'ts',
0: 'x',
1: 'y',
realField: 'keep',
[SPLAT_SYMBOL]: [{ realField: 'keep' }],
};
const out = runFormatter(info);
expect(out).toContain('realField');
expect(out).toContain('keep');
expect(out).not.toMatch(/^\s*0:/m);
expect(out).not.toMatch(/^\s*1:/m);
});
it('surfaces unconsumed primitive SPLAT[0] (no %s in message) for debug level', () => {
const info = {
level: 'debug',
message: 'prefix:',
timestamp: 'ts',
[SPLAT_SYMBOL]: ['detailValueXYZ'],
};
const out = runFormatter(info);
expect(out).toContain('detailValueXYZ');
});
it('still surfaces array metadata in SPLAT[0] when no object is extracted', () => {
const info = {
level: 'debug',
message: 'list',
timestamp: 'ts',
[SPLAT_SYMBOL]: [['alpha', 'beta', 'gamma']],
};
const out = runFormatter(info);
expect(out).toContain('alpha');
expect(out).toContain('beta');
expect(out).toContain('gamma');
});
});
+1 -1
View File
@@ -5,7 +5,7 @@ const {
FlowStateManager,
MCPServersRegistry,
OAuthReconnectionManager,
} = require('@hanzochat/api');
} = require('@librechat/api');
const logger = require('./winston');
global.EventSource = EventSource;
+63
View File
@@ -0,0 +1,63 @@
'use strict';
/**
* Hanzo Base data-layer adapter — entry point.
*
* Exposes:
* - `mongoose` : the mongoose-compatible facade (feed to createModels/createMethods)
* - `connectDb()`: initialise the Base client and provision collections
* - `store` : the underlying DocumentStore (for tests/introspection)
*/
const { logger } = require('@librechat/data-schemas');
const mongoose = require('./mongoose');
const store = require('./store');
function resolveConfig() {
const url = process.env.HANZO_BASE_URL || process.env.BASE_URL || null;
const token = process.env.HANZO_BASE_TOKEN || process.env.BASE_TOKEN || null;
return { url, token };
}
let ready = null;
/**
* Connect to Hanzo Base and ensure every registered model's collection exists.
* Idempotent: repeated calls return the same in-flight/settled promise.
*/
async function connectDb() {
if (ready) {
return ready;
}
ready = (async () => {
const { url, token } = resolveConfig();
if (!url) {
throw new Error(
'[BaseAdapter] Please define HANZO_BASE_URL (Hanzo Base instance URL) for the data layer',
);
}
await store.init({ url, token });
try {
await store.health();
logger.info(`[BaseAdapter] Connected to Hanzo Base at ${url}`);
} catch (err) {
logger.error('[BaseAdapter] Base health check failed', err);
throw err;
}
await ensureCollections();
return mongoose;
})();
return ready;
}
/** Provision the Base collection for every model registered on the facade. */
async function ensureCollections() {
const registered = mongoose.models;
const names = Object.keys(registered);
for (const name of names) {
await registered[name].ensure();
}
logger.info(`[BaseAdapter] Provisioned ${names.length} Base collections`);
}
module.exports = { mongoose, store, connectDb, ensureCollections };
+828
View File
@@ -0,0 +1,828 @@
'use strict';
/**
* BaseModel — a Mongoose-Model-compatible facade backed by Hanzo Base.
*
* One instance per collection. It presents the subset of the Mongoose Model
* static API that the LibreChat data layer actually uses (find / findOne /
* findOneAndUpdate / create / updateOne / deleteMany / countDocuments /
* bulkWrite / aggregate …) and executes it against a document `store`
* (see store.js) whose records hold the full Mongo document as JSON.
*
* Correctness comes from query.js: candidate records are fetched from Base
* (with best-effort filter pushdown) and every predicate, update operator,
* projection and sort is then evaluated in JS.
*/
const {
matches,
applyUpdate,
parseProjection,
applyProjection,
sortDocs,
getPath,
setPath,
idString,
} = require('./query');
const { describeSchema, resolveDefault } = require('./schema');
const { generateObjectId } = require('./objectId');
function now() {
return new Date();
}
/**
* Parse a MeiliSearch-style filter string (e.g. `user = "abc"`) into a Mongo
* equality filter for scoping. Supports the `field = "value"` / `field = 'value'`
* clauses the chat app emits; unknown syntax is ignored (no scoping).
*/
function parseMeiliFilter(filter) {
const scope = {};
if (typeof filter !== 'string') {
return scope;
}
const re = /(\w+)\s*=\s*(?:"([^"]*)"|'([^']*)')/g;
let m;
while ((m = re.exec(filter)) !== null) {
scope[m[1]] = m[2] !== undefined ? m[2] : m[3];
}
return scope;
}
/** Extract top-level equality conditions from a filter (for upsert seeding). */
function equalitySeed(filter) {
const seed = {};
if (!filter || typeof filter !== 'object') {
return seed;
}
for (const [k, v] of Object.entries(filter)) {
if (k.startsWith('$')) {
continue;
}
if (v === null || typeof v !== 'object' || v instanceof Date || Array.isArray(v)) {
seed[k] = v;
}
}
return seed;
}
class BaseModel {
/**
* @param {string} name - Mongoose model name (e.g. "Conversation")
* @param {import('mongoose').Schema} schema
* @param {import('./store').DocumentStore} store
*/
constructor(name, schema, store) {
this.modelName = name;
this.schema = schema;
this.store = store;
this.collection = { collectionName: name.toLowerCase() };
this.collectionName = name.toLowerCase();
const desc = describeSchema(schema);
this._defaults = desc.defaults;
this._timestamps = desc.timestamps;
this._promoted = desc.promoted;
this._promotedNames = desc.promoted.map((p) => p.name);
this._deselected = desc.deselected;
this._dateFields = desc.dateFields;
this._searchable = desc.searchable;
}
/** Register/verify this collection's schema in Base. Idempotent. */
async ensure() {
await this.store.ensureCollection(this.collectionName, this._promoted);
}
// ---- internal helpers -------------------------------------------------
/** Build the persisted record { promoted columns…, data } from a Mongo doc. */
_toRecord(doc) {
const record = { data: doc };
for (const col of this._promoted) {
let v = idString(getPath(doc, col.name));
if (v === undefined) {
v = null;
}
if (v instanceof Date) {
v = v.toISOString();
}
record[col.name] = v;
}
return record;
}
/** Materialize a stored Mongo doc: all Date-typed fields become Date objects. */
_hydrate(doc) {
for (const f of this._dateFields) {
const v = getPath(doc, f);
// Cast ISO strings and epoch numbers (e.g. `default: Date.now`) to Date.
if (typeof v === 'string' || typeof v === 'number') {
setPath(doc, f, new Date(v));
}
}
return doc;
}
/** Apply schema defaults for keys absent from doc. */
_applyDefaults(doc) {
for (const [name, def] of this._defaults) {
if (doc[name] === undefined) {
const value = resolveDefault(def);
if (value !== undefined) {
doc[name] = value;
}
}
}
}
/** Fetch candidate {baseId, doc} records for a filter (pushdown + always safe). */
async _candidates(filter) {
const raw = await this.store.list(this.collectionName, filter, this._promotedNames);
return raw;
}
_newDoc(seed) {
const doc = { ...seed };
if (!doc._id) {
doc._id = generateObjectId();
}
this._applyDefaults(doc);
if (this._timestamps) {
const ts = now();
if (doc[this._timestamps.createdAt] === undefined) {
doc[this._timestamps.createdAt] = ts;
}
doc[this._timestamps.updatedAt] = ts;
}
return doc;
}
async _insert(doc) {
const stored = { ...doc };
// Persist Dates as ISO strings inside the JSON blob.
if (this._timestamps) {
for (const f of [this._timestamps.createdAt, this._timestamps.updatedAt]) {
if (stored[f] instanceof Date) {
stored[f] = stored[f].toISOString();
}
}
}
const rec = await this.store.create(this.collectionName, this._toRecord(stored));
return rec.baseId;
}
async _replace(baseId, doc, { touch = true } = {}) {
if (this._timestamps && touch) {
doc[this._timestamps.updatedAt] = now(); // reflect fresh updatedAt on the caller's doc
}
const stored = { ...doc };
if (this._timestamps) {
for (const f of [this._timestamps.createdAt, this._timestamps.updatedAt]) {
if (stored[f] instanceof Date) {
stored[f] = stored[f].toISOString();
}
}
}
await this.store.update(this.collectionName, baseId, this._toRecord(stored));
}
// ---- write operations -------------------------------------------------
async _create(input) {
const doc = this._newDoc(input);
await this._insert(doc);
return this._hydrate({ ...doc });
}
/** `new Model(data)` — build an unsaved document (defaults + _id) with .save(). */
_newDocument(data = {}) {
return makeDocument(this, this._hydrate(this._newDoc(data)));
}
async create(input) {
if (Array.isArray(input)) {
const out = [];
for (const item of input) {
out.push(makeDocument(this, await this._create(item)));
}
return out;
}
return makeDocument(this, await this._create(input));
}
async insertMany(docs, options = {}) {
const created = [];
for (const d of docs) {
created.push(await this._create(d));
}
if (options.lean) {
return created;
}
return created.map((d) => makeDocument(this, d));
}
/**
* Core upsert/update primitive shared by findOneAndUpdate/updateOne/updateMany.
* @returns {{ doc: object|null, matched: number, modified: number, upserted: boolean }}
*/
async _updateOneInternal(filter, update, options = {}) {
const candidates = await this._candidates(filter);
const hit = candidates.find((c) => matches(c.doc, filter));
if (hit) {
const before = this._hydrate({ ...hit.doc });
const next = { ...hit.doc };
applyUpdate(next, update, { isInsert: false });
await this._replace(hit.baseId, next, { touch: options.timestamps !== false });
return { doc: this._hydrate(next), before, matched: 1, modified: 1, upserted: false };
}
if (options.upsert) {
const seed = equalitySeed(filter);
const doc = { ...seed };
applyUpdate(doc, update, { isInsert: true });
const full = this._newDoc(doc);
await this._insert(full);
return { doc: this._hydrate(full), before: null, matched: 0, modified: 0, upserted: true };
}
return { doc: null, before: null, matched: 0, modified: 0, upserted: false };
}
findOneAndUpdate(filter, update, options = {}) {
return new Query(this, 'findOneAndUpdate', { filter, update, options });
}
findByIdAndUpdate(id, update, options = {}) {
return this.findOneAndUpdate({ _id: id }, update, options);
}
findOneAndDelete(filter) {
return new Query(this, 'findOneAndDelete', { filter });
}
async updateOne(filter, update, options = {}) {
const r = await this._updateOneInternalPublic(filter, update, options);
return r;
}
async _updateOneInternalPublic(filter, update, options) {
const r = await this._updateOneInternal(filter, update, options);
return {
acknowledged: true,
matchedCount: r.matched,
modifiedCount: r.modified,
upsertedCount: r.upserted ? 1 : 0,
upsertedId: r.upserted ? r.doc._id : null,
};
}
async updateMany(filter, update, options = {}) {
const candidates = await this._candidates(filter);
const hits = candidates.filter((c) => matches(c.doc, filter));
for (const hit of hits) {
const next = { ...hit.doc };
applyUpdate(next, update, { isInsert: false });
await this._replace(hit.baseId, next, { touch: options.timestamps !== false });
}
if (!hits.length && options.upsert) {
const seed = equalitySeed(filter);
const doc = { ...seed };
applyUpdate(doc, update, { isInsert: true });
await this._insert(this._newDoc(doc));
return { acknowledged: true, matchedCount: 0, modifiedCount: 0, upsertedCount: 1 };
}
return {
acknowledged: true,
matchedCount: hits.length,
modifiedCount: hits.length,
upsertedCount: 0,
};
}
async _deleteWhere(filter, limit) {
const candidates = await this._candidates(filter);
const hits = candidates.filter((c) => matches(c.doc, filter));
const toDelete = limit ? hits.slice(0, limit) : hits;
for (const hit of toDelete) {
await this.store.delete(this.collectionName, hit.baseId);
}
return { acknowledged: true, deletedCount: toDelete.length };
}
deleteOne(filter) {
return this._deleteWhere(filter, 1);
}
deleteMany(filter) {
return this._deleteWhere(filter || {}, 0);
}
// ---- read operations --------------------------------------------------
find(filter = {}, projection, options) {
return new Query(this, 'find', { filter, projection, options });
}
findOne(filter = {}, projection, options) {
return new Query(this, 'findOne', { filter, projection, options });
}
findById(id, projection, options) {
return new Query(this, 'findOne', { filter: { _id: id }, projection, options });
}
async countDocuments(filter = {}) {
const candidates = await this._candidates(filter);
return candidates.filter((c) => matches(c.doc, filter)).length;
}
async estimatedDocumentCount() {
const candidates = await this._candidates({});
return candidates.length;
}
async distinct(field, filter = {}) {
const candidates = await this._candidates(filter);
const set = new Set();
for (const c of candidates) {
if (!matches(c.doc, filter)) {
continue;
}
const v = getPath(c.doc, field);
if (Array.isArray(v)) {
v.forEach((x) => set.add(x));
} else if (v !== undefined) {
set.add(v);
}
}
return [...set];
}
async exists(filter) {
const candidates = await this._candidates(filter);
const hit = candidates.find((c) => matches(c.doc, filter));
return hit ? { _id: hit.doc._id } : null;
}
/**
* Full-text search over Base/SQLite (replaces MeiliSearch).
* Case-insensitive substring match across the model's searchable content
* fields (title, text, …), scoped by the Meili-style `opts.filter` string
* (e.g. `user = "abc"`). Returns MeiliSearch-shaped `{ hits }`.
*/
async meiliSearch(query, opts = {}) {
const terms = (query || '').trim().toLowerCase();
if (!terms || !this._searchable.size) {
return { hits: [] };
}
const scope = parseMeiliFilter(opts.filter);
const candidates = await this._candidates(scope);
const hits = [];
for (const c of candidates) {
if (!matches(c.doc, scope)) {
continue;
}
for (const field of this._searchable) {
const value = getPath(c.doc, field);
if (typeof value === 'string' && value.toLowerCase().includes(terms)) {
hits.push(applyProjection(this._hydrate({ ...c.doc }), null, this._deselected));
break;
}
}
}
return { hits };
}
async syncWithMeili() {
/* no-op: MeiliSearch sync disabled under Base adapter (Phase 2 FTS). */
}
async bulkWrite(operations = []) {
const result = {
insertedCount: 0,
matchedCount: 0,
modifiedCount: 0,
deletedCount: 0,
upsertedCount: 0,
upsertedIds: {},
insertedIds: {},
};
for (const op of operations) {
if (op.insertOne) {
const doc = await this._create(op.insertOne.document);
result.insertedIds[result.insertedCount] = doc._id;
result.insertedCount += 1;
} else if (op.updateOne) {
const { filter, update, upsert, timestamps } = op.updateOne;
const r = await this._updateOneInternal(filter, update, { upsert, timestamps });
result.matchedCount += r.matched;
result.modifiedCount += r.modified;
if (r.upserted) {
result.upsertedIds[result.upsertedCount] = r.doc._id;
result.upsertedCount += 1;
}
} else if (op.updateMany) {
const { filter, update, upsert, timestamps } = op.updateMany;
const r = await this.updateMany(filter, update, { upsert, timestamps });
result.matchedCount += r.matchedCount;
result.modifiedCount += r.modifiedCount;
result.upsertedCount += r.upsertedCount || 0;
} else if (op.deleteOne) {
const r = await this._deleteWhere(op.deleteOne.filter, 1);
result.deletedCount += r.deletedCount;
} else if (op.deleteMany) {
const r = await this._deleteWhere(op.deleteMany.filter, 0);
result.deletedCount += r.deletedCount;
}
}
return result;
}
/** Minimal aggregation engine (used by Prompt / agentCategory, not the hot path). */
async aggregate(pipeline = []) {
const candidates = await this._candidates({});
let docs = candidates.map((c) => this._hydrate({ ...c.doc }));
for (const stage of pipeline) {
const op = Object.keys(stage)[0];
const arg = stage[op];
switch (op) {
case '$match':
docs = docs.filter((d) => matches(d, arg));
break;
case '$sort':
docs = sortDocs(docs, arg);
break;
case '$limit':
docs = docs.slice(0, arg);
break;
case '$skip':
docs = docs.slice(arg);
break;
case '$count':
docs = [{ [arg]: docs.length }];
break;
case '$project':
docs = docs.map((d) => applyProjection(d, parseProjection(arg)));
break;
case '$unwind': {
const field = (typeof arg === 'string' ? arg : arg.path).replace(/^\$/, '');
const next = [];
for (const d of docs) {
const v = getPath(d, field);
if (Array.isArray(v)) {
for (const item of v) {
next.push({ ...d, [field]: item });
}
} else if (v !== undefined) {
next.push(d);
}
}
docs = next;
break;
}
case '$group':
docs = groupStage(docs, arg);
break;
default:
throw new Error(`[BaseModel.aggregate] Unsupported stage: ${op}`);
}
}
return docs;
}
}
/** $group implementation for the aggregation engine. */
function groupStage(docs, spec) {
const { _id: idSpec, ...accs } = spec;
const groups = new Map();
const keyOf = (d) => {
if (idSpec === null) {
return '__null__';
}
if (typeof idSpec === 'string' && idSpec.startsWith('$')) {
return JSON.stringify(getPath(d, idSpec.slice(1)) ?? null);
}
return JSON.stringify(idSpec);
};
const idValue = (d) => {
if (idSpec === null) {
return null;
}
if (typeof idSpec === 'string' && idSpec.startsWith('$')) {
return getPath(d, idSpec.slice(1)) ?? null;
}
return idSpec;
};
for (const d of docs) {
const k = keyOf(d);
if (!groups.has(k)) {
groups.set(k, { _id: idValue(d), _docs: [] });
}
groups.get(k)._docs.push(d);
}
const out = [];
for (const g of groups.values()) {
const row = { _id: g._id };
for (const [field, acc] of Object.entries(accs)) {
const accOp = Object.keys(acc)[0];
const expr = acc[accOp];
const values = g._docs.map((d) =>
typeof expr === 'string' && expr.startsWith('$') ? getPath(d, expr.slice(1)) : expr,
);
switch (accOp) {
case '$sum':
row[field] = values.reduce((s, v) => s + (Number(v) || (expr === 1 ? 1 : 0)), 0);
break;
case '$count':
row[field] = g._docs.length;
break;
case '$avg':
row[field] = values.reduce((s, v) => s + (Number(v) || 0), 0) / (values.length || 1);
break;
case '$min':
row[field] = values.reduce((m, v) => (m == null || v < m ? v : m), null);
break;
case '$max':
row[field] = values.reduce((m, v) => (m == null || v > m ? v : m), null);
break;
case '$first':
row[field] = values[0];
break;
case '$last':
row[field] = values[values.length - 1];
break;
case '$push':
row[field] = values;
break;
case '$addToSet':
row[field] = [...new Set(values)];
break;
default:
throw new Error(`[BaseModel.aggregate] Unsupported accumulator: ${accOp}`);
}
}
out.push(row);
}
return out;
}
/**
* Query — a lazy, chainable, thenable wrapper mirroring the Mongoose Query API
* subset in use: select / sort / limit / skip / lean / populate / collation,
* plus terminal execution when awaited (or via .then/.exec).
*/
class Query {
constructor(model, op, params) {
this.model = model;
this.op = op;
this.params = params;
this._projection = params.projection;
this._sort = null;
this._limit = null;
this._skip = null;
this._lean = false;
}
select(projection) {
this._projection = projection;
return this;
}
sort(spec) {
this._sort = spec;
return this;
}
limit(n) {
this._limit = n;
return this;
}
skip(n) {
this._skip = n;
return this;
}
lean() {
this._lean = true;
return this;
}
populate() {
return this; // refs are stored inline as ids; no server-side population
}
collation() {
return this;
}
session() {
return this; // Base has no transactions; sessions are inert
}
hint() {
return this;
}
read() {
return this;
}
/** Merge an additional filter (Mongoose `Model.find(A).find(B)` / `.deleteMany(B)`). */
find(extra) {
this.params.filter = mergeFilter(this.params.filter, extra);
return this;
}
/** `Model.find(A).deleteMany(B)` — delete with merged conditions. */
deleteMany(extra) {
const filter = mergeFilter(this.params.filter, extra);
return this.model._deleteWhere(filter, 0);
}
deleteOne(extra) {
const filter = mergeFilter(this.params.filter, extra);
return this.model._deleteWhere(filter, 1);
}
countDocuments() {
return this.model.countDocuments(this.params.filter);
}
async _run() {
const model = this.model;
if (this.op === 'findOneAndUpdate') {
const r = await model._updateOneInternal(
this.params.filter,
this.params.update,
this.params.options,
);
// Mongoose returns the pre-update doc when `new` is not truthy (default),
// and the post-update doc when `{ new: true }`. Upserts with `new:false`
// return null (there was no prior document).
const wantNew = !!(this.params.options && this.params.options.new);
let doc = wantNew ? r.doc : r.upserted ? null : r.before;
if (!doc) {
return null;
}
doc = applyProjection(doc, parseProjection(this._projection), model._deselected);
return this._lean ? doc : makeDocument(model, doc);
}
if (this.op === 'findOneAndDelete') {
const candidates = await model._candidates(this.params.filter);
const hit = candidates.find((c) => matches(c.doc, this.params.filter));
if (!hit) {
return null;
}
await model.store.delete(model.collectionName, hit.baseId);
const doc = applyProjection(
model._hydrate({ ...hit.doc }),
parseProjection(this._projection),
model._deselected,
);
return this._lean ? doc : makeDocument(model, doc);
}
// read path (find / findOne)
const candidates = await model._candidates(this.params.filter);
let docs = candidates.filter((c) => matches(c.doc, this.params.filter)).map((c) => c.doc);
if (this._sort) {
docs = sortDocs(docs, this._sort);
}
if (this._skip) {
docs = docs.slice(this._skip);
}
if (this.op === 'findOne') {
const doc = docs[0];
if (!doc) {
return null;
}
const out = applyProjection(
model._hydrate({ ...doc }),
parseProjection(this._projection),
model._deselected,
);
return this._lean ? out : makeDocument(model, out);
}
if (this._limit != null) {
docs = docs.slice(0, this._limit);
}
const projection = parseProjection(this._projection);
const out = docs.map((d) =>
applyProjection(model._hydrate({ ...d }), projection, model._deselected),
);
return this._lean ? out : out.map((d) => makeDocument(model, d));
}
exec() {
return this._run();
}
then(onFulfilled, onRejected) {
return this._run().then(onFulfilled, onRejected);
}
catch(onRejected) {
return this._run().catch(onRejected);
}
finally(onFinally) {
return this._run().finally(onFinally);
}
}
function mergeFilter(a, b) {
if (!b || !Object.keys(b).length) {
return a;
}
if (!a || !Object.keys(a).length) {
return b;
}
return { $and: [a, b] };
}
/**
* Wrap a plain doc as a lightweight "document": all data fields are enumerable
* (so spreads, `.map`, JSON all behave), while toObject/save/etc. are hidden.
*/
function makeDocument(model, doc) {
if (doc == null) {
return doc;
}
const defineHidden = (name, value) =>
Object.defineProperty(doc, name, { value, enumerable: false, writable: true, configurable: true });
defineHidden('toObject', function toObject() {
const plain = {};
for (const k of Object.keys(this)) {
plain[k] = this[k];
}
return plain;
});
defineHidden('toJSON', doc.toObject);
defineHidden('save', async function save() {
const candidates = await model._candidates({ _id: this._id });
const plain = this.toObject();
const hit = candidates.find((c) => c.doc._id === this._id);
if (hit) {
// Merge over the stored record so a doc loaded with a narrowed projection
// (e.g. without select:false secrets) never erases the unloaded fields.
await model._replace(hit.baseId, { ...hit.doc, ...plain }, { touch: true });
} else {
await model._insert(model._newDoc(plain));
}
return this;
});
defineHidden('deleteOne', async function deleteOne() {
return model._deleteWhere({ _id: this._id }, 1);
});
if (doc._id !== undefined) {
defineHidden('id', String(doc._id));
}
return doc;
}
/** Static methods exposed on the constructor returned by `mongoose.model()`. */
const MODEL_STATICS = [
'find',
'findOne',
'findById',
'findOneAndUpdate',
'findByIdAndUpdate',
'findOneAndDelete',
'create',
'insertMany',
'updateOne',
'updateMany',
'deleteOne',
'deleteMany',
'countDocuments',
'estimatedDocumentCount',
'distinct',
'exists',
'bulkWrite',
'aggregate',
'meiliSearch',
'syncWithMeili',
'ensure',
];
/**
* Wrap a BaseModel as a Mongoose-style model constructor:
* - `new Model(data)` creates an unsaved document (with `.save()`)
* - `Model.find(...)`, `Model.findOneAndUpdate(...)`, … are the statics
*/
function makeModelCtor(baseModel) {
function ModelCtor(data) {
return baseModel._newDocument(data);
}
for (const name of MODEL_STATICS) {
ModelCtor[name] = baseModel[name].bind(baseModel);
}
ModelCtor.modelName = baseModel.modelName;
ModelCtor.schema = baseModel.schema;
ModelCtor.collection = baseModel.collection;
ModelCtor.base = baseModel;
return ModelCtor;
}
module.exports = { BaseModel, Query, makeDocument, makeModelCtor };
+284
View File
@@ -0,0 +1,284 @@
'use strict';
/*
* BaseModel behaviour over an in-memory store (no Base/Mongo required).
* Exercises the exact Mongoose call patterns the chat hot path relies on.
*/
const { Schema } = require('mongoose');
const { BaseModel } = require('./model');
/** In-memory backend mirroring DocumentStore's interface. */
function memStore() {
const cols = {};
let seq = 0;
const clone = (x) => JSON.parse(JSON.stringify(x));
return {
async ensureCollection(name) {
cols[name] = cols[name] || new Map();
},
async list(name) {
cols[name] = cols[name] || new Map();
return [...cols[name].entries()].map(([baseId, doc]) => ({ baseId, doc: clone(doc) }));
},
async create(name, record) {
cols[name] = cols[name] || new Map();
const baseId = 'b' + ++seq;
cols[name].set(baseId, clone(record.data));
return { baseId, doc: clone(record.data) };
},
async update(name, baseId, record) {
cols[name].set(baseId, clone(record.data));
return { baseId, doc: clone(record.data) };
},
async delete(name, baseId) {
cols[name].delete(baseId);
},
};
}
const convoSchema = new Schema(
{
conversationId: { type: String, unique: true, required: true, index: true },
title: { type: String, default: 'New Chat', meiliIndex: true },
user: { type: String, index: true },
tags: { type: [String], default: [] },
expiredAt: { type: Date },
},
{ timestamps: true },
);
const messageSchema = new Schema(
{
messageId: { type: String, unique: true, required: true, index: true },
conversationId: { type: String, index: true, required: true },
user: { type: String, index: true, required: true, default: null },
text: { type: String, meiliIndex: true },
},
{ timestamps: true },
);
const userSchema = new Schema(
{
email: { type: String, required: true, unique: true },
role: { type: String, default: 'USER' },
password: { type: String, select: false },
totpSecret: { type: String, select: false },
},
{ timestamps: true },
);
const balanceSchema = new Schema({
user: { type: Schema.Types.ObjectId, ref: 'User', index: true, required: true },
tokenCredits: { type: Number, default: 0 },
expiresAt: { type: Date, default: null },
lastRefill: { type: Date, default: Date.now },
});
const sessionSchema = new Schema({
refreshTokenHash: { type: String, index: true },
user: { type: Schema.Types.ObjectId, ref: 'User', index: true },
expiration: { type: Date, required: true, index: true },
});
let store, Conversation, Message, User, Balance, Session;
beforeEach(async () => {
store = memStore();
Conversation = new BaseModel('Conversation', convoSchema, store);
Message = new BaseModel('Message', messageSchema, store);
User = new BaseModel('User', userSchema, store);
Balance = new BaseModel('Balance', balanceSchema, store);
Session = new BaseModel('Session', sessionSchema, store);
for (const m of [Conversation, Message, User, Balance, Session]) await m.ensure();
});
describe('schema introspection', () => {
test('promotes indexed/unique/timestamp fields and _id', () => {
expect(Conversation._promotedNames.sort()).toEqual(
['_id', 'conversationId', 'createdAt', 'updatedAt', 'user'].sort(),
);
});
test('timestamps detected only when configured', () => {
expect(Conversation._timestamps.createdAt).toBe('createdAt');
expect(Balance._timestamps).toBeNull();
});
});
describe('create + defaults + ids', () => {
test('User.create applies defaults, ObjectId _id, Date timestamps', async () => {
const u = await User.create({ email: 'z@zoo.ngo', password: 'hashed' });
expect(u._id).toMatch(/^[0-9a-f]{24}$/);
expect(u.role).toBe('USER');
expect(u.createdAt).toBeInstanceOf(Date);
});
});
describe('login-path reads', () => {
test('findOne + select projection + lean', async () => {
await User.create({ email: 'z@zoo.ngo', password: 'hashed' });
const found = await User.findOne({ email: 'z@zoo.ngo' }).select('email password').lean();
expect(found.password).toBe('hashed');
expect(found.role).toBeUndefined();
expect(found._id).toBeTruthy();
});
test('select:false secrets (password/totpSecret) hidden by default', async () => {
await User.create({ email: 'z@zoo.ngo', password: 'hashed', totpSecret: 'seed' });
const def = await User.findOne({ email: 'z@zoo.ngo' }).lean();
expect(def.password).toBeUndefined();
expect(def.totpSecret).toBeUndefined();
expect(def.email).toBe('z@zoo.ngo');
// login path explicitly requests +password
const withPw = await User.findOne({ email: 'z@zoo.ngo' }, '+password').lean();
expect(withPw.password).toBe('hashed');
expect(withPw.totpSecret).toBeUndefined(); // still hidden
});
});
describe('date-equality reads (H1 — no broken date pushdown)', () => {
test('find by an exact Date value returns the record', async () => {
const exp = new Date('2026-07-04T12:34:56.789Z');
await Conversation.findOneAndUpdate(
{ conversationId: 'd1', user: '507f1f77bcf86cd799439011' },
{ $set: { expiredAt: exp } },
{ upsert: true, new: true },
);
const byDate = await Conversation.find({ expiredAt: exp }).lean();
expect(byDate.map((c) => c.conversationId)).toEqual(['d1']);
const gt = await Conversation.find({ expiredAt: { $gt: new Date('2026-07-01T00:00:00Z') } }).lean();
expect(gt.some((c) => c.conversationId === 'd1')).toBe(true);
});
});
describe('save() preserves select:false fields after a projected load (M2)', () => {
test('mutate a default-loaded user + save keeps password/totpSecret', async () => {
await User.create({ email: 'z@zoo.ngo', password: 'hashed', totpSecret: 'seed' });
const u = await User.findOne({ email: 'z@zoo.ngo' }); // default view: secrets hidden
expect(u.password).toBeUndefined();
u.name = 'Renamed';
await u.save();
const reloaded = await User.findOne({ email: 'z@zoo.ngo' }, '+password').lean();
expect(reloaded.password).toBe('hashed'); // not erased by save()
expect(reloaded.name).toBe('Renamed');
});
});
describe('date hydration (non-timestamp Date fields)', () => {
test('session.expiration and balance dates hydrate to Date', async () => {
const exp = new Date(Date.now() + 3600_000);
await Session.create({ user: '507f1f77bcf86cd799439011', expiration: exp });
const s = await Session.findOne({ user: '507f1f77bcf86cd799439011' }).lean();
expect(s.expiration).toBeInstanceOf(Date);
expect(typeof s.expiration.getTime()).toBe('number'); // would throw on a string
const bal = await Balance.findOneAndUpdate(
{ user: '507f1f77bcf86cd799439011' },
{ $inc: { tokenCredits: 1 } },
{ upsert: true, new: true },
).lean();
expect(bal.lastRefill).toBeInstanceOf(Date);
});
});
describe('balance upsert', () => {
test('$inc + $set upsert then accumulate', async () => {
const uid = '507f1f77bcf86cd799439011';
const bal = await Balance.findOneAndUpdate(
{ user: uid },
{ $inc: { tokenCredits: 1000 } },
{ upsert: true, new: true },
).lean();
expect(bal.tokenCredits).toBe(1000);
expect(String(bal.user)).toBe(uid);
const bal2 = await Balance.findOneAndUpdate(
{ user: uid },
{ $inc: { tokenCredits: 500 } },
{ new: true },
).lean();
expect(bal2.tokenCredits).toBe(1500);
});
});
describe('chat hot path', () => {
const uid = '507f1f77bcf86cd799439011';
test('saveConvo upsert + toObject', async () => {
const c = await Conversation.findOneAndUpdate(
{ conversationId: 'c1', user: uid },
{ $set: { title: 'Hello', endpoint: 'openAI' } },
{ new: true, upsert: true },
);
expect(c.conversationId).toBe('c1');
expect(c.toObject().title).toBe('Hello');
});
test('saveMessage upsert + partial merge, sorted read', async () => {
const base = Date.now();
for (let i = 0; i < 3; i++) {
await Message.findOneAndUpdate(
{ messageId: 'm' + i, user: uid },
{ conversationId: 'c1', text: 'msg' + i, createdAt: new Date(base + i * 1000) },
{ upsert: true, new: true },
);
}
await Message.findOneAndUpdate({ messageId: 'm1', user: uid }, { text: 'edited' }, { new: true });
const msgs = await Message.find({ conversationId: 'c1', user: uid }).sort({ createdAt: 1 }).lean();
expect(msgs.map((m) => m.messageId)).toEqual(['m0', 'm1', 'm2']);
expect(msgs[1].text).toBe('edited');
expect(msgs[1].conversationId).toBe('c1'); // merge preserved
});
test('find(A).deleteMany(B) merges conditions', async () => {
const base = Date.now();
for (let i = 0; i < 3; i++) {
await Message.create({ messageId: 'm' + i, conversationId: 'c1', user: uid, createdAt: new Date(base + i * 1000) });
}
const res = await Message.find({ conversationId: 'c1', user: uid }).deleteMany({
createdAt: { $gt: new Date(base + 500) },
});
expect(res.deletedCount).toBe(2);
const left = await Message.find({ conversationId: 'c1' }).lean();
expect(left).toHaveLength(1);
expect(left[0].messageId).toBe('m0');
});
test('deleteMany $or/$exists', async () => {
await Message.create({ messageId: 'mx', conversationId: '', user: uid });
const res = await Message.deleteMany({
$or: [{ conversationId: '' }, { conversationId: { $exists: false } }],
});
expect(res.deletedCount).toBe(1);
});
test('bulkWrite updateOne upsert', async () => {
const res = await Conversation.bulkWrite([
{ updateOne: { filter: { conversationId: 'c1', user: uid }, update: { title: 'A' }, upsert: true } },
{ updateOne: { filter: { conversationId: 'c2', user: uid }, update: { title: 'B' }, upsert: true } },
]);
expect(res.upsertedCount).toBe(2);
expect(await Conversation.countDocuments({ conversationId: { $in: ['c1', 'c2'] } })).toBe(2);
});
test('meiliSearch (Base/SQLite FTS) — title + text, user-scoped', async () => {
const other = '507f1f77bcf86cd799439099';
await Conversation.findOneAndUpdate({ conversationId: 'c1', user: uid }, { $set: { title: 'Quantum physics notes' } }, { upsert: true, new: true });
await Conversation.findOneAndUpdate({ conversationId: 'c2', user: uid }, { $set: { title: 'Grocery list' } }, { upsert: true, new: true });
await Conversation.findOneAndUpdate({ conversationId: 'c3', user: other }, { $set: { title: 'Quantum leaps' } }, { upsert: true, new: true });
const conv = await Conversation.meiliSearch('quantum', { filter: `user = "${uid}"` });
expect(conv.hits.map((h) => h.conversationId)).toEqual(['c1']); // case-insensitive, user-scoped
await Message.create({ messageId: 'm1', conversationId: 'c1', user: uid, text: 'the WAVEFUNCTION collapses' });
await Message.create({ messageId: 'm2', conversationId: 'c1', user: uid, text: 'unrelated chatter' });
const msg = await Message.meiliSearch('wavefunction', { filter: `user = "${uid}"` });
expect(msg.hits.map((h) => h.messageId)).toEqual(['m1']);
expect((await Conversation.meiliSearch('', { filter: `user = "${uid}"` })).hits).toEqual([]);
});
test('findOneAndUpdate new:false returns pre-update doc', async () => {
await Conversation.findOneAndUpdate({ conversationId: 'c1', user: uid }, { title: 'first' }, { upsert: true, new: true });
const before = await Conversation.findOneAndUpdate(
{ conversationId: 'c1', user: uid },
{ title: 'second' },
{ new: false },
);
expect(before.title).toBe('first');
const now = await Conversation.findOne({ conversationId: 'c1' }).lean();
expect(now.title).toBe('second');
});
});
+122
View File
@@ -0,0 +1,122 @@
'use strict';
/**
* A self-contained, `mongoose`-shaped facade backed by Hanzo Base.
*
* `@librechat/data-schemas` builds its schemas with real `mongoose` (a pure
* schema DSL inside that package) and then asks a mongoose instance to turn
* those schemas into models (`createModels`/`createMethods`). We hand it THIS
* facade instead: it registers every model as a BaseModel persisting to Base —
* never MongoDB — and provides just the mongoose surface the data layer uses at
* runtime (`model`, `models`, `Types.ObjectId`, `Schema.Types`, a no-op session
* / connection). It carries **no `mongoose` runtime dependency**; schema objects
* are passed in already-built, so this module never imports the driver.
*
* Result: one adapter, and the entire data-schemas model+method surface
* (User, Session, Token, Role, Balance, Conversation, Message, Agent, …) runs
* on Base with zero per-model porting.
*/
const { BaseModel, makeModelCtor } = require('./model');
const { ObjectId, isValidObjectId } = require('./objectId');
const store = require('./store');
/** Shared model registry (mirrors `mongoose.models`) — values are model ctors. */
const models = {};
/** BaseModel instances keyed by name (for connectDb collection provisioning). */
const baseModels = {};
function model(name, schema) {
if (models[name]) {
return models[name];
}
if (!schema) {
return undefined;
}
const base = new BaseModel(name, schema, store);
const ctor = makeModelCtor(base);
baseModels[name] = base;
models[name] = ctor;
return ctor;
}
/** No-op session: Base has no multi-document transactions (see adapter notes). */
function makeSession() {
return {
startTransaction() {},
async commitTransaction() {},
async abortTransaction() {},
async endSession() {},
async withTransaction(fn) {
return fn();
},
inTransaction() {
return false;
},
};
}
/** Connection stub — the real Base connection lives in ./index.js connectDb(). */
const connection = {
readyState: 1,
_readyState: 1,
on() {},
once() {},
model,
models,
collections: {},
async dropDatabase() {},
async close() {},
db: {
async dropDatabase() {},
collection() {
return {};
},
},
};
/** Minimal Schema.Types surface for the few `mongoose.Schema.Types.*` runtime uses. */
class Mixed {}
const SchemaTypes = {
ObjectId,
Mixed,
String,
Number,
Boolean,
Date,
Array,
Buffer,
Map,
};
const facade = {
__isBaseFacade: true,
model,
models,
connection,
connections: [connection],
Types: { ObjectId, Decimal128: Number, Mixed },
Schema: { Types: SchemaTypes },
Model: BaseModel,
isValidObjectId,
async connect() {
return facade;
},
createConnection() {
return connection;
},
async disconnect() {},
startSession() {
return makeSession();
},
set() {
return facade;
},
get() {
return undefined;
},
/** Access the underlying BaseModel instances (used by connectDb). */
baseModels,
};
module.exports = facade;
+93
View File
@@ -0,0 +1,93 @@
'use strict';
/**
* Mongo-compatible ObjectId generation.
*
* Hanzo Base stores documents keyed by its own 15-char id, but LibreChat code
* treats `_id` as a 24-hex Mongo ObjectId string (equality, `.toString()`,
* refs). We generate real ObjectId-shaped values so that behaviour is
* preserved without a live MongoDB.
*
* Layout (12 bytes -> 24 hex): 4-byte timestamp | 5-byte process random | 3-byte counter.
*/
const crypto = require('crypto');
const PROCESS_RANDOM = crypto.randomBytes(5);
let COUNTER = crypto.randomBytes(3).readUIntBE(0, 3);
/** @returns {string} 24-char lowercase hex ObjectId string. */
function generateObjectId() {
const buf = Buffer.allocUnsafe(12);
buf.writeUInt32BE(Math.floor(Date.now() / 1000), 0);
PROCESS_RANDOM.copy(buf, 4);
COUNTER = (COUNTER + 1) % 0xffffff;
buf.writeUIntBE(COUNTER, 9, 3);
return buf.toString('hex');
}
const HEX_24 = /^[0-9a-fA-F]{24}$/;
/** @param {unknown} value @returns {boolean} */
function isValidObjectId(value) {
if (value == null) {
return false;
}
if (typeof value === 'string') {
return HEX_24.test(value);
}
if (typeof value === 'object' && typeof value.toString === 'function') {
return HEX_24.test(value.toString());
}
return false;
}
/**
* A minimal, dependency-free ObjectId compatible with the surface the chat code
* uses from `mongoose.Types.ObjectId`: `new ObjectId()`, `new ObjectId(hex)`,
* `.isValid()`, `.createFromHexString()`, and instances that stringify / JSON /
* compare as their 24-hex value. Lets us drop the `mongoose` runtime dependency.
*/
class ObjectId {
constructor(id) {
if (id == null) {
this._id = generateObjectId();
} else if (typeof id === 'string') {
this._id = id;
} else if (id instanceof ObjectId) {
this._id = id._id;
} else if (typeof id.toHexString === 'function') {
this._id = id.toHexString();
} else {
this._id = String(id);
}
}
toString() {
return this._id;
}
toHexString() {
return this._id;
}
toJSON() {
return this._id;
}
valueOf() {
return this._id;
}
equals(other) {
if (other == null) {
return false;
}
return String(typeof other.toHexString === 'function' ? other.toHexString() : other) === this._id;
}
static isValid(value) {
return isValidObjectId(value);
}
static createFromHexString(hex) {
return new ObjectId(hex);
}
}
module.exports = { generateObjectId, isValidObjectId, HEX_24, ObjectId };
+537
View File
@@ -0,0 +1,537 @@
'use strict';
/**
* A small, correct MongoDB query/update engine used by the Hanzo Base adapter.
*
* The Base document store keeps each document as a JSON blob; filtering,
* update-operator application, projection and sorting all run here in JS.
* This is the correctness backstop for the whole adapter: any Mongo filter the
* LibreChat data layer produces is evaluated here, so translation to Base's
* filter DSL (see store.js) is only ever a best-effort pushdown optimisation.
*/
/**
* Path segments that must never be traversed or written — guards against
* prototype pollution via attacker-controlled dotted keys (e.g. an update or
* imported document containing `__proto__.x`). Mongoose sanitized these; the
* adapter must too.
*/
const FORBIDDEN_SEGMENTS = new Set(['__proto__', 'constructor', 'prototype']);
/** Resolve a possibly-dotted path against a document. */
function getPath(doc, path) {
if (doc == null) {
return undefined;
}
if (!path.includes('.')) {
return FORBIDDEN_SEGMENTS.has(path) ? undefined : doc[path];
}
let cur = doc;
for (const part of path.split('.')) {
if (cur == null || FORBIDDEN_SEGMENTS.has(part)) {
return undefined;
}
cur = cur[part];
}
return cur;
}
/** Set a possibly-dotted path on a document (mutates). No-op on forbidden segments. */
function setPath(doc, path, value) {
if (!path.includes('.')) {
if (!FORBIDDEN_SEGMENTS.has(path)) {
doc[path] = value;
}
return;
}
const parts = path.split('.');
if (parts.some((p) => FORBIDDEN_SEGMENTS.has(p))) {
return;
}
let cur = doc;
for (let i = 0; i < parts.length - 1; i++) {
if (cur[parts[i]] == null || typeof cur[parts[i]] !== 'object') {
cur[parts[i]] = {};
}
cur = cur[parts[i]];
}
cur[parts[parts.length - 1]] = value;
}
/** Delete a possibly-dotted path (mutates). No-op on forbidden segments. */
function unsetPath(doc, path) {
if (!path.includes('.')) {
if (!FORBIDDEN_SEGMENTS.has(path)) {
delete doc[path];
}
return;
}
const parts = path.split('.');
if (parts.some((p) => FORBIDDEN_SEGMENTS.has(p))) {
return;
}
let cur = doc;
for (let i = 0; i < parts.length - 1; i++) {
if (cur[parts[i]] == null) {
return;
}
cur = cur[parts[i]];
}
delete cur[parts[parts.length - 1]];
}
/** Coerce ObjectId-like objects (Mongo ObjectId, wrappers) to their hex string. */
function idString(value) {
if (value && typeof value === 'object' && typeof value.toHexString === 'function') {
return value.toHexString();
}
return value;
}
/** Coerce a value to a comparable primitive (Date -> ms, ObjectId -> hex). */
function comparable(value) {
value = idString(value);
if (value instanceof Date) {
return value.getTime();
}
if (typeof value === 'string') {
// ISO date strings sort correctly lexicographically, but to compare against
// Date values we normalise anything that parses as a date to ms.
const t = Date.parse(value);
if (!Number.isNaN(t) && /\d{4}-\d{2}-\d{2}T/.test(value)) {
return t;
}
}
return value;
}
/** Deep-ish equality sufficient for query matching (primitives, dates, arrays, plain objects). */
function valueEquals(a, b) {
a = idString(a);
b = idString(b);
if (a === b) {
return true;
}
if (a instanceof Date || b instanceof Date) {
return comparable(a) === comparable(b);
}
if (a == null || b == null) {
return a == null && b == null;
}
if (Array.isArray(a) && Array.isArray(b)) {
return a.length === b.length && a.every((x, i) => valueEquals(x, b[i]));
}
return false;
}
function compare(a, b) {
const ca = comparable(a);
const cb = comparable(b);
if (ca == null && cb == null) {
return 0;
}
if (ca == null) {
return -1;
}
if (cb == null) {
return 1;
}
if (ca < cb) {
return -1;
}
if (ca > cb) {
return 1;
}
return 0;
}
function toRegExp(spec, options) {
if (spec instanceof RegExp) {
return spec;
}
return new RegExp(spec, options || '');
}
/** Evaluate a single field's operator expression against a document value. */
function matchOperators(fieldValue, expr) {
// Array-aware equality helper: a field matches a scalar if it equals it, or
// (when the field is an array) if the array contains it — Mongo semantics.
const eq = (target) => {
if (Array.isArray(fieldValue) && !Array.isArray(target)) {
return fieldValue.some((v) => valueEquals(v, target));
}
return valueEquals(fieldValue, target);
};
if (expr instanceof RegExp) {
return typeof fieldValue === 'string' && expr.test(fieldValue);
}
if (expr === null || typeof expr !== 'object' || expr instanceof Date || Array.isArray(expr)) {
return eq(expr);
}
const keys = Object.keys(expr);
const isOperatorExpr = keys.some((k) => k.startsWith('$'));
if (!isOperatorExpr) {
return eq(expr);
}
for (const op of keys) {
const operand = expr[op];
switch (op) {
case '$eq':
if (!eq(operand)) return false;
break;
case '$ne':
if (eq(operand)) return false;
break;
case '$in': {
const arr = operand || [];
const hit = Array.isArray(fieldValue)
? fieldValue.some((v) => arr.some((o) => valueEquals(v, o)))
: arr.some((o) => valueEquals(fieldValue, o));
if (!hit) return false;
break;
}
case '$nin': {
const arr = operand || [];
const hit = Array.isArray(fieldValue)
? fieldValue.some((v) => arr.some((o) => valueEquals(v, o)))
: arr.some((o) => valueEquals(fieldValue, o));
if (hit) return false;
break;
}
case '$gt':
if (fieldValue === undefined || compare(fieldValue, operand) <= 0) return false;
break;
case '$gte':
if (fieldValue === undefined || compare(fieldValue, operand) < 0) return false;
break;
case '$lt':
if (fieldValue === undefined || compare(fieldValue, operand) >= 0) return false;
break;
case '$lte':
if (fieldValue === undefined || compare(fieldValue, operand) > 0) return false;
break;
case '$exists':
if ((fieldValue !== undefined) !== !!operand) return false;
break;
case '$regex': {
const re = toRegExp(operand, expr.$options);
if (typeof fieldValue !== 'string' || !re.test(fieldValue)) return false;
break;
}
case '$options':
break; // handled with $regex
case '$not':
if (matchOperators(fieldValue, operand)) return false;
break;
case '$size':
if (!Array.isArray(fieldValue) || fieldValue.length !== operand) return false;
break;
case '$all': {
if (!Array.isArray(fieldValue)) return false;
const ok = (operand || []).every((o) => fieldValue.some((v) => valueEquals(v, o)));
if (!ok) return false;
break;
}
case '$elemMatch': {
if (!Array.isArray(fieldValue)) return false;
if (!fieldValue.some((v) => matches(v, operand))) return false;
break;
}
default:
// Unknown operator: treat the whole expression as an equality target.
return eq(expr);
}
}
return true;
}
/**
* Does `doc` match Mongo filter `query`?
* @param {Record<string, unknown>} doc
* @param {Record<string, unknown>} query
* @returns {boolean}
*/
function matches(doc, query) {
if (!query || typeof query !== 'object') {
return true;
}
for (const key of Object.keys(query)) {
const val = query[key];
if (key === '$and') {
if (!(val || []).every((sub) => matches(doc, sub))) return false;
continue;
}
if (key === '$or') {
if (!(val || []).some((sub) => matches(doc, sub))) return false;
continue;
}
if (key === '$nor') {
if ((val || []).some((sub) => matches(doc, sub))) return false;
continue;
}
if (key === '$not') {
if (matches(doc, val)) return false;
continue;
}
if (key.startsWith('$')) {
continue; // unsupported top-level operator -> ignore (permissive)
}
if (!matchOperators(getPath(doc, key), val)) {
return false;
}
}
return true;
}
const UPDATE_OPERATORS = new Set([
'$set',
'$unset',
'$inc',
'$push',
'$addToSet',
'$pull',
'$setOnInsert',
'$min',
'$max',
'$mul',
'$rename',
]);
/** Split a Mongoose-style update into operator groups, wrapping bare fields as $set. */
function normalizeUpdate(update) {
if (!update || typeof update !== 'object') {
return { $set: {} };
}
const keys = Object.keys(update);
const hasOperator = keys.some((k) => UPDATE_OPERATORS.has(k));
if (!hasOperator) {
return { $set: { ...update } };
}
const out = {};
const bareSet = {};
for (const k of keys) {
if (UPDATE_OPERATORS.has(k)) {
out[k] = update[k];
} else {
bareSet[k] = update[k];
}
}
if (Object.keys(bareSet).length) {
out.$set = { ...bareSet, ...(out.$set || {}) };
}
return out;
}
/**
* Apply a normalized/raw update to a document in place and return it.
* @param {Record<string, unknown>} doc - target document (mutated)
* @param {Record<string, unknown>} update
* @param {{ isInsert?: boolean }} [opts]
*/
function applyUpdate(doc, update, opts = {}) {
const ops = normalizeUpdate(update);
if (ops.$set) {
for (const [k, v] of Object.entries(ops.$set)) {
setPath(doc, k, v);
}
}
if (ops.$setOnInsert && opts.isInsert) {
for (const [k, v] of Object.entries(ops.$setOnInsert)) {
setPath(doc, k, v);
}
}
if (ops.$unset) {
for (const k of Object.keys(ops.$unset)) {
unsetPath(doc, k);
}
}
if (ops.$inc) {
for (const [k, v] of Object.entries(ops.$inc)) {
setPath(doc, k, (Number(getPath(doc, k)) || 0) + Number(v));
}
}
if (ops.$mul) {
for (const [k, v] of Object.entries(ops.$mul)) {
setPath(doc, k, (Number(getPath(doc, k)) || 0) * Number(v));
}
}
if (ops.$min) {
for (const [k, v] of Object.entries(ops.$min)) {
const cur = getPath(doc, k);
if (cur === undefined || compare(v, cur) < 0) setPath(doc, k, v);
}
}
if (ops.$max) {
for (const [k, v] of Object.entries(ops.$max)) {
const cur = getPath(doc, k);
if (cur === undefined || compare(v, cur) > 0) setPath(doc, k, v);
}
}
if (ops.$push) {
for (const [k, v] of Object.entries(ops.$push)) {
const arr = Array.isArray(getPath(doc, k)) ? getPath(doc, k) : [];
if (v && typeof v === 'object' && Array.isArray(v.$each)) {
arr.push(...v.$each);
} else {
arr.push(v);
}
setPath(doc, k, arr);
}
}
if (ops.$addToSet) {
for (const [k, v] of Object.entries(ops.$addToSet)) {
const arr = Array.isArray(getPath(doc, k)) ? getPath(doc, k) : [];
const items = v && typeof v === 'object' && Array.isArray(v.$each) ? v.$each : [v];
for (const item of items) {
if (!arr.some((x) => valueEquals(x, item))) arr.push(item);
}
setPath(doc, k, arr);
}
}
if (ops.$pull) {
for (const [k, cond] of Object.entries(ops.$pull)) {
const arr = getPath(doc, k);
if (Array.isArray(arr)) {
setPath(
doc,
k,
arr.filter((x) =>
cond && typeof cond === 'object' && !Array.isArray(cond)
? !matchOperators(x, cond)
: !valueEquals(x, cond),
),
);
}
}
}
if (ops.$rename) {
for (const [from, to] of Object.entries(ops.$rename)) {
const v = getPath(doc, from);
unsetPath(doc, from);
setPath(doc, to, v);
}
}
return doc;
}
/**
* Parse a projection spec (space string or object) into
* { plain:Set, plus:Set, exclude:Set } or null for no projection.
* - `field` -> plain inclusion
* - `+field` -> force-include a `select:false` field on top of the default set
* - `-field` -> exclusion
*/
function parseProjection(spec) {
if (!spec) {
return null;
}
const plain = new Set();
const plus = new Set();
const exclude = new Set();
if (typeof spec === 'string') {
for (const token of spec.split(/\s+/).filter(Boolean)) {
if (token.startsWith('-')) {
exclude.add(token.slice(1));
} else if (token.startsWith('+')) {
plus.add(token.slice(1));
} else {
plain.add(token);
}
}
} else if (typeof spec === 'object') {
for (const [k, v] of Object.entries(spec)) {
if (v) {
plain.add(k);
} else {
exclude.add(k);
}
}
}
if (!plain.size && !plus.size && !exclude.size) {
return null;
}
return { plain, plus, exclude };
}
/**
* Apply a parsed projection with Mongoose `select:false` semantics, returning a
* new object. `deselected` are fields excluded from reads by default (secrets).
*/
function applyProjection(doc, projection, deselected) {
const hidden = deselected || EMPTY_SET;
// No projection: default view = everything minus deselected (secrets).
if (!projection) {
if (!hidden.size) {
return doc;
}
const out = { ...doc };
for (const f of hidden) {
unsetPath(out, f);
}
return out;
}
// Inclusion mode: return only the named fields (+plus), _id unless -_id.
if (projection.plain.size) {
const fields = new Set([...projection.plain, ...projection.plus]);
if (!projection.exclude.has('_id')) {
fields.add('_id');
}
const out = {};
for (const f of fields) {
const v = getPath(doc, f);
if (v !== undefined) {
setPath(out, f, v);
}
}
return out;
}
// Default + adjustments: base minus deselected (unless +included) minus -excluded.
const out = { ...doc };
for (const f of hidden) {
if (!projection.plus.has(f)) {
unsetPath(out, f);
}
}
for (const f of projection.exclude) {
unsetPath(out, f);
}
return out;
}
const EMPTY_SET = new Set();
/** Sort documents by a Mongo sort spec ({ field: 1|-1 }). Returns a new array. */
function sortDocs(docs, sortSpec) {
if (!sortSpec || !Object.keys(sortSpec).length) {
return docs;
}
const keys = Object.entries(sortSpec).map(([k, v]) => [k, v === -1 || v === 'desc' ? -1 : 1]);
return [...docs].sort((a, b) => {
for (const [k, dir] of keys) {
const c = compare(getPath(a, k), getPath(b, k));
if (c !== 0) {
return c * dir;
}
}
return 0;
});
}
module.exports = {
matches,
applyUpdate,
normalizeUpdate,
parseProjection,
applyProjection,
sortDocs,
getPath,
setPath,
unsetPath,
valueEquals,
compare,
idString,
};
+124
View File
@@ -0,0 +1,124 @@
'use strict';
const { matches, applyUpdate, parseProjection, applyProjection, sortDocs } = require('./query');
describe('base/query matcher', () => {
const doc = {
_id: 'a1',
user: 'u1',
conversationId: 'c1',
tags: ['x', 'y'],
tokenCount: 10,
createdAt: '2026-07-04T00:00:00.000Z',
};
test('equality + implicit array-contains', () => {
expect(matches(doc, { user: 'u1' })).toBe(true);
expect(matches(doc, { user: 'u2' })).toBe(false);
expect(matches(doc, { tags: 'x' })).toBe(true); // array contains
expect(matches(doc, { tags: 'z' })).toBe(false);
});
test('operators $in/$nin/$ne/$exists', () => {
expect(matches(doc, { conversationId: { $in: ['c1', 'c2'] } })).toBe(true);
expect(matches(doc, { conversationId: { $nin: ['c2'] } })).toBe(true);
expect(matches(doc, { user: { $ne: 'u2' } })).toBe(true);
expect(matches(doc, { missing: { $exists: false } })).toBe(true);
expect(matches(doc, { user: { $exists: true } })).toBe(true);
});
test('comparison operators with numbers and dates', () => {
expect(matches(doc, { tokenCount: { $gt: 5, $lte: 10 } })).toBe(true);
expect(matches(doc, { tokenCount: { $gte: 11 } })).toBe(false);
expect(matches(doc, { createdAt: { $gt: new Date('2026-07-03T00:00:00Z') } })).toBe(true);
expect(matches(doc, { createdAt: { $lt: new Date('2026-07-03T00:00:00Z') } })).toBe(false);
});
test('logical $or / $and / $nor', () => {
expect(matches(doc, { $or: [{ user: 'nope' }, { conversationId: 'c1' }] })).toBe(true);
expect(matches(doc, { $and: [{ user: 'u1' }, { conversationId: 'c1' }] })).toBe(true);
expect(matches(doc, { $nor: [{ user: 'u1' }] })).toBe(false);
});
test('$exists false / null equivalence for absent fields', () => {
expect(matches({ a: 1 }, { $or: [{ b: null }, { b: { $exists: false } }] })).toBe(true);
});
});
describe('base/query applyUpdate', () => {
test('$set / implicit set / $unset', () => {
expect(applyUpdate({ a: 1 }, { $set: { b: 2 } })).toEqual({ a: 1, b: 2 });
expect(applyUpdate({ a: 1 }, { b: 2 })).toEqual({ a: 1, b: 2 }); // implicit $set (merge)
expect(applyUpdate({ a: 1, b: 2 }, { $unset: { b: 1 } })).toEqual({ a: 1 });
});
test('$inc accumulates', () => {
expect(applyUpdate({ n: 5 }, { $inc: { n: 3 } })).toEqual({ n: 8 });
expect(applyUpdate({}, { $inc: { n: 2 } })).toEqual({ n: 2 });
});
test('$push / $addToSet / $pull', () => {
expect(applyUpdate({ a: [1] }, { $push: { a: 2 } })).toEqual({ a: [1, 2] });
expect(applyUpdate({ a: [1] }, { $addToSet: { a: 1 } })).toEqual({ a: [1] });
expect(applyUpdate({ a: [1, 2, 3] }, { $pull: { a: 2 } })).toEqual({ a: [1, 3] });
});
test('$setOnInsert only applies on insert', () => {
expect(applyUpdate({}, { $setOnInsert: { a: 1 } }, { isInsert: true })).toEqual({ a: 1 });
expect(applyUpdate({}, { $setOnInsert: { a: 1 } }, { isInsert: false })).toEqual({});
});
test('prototype pollution is blocked (dotted + bare keys, every operator)', () => {
applyUpdate({}, { $set: { '__proto__.polluted': 'x' } });
applyUpdate({}, { '__proto__.polluted2': 'y' }); // implicit $set
applyUpdate({}, { $inc: { '__proto__.count': 5 } });
applyUpdate({}, { $set: { 'constructor.prototype.polluted3': 'z' } });
applyUpdate({}, { $set: { __proto__: { polluted4: 'w' } } });
applyUpdate({}, { $rename: { a: '__proto__.x' } });
expect({}.polluted).toBeUndefined();
expect({}.polluted2).toBeUndefined();
expect({}.count).toBeUndefined();
expect({}.polluted3).toBeUndefined();
expect({}.polluted4).toBeUndefined();
expect({}.x).toBeUndefined();
expect(Object.prototype.polluted).toBeUndefined();
});
});
describe('base/query projection + sort', () => {
const d = { _id: '1', a: 1, b: 2, c: 3, password: 'secret' };
const hidden = new Set(['password']);
test('inclusion projection keeps _id', () => {
expect(applyProjection(d, parseProjection('a b'), hidden)).toEqual({ _id: '1', a: 1, b: 2 });
});
test('exclusion projection', () => {
expect(applyProjection(d, parseProjection('-b -c -password'), hidden)).toEqual({ _id: '1', a: 1 });
});
test('select:false field hidden by default', () => {
expect(applyProjection(d, null, hidden)).toEqual({ _id: '1', a: 1, b: 2, c: 3 });
});
test('+field re-includes a select:false field on top of default', () => {
expect(applyProjection(d, parseProjection('+password'), hidden)).toEqual({
_id: '1',
a: 1,
b: 2,
c: 3,
password: 'secret',
});
});
test('explicit inclusion can return a select:false field', () => {
expect(applyProjection(d, parseProjection('a password'), hidden)).toEqual({
_id: '1',
a: 1,
password: 'secret',
});
});
test('sort by multiple keys / directions', () => {
const rows = [
{ k: 2, t: 'b' },
{ k: 1, t: 'a' },
{ k: 2, t: 'a' },
];
expect(sortDocs(rows, { k: 1, t: 1 }).map((r) => `${r.k}${r.t}`)).toEqual(['1a', '2a', '2b']);
expect(sortDocs(rows, { k: -1 }).map((r) => r.k)).toEqual([2, 2, 1]);
});
});
+167
View File
@@ -0,0 +1,167 @@
'use strict';
/**
* Introspects a real Mongoose Schema (built by @librechat/data-schemas as a
* pure schema DSL — never connected to Mongo) and distills exactly what the
* Base adapter needs:
* - default values to apply on insert
* - whether timestamps (createdAt/updatedAt) are managed
* - which primitive fields to promote to real Base columns for filter/sort
*
* Everything else about the Mongoose schema (validators, casting, refs) is
* intentionally ignored: the adapter trusts the application layer and stores
* the full document as JSON.
*/
const PRIMITIVE_INSTANCES = new Set(['String', 'Number', 'Date', 'Boolean', 'ObjectID', 'ObjectId']);
/** Identifier / scoping fields that are `meiliIndex` but not useful free-text search targets. */
const SEARCH_STOPLIST = new Set([
'_id',
'_meiliIndex',
'conversationId',
'messageId',
'user',
'organization',
'endpoint',
'model',
'sender',
]);
/** Map a Mongoose SchemaType instance to a Base column type. */
function baseColumnType(instance) {
switch (instance) {
case 'Number':
return 'number';
case 'Boolean':
return 'bool';
case 'Date':
return 'date';
default:
return 'text';
}
}
/**
* @param {import('mongoose').Schema} schema
* @returns {{
* defaults: Array<[string, unknown]>,
* timestamps: { createdAt: string, updatedAt: string } | null,
* promoted: Array<{ name: string, type: string, unique: boolean }>,
* deselected: Set<string>,
* dateFields: Set<string>,
* }}
*/
function describeSchema(schema) {
const defaults = [];
const promoted = new Map();
const deselected = new Set();
const dateFields = new Set();
const searchable = new Set();
// Always promote _id so document lookup by id is a real, unique Base column.
promoted.set('_id', { name: '_id', type: 'text', unique: true });
const paths = schema.paths || {};
for (const name of Object.keys(paths)) {
if (name === '_id' || name === '__v') {
continue;
}
const type = paths[name];
const options = type.options || {};
// Fields marked `select: false` are excluded from reads by default
// (Mongoose semantics) — critical for secrets: password, totpSecret,
// backupCodes, keyHash. Only returned when explicitly `+selected`.
if (options.select === false) {
deselected.add(name);
}
// All Date-typed paths must be re-hydrated to Date objects on read so
// callers can safely call `.getTime()` / date methods (e.g. session.expiration).
if (type.instance === 'Date') {
dateFields.add(name);
}
// Free-text search targets: fields flagged for MeiliSearch that are actual
// content (title, text, …), not identifiers. Used by Base/SQLite FTS.
if (options.meiliIndex && type.instance === 'String' && !SEARCH_STOPLIST.has(name)) {
searchable.add(name);
}
// Collect declared defaults for primitive top-level paths (no dots).
if (!name.includes('.') && Object.prototype.hasOwnProperty.call(options, 'default')) {
defaults.push([name, options.default]);
}
// Promote indexed / unique primitive fields to Base columns for filtering.
// A DB-level UNIQUE index is only safe for always-present unique keys
// (unique && required) — e.g. conversationId, messageId — to avoid the
// sparse-null conflicts of optional unique fields (googleId, openidId, …),
// whose logical uniqueness stays enforced by the adapter-level upsert.
if (!name.includes('.') && PRIMITIVE_INSTANCES.has(type.instance)) {
if ((options.index || options.unique) && !promoted.has(name)) {
const columnType = baseColumnType(type.instance);
promoted.set(name, {
name,
type: columnType,
// Full UNIQUE for always-present keys (conversationId, messageId, email).
unique: !!(options.unique && options.required),
// Partial UNIQUE (WHERE != '') for optional unique text keys
// (googleId, openidId, username, …): DB-level race safety without the
// sparse-null conflict.
sparseUnique: !!(options.unique && !options.required && columnType === 'text'),
});
}
}
}
// Single-field indexes declared via schema.index({ field: 1 }).
try {
for (const [spec] of schema.indexes()) {
const fields = Object.keys(spec || {});
if (fields.length === 1) {
const f = fields[0];
if (f !== '_id' && !f.includes('.') && paths[f] && PRIMITIVE_INSTANCES.has(paths[f].instance) && !promoted.has(f)) {
promoted.set(f, { name: f, type: baseColumnType(paths[f].instance), unique: false });
}
}
}
} catch {
/* schema.indexes() unavailable — ignore */
}
let timestamps = null;
const tsOpt = schema.options && schema.options.timestamps;
if (tsOpt) {
timestamps =
tsOpt === true
? { createdAt: 'createdAt', updatedAt: 'updatedAt' }
: {
createdAt: tsOpt.createdAt || 'createdAt',
updatedAt: tsOpt.updatedAt || 'updatedAt',
};
for (const f of [timestamps.createdAt, timestamps.updatedAt]) {
dateFields.add(f);
if (!promoted.has(f)) {
promoted.set(f, { name: f, type: 'date', unique: false });
}
}
}
return {
defaults,
timestamps,
promoted: [...promoted.values()],
deselected,
dateFields,
searchable,
};
}
/** Resolve a schema default (calling it if it is a function). */
function resolveDefault(def) {
return typeof def === 'function' ? def() : def;
}
module.exports = { describeSchema, resolveDefault, baseColumnType };
+44
View File
@@ -0,0 +1,44 @@
'use strict';
/*
* Boot smoke: drive the REAL wired boot entry points against Hanzo Base, the
* same way api/server/index.js does:
* require('~/db').connectDb() -> Base facade + collection provisioning
* require('~/models').seedDatabase() -> roles + default categories
*
* Proves the wired data layer boots on Base and that non-core models
* (Role, AgentCategory) come along through the same adapter.
*
* Run: HANZO_BASE_URL=... HANZO_BASE_TOKEN=... node api/db/base/__tests__/boot-smoke.js
*/
const path = require('path');
require('module-alias')({ base: path.resolve(__dirname, '../../..') }); // base = api/
const assert = require('assert');
async function main() {
const { connectDb } = require('~/db');
const { seedDatabase } = require('~/models');
const { Role } = require('~/db/models');
await connectDb();
console.log('[boot] connectDb() OK — Base connected + collections provisioned');
await seedDatabase();
console.log('[boot] seedDatabase() OK — roles + categories seeded');
const adminRole = await Role.findOne({ name: 'ADMIN' }).lean();
assert(adminRole && adminRole.name === 'ADMIN', 'ADMIN role seeded to Base');
console.log(' ok - ADMIN role present on Base:', adminRole.name);
const userRole = await Role.findOne({ name: 'USER' }).lean();
assert(userRole && userRole.name === 'USER', 'USER role seeded to Base');
console.log(' ok - USER role present on Base:', userRole.name);
console.log('\nBOOT SMOKE PASSED — wired data layer boots on Hanzo Base.');
process.exit(0);
}
main().catch((e) => {
console.error('\nBOOT SMOKE FAILED:', e && e.stack ? e.stack : e);
process.exit(1);
});
+141
View File
@@ -0,0 +1,141 @@
'use strict';
/*
* LIVE proof: the REAL @librechat/data-schemas model+method factories, driven
* through the Hanzo Base facade, persisting to a running Base instance.
*
* Exercises the login path (createUser + balance grant + findUser +
* bcrypt.compare) and the chat hot path (saveConvo + saveMessage shapes),
* then reads the documents straight back out of Base to prove they landed
* there — not in MongoDB.
*
* Run: HANZO_BASE_URL=... HANZO_BASE_TOKEN=... node api/db/base/__tests__/live.js
*/
const assert = require('assert');
const bcrypt = require('bcryptjs');
const { createModels, createMethods } = require('@librechat/data-schemas');
const { mongoose, connectDb, store } = require('..');
async function main() {
const models = createModels(mongoose);
const methods = createMethods(mongoose);
await connectDb();
console.log('[live] connected + collections provisioned');
const ok = (m) => console.log(' ok -', m);
const email = `z+${Date.now()}@zoo.ngo`;
const plain = 'IloveChat2026!!';
const password = await bcrypt.hash(plain, 10);
// ---- LOGIN PATH ----------------------------------------------------------
const userId = await methods.createUser(
{ email, password, name: 'Z', username: 'z', provider: 'local' },
{ enabled: true, startBalance: 100000 },
true,
false,
);
assert(userId, 'createUser returned an id');
ok('createUser (User + Balance grant)');
// Default read hides select:false secrets (password/totpSecret) — regression guard.
const safeUser = await methods.findUser({ email: email.toUpperCase() }); // email normalized
assert(safeUser && safeUser.email === email, 'findUser by (normalized) email');
assert.strictEqual(safeUser.password, undefined, 'password NOT returned by default (select:false)');
ok('findUser default view hides the password hash (select:false honored)');
// Real login path selects +password explicitly (see api/strategies/localStrategy.js).
const user = await methods.findUser({ email }, '+password');
const match = await bcrypt.compare(plain, user.password);
assert.strictEqual(match, true, 'bcrypt.compare succeeds — password stored hashed');
assert.notStrictEqual(user.password, plain, 'password is NOT plaintext');
ok('findUser(+password) + bcrypt verification (hashed, never plaintext)');
const balance = await models.Balance.findOne({ user: user._id }).lean();
assert.strictEqual(balance.tokenCredits, 100000, 'balance granted via $inc upsert');
ok('Balance record created on Base with $inc grant');
// ---- CHAT HOT PATH -------------------------------------------------------
const uid = String(user._id);
const convo = await models.Conversation.findOneAndUpdate(
{ conversationId: 'conv-live-1', user: uid },
{ $set: { title: 'Hello Base', endpoint: 'openAI', model: 'gpt-4o', messages: [] } },
{ new: true, upsert: true },
);
assert.strictEqual(convo.conversationId, 'conv-live-1');
assert.strictEqual(convo.title, 'Hello Base');
ok('saveConvo — conversation upserted to Base');
const msg = await models.Message.findOneAndUpdate(
{ messageId: 'msg-live-1', user: uid },
{
conversationId: 'conv-live-1',
text: 'Persisted to Hanzo Base (SQLite), not Mongo.',
sender: 'User',
isCreatedByUser: true,
},
{ new: true, upsert: true },
);
assert.strictEqual(msg.messageId, 'msg-live-1');
ok('saveMessage — message upserted to Base');
const msgs = await models.Message.find({ conversationId: 'conv-live-1', user: uid })
.sort({ createdAt: 1 })
.lean();
assert.strictEqual(msgs.length, 1);
assert.ok(msgs[0].text.includes('Hanzo Base'));
ok('getMessages — read back from Base, sorted');
// ---- VERIFY THE BYTES ARE IN BASE (raw REST, bypassing the adapter) ------
const raw = await store.client.send(
`/v1/collections/message/records?filter=${encodeURIComponent(`messageId='msg-live-1' && user='${uid}'`)}`,
);
assert.ok(raw.items && raw.items.length === 1, 'message present in Base via raw REST');
const storedDoc = raw.items[0].data;
const parsed = typeof storedDoc === 'string' ? JSON.parse(storedDoc) : storedDoc;
assert.strictEqual(parsed.messageId, 'msg-live-1', 'Base record.data holds the full document');
assert.strictEqual(raw.items[0].conversationId, 'conv-live-1', 'promoted column mirrored for filter');
ok('RAW Base REST confirms the document physically persisted to Base/SQLite');
// ---- FTS (Base/SQLite search replaces MeiliSearch) -----------------------
const convoHits = await models.Conversation.meiliSearch('hello', { filter: `user = "${uid}"` });
assert.ok(
convoHits.hits.some((h) => h.conversationId === 'conv-live-1'),
'conversation search returns the matching convo by title',
);
ok('Conversation.meiliSearch — real title hit from Base/SQLite');
const msgHits = await models.Message.meiliSearch('hanzo base', { filter: `user = "${uid}"` });
assert.ok(
msgHits.hits.some((h) => h.messageId === 'msg-live-1'),
'message search returns the matching message by text',
);
assert.strictEqual(
(await models.Conversation.meiliSearch('zzz-no-such-term', { filter: `user = "${uid}"` })).hits.length,
0,
'no false positives',
);
ok('Message.meiliSearch — real text hit from Base/SQLite (Meili dropped)');
// ---- Base UNIQUE index on natural key rejects a duplicate (race-safety) ----
let rejected = false;
try {
await store.create('conversation', {
_id: '0'.repeat(24),
conversationId: 'conv-live-1', // duplicate of the one saved above
user: uid,
data: { _id: '0'.repeat(24), conversationId: 'conv-live-1', user: uid },
});
} catch (err) {
rejected = true;
}
assert.ok(rejected, 'Base UNIQUE index on conversationId rejects a duplicate insert');
ok('Base UNIQUE index enforces conversationId uniqueness (race-safety)');
console.log('\nLIVE proof PASSED — login + conversation + message + SEARCH on Hanzo Base.');
console.log(` user collection id space, email=${email}`);
process.exit(0);
}
main().catch((e) => {
console.error('\nLIVE proof FAILED:', e && e.stack ? e.stack : e);
process.exit(1);
});
+233
View File
@@ -0,0 +1,233 @@
'use strict';
/**
* DocumentStore — the Hanzo Base backend for the Mongoose adapter.
*
* Each Mongo model maps to one Base collection whose records carry the full
* document as a JSON `data` field plus a handful of promoted scalar columns
* (derived from indexed/unique schema paths) used only for filter pushdown.
*
* Filter pushdown builds a Base filter-DSL string that is guaranteed to be a
* SUPERSET of the true result; query.js then narrows it in JS, so pushdown can
* be as conservative as needed without ever being wrong.
*/
const { idString } = require('./query');
// `@hanzo/base` ships as an ESM-only package (exports expose only the `import`
// condition), so it must be loaded from this CommonJS module via dynamic import.
let _baseModulePromise = null;
function loadBase() {
if (!_baseModulePromise) {
_baseModulePromise = import('@hanzo/base');
}
return _baseModulePromise;
}
const DATA_FIELD_MAX_SIZE = 8 * 1024 * 1024; // 8 MiB per document
/** Escape a string value for the Base filter DSL (single-quoted). */
function quote(value) {
return `'${String(value).replace(/\\/g, '\\\\').replace(/'/g, "\\'")}'`;
}
/** Format a scalar for the filter DSL, or return undefined if not pushable. */
function literal(value) {
const v = idString(value);
if (v === null || v === undefined) {
return undefined;
}
if (typeof v === 'number') {
return Number.isFinite(v) ? String(v) : undefined; // never emit `= NaN/Infinity`
}
if (typeof v === 'boolean') {
return String(v);
}
// Dates are intentionally NOT pushed down: Base normalizes its date columns to
// a `YYYY-MM-DD HH:MM:SS.sssZ` (space) form that never string-matches an ISO
// `T` literal, which would wrongly EXCLUDE the record (superset invariant
// violation). The JS matcher (query.js) handles all date predicates.
if (v instanceof Date) {
return undefined;
}
if (typeof v === 'string') {
return quote(v);
}
return undefined;
}
/**
* Collect conjunctive (AND-ed) constraints on promoted fields into a Base DSL
* filter. Disjunctions and non-promoted / operator predicates are skipped —
* the JS matcher enforces them — keeping the result a safe superset.
*/
function buildFilter(query, promoted) {
const promotedSet = new Set(promoted);
const clauses = [];
const walk = (q) => {
if (!q || typeof q !== 'object') {
return;
}
for (const [key, val] of Object.entries(q)) {
if (key === '$and' && Array.isArray(val)) {
val.forEach(walk);
continue;
}
if (key.startsWith('$')) {
continue; // $or/$nor/etc — not safe to push
}
if (!promotedSet.has(key)) {
continue;
}
if (val === null || typeof val !== 'object' || val instanceof Date) {
const lit = literal(val);
if (lit !== undefined) {
clauses.push(`${key} = ${lit}`);
}
continue;
}
if (Array.isArray(val)) {
continue;
}
// operator expression on a promoted field
if (Array.isArray(val.$in) && val.$in.length) {
const parts = val.$in.map((v) => literal(v)).filter((l) => l !== undefined);
if (parts.length === val.$in.length) {
clauses.push(`(${parts.map((l) => `${key} = ${l}`).join(' || ')})`);
}
continue;
}
if (val.$eq !== undefined) {
const lit = literal(val.$eq);
if (lit !== undefined) {
clauses.push(`${key} = ${lit}`);
}
}
}
};
walk(query);
return clauses.length ? clauses.join(' && ') : undefined;
}
class DocumentStore {
constructor() {
this.client = null;
this.url = null;
this._ensured = new Set();
}
/** @param {{ url: string, token?: string }} config */
async init({ url, token }) {
if (!url) {
throw new Error('[BaseStore] Base URL is required (HANZO_BASE_URL / BASE_URL)');
}
const { BaseClient } = await loadBase();
this.url = url.replace(/\/$/, '');
this.client = new BaseClient(this.url);
if (token) {
this.client.authStore.save(token, null);
}
return this;
}
async health() {
return this.client.send('/v1/health');
}
/** Idempotently ensure the Base collection exists with the required schema. */
async ensureCollection(name, promoted) {
if (this._ensured.has(name)) {
return;
}
try {
await this.client.send(`/v1/collections/${name}`, { method: 'GET' });
this._ensured.add(name);
return;
} catch (err) {
if (!isNotFound(err)) {
throw err;
}
}
const fields = [{ name: 'data', type: 'json', maxSize: DATA_FIELD_MAX_SIZE }];
const indexes = [];
for (const col of promoted) {
fields.push({ name: col.name, type: col.type });
const safe = col.name.replace(/[^A-Za-z0-9_]/g, '_');
const idx = `\`idx_${name}_${safe}\` ON \`${name}\` (\`${col.name}\`)`;
if (col.unique) {
indexes.push(`CREATE UNIQUE INDEX ${idx}`);
} else if (col.sparseUnique) {
// Partial unique: allow many empty values, enforce uniqueness on real ones.
indexes.push(`CREATE UNIQUE INDEX ${idx} WHERE \`${col.name}\` != ''`);
} else {
indexes.push(`CREATE INDEX ${idx}`);
}
}
const body = { name, type: 'base', fields, indexes };
try {
await this.client.send('/v1/collections', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
});
} catch (err) {
// Tolerate a concurrent creator (unique-name violation).
if (!isConflict(err) && !isNotFound(err)) {
throw err;
}
}
this._ensured.add(name);
}
_parse(record) {
let doc = record.data;
if (typeof doc === 'string') {
try {
doc = JSON.parse(doc);
} catch {
doc = {};
}
}
return { baseId: record.id, doc: doc || {} };
}
/** List candidate docs for a Mongo filter (superset via pushdown). */
async list(collectionName, mongoFilter, promotedNames) {
const filter = buildFilter(mongoFilter, promotedNames || []);
const items = await this.client
.collection(collectionName)
.getFullList(filter ? { filter } : {});
return items.map((r) => this._parse(r));
}
async create(collectionName, record) {
const created = await this.client.collection(collectionName).create(record);
return this._parse(created);
}
async update(collectionName, baseId, record) {
const updated = await this.client.collection(collectionName).update(baseId, record);
return this._parse(updated);
}
async delete(collectionName, baseId) {
await this.client.collection(collectionName).delete(baseId);
}
}
function statusOf(err) {
return (err && (err.status || (err.data && err.data.status))) || 0;
}
function isNotFound(err) {
return statusOf(err) === 404;
}
function isConflict(err) {
return statusOf(err) === 400 || statusOf(err) === 409;
}
module.exports = new DocumentStore();
module.exports.DocumentStore = DocumentStore;
module.exports.buildFilter = buildFilter;
+43
View File
@@ -0,0 +1,43 @@
'use strict';
/* buildFilter — the Base filter-DSL pushdown. Must only ever emit a SUPERSET. */
const { buildFilter } = require('./store');
const promoted = ['_id', 'conversationId', 'user', 'expiredAt', 'tokenCount'];
describe('store/buildFilter pushdown', () => {
test('promoted equality + $in', () => {
expect(buildFilter({ conversationId: 'c1', user: 'u1' }, promoted)).toBe(
"conversationId = 'c1' && user = 'u1'",
);
expect(buildFilter({ conversationId: { $in: ['a', 'b'] } }, promoted)).toBe(
"(conversationId = 'a' || conversationId = 'b')",
);
});
test('DSL injection cannot break out of the quoted literal', () => {
const f = buildFilter({ conversationId: "x' || user='victim" }, promoted);
expect(f).toBe("conversationId = 'x\\' || user=\\'victim'");
expect(f).not.toMatch(/ \|\| user='victim/); // no un-escaped injected clause
});
test('Date predicates are NOT pushed down (superset invariant — H1)', () => {
// Base date columns normalize to a space separator; pushing an ISO literal
// would wrongly EXCLUDE the record. Dates must fall through to the JS matcher.
expect(buildFilter({ expiredAt: new Date('2026-07-04T12:00:00Z') }, promoted)).toBeUndefined();
expect(buildFilter({ expiredAt: { $eq: new Date() } }, promoted)).toBeUndefined();
// a date predicate must never partially push while dropping the date clause
expect(buildFilter({ user: 'u1', expiredAt: new Date() }, promoted)).toBe("user = 'u1'");
});
test('non-finite numbers are not pushed (invalid DSL guard — L1)', () => {
expect(buildFilter({ tokenCount: NaN }, promoted)).toBeUndefined();
expect(buildFilter({ tokenCount: Infinity }, promoted)).toBeUndefined();
expect(buildFilter({ tokenCount: 5 }, promoted)).toBe('tokenCount = 5');
});
test('non-promoted / $or / empty-$in are not pushed (JS matcher handles them)', () => {
expect(buildFilter({ title: 'x' }, promoted)).toBeUndefined();
expect(buildFilter({ $or: [{ user: 'a' }, { user: 'b' }] }, promoted)).toBeUndefined();
expect(buildFilter({ conversationId: { $in: [] } }, promoted)).toBeUndefined();
});
});
+7 -92
View File
@@ -1,95 +1,10 @@
require('dotenv').config();
const { isEnabled } = require('@hanzochat/api');
const { logger } = require('@librechat/data-schemas');
const mongoose = require('mongoose');
const MONGO_URI = process.env.MONGO_URI;
// MONGO_URI is intentionally optional: in SQLite-only mode (every collection
// served from CHAT_STORE_SQLITE, chat-docdb deleted) there is no Mongo to
// connect to. connectDb() below handles the unset case by skipping the
// connection rather than failing boot. When MONGO_URI IS set (default and the
// dual-write migration window) the connection is required as before.
/** The maximum number of connections in the connection pool. */
const maxPoolSize = parseInt(process.env.MONGO_MAX_POOL_SIZE) || undefined;
/** The minimum number of connections in the connection pool. */
const minPoolSize = parseInt(process.env.MONGO_MIN_POOL_SIZE) || undefined;
/** The maximum number of connections that may be in the process of being established concurrently by the connection pool. */
const maxConnecting = parseInt(process.env.MONGO_MAX_CONNECTING) || undefined;
/** The maximum number of milliseconds that a connection can remain idle in the pool before being removed and closed. */
const maxIdleTimeMS = parseInt(process.env.MONGO_MAX_IDLE_TIME_MS) || undefined;
/** The maximum time in milliseconds that a thread can wait for a connection to become available. */
const waitQueueTimeoutMS = parseInt(process.env.MONGO_WAIT_QUEUE_TIMEOUT_MS) || undefined;
/** Set to false to disable automatic index creation for all models associated with this connection. */
const autoIndex =
process.env.MONGO_AUTO_INDEX != undefined
? isEnabled(process.env.MONGO_AUTO_INDEX) || false
: undefined;
/** Set to `false` to disable Mongoose automatically calling `createCollection()` on every model created on this connection. */
const autoCreate =
process.env.MONGO_AUTO_CREATE != undefined
? isEnabled(process.env.MONGO_AUTO_CREATE) || false
: undefined;
/**
* Global is used here to maintain a cached connection across hot reloads
* in development. This prevents connections growing exponentially
* during API Route usage.
* Database connection.
*
* MongoDB has been dropped from Hanzo Chat — the data layer runs on Hanzo Base
* (SQLite embedded / Postgres for prod multi-instance). `connectDb` initialises
* the Base client and provisions every registered model's collection. See ./base.
*/
let cached = global.mongoose;
const { connectDb } = require('./base');
if (!cached) {
cached = global.mongoose = { conn: null, promise: null };
}
mongoose.connection.on('error', (err) => {
logger.error('[connectDb] MongoDB connection error:', err);
});
async function connectDb() {
if (!MONGO_URI) {
// SQLite-only mode: skip Mongo entirely. Disable command buffering so any
// stray mongoose query (e.g. Meili indexSync, which still references
// mongoose.models directly) fails fast and is swallowed by its caller
// instead of hanging forever on a pool that will never connect.
mongoose.set('bufferCommands', false);
logger.info('[connectDb] MONGO_URI unset — SQLite-only mode, skipping MongoDB connection');
return null;
}
if (cached.conn && cached.conn?._readyState === 1) {
return cached.conn;
}
const disconnected = cached.conn && cached.conn?._readyState !== 1;
if (!cached.promise || disconnected) {
const opts = {
bufferCommands: false,
...(maxPoolSize ? { maxPoolSize } : {}),
...(minPoolSize ? { minPoolSize } : {}),
...(maxConnecting ? { maxConnecting } : {}),
...(maxIdleTimeMS ? { maxIdleTimeMS } : {}),
...(waitQueueTimeoutMS ? { waitQueueTimeoutMS } : {}),
...(autoIndex != undefined ? { autoIndex } : {}),
...(autoCreate != undefined ? { autoCreate } : {}),
// useNewUrlParser: true,
// useUnifiedTopology: true,
// bufferMaxEntries: 0,
// useFindAndModify: true,
// useCreateIndex: true
};
logger.info('Mongo Connection options');
logger.info(JSON.stringify(opts, null, 2));
mongoose.set('strictQuery', true);
cached.promise = mongoose.connect(MONGO_URI, opts).then((mongoose) => {
return mongoose;
});
}
cached.conn = await cached.promise;
return cached.conn;
}
module.exports = {
connectDb,
};
module.exports = { connectDb };
+2 -2
View File
@@ -1,8 +1,8 @@
const mongoose = require('mongoose');
const { mongoose, connectDb } = require('./base');
const { createModels } = require('@librechat/data-schemas');
const { connectDb } = require('./connect');
const indexSync = require('./indexSync');
// Register every model on the Base-backed mongoose facade.
createModels(mongoose);
module.exports = { connectDb, indexSync };
-26
View File
@@ -1,26 +0,0 @@
describe('api/db/index.js', () => {
test('createModels is called before indexSync is loaded', () => {
jest.resetModules();
const callOrder = [];
jest.mock('@librechat/data-schemas', () => ({
createModels: jest.fn((m) => {
callOrder.push('createModels');
m.models.Message = { name: 'Message' };
m.models.Conversation = { name: 'Conversation' };
}),
}));
jest.mock('./indexSync', () => {
callOrder.push('indexSync');
return jest.fn();
});
jest.mock('./connect', () => ({ connectDb: jest.fn() }));
require('./index');
expect(callOrder).toEqual(['createModels', 'indexSync']);
});
});
+8 -355
View File
@@ -1,363 +1,16 @@
const mongoose = require('mongoose');
const { MeiliSearch } = require('meilisearch');
const { logger } = require('@librechat/data-schemas');
const { CacheKeys } = require('librechat-data-provider');
const { isEnabled, FlowStateManager } = require('@hanzochat/api');
const { getLogStores } = require('~/cache');
const { batchResetMeiliFlags } = require('./utils');
const Conversation = mongoose.models.Conversation;
const Message = mongoose.models.Message;
const searchEnabled = isEnabled(process.env.SEARCH);
const indexingDisabled = isEnabled(process.env.MEILI_NO_SYNC);
let currentTimeout = null;
const defaultSyncThreshold = 1000;
const syncThreshold = process.env.MEILI_SYNC_THRESHOLD
? parseInt(process.env.MEILI_SYNC_THRESHOLD, 10)
: defaultSyncThreshold;
class MeiliSearchClient {
static instance = null;
static getInstance() {
if (!MeiliSearchClient.instance) {
if (!process.env.MEILI_HOST || !process.env.MEILI_MASTER_KEY) {
throw new Error('Meilisearch configuration is missing.');
}
MeiliSearchClient.instance = new MeiliSearch({
host: process.env.MEILI_HOST,
apiKey: process.env.MEILI_MASTER_KEY,
});
}
return MeiliSearchClient.instance;
}
}
/**
* Deletes documents from MeiliSearch index that are missing the user field
* @param {import('meilisearch').Index} index - MeiliSearch index instance
* @param {string} indexName - Name of the index for logging
* @returns {Promise<number>} - Number of documents deleted
*/
async function deleteDocumentsWithoutUserField(index, indexName) {
let deletedCount = 0;
let offset = 0;
const batchSize = 1000;
try {
while (true) {
const searchResult = await index.search('', {
limit: batchSize,
offset: offset,
});
if (searchResult.hits.length === 0) {
break;
}
const idsToDelete = searchResult.hits.filter((hit) => !hit.user).map((hit) => hit.id);
if (idsToDelete.length > 0) {
logger.info(
`[indexSync] Deleting ${idsToDelete.length} documents without user field from ${indexName} index`,
);
await index.deleteDocuments(idsToDelete);
deletedCount += idsToDelete.length;
}
if (searchResult.hits.length < batchSize) {
break;
}
offset += batchSize;
}
if (deletedCount > 0) {
logger.info(`[indexSync] Deleted ${deletedCount} orphaned documents from ${indexName} index`);
}
} catch (error) {
logger.error(`[indexSync] Error deleting documents from ${indexName}:`, error);
}
return deletedCount;
}
/**
* Ensures indexes have proper filterable attributes configured and checks if documents have user field
* @param {MeiliSearch} client - MeiliSearch client instance
* @returns {Promise<{settingsUpdated: boolean, orphanedDocsFound: boolean}>} - Status of what was done
*/
async function ensureFilterableAttributes(client) {
let settingsUpdated = false;
let hasOrphanedDocs = false;
try {
// Check and update messages index
try {
const messagesIndex = client.index('messages');
const settings = await messagesIndex.getSettings();
if (!settings.filterableAttributes || !settings.filterableAttributes.includes('user')) {
logger.info('[indexSync] Configuring messages index to filter by user...');
await messagesIndex.updateSettings({
filterableAttributes: ['user'],
});
logger.info('[indexSync] Messages index configured for user filtering');
settingsUpdated = true;
}
// Check if existing documents have user field indexed
try {
const searchResult = await messagesIndex.search('', { limit: 1 });
if (searchResult.hits.length > 0 && !searchResult.hits[0].user) {
logger.info(
'[indexSync] Existing messages missing user field, will clean up orphaned documents...',
);
hasOrphanedDocs = true;
}
} catch (searchError) {
logger.debug('[indexSync] Could not check message documents:', searchError.message);
}
} catch (error) {
if (error.code !== 'index_not_found') {
logger.warn('[indexSync] Could not check/update messages index settings:', error.message);
}
}
// Check and update conversations index
try {
const convosIndex = client.index('convos');
const settings = await convosIndex.getSettings();
if (!settings.filterableAttributes || !settings.filterableAttributes.includes('user')) {
logger.info('[indexSync] Configuring convos index to filter by user...');
await convosIndex.updateSettings({
filterableAttributes: ['user'],
});
logger.info('[indexSync] Convos index configured for user filtering');
settingsUpdated = true;
}
// Check if existing documents have user field indexed
try {
const searchResult = await convosIndex.search('', { limit: 1 });
if (searchResult.hits.length > 0 && !searchResult.hits[0].user) {
logger.info(
'[indexSync] Existing conversations missing user field, will clean up orphaned documents...',
);
hasOrphanedDocs = true;
}
} catch (searchError) {
logger.debug('[indexSync] Could not check conversation documents:', searchError.message);
}
} catch (error) {
if (error.code !== 'index_not_found') {
logger.warn('[indexSync] Could not check/update convos index settings:', error.message);
}
}
// If either index has orphaned documents, clean them up (but don't force resync)
if (hasOrphanedDocs) {
try {
const messagesIndex = client.index('messages');
await deleteDocumentsWithoutUserField(messagesIndex, 'messages');
} catch (error) {
logger.debug('[indexSync] Could not clean up messages:', error.message);
}
try {
const convosIndex = client.index('convos');
await deleteDocumentsWithoutUserField(convosIndex, 'convos');
} catch (error) {
logger.debug('[indexSync] Could not clean up convos:', error.message);
}
logger.info('[indexSync] Orphaned documents cleaned up without forcing resync.');
}
if (settingsUpdated) {
logger.info('[indexSync] Index settings updated. Full re-sync will be triggered.');
}
} catch (error) {
logger.error('[indexSync] Error ensuring filterable attributes:', error);
}
return { settingsUpdated, orphanedDocsFound: hasOrphanedDocs };
}
/**
* Performs the actual sync operations for messages and conversations
* @param {FlowStateManager} flowManager - Flow state manager instance
* @param {string} flowId - Flow identifier
* @param {string} flowType - Flow type
*/
async function performSync(flowManager, flowId, flowType) {
try {
if (indexingDisabled === true) {
logger.info('[indexSync] Indexing is disabled, skipping...');
return { messagesSync: false, convosSync: false };
}
const client = MeiliSearchClient.getInstance();
const { status } = await client.health();
if (status !== 'available') {
throw new Error('Meilisearch not available');
}
/** Ensures indexes have proper filterable attributes configured */
const { settingsUpdated, orphanedDocsFound: _orphanedDocsFound } =
await ensureFilterableAttributes(client);
let messagesSync = false;
let convosSync = false;
// Only reset flags if settings were actually updated (not just for orphaned doc cleanup)
if (settingsUpdated) {
logger.info(
'[indexSync] Settings updated. Forcing full re-sync to reindex with new configuration...',
);
// Reset sync flags to force full re-sync
await batchResetMeiliFlags(Message.collection);
await batchResetMeiliFlags(Conversation.collection);
}
// Check if we need to sync messages
logger.info('[indexSync] Requesting message sync progress...');
const messageProgress = await Message.getSyncProgress();
if (!messageProgress.isComplete || settingsUpdated) {
logger.info(
`[indexSync] Messages need syncing: ${messageProgress.totalProcessed}/${messageProgress.totalDocuments} indexed`,
);
const messageCount = messageProgress.totalDocuments;
const messagesIndexed = messageProgress.totalProcessed;
const unindexedMessages = messageCount - messagesIndexed;
if (settingsUpdated || unindexedMessages > syncThreshold) {
logger.info(`[indexSync] Starting message sync (${unindexedMessages} unindexed)`);
await Message.syncWithMeili();
messagesSync = true;
} else if (unindexedMessages > 0) {
logger.info(
`[indexSync] ${unindexedMessages} messages unindexed (below threshold: ${syncThreshold}, skipping)`,
);
}
} else {
logger.info(
`[indexSync] Messages are fully synced: ${messageProgress.totalProcessed}/${messageProgress.totalDocuments}`,
);
}
// Check if we need to sync conversations
const convoProgress = await Conversation.getSyncProgress();
if (!convoProgress.isComplete || settingsUpdated) {
logger.info(
`[indexSync] Conversations need syncing: ${convoProgress.totalProcessed}/${convoProgress.totalDocuments} indexed`,
);
const convoCount = convoProgress.totalDocuments;
const convosIndexed = convoProgress.totalProcessed;
const unindexedConvos = convoCount - convosIndexed;
if (settingsUpdated || unindexedConvos > syncThreshold) {
logger.info(`[indexSync] Starting convos sync (${unindexedConvos} unindexed)`);
await Conversation.syncWithMeili();
convosSync = true;
} else if (unindexedConvos > 0) {
logger.info(
`[indexSync] ${unindexedConvos} convos unindexed (below threshold: ${syncThreshold}, skipping)`,
);
}
} else {
logger.info(
`[indexSync] Conversations are fully synced: ${convoProgress.totalProcessed}/${convoProgress.totalDocuments}`,
);
}
return { messagesSync, convosSync };
} finally {
if (indexingDisabled === true) {
logger.info('[indexSync] Indexing is disabled, skipping cleanup...');
} else if (flowManager && flowId && flowType) {
try {
await flowManager.deleteFlow(flowId, flowType);
logger.debug('[indexSync] Flow state cleaned up');
} catch (cleanupErr) {
logger.debug('[indexSync] Could not clean up flow state:', cleanupErr.message);
}
}
}
}
/**
* Main index sync function that uses FlowStateManager to prevent concurrent execution
* Search index sync.
*
* MongoDB + MeiliSearch have been dropped. Full-text search now runs directly
* on Hanzo Base/SQLite via the data-layer adapter (the model `meiliSearch()`
* method in api/db/base searches content fields on Base). There is no separate
* index to sync, so this is a no-op kept for boot-sequence compatibility
* (api/server/index.js calls `indexSync()`).
*/
async function indexSync() {
if (!searchEnabled) {
return;
}
logger.info('[indexSync] Starting index synchronization check...');
// Get or create FlowStateManager instance
const flowsCache = getLogStores(CacheKeys.FLOWS);
if (!flowsCache) {
logger.warn('[indexSync] Flows cache not available, falling back to direct sync');
return await performSync(null, null, null);
}
const flowManager = new FlowStateManager(flowsCache, {
ttl: 60000 * 10, // 10 minutes TTL for sync operations
});
// Use a unique flow ID for the sync operation
const flowId = 'meili-index-sync';
const flowType = 'MEILI_SYNC';
try {
// This will only execute the handler if no other instance is running the sync
const result = await flowManager.createFlowWithHandler(flowId, flowType, () =>
performSync(flowManager, flowId, flowType),
);
if (result.messagesSync || result.convosSync) {
logger.info('[indexSync] Sync completed successfully');
} else {
logger.debug('[indexSync] No sync was needed');
}
return result;
} catch (err) {
if (err.message.includes('flow already exists')) {
logger.info('[indexSync] Sync already running on another instance');
return;
}
if (err.message.includes('not found')) {
logger.debug('[indexSync] Creating indices...');
currentTimeout = setTimeout(async () => {
try {
await Message.syncWithMeili();
await Conversation.syncWithMeili();
} catch (err) {
logger.error('[indexSync] Trouble creating indices, try restarting the server.', err);
}
}, 750);
} else if (err.message.includes('Meilisearch not configured')) {
logger.info('[indexSync] Meilisearch not configured, search will be disabled.');
} else {
logger.error('[indexSync] error', err);
}
}
logger.info('[indexSync] Base/SQLite search active — no external index to sync');
}
process.on('exit', () => {
logger.debug('[indexSync] Clearing sync timeouts before exiting...');
clearTimeout(currentTimeout);
});
module.exports = indexSync;
-465
View File
@@ -1,465 +0,0 @@
/**
* Unit tests for performSync() function in indexSync.js
*
* Tests use real mongoose with mocked model methods, only mocking external calls.
*/
const mongoose = require('mongoose');
// Mock only external dependencies (not internal classes/models)
const mockLogger = {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
};
const mockMeiliHealth = jest.fn();
const mockMeiliIndex = jest.fn();
const mockBatchResetMeiliFlags = jest.fn();
const mockIsEnabled = jest.fn();
const mockGetLogStores = jest.fn();
// Create mock models that will be reused
const createMockModel = (collectionName) => ({
collection: { name: collectionName },
getSyncProgress: jest.fn(),
syncWithMeili: jest.fn(),
countDocuments: jest.fn(),
});
const originalMessageModel = mongoose.models.Message;
const originalConversationModel = mongoose.models.Conversation;
// Mock external modules
jest.mock('@librechat/data-schemas', () => ({
logger: mockLogger,
}));
jest.mock('meilisearch', () => ({
MeiliSearch: jest.fn(() => ({
health: mockMeiliHealth,
index: mockMeiliIndex,
})),
}));
jest.mock('./utils', () => ({
batchResetMeiliFlags: mockBatchResetMeiliFlags,
}));
jest.mock('@hanzochat/api', () => ({
isEnabled: mockIsEnabled,
FlowStateManager: jest.fn(),
}));
jest.mock('~/cache', () => ({
getLogStores: mockGetLogStores,
}));
// Set environment before module load
process.env.MEILI_HOST = 'http://localhost:7700';
process.env.MEILI_MASTER_KEY = 'test-key';
process.env.SEARCH = 'true';
process.env.MEILI_SYNC_THRESHOLD = '1000'; // Set threshold before module loads
describe('performSync() - syncThreshold logic', () => {
const ORIGINAL_ENV = process.env;
let Message;
let Conversation;
beforeAll(() => {
Message = createMockModel('messages');
Conversation = createMockModel('conversations');
mongoose.models.Message = Message;
mongoose.models.Conversation = Conversation;
});
beforeEach(() => {
// Reset all mocks
jest.clearAllMocks();
// Reset modules to ensure fresh load of indexSync.js and its top-level consts (like syncThreshold)
jest.resetModules();
// Set up environment
process.env = { ...ORIGINAL_ENV };
process.env.MEILI_HOST = 'http://localhost:7700';
process.env.MEILI_MASTER_KEY = 'test-key';
process.env.SEARCH = 'true';
delete process.env.MEILI_NO_SYNC;
// Re-ensure models are available in mongoose after resetModules
// We must require mongoose again to get the fresh instance that indexSync will use
const mongoose = require('mongoose');
mongoose.models.Message = Message;
mongoose.models.Conversation = Conversation;
// Mock isEnabled
mockIsEnabled.mockImplementation((val) => val === 'true' || val === true);
// Mock MeiliSearch client responses
mockMeiliHealth.mockResolvedValue({ status: 'available' });
mockMeiliIndex.mockReturnValue({
getSettings: jest.fn().mockResolvedValue({ filterableAttributes: ['user'] }),
updateSettings: jest.fn().mockResolvedValue({}),
search: jest.fn().mockResolvedValue({ hits: [] }),
});
mockBatchResetMeiliFlags.mockResolvedValue(undefined);
});
afterEach(() => {
process.env = ORIGINAL_ENV;
});
afterAll(() => {
mongoose.models.Message = originalMessageModel;
mongoose.models.Conversation = originalConversationModel;
});
test('triggers sync when unindexed messages exceed syncThreshold', async () => {
// Arrange: Set threshold before module load
process.env.MEILI_SYNC_THRESHOLD = '1000';
// Arrange: 1050 unindexed messages > 1000 threshold
Message.getSyncProgress.mockResolvedValue({
totalProcessed: 100,
totalDocuments: 1150, // 1050 unindexed
isComplete: false,
});
Conversation.getSyncProgress.mockResolvedValue({
totalProcessed: 50,
totalDocuments: 50,
isComplete: true,
});
Message.syncWithMeili.mockResolvedValue(undefined);
// Act
const indexSync = require('./indexSync');
await indexSync();
// Assert: No countDocuments calls
expect(Message.countDocuments).not.toHaveBeenCalled();
expect(Conversation.countDocuments).not.toHaveBeenCalled();
// Assert: Message sync triggered because 1050 > 1000
expect(Message.syncWithMeili).toHaveBeenCalledTimes(1);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Messages need syncing: 100/1150 indexed',
);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Starting message sync (1050 unindexed)',
);
// Assert: Conversation sync NOT triggered (already complete)
expect(Conversation.syncWithMeili).not.toHaveBeenCalled();
});
test('skips sync when unindexed messages are below syncThreshold', async () => {
// Arrange: 50 unindexed messages < 1000 threshold
Message.getSyncProgress.mockResolvedValue({
totalProcessed: 100,
totalDocuments: 150, // 50 unindexed
isComplete: false,
});
Conversation.getSyncProgress.mockResolvedValue({
totalProcessed: 50,
totalDocuments: 50,
isComplete: true,
});
process.env.MEILI_SYNC_THRESHOLD = '1000';
// Act
const indexSync = require('./indexSync');
await indexSync();
// Assert: No countDocuments calls
expect(Message.countDocuments).not.toHaveBeenCalled();
expect(Conversation.countDocuments).not.toHaveBeenCalled();
// Assert: Message sync NOT triggered because 50 < 1000
expect(Message.syncWithMeili).not.toHaveBeenCalled();
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Messages need syncing: 100/150 indexed',
);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] 50 messages unindexed (below threshold: 1000, skipping)',
);
// Assert: Conversation sync NOT triggered (already complete)
expect(Conversation.syncWithMeili).not.toHaveBeenCalled();
});
test('respects syncThreshold at boundary (exactly at threshold)', async () => {
// Arrange: 1000 unindexed messages = 1000 threshold (NOT greater than)
Message.getSyncProgress.mockResolvedValue({
totalProcessed: 100,
totalDocuments: 1100, // 1000 unindexed
isComplete: false,
});
Conversation.getSyncProgress.mockResolvedValue({
totalProcessed: 0,
totalDocuments: 0,
isComplete: true,
});
process.env.MEILI_SYNC_THRESHOLD = '1000';
// Act
const indexSync = require('./indexSync');
await indexSync();
// Assert: No countDocuments calls
expect(Message.countDocuments).not.toHaveBeenCalled();
// Assert: Message sync NOT triggered because 1000 is NOT > 1000
expect(Message.syncWithMeili).not.toHaveBeenCalled();
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Messages need syncing: 100/1100 indexed',
);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] 1000 messages unindexed (below threshold: 1000, skipping)',
);
});
test('triggers sync when unindexed is threshold + 1', async () => {
// Arrange: 1001 unindexed messages > 1000 threshold
Message.getSyncProgress.mockResolvedValue({
totalProcessed: 100,
totalDocuments: 1101, // 1001 unindexed
isComplete: false,
});
Conversation.getSyncProgress.mockResolvedValue({
totalProcessed: 0,
totalDocuments: 0,
isComplete: true,
});
Message.syncWithMeili.mockResolvedValue(undefined);
process.env.MEILI_SYNC_THRESHOLD = '1000';
// Act
const indexSync = require('./indexSync');
await indexSync();
// Assert: No countDocuments calls
expect(Message.countDocuments).not.toHaveBeenCalled();
// Assert: Message sync triggered because 1001 > 1000
expect(Message.syncWithMeili).toHaveBeenCalledTimes(1);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Messages need syncing: 100/1101 indexed',
);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Starting message sync (1001 unindexed)',
);
});
test('uses totalDocuments from convoProgress for conversation sync decisions', async () => {
// Arrange: Messages complete, conversations need sync
Message.getSyncProgress.mockResolvedValue({
totalProcessed: 100,
totalDocuments: 100,
isComplete: true,
});
Conversation.getSyncProgress.mockResolvedValue({
totalProcessed: 50,
totalDocuments: 1100, // 1050 unindexed > 1000 threshold
isComplete: false,
});
Conversation.syncWithMeili.mockResolvedValue(undefined);
process.env.MEILI_SYNC_THRESHOLD = '1000';
// Act
const indexSync = require('./indexSync');
await indexSync();
// Assert: No countDocuments calls (the optimization)
expect(Message.countDocuments).not.toHaveBeenCalled();
expect(Conversation.countDocuments).not.toHaveBeenCalled();
// Assert: Only conversation sync triggered
expect(Message.syncWithMeili).not.toHaveBeenCalled();
expect(Conversation.syncWithMeili).toHaveBeenCalledTimes(1);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Conversations need syncing: 50/1100 indexed',
);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Starting convos sync (1050 unindexed)',
);
});
test('skips sync when collections are fully synced', async () => {
// Arrange: Everything already synced
Message.getSyncProgress.mockResolvedValue({
totalProcessed: 100,
totalDocuments: 100,
isComplete: true,
});
Conversation.getSyncProgress.mockResolvedValue({
totalProcessed: 50,
totalDocuments: 50,
isComplete: true,
});
// Act
const indexSync = require('./indexSync');
await indexSync();
// Assert: No countDocuments calls
expect(Message.countDocuments).not.toHaveBeenCalled();
expect(Conversation.countDocuments).not.toHaveBeenCalled();
// Assert: No sync triggered
expect(Message.syncWithMeili).not.toHaveBeenCalled();
expect(Conversation.syncWithMeili).not.toHaveBeenCalled();
// Assert: Correct logs
expect(mockLogger.info).toHaveBeenCalledWith('[indexSync] Messages are fully synced: 100/100');
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Conversations are fully synced: 50/50',
);
});
test('triggers message sync when settingsUpdated even if below syncThreshold', async () => {
// Arrange: Only 50 unindexed messages (< 1000 threshold), but settings were updated
Message.getSyncProgress.mockResolvedValue({
totalProcessed: 100,
totalDocuments: 150, // 50 unindexed
isComplete: false,
});
Conversation.getSyncProgress.mockResolvedValue({
totalProcessed: 50,
totalDocuments: 50,
isComplete: true,
});
Message.syncWithMeili.mockResolvedValue(undefined);
// Mock settings update scenario
mockMeiliIndex.mockReturnValue({
getSettings: jest.fn().mockResolvedValue({ filterableAttributes: [] }), // No user field
updateSettings: jest.fn().mockResolvedValue({}),
search: jest.fn().mockResolvedValue({ hits: [] }),
});
process.env.MEILI_SYNC_THRESHOLD = '1000';
// Act
const indexSync = require('./indexSync');
await indexSync();
// Assert: Flags were reset due to settings update
expect(mockBatchResetMeiliFlags).toHaveBeenCalledWith(Message.collection);
expect(mockBatchResetMeiliFlags).toHaveBeenCalledWith(Conversation.collection);
// Assert: Message sync triggered despite being below threshold (50 < 1000)
expect(Message.syncWithMeili).toHaveBeenCalledTimes(1);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Settings updated. Forcing full re-sync to reindex with new configuration...',
);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Starting message sync (50 unindexed)',
);
});
test('triggers conversation sync when settingsUpdated even if below syncThreshold', async () => {
// Arrange: Messages complete, conversations have 50 unindexed (< 1000 threshold), but settings were updated
Message.getSyncProgress.mockResolvedValue({
totalProcessed: 100,
totalDocuments: 100,
isComplete: true,
});
Conversation.getSyncProgress.mockResolvedValue({
totalProcessed: 50,
totalDocuments: 100, // 50 unindexed
isComplete: false,
});
Conversation.syncWithMeili.mockResolvedValue(undefined);
// Mock settings update scenario
mockMeiliIndex.mockReturnValue({
getSettings: jest.fn().mockResolvedValue({ filterableAttributes: [] }), // No user field
updateSettings: jest.fn().mockResolvedValue({}),
search: jest.fn().mockResolvedValue({ hits: [] }),
});
process.env.MEILI_SYNC_THRESHOLD = '1000';
// Act
const indexSync = require('./indexSync');
await indexSync();
// Assert: Flags were reset due to settings update
expect(mockBatchResetMeiliFlags).toHaveBeenCalledWith(Message.collection);
expect(mockBatchResetMeiliFlags).toHaveBeenCalledWith(Conversation.collection);
// Assert: Conversation sync triggered despite being below threshold (50 < 1000)
expect(Conversation.syncWithMeili).toHaveBeenCalledTimes(1);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Settings updated. Forcing full re-sync to reindex with new configuration...',
);
expect(mockLogger.info).toHaveBeenCalledWith('[indexSync] Starting convos sync (50 unindexed)');
});
test('triggers both message and conversation sync when settingsUpdated even if both below syncThreshold', async () => {
// Arrange: Set threshold before module load
process.env.MEILI_SYNC_THRESHOLD = '1000';
// Arrange: Both have documents below threshold (50 each), but settings were updated
Message.getSyncProgress.mockResolvedValue({
totalProcessed: 100,
totalDocuments: 150, // 50 unindexed
isComplete: false,
});
Conversation.getSyncProgress.mockResolvedValue({
totalProcessed: 50,
totalDocuments: 100, // 50 unindexed
isComplete: false,
});
Message.syncWithMeili.mockResolvedValue(undefined);
Conversation.syncWithMeili.mockResolvedValue(undefined);
// Mock settings update scenario
mockMeiliIndex.mockReturnValue({
getSettings: jest.fn().mockResolvedValue({ filterableAttributes: [] }), // No user field
updateSettings: jest.fn().mockResolvedValue({}),
search: jest.fn().mockResolvedValue({ hits: [] }),
});
// Act
const indexSync = require('./indexSync');
await indexSync();
// Assert: Flags were reset due to settings update
expect(mockBatchResetMeiliFlags).toHaveBeenCalledWith(Message.collection);
expect(mockBatchResetMeiliFlags).toHaveBeenCalledWith(Conversation.collection);
// Assert: Both syncs triggered despite both being below threshold
expect(Message.syncWithMeili).toHaveBeenCalledTimes(1);
expect(Conversation.syncWithMeili).toHaveBeenCalledTimes(1);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Settings updated. Forcing full re-sync to reindex with new configuration...',
);
expect(mockLogger.info).toHaveBeenCalledWith(
'[indexSync] Starting message sync (50 unindexed)',
);
expect(mockLogger.info).toHaveBeenCalledWith('[indexSync] Starting convos sync (50 unindexed)');
});
});
+1 -1
View File
@@ -1,4 +1,4 @@
const mongoose = require('mongoose');
const { mongoose } = require('./base');
const { createModels } = require('@librechat/data-schemas');
const models = createModels(mongoose);
-90
View File
@@ -1,90 +0,0 @@
const { logger } = require('@librechat/data-schemas');
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
/**
* Batch update documents in chunks to avoid timeouts on weak instances
* @param {mongoose.Collection} collection - MongoDB collection
* @returns {Promise<number>} - Total modified count
* @throws {Error} - Throws if database operations fail (e.g., network issues, connection loss, permission problems)
*/
async function batchResetMeiliFlags(collection) {
const DEFAULT_BATCH_SIZE = 1000;
let BATCH_SIZE = parseEnvInt('MEILI_SYNC_BATCH_SIZE', DEFAULT_BATCH_SIZE);
if (BATCH_SIZE === 0) {
logger.warn(
`[batchResetMeiliFlags] MEILI_SYNC_BATCH_SIZE cannot be 0. Using default: ${DEFAULT_BATCH_SIZE}`,
);
BATCH_SIZE = DEFAULT_BATCH_SIZE;
}
const BATCH_DELAY_MS = parseEnvInt('MEILI_SYNC_DELAY_MS', 100);
let totalModified = 0;
let hasMore = true;
try {
while (hasMore) {
const docs = await collection
.find({ expiredAt: null, _meiliIndex: { $ne: false } }, { projection: { _id: 1 } })
.limit(BATCH_SIZE)
.toArray();
if (docs.length === 0) {
break;
}
const ids = docs.map((doc) => doc._id);
const result = await collection.updateMany(
{ _id: { $in: ids } },
{ $set: { _meiliIndex: false } },
);
totalModified += result.modifiedCount;
process.stdout.write(
`\r Updating ${collection.collectionName}: ${totalModified} documents...`,
);
if (docs.length < BATCH_SIZE) {
hasMore = false;
}
if (hasMore && BATCH_DELAY_MS > 0) {
await sleep(BATCH_DELAY_MS);
}
}
return totalModified;
} catch (error) {
throw new Error(
`Failed to batch reset Meili flags for collection '${collection.collectionName}' after processing ${totalModified} documents: ${error.message}`,
);
}
}
/**
* Parse and validate an environment variable as a positive integer
* @param {string} varName - Environment variable name
* @param {number} defaultValue - Default value to use if invalid or missing
* @returns {number} - Parsed value or default
*/
function parseEnvInt(varName, defaultValue) {
const value = process.env[varName];
if (!value) {
return defaultValue;
}
const parsed = parseInt(value, 10);
if (isNaN(parsed) || parsed < 0) {
logger.warn(
`[batchResetMeiliFlags] Invalid value for ${varName}="${value}". Expected a positive integer. Using default: ${defaultValue}`,
);
return defaultValue;
}
return parsed;
}
module.exports = {
batchResetMeiliFlags,
};
-523
View File
@@ -1,523 +0,0 @@
const mongoose = require('mongoose');
const { MongoMemoryServer } = require('mongodb-memory-server');
const { batchResetMeiliFlags } = require('./utils');
describe('batchResetMeiliFlags', () => {
let mongoServer;
let testCollection;
const ORIGINAL_BATCH_SIZE = process.env.MEILI_SYNC_BATCH_SIZE;
const ORIGINAL_BATCH_DELAY = process.env.MEILI_SYNC_DELAY_MS;
beforeAll(async () => {
mongoServer = await MongoMemoryServer.create();
const mongoUri = mongoServer.getUri();
await mongoose.connect(mongoUri);
});
afterAll(async () => {
await mongoose.disconnect();
await mongoServer.stop();
// Restore original env variables
if (ORIGINAL_BATCH_SIZE !== undefined) {
process.env.MEILI_SYNC_BATCH_SIZE = ORIGINAL_BATCH_SIZE;
} else {
delete process.env.MEILI_SYNC_BATCH_SIZE;
}
if (ORIGINAL_BATCH_DELAY !== undefined) {
process.env.MEILI_SYNC_DELAY_MS = ORIGINAL_BATCH_DELAY;
} else {
delete process.env.MEILI_SYNC_DELAY_MS;
}
});
beforeEach(async () => {
// Create a fresh collection for each test
testCollection = mongoose.connection.db.collection('test_meili_batch');
await testCollection.deleteMany({});
// Reset env variables to defaults
delete process.env.MEILI_SYNC_BATCH_SIZE;
delete process.env.MEILI_SYNC_DELAY_MS;
});
afterEach(async () => {
if (testCollection) {
await testCollection.deleteMany({});
}
});
describe('basic functionality', () => {
it('should reset _meiliIndex flag for documents with expiredAt: null and _meiliIndex: true', async () => {
// Insert test documents
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true, name: 'doc1' },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true, name: 'doc2' },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true, name: 'doc3' },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(3);
const updatedDocs = await testCollection.find({ _meiliIndex: false }).toArray();
expect(updatedDocs).toHaveLength(3);
const notUpdatedDocs = await testCollection.find({ _meiliIndex: true }).toArray();
expect(notUpdatedDocs).toHaveLength(0);
});
it('should not modify documents with expiredAt set', async () => {
const expiredDate = new Date();
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: expiredDate, _meiliIndex: true },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(1);
const expiredDoc = await testCollection.findOne({ expiredAt: expiredDate });
expect(expiredDoc._meiliIndex).toBe(true);
});
it('should not modify documents with _meiliIndex: false', async () => {
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: false },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(1);
});
it('should return 0 when no documents match the criteria', async () => {
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: new Date(), _meiliIndex: true },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: false },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(0);
});
it('should return 0 when collection is empty', async () => {
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(0);
});
});
describe('batch processing', () => {
it('should process documents in batches according to MEILI_SYNC_BATCH_SIZE', async () => {
process.env.MEILI_SYNC_BATCH_SIZE = '2';
const docs = [];
for (let i = 0; i < 5; i++) {
docs.push({
_id: new mongoose.Types.ObjectId(),
expiredAt: null,
_meiliIndex: true,
name: `doc${i}`,
});
}
await testCollection.insertMany(docs);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(5);
const updatedDocs = await testCollection.find({ _meiliIndex: false }).toArray();
expect(updatedDocs).toHaveLength(5);
});
it('should handle large datasets with small batch sizes', async () => {
process.env.MEILI_SYNC_BATCH_SIZE = '10';
const docs = [];
for (let i = 0; i < 25; i++) {
docs.push({
_id: new mongoose.Types.ObjectId(),
expiredAt: null,
_meiliIndex: true,
});
}
await testCollection.insertMany(docs);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(25);
});
it('should use default batch size of 1000 when env variable is not set', async () => {
// Create exactly 1000 documents to verify default batch behavior
const docs = [];
for (let i = 0; i < 1000; i++) {
docs.push({
_id: new mongoose.Types.ObjectId(),
expiredAt: null,
_meiliIndex: true,
});
}
await testCollection.insertMany(docs);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(1000);
});
});
describe('return value', () => {
it('should return correct modified count', async () => {
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
await expect(batchResetMeiliFlags(testCollection)).resolves.toBe(1);
});
});
describe('batch delay', () => {
it('should respect MEILI_SYNC_DELAY_MS between batches', async () => {
process.env.MEILI_SYNC_BATCH_SIZE = '2';
process.env.MEILI_SYNC_DELAY_MS = '50';
const docs = [];
for (let i = 0; i < 5; i++) {
docs.push({
_id: new mongoose.Types.ObjectId(),
expiredAt: null,
_meiliIndex: true,
});
}
await testCollection.insertMany(docs);
const startTime = Date.now();
await batchResetMeiliFlags(testCollection);
const endTime = Date.now();
// With 5 documents and batch size 2, we need 3 batches
// That means 2 delays between batches (not after the last one)
// So minimum time should be around 100ms (2 * 50ms)
// Using a slightly lower threshold to account for timing variations
const elapsed = endTime - startTime;
expect(elapsed).toBeGreaterThanOrEqual(80);
});
it('should not delay when MEILI_SYNC_DELAY_MS is 0', async () => {
process.env.MEILI_SYNC_BATCH_SIZE = '2';
process.env.MEILI_SYNC_DELAY_MS = '0';
const docs = [];
for (let i = 0; i < 5; i++) {
docs.push({
_id: new mongoose.Types.ObjectId(),
expiredAt: null,
_meiliIndex: true,
});
}
await testCollection.insertMany(docs);
const startTime = Date.now();
await batchResetMeiliFlags(testCollection);
const endTime = Date.now();
const elapsed = endTime - startTime;
// Should complete without intentional delays, but database operations still take time
// Just verify it completes and returns the correct count
expect(elapsed).toBeLessThan(1000); // More reasonable upper bound
const result = await testCollection.countDocuments({ _meiliIndex: false });
expect(result).toBe(5);
});
it('should not delay after the last batch', async () => {
process.env.MEILI_SYNC_BATCH_SIZE = '3';
process.env.MEILI_SYNC_DELAY_MS = '100';
// Exactly 3 documents - should fit in one batch, no delay
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
// Verify all 3 documents were processed in a single batch
expect(result).toBe(3);
const updatedDocs = await testCollection.countDocuments({ _meiliIndex: false });
expect(updatedDocs).toBe(3);
});
});
describe('edge cases', () => {
it('should handle documents without _meiliIndex field', async () => {
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
// both documents should be updated
expect(result).toBe(2);
});
it('should handle mixed document states correctly', async () => {
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: false },
{ _id: new mongoose.Types.ObjectId(), expiredAt: new Date(), _meiliIndex: true },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: null },
{ _id: new mongoose.Types.ObjectId(), expiredAt: null },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(4);
const flaggedDocs = await testCollection
.find({ expiredAt: null, _meiliIndex: false })
.toArray();
expect(flaggedDocs).toHaveLength(5); // 4 were updated, 1 was already false
});
});
describe('error handling', () => {
it('should throw error with context when find operation fails', async () => {
const mockCollection = {
collectionName: 'test_meili_batch',
find: jest.fn().mockReturnValue({
limit: jest.fn().mockReturnValue({
toArray: jest.fn().mockRejectedValue(new Error('Network error')),
}),
}),
};
await expect(batchResetMeiliFlags(mockCollection)).rejects.toThrow(
"Failed to batch reset Meili flags for collection 'test_meili_batch' after processing 0 documents: Network error",
);
});
it('should throw error with context when updateMany operation fails', async () => {
const mockCollection = {
collectionName: 'test_meili_batch',
find: jest.fn().mockReturnValue({
limit: jest.fn().mockReturnValue({
toArray: jest
.fn()
.mockResolvedValue([
{ _id: new mongoose.Types.ObjectId() },
{ _id: new mongoose.Types.ObjectId() },
]),
}),
}),
updateMany: jest.fn().mockRejectedValue(new Error('Connection lost')),
};
await expect(batchResetMeiliFlags(mockCollection)).rejects.toThrow(
"Failed to batch reset Meili flags for collection 'test_meili_batch' after processing 0 documents: Connection lost",
);
});
it('should include documents processed count in error when failure occurs mid-batch', async () => {
// Set batch size to 2 to force multiple batches
process.env.MEILI_SYNC_BATCH_SIZE = '2';
process.env.MEILI_SYNC_DELAY_MS = '0'; // No delay for faster test
let findCallCount = 0;
let updateCallCount = 0;
const mockCollection = {
collectionName: 'test_meili_batch',
find: jest.fn().mockReturnValue({
limit: jest.fn().mockReturnValue({
toArray: jest.fn().mockImplementation(() => {
findCallCount++;
// Return 2 documents for first two calls (to keep loop going)
// Return 2 documents for third call (to trigger third update which will fail)
if (findCallCount <= 3) {
return Promise.resolve([
{ _id: new mongoose.Types.ObjectId() },
{ _id: new mongoose.Types.ObjectId() },
]);
}
// Should not reach here due to error
return Promise.resolve([]);
}),
}),
}),
updateMany: jest.fn().mockImplementation(() => {
updateCallCount++;
if (updateCallCount === 1) {
return Promise.resolve({ modifiedCount: 2 });
} else if (updateCallCount === 2) {
return Promise.resolve({ modifiedCount: 2 });
} else {
return Promise.reject(new Error('Database timeout'));
}
}),
};
await expect(batchResetMeiliFlags(mockCollection)).rejects.toThrow(
"Failed to batch reset Meili flags for collection 'test_meili_batch' after processing 4 documents: Database timeout",
);
});
it('should use collection.collectionName in error messages', async () => {
const mockCollection = {
collectionName: 'messages',
find: jest.fn().mockReturnValue({
limit: jest.fn().mockReturnValue({
toArray: jest.fn().mockRejectedValue(new Error('Permission denied')),
}),
}),
};
await expect(batchResetMeiliFlags(mockCollection)).rejects.toThrow(
"Failed to batch reset Meili flags for collection 'messages' after processing 0 documents: Permission denied",
);
});
});
describe('environment variable validation', () => {
let warnSpy;
beforeEach(() => {
// Mock logger.warn to track warning calls
const { logger } = require('@librechat/data-schemas');
warnSpy = jest.spyOn(logger, 'warn').mockImplementation(() => {});
});
afterEach(() => {
if (warnSpy) {
warnSpy.mockRestore();
}
});
it('should log warning and use default when MEILI_SYNC_BATCH_SIZE is not a number', async () => {
process.env.MEILI_SYNC_BATCH_SIZE = 'abc';
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(1);
expect(warnSpy).toHaveBeenCalledWith(
expect.stringContaining('Invalid value for MEILI_SYNC_BATCH_SIZE="abc"'),
);
expect(warnSpy).toHaveBeenCalledWith(expect.stringContaining('Using default: 1000'));
});
it('should log warning and use default when MEILI_SYNC_DELAY_MS is not a number', async () => {
process.env.MEILI_SYNC_DELAY_MS = 'xyz';
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(1);
expect(warnSpy).toHaveBeenCalledWith(
expect.stringContaining('Invalid value for MEILI_SYNC_DELAY_MS="xyz"'),
);
expect(warnSpy).toHaveBeenCalledWith(expect.stringContaining('Using default: 100'));
});
it('should log warning and use default when MEILI_SYNC_BATCH_SIZE is negative', async () => {
process.env.MEILI_SYNC_BATCH_SIZE = '-50';
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(1);
expect(warnSpy).toHaveBeenCalledWith(
expect.stringContaining('Invalid value for MEILI_SYNC_BATCH_SIZE="-50"'),
);
});
it('should log warning and use default when MEILI_SYNC_DELAY_MS is negative', async () => {
process.env.MEILI_SYNC_DELAY_MS = '-100';
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(1);
expect(warnSpy).toHaveBeenCalledWith(
expect.stringContaining('Invalid value for MEILI_SYNC_DELAY_MS="-100"'),
);
});
it('should accept valid positive integer values without warnings', async () => {
process.env.MEILI_SYNC_BATCH_SIZE = '500';
process.env.MEILI_SYNC_DELAY_MS = '50';
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(1);
expect(warnSpy).not.toHaveBeenCalled();
});
it('should log warning and use default when MEILI_SYNC_BATCH_SIZE is zero', async () => {
process.env.MEILI_SYNC_BATCH_SIZE = '0';
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(1);
expect(warnSpy).toHaveBeenCalledWith(
expect.stringContaining('MEILI_SYNC_BATCH_SIZE cannot be 0. Using default: 1000'),
);
});
it('should accept zero as a valid value for MEILI_SYNC_DELAY_MS without warnings', async () => {
process.env.MEILI_SYNC_DELAY_MS = '0';
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(1);
expect(warnSpy).not.toHaveBeenCalled();
});
it('should not log warnings when environment variables are not set', async () => {
delete process.env.MEILI_SYNC_BATCH_SIZE;
delete process.env.MEILI_SYNC_DELAY_MS;
await testCollection.insertMany([
{ _id: new mongoose.Types.ObjectId(), expiredAt: null, _meiliIndex: true },
]);
const result = await batchResetMeiliFlags(testCollection);
expect(result).toBe(1);
expect(warnSpy).not.toHaveBeenCalled();
});
});
});
@@ -0,0 +1,15 @@
{
"keep": {
"days": true,
"amount": 14
},
"auditLog": "/Users/z/work/hanzo/chat/api/logs/.124163c776d287793643ac0e08ac1d35d67ad894-audit.json",
"files": [
{
"date": 1771557162318,
"name": "/Users/z/work/hanzo/chat/api/logs/meiliSync-2026-02-19.log",
"hash": "a54018af15e4552979b0e91a2379ef40b95019fe56fe70074bb13f91952d908f"
}
],
"hashType": "sha256"
}
@@ -0,0 +1,15 @@
{
"keep": {
"days": true,
"amount": 14
},
"auditLog": "/Users/z/work/hanzo/chat/api/logs/.35252977fe148e605b7b92f19cd71e590a6f0a53-audit.json",
"files": [
{
"date": 1771557162297,
"name": "/Users/z/work/hanzo/chat/api/logs/error-2026-02-19.log",
"hash": "63688154da6c0a28cba1e30ddeef3eb867682460096d9b007dcfd2ddc25202b0"
}
],
"hashType": "sha256"
}

Some files were not shown because too many files have changed in this diff Show More