hanzo-docs/docs
1
0
Fork 0
mirror of https://github.com/hanzo-docs/docs.git synced 2026-07-27 04:31:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

Name the OLAP store Hanzo Datastore

Browse Source 
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
This commit is contained in:
Hanzo Dev
2026-07-26 16:17:47 -07:00
parent 97059fe0d0 c62b8f61a9
commit 8f74e3d4f2
3 changed files with 41 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/docs/content/docs/services/insights/self-hosting.mdx
+10 -10
View File
@@ -16,7 +16,7 @@ Hanzo Insights is fully self-hostable. Deploy on your infrastructure for complet
| **Insights Plugin** | Event processing | 2 CPU, 4GB RAM |
| **Insights Worker** | Async jobs | 1 CPU, 2GB RAM |
| **PostgreSQL** | Metadata storage | 2 CPU, 4GB RAM, 50GB disk |
| **ClickHouse** | Event analytics | 4 CPU, 8GB RAM, 100GB+ disk |
| **Datastore** | Event analytics | 4 CPU, 8GB RAM, 100GB+ disk |
| **Redis** | Cache and queues | 1 CPU, 2GB RAM |
| **Kafka/Stream** | Event streaming | 2 CPU, 4GB RAM |
@@ -31,14 +31,14 @@ services:
- "8000:8000"
environment:
DATABASE_URL: postgres://insights:password@postgres:5432/insights
CLICKHOUSE_HOST: clickhouse
DATASTORE_HOST: datastore
REDIS_URL: redis://redis:6379
KAFKA_HOSTS: stream:9092
SECRET_KEY: your-secret-key
SITE_URL: https://insights.yourdomain.com
depends_on:
- postgres
- clickhouse
- datastore
- redis
- stream
@@ -55,7 +55,7 @@ services:
command: ./bin/plugin-server
environment:
DATABASE_URL: postgres://insights:password@postgres:5432/insights
CLICKHOUSE_HOST: clickhouse
DATASTORE_HOST: datastore
KAFKA_HOSTS: stream:9092
REDIS_URL: redis://redis:6379
@@ -64,7 +64,7 @@ services:
command: ./bin/worker
environment:
DATABASE_URL: postgres://insights:password@postgres:5432/insights
CLICKHOUSE_HOST: clickhouse
DATASTORE_HOST: datastore
REDIS_URL: redis://redis:6379
postgres:
@@ -76,10 +76,10 @@ services:
volumes:
- postgres_data:/var/lib/postgresql/data
clickhouse:
image: clickhouse/clickhouse-server:24.3
datastore:
image: ghcr.io/hanzoai/datastore:26.6.1.1
volumes:
- clickhouse_data:/var/lib/clickhouse
- datastore_data:/var/lib/clickhouse
redis:
image: redis:7-alpine
@@ -99,7 +99,7 @@ services:
volumes:
postgres_data:
clickhouse_data:
datastore_data:
redis_data:
```
@@ -145,7 +145,7 @@ spec:
| Variable | Description |
|----------|-------------|
| `DATABASE_URL` | PostgreSQL connection string |
| `CLICKHOUSE_HOST` | ClickHouse hostname |
| `DATASTORE_HOST` | Datastore hostname |
| `REDIS_URL` | Redis connection string |
| `KAFKA_HOSTS` | Kafka broker addresses (comma-separated) |
| `SECRET_KEY` | Django secret key for session encryption |
apps/docs/content/docs/skills/hanzo-analytics.mdx
+11 -11
View File
@@ -1,17 +1,17 @@
---
title: "Hanzo Analytics - Privacy-Focused Web Analytics"
description: "Hanzo Analytics is a privacy-focused web analytics platform -- an alternative to Google Analytics. Next.js 15 dashboard (port 3000) with Prisma ORM, PostgreSQL/ClickHouse storage, and a Go-based hi..."
description: "Hanzo Analytics is a privacy-focused web analytics platform -- an alternative to Google Analytics. Next.js 15 dashboard (port 3000) with Prisma ORM, PostgreSQL/Datastore storage, and a Go-based hi..."
---
## Overview
Hanzo Analytics is a **privacy-focused web analytics platform** -- an alternative to Google Analytics. Next.js 15 dashboard (port 3000) with Prisma ORM, PostgreSQL/ClickHouse storage, and a Go-based high-performance event collector. Includes anti-ad-blocker technology (encrypted tracker + WASM decryption). Live at `analytics.hanzo.ai`.
Hanzo Analytics is a **privacy-focused web analytics platform** -- an alternative to Google Analytics. Next.js 15 dashboard (port 3000) with Prisma ORM, PostgreSQL/Hanzo Datastore storage, and a Go-based high-performance event collector. Includes anti-ad-blocker technology (encrypted tracker + WASM decryption). Live at `analytics.hanzo.ai`.
### Why Hanzo Analytics?
- **Privacy-first**: No cookies required, GDPR/HIPAA compliant by design
- **Anti-ad-blocker**: Encrypted tracker script + WASM decryptor defeats blockers
- **Dual database**: PostgreSQL for metadata, optional ClickHouse for event scale
- **Dual database**: PostgreSQL for metadata, optional Datastore for event scale
- **Real-time**: Live visitor tracking, real-time dashboards
- **Multi-tenant**: Teams, users, roles, shared dashboards
- **Revenue tracking**: Built-in e-commerce revenue attribution
@@ -22,7 +22,7 @@ Hanzo Analytics is a **privacy-focused web analytics platform** -- an alternativ
- **Dashboard**: Next.js 15 + React 19 + Chart.js + react-intl (i18n)
- **ORM**: Prisma 6 with PostgreSQL adapter
- **State**: Zustand + TanStack React Query
- **Collector**: Go (Gin) + ClickHouse for high-throughput event ingestion
- **Collector**: Go (Gin) + Datastore for high-throughput event ingestion
- **Tracker**: Rollup-bundled JS, encrypted delivery via WASM
- **Validation**: Zod
- **Auth**: JWT (jsonwebtoken) + bcryptjs password hashing
@@ -48,8 +48,8 @@ Repo: `hanzoai/analytics` (Umami fork, v3.0.3).
1. **PostgreSQL** (minimum v12.14) for metadata and event storage
2. **Node.js 18.18+** for the dashboard application
3. **Go 1.26+** for the collector service (optional, for ClickHouse pipeline)
4. **ClickHouse** (optional) for high-volume event analytics
3. **Go 1.26+** for the collector service (optional, for the Datastore pipeline)
4. **Datastore** (optional) for high-volume event analytics
## Quick reference
@@ -60,9 +60,9 @@ Repo: `hanzoai/analytics` (Umami fork, v3.0.3).
| Dev port | `3001` (Next.js dev) / `3000` (production) |
| Image | `ghcr.io/hanzoai/analytics:latest` |
| Package | `@hanzo/analytics` (v3.0.3) |
| Database | PostgreSQL (required), ClickHouse (optional) |
| Database | PostgreSQL (required), Datastore (optional) |
| ORM | Prisma 6 |
| Collector | Go + Gin + ClickHouse (`collector/`) |
| Collector | Go + Gin + Datastore (`collector/`) |
| Health check | `/api/heartbeat` |
| Repo | `github.com/hanzoai/analytics` |
| License | MIT |
@@ -149,7 +149,7 @@ pnpm start # http://localhost:3000
┌──────────────────────┐
│ collector (Go) │
High-volume ─────▶│ Gin HTTP server │──── ClickHouse
High-volume ─────▶│ Gin HTTP server │──── Datastore
events │ event.go │ (event_data,
│ forward/ │ session_data)
│ writer/ │
@@ -199,7 +199,7 @@ pnpm start # http://localhost:3000
```bash
DATABASE_URL=postgresql://user:pass@localhost:5432/analytics # Required
APP_SECRET=random-secret-string # Required
CLICKHOUSE_URL=http://localhost:8123 # Optional
DATASTORE_URL=http://localhost:8123 # Optional
```
## Troubleshooting
@@ -209,7 +209,7 @@ CLICKHOUSE_URL=http://localhost:8123 # Optional
| Build fails on first run | Missing DATABASE_URL | Set `DATABASE_URL` in `.env` before `pnpm build` |
| Default login not working | Changed by script | Default is `admin` / `analytics`, use `npm run change-password` |
| Ad blocker blocks events | Tracker detected | Use encrypted tracker pipeline (enabled by default) |
| High event volume slow | PostgreSQL bottleneck | Deploy Go collector with ClickHouse backend |
| High event volume slow | PostgreSQL bottleneck | Deploy Go collector with Datastore backend |
| Geo data missing | No GeoIP database | Run `pnpm build-geo` to download MaxMind DB |
| Docker health check fails | Port not ready | Increase `retries` or wait for DB initialization |
apps/docs/content/docs/skills/hanzo-o11y.mdx
+20 -20
View File
@@ -1,17 +1,17 @@
---
title: "Hanzo O11y - Full-Stack Observability Platform"
description: "Hanzo O11y is a full-stack observability platform for logs, metrics, and traces. Go 1.25 backend (Gin/gorilla-mux) with a React 18 + Vite frontend. Uses ClickHouse as the telemetry datastore and Op..."
description: "Hanzo O11y is a full-stack observability platform for logs, metrics, and traces. Go 1.25 backend (Gin/gorilla-mux) with a React 18 + Vite frontend. Uses Hanzo Datastore as the telemetry store and Op..."
---
## Overview
Hanzo O11y is a **full-stack observability platform** for logs, metrics, and traces. Go 1.25 backend (Gin/gorilla-mux) with a React 18 + Vite frontend. Uses **ClickHouse** as the telemetry datastore and **OpenTelemetry** for ingestion. Fork of SigNoz. Live at `o11y.hanzo.ai`.
Hanzo O11y is a **full-stack observability platform** for logs, metrics, and traces. Go 1.25 backend (Gin/gorilla-mux) with a React 18 + Vite frontend. Uses **Hanzo Datastore** (Hanzo's Apache-2.0 ClickHouse fork) as the telemetry store and **OpenTelemetry** for ingestion. Fork of SigNoz. Live at `o11y.hanzo.ai`.
### Why Hanzo O11y?
- **Single pane of glass**: Logs, metrics, traces, and LLM observability in one UI
- **OpenTelemetry native**: No vendor lock-in, standard instrumentation
- **ClickHouse backend**: Columnar storage optimized for observability queries (50% less resources than Elastic)
- **Datastore backend**: Columnar storage optimized for observability queries (50% less resources than Elastic)
- **Self-hostable**: Docker Compose or Helm, community or enterprise edition
- **LLM Observability**: Track LLM calls, token usage, costs, prompt/response analysis
- **Built-in alerting**: Alert on any telemetry signal (logs, metrics, traces)
@@ -20,7 +20,7 @@ Hanzo O11y is a **full-stack observability platform** for logs, metrics, and tra
- **Backend**: Go 1.25, Gin, gorilla/mux, go-redis, cobra CLI
- **Frontend**: React 18, TypeScript, Vite (rolldown-vite), Ant Design, `@hanzo/ui`, `@hanzo/insights`
- **Telemetry Store**: ClickHouse (`ghcr.io/hanzoai/datastore`)
- **Telemetry Store**: Datastore (`ghcr.io/hanzoai/datastore`)
- **Ingestion**: OpenTelemetry Collector fork (`ghcr.io/hanzoai/otel-collector`)
- **Metadata Store**: SQLite (community) or PostgreSQL (enterprise)
- **Auth**: JWT tokens (built-in), OIDC + SAML (enterprise), OpenFGA for authz
@@ -41,8 +41,8 @@ Fork of `signoz/signoz`. Repo: `hanzoai/o11y`.
## Hard requirements
1. **ClickHouse** (via `ghcr.io/hanzoai/datastore`) for telemetry storage
2. **ZooKeeper** for ClickHouse coordination (single-node or cluster)
1. **Datastore** (via `ghcr.io/hanzoai/datastore`) for telemetry storage
2. **ZooKeeper** for Datastore coordination (single-node or cluster)
3. **OpenTelemetry Collector** (`ghcr.io/hanzoai/otel-collector`) for data ingestion
4. **Docker** or **Kubernetes** for deployment
@@ -69,8 +69,8 @@ Fork of `signoz/signoz`. Repo: `hanzoai/o11y`.
```
┌──────────────┐ ┌───────────────────┐ ┌──────────────┐
│ Your Apps │────>│ OTel Collector │────>│ ClickHouse │
│ (OTel SDK) │ │ (gRPC/HTTP) │ │ (Datastore) │
│ Your Apps │────>│ OTel Collector │────>│ Datastore │
│ (OTel SDK) │ │ (gRPC/HTTP) │ │ (:9000) │
└──────────────┘ │ :4317 / :4318 │ └──────┬───────┘
└───────────────────┘ │
│
@@ -83,8 +83,8 @@ Fork of `signoz/signoz`. Repo: `hanzoai/o11y`.
### Data Flow
1. Applications instrumented with OpenTelemetry SDKs send traces, metrics, and logs to the OTel Collector
2. OTel Collector processes and exports to ClickHouse databases: `observe_traces`, `observe_metrics`, `observe_logs`, `observe_meter`, `observe_metadata`
3. The Go backend queries ClickHouse for telemetry data and serves the React frontend
2. OTel Collector processes and exports to Datastore databases: `observe_traces`, `observe_metrics`, `observe_logs`, `observe_meter`, `observe_metadata`
3. The Go backend queries Datastore for telemetry data and serves the React frontend
4. SQLite (or PostgreSQL in enterprise) stores metadata: users, orgs, dashboards, alerts, saved views
### Directory Structure
@@ -100,14 +100,14 @@ pkg/
authz/ # Authorization
cache/ # In-memory or Redis cache
prometheus/ # PromQL query engine
querier/ # Query layer over ClickHouse
querier/ # Query layer over Datastore
query-service/ # Legacy query service
querybuilder/ # Query builder for UI
sqlstore/ # SQLite/PostgreSQL metadata store
telemetrylogs/ # Log query handlers
telemetrymetrics/# Metric query handlers
telemetrytraces/ # Trace query handlers
telemetrystore/ # ClickHouse connection management
telemetrystore/ # Datastore connection management
web/ # Embedded frontend serving
ee/
anomaly/ # Anomaly detection (enterprise)
@@ -121,7 +121,7 @@ frontend/
deploy/
docker/ # Docker Compose deployment
docker-swarm/ # Docker Swarm deployment
common/ # Shared ClickHouse configs, dashboards
common/ # Shared Datastore configs, dashboards
conf/
example.yaml # Full configuration reference
tests/
@@ -135,7 +135,7 @@ git clone https://github.com/hanzoai/o11y.git && cd o11y/deploy/docker
docker compose -f docker-compose.yaml up -d
```
This starts: ZooKeeper, ClickHouse (datastore), O11y backend, OTel Collector, and a schema migrator.
This starts: ZooKeeper, Datastore, O11y backend, OTel Collector, and a schema migrator.
Access the UI at `http://localhost:8080`.
@@ -146,7 +146,7 @@ Send telemetry to `localhost:4317` (gRPC) or `localhost:4318` (HTTP).
### Backend (Go)
```bash
# Start dependencies (ClickHouse + OTel Collector)
# Start dependencies (Datastore + OTel Collector)
make devenv-up
# Run enterprise server locally
@@ -189,7 +189,7 @@ make docker-build-enterprise
## Key Environment Variables
```bash
# Telemetry store (ClickHouse)
# Telemetry store (Datastore)
HANZO_TELEMETRYSTORE_DATASTORE_DSN=tcp://127.0.0.1:9000
HANZO_TELEMETRYSTORE_DATASTORE_CLUSTER=cluster
@@ -209,8 +209,8 @@ HANZO_WEB_ENABLED=true
HANZO_INSTRUMENTATION_LOGS_LEVEL=info
# OTel Collector (separate process)
HANZO_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000
HANZO_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER=cluster
HANZO_OTEL_COLLECTOR_DATASTORE_DSN=tcp://datastore:9000
HANZO_OTEL_COLLECTOR_DATASTORE_CLUSTER=cluster
```
## OTel Collector Pipeline
@@ -224,7 +224,7 @@ The bundled OTel Collector config (`deploy/docker/otel-collector-config.yaml`) d
| metrics/prometheus | prometheus scraper | observeclickhousemetrics, metadataexporter, observemeter |
| logs | OTLP | clickhouselogsexporter, metadataexporter, observemeter |
ClickHouse databases: `observe_traces`, `observe_metrics`, `observe_logs`, `observe_meter`, `observe_metadata`.
Datastore databases: `observe_traces`, `observe_metrics`, `observe_logs`, `observe_meter`, `observe_metadata`.
## Instrumenting Your Application
@@ -255,7 +255,7 @@ All languages supported by OpenTelemetry work: Java, Python, Node.js, Go, .NET,
| Issue | Cause | Solution |
|-------|-------|----------|
| No data in UI | OTel Collector not receiving data | Check `localhost:4317` is reachable, verify SDK config |
| ClickHouse OOM | High cardinality queries | Tune `max_execution_time` in conf, add resource limits |
| Datastore OOM | High cardinality queries | Tune `max_execution_time` in conf, add resource limits |
| Slow log queries | Missing indexes | Use the query builder, avoid unbounded time ranges |
| OTel Collector crash loops | Schema migration incomplete | Run migrator: `/otel-collector migrate sync up` |
| Frontend 502 | Backend not ready | Check `localhost:8080/api/v1/health` |