This repo holds the network-layer Rust crates previously living under github.com/hanzoai/node:hanzo-libs/. Breaking them out gives each crate its own version + publish cadence and lets downstream consumers depend on them without pulling the whole node tree. Highlights: - hanzo-hmm — Hidden Markov Model primitives + Hamiltonian MarketMaker for pricing heterogeneous compute. 17 lib tests pass. Earlier was named hanzo-llm (misleading) and the standalone hmm stub; both merged here with HLLM* identifier prefixes scrubbed. - hanzo-vm — Hanzo EVM with precompiles wired to canonical libluxprecompile.dylib (PQ verify, Quasar) and hanzo-engine (AI inference, embedding). 37 lib tests pass (DYLD_LIBRARY_PATH= /Users/z/work/lux/precompile/dist). - hanzo-machine — VM lifecycle (Apple Virtualization.framework via vfkit on macOS; KVM on Linux), thin Rust wrapper over canonical C++ impl in github.com/luxfi/luxcpp/machine. - hanzo-pqc / hanzo-did / hanzo-identity / hanzo-libp2p* — crypto + identity + networking primitives. - hanzo-l2 / hanzo-consensus / hanzo-mining — chain layer. - hanzo-api / hanzo-http-api / hanzo-mcp / hanzo-tools / hanzo-tools-runner — service + tooling crates. Build: `cargo build --workspace` succeeds in ~25min cold (44 crates).
3.2 KiB
3.2 KiB
Hanzo DB - Multi-Backend Database Abstraction
A unified database abstraction layer for Hanzo Node that supports multiple backend databases, each optimized for different workloads.
Supported Backends
| Backend | Best For | Features |
|---|---|---|
| LanceDB | Vector Search, AI/ML | • Native vector operations • Multimodal storage • Columnar format • Fast similarity search |
| DuckDB | Analytics, OLAP | • In-process analytical database • SQL support • Columnar storage • Fast aggregations |
| PostgreSQL | Transactional, OLTP | • ACID compliance • Rich SQL features • Extensions (pgvector) • Battle-tested |
| Redis | Caching, Real-time | • In-memory storage • Pub/sub support • TTL/expiration • Extremely fast |
| SQLite | Embedded, Lightweight | • Zero-configuration • Single file • Serverless • Wide compatibility |
Usage
use hanzo_db::{connect, HanzoDbConfig, DatabaseBackend, WorkloadType};
// Automatically select backend based on workload
let config = HanzoDbConfig {
backend: DatabaseBackend::for_workload(WorkloadType::VectorSearch),
..Default::default()
};
// Or explicitly choose a backend
let config = HanzoDbConfig {
backend: DatabaseBackend::LanceDB,
path: Some("./data/vectors".into()),
..Default::default()
};
let db = connect(config).await?;
// Use unified interface regardless of backend
db.create_table("embeddings", schema).await?;
db.insert("embeddings", &records).await?;
let results = db.vector_search(query).await?;
Features
- Unified Interface: Same API across all backends
- Automatic Backend Selection: Choose optimal backend based on workload
- Migration Support: Move data between backends
- Connection Pooling: Efficient resource management
- Transaction Support: ACID guarantees where supported
- Vector Operations: Native support in LanceDB, extension support in PostgreSQL
Configuration
Environment Variables
# Select default backend
HANZO_DB_BACKEND=lancedb # lancedb, duckdb, postgresql, redis, sqlite
# Backend-specific configuration
HANZO_DB_PATH=./storage/hanzo-db
HANZO_DB_URL=postgresql://user:pass@localhost/hanzo
HANZO_REDIS_URL=redis://localhost:6379
Feature Flags
[dependencies]
hanzo_db = { version = "1.0", features = ["lancedb", "duckdb", "postgres"] }
Migration
Migrate from one backend to another:
# From SQLite to LanceDB
hanzo-migrate --from sqlite://old.db --to lancedb://./vectors
# From LanceDB to PostgreSQL
hanzo-migrate --from lancedb://./vectors --to postgresql://localhost/hanzo
Performance Characteristics
| Operation | LanceDB | DuckDB | PostgreSQL | Redis | SQLite |
|---|---|---|---|---|---|
| Vector Search | ⭐⭐⭐⭐⭐ | ⭐⭐ | ⭐⭐⭐ | ⭐ | ⭐ |
| Analytics | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐ | ⭐⭐ |
| Transactions | ⭐⭐⭐ | ⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐ | ⭐⭐⭐⭐ |
| Caching | ⭐⭐ | ⭐ | ⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ |
| Embedded | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐ | ⭐ | ⭐⭐⭐⭐⭐ |
License
Apache 2.0