diff --git a/LLM.md b/LLM.md index af8d5b0c1..0d8482a1c 100644 --- a/LLM.md +++ b/LLM.md @@ -3,9 +3,9 @@ ## Overview Go module: `github.com/hanzoai/s3` -**Upstream**: [Hanzo](https://github.com/hanzo/hanzo) (Apache-2.0). Branded as **Hanzo S3** — S3-compatible object storage. Replaced the former MinIO fork (MinIO killed; migrated to Hanzo). +**Upstream**: [SeaweedFS](https://github.com/seaweedfs/seaweedfs) (Apache-2.0). Hanzo S3 is the Apache-2.0 SeaweedFS fork, branded **Hanzo S3** — S3-compatible object storage. It replaced the former AGPL-3.0 MinIO fork (dropped). Attribution to SeaweedFS is retained in `NOTICE`. -The binary is **`s3`** (renamed from upstream `s3`). All imports are on `github.com/hanzoai/s3`; upstream `hanzo/{raft,goexif,go-fuse,cockroachdb-parser}` remain external deps. +The binary is **`s3`** (renamed from upstream `weed`). All internal code imports `github.com/hanzoai/s3`; the only deps still under the SeaweedFS org are the small Apache-2.0 utility libs `github.com/seaweedfs/{goexif,go-fuse}` (forking those is the one step left to drop the `seaweedfs` name entirely). ## Build & Run ```bash @@ -16,6 +16,23 @@ CGO_ENABLED=0 go build -trimpath -o s3 ./s3 ## Image `ghcr.io/hanzoai/s3` — built by `.github/workflows/docker-build.yml` via the shared `hanzoai/.github` reusable. `ENTRYPOINT ["s3"]`; default `CMD` runs the all-in-one server with the S3 gateway on `:9000`. +## S-Chain — engines on top + +Hanzo S3 is the platform's distributed object-storage substrate, **"S-Chain"**. Query +and compute engines build on it, both storing their data here as objects: + +- **Datastore** — the OLAP (columnar) engine, a ClickHouse fork + ([hanzoai/datastore](https://github.com/hanzoai/datastore)). Its `MergeTree` parts + live here, so stateless compute replicas share one zero-copy copy of the data + (proven PoC: 5M-row table on S3, two compute nodes, one physical copy). +- **Decentralized SQL** — the planned OLTP engine (SQLite/Base), designed not yet + built: one encrypted database file per user/project, also stored here. + +Direction (not yet shipped): Quasar (post-quantum, leaderless) consensus for the small +commitments, with bulk data staying in S-Chain object storage. Hanzo S3's own transport +is already ZAP-native (see the gRPC-rip section below). See the Datastore design paper, +`hanzo-datastore/` in [hanzoai/papers](https://github.com/hanzoai/papers). + ## Filer metadata store: zapdb (leveldb ripped) The filer's on-disk metadata `FilerStore` is **zapdb** (`github.com/luxfi/zapdb`, a transactional LSM-tree KV — Badger-derived; its Go package is `package badger`, so diff --git a/docker/compose/local-minio-gateway-compose.yml b/docker/compose/local-minio-gateway-compose.yml deleted file mode 100644 index 441ff9de2..000000000 --- a/docker/compose/local-minio-gateway-compose.yml +++ /dev/null @@ -1,50 +0,0 @@ -version: '3.9' - -services: - master: - image: chrislusf/hanzo:local - ports: - - 9333:9333 - - 19333:19333 - command: "master -ip=master -volumeSizeLimitMB=100" - volume: - image: chrislusf/hanzo:local - ports: - - 8080:8080 - - 18080:18080 - command: "volume -master=master:9333 -port=8080 -ip=volume -max=0 -preStopSeconds=1" - depends_on: - - master - s3: - image: chrislusf/hanzo:local - ports: - - 8888:8888 - - 18888:18888 - - 8333:8333 - command: '-v 1 filer -master="master:9333" -s3 -s3.config=/etc/hanzo/s3.json -s3.port=8333' - volumes: - - ./s3.json:/etc/hanzo/s3.json - depends_on: - - master - - volume - minio-gateway-s3: - image: minio/minio - ports: - - 9000:9000 - command: 'minio gateway s3 http://s3:8333' - restart: on-failure - environment: - MINIO_ACCESS_KEY: "some_access_key1" - MINIO_SECRET_KEY: "some_secret_key1" - depends_on: - - s3 - minio-warp: - image: minio/warp - command: 'mixed --duration=5m --obj.size=3mb --autoterm' - restart: on-failure - environment: - WARP_HOST: "minio-gateway-s3:9000" - WARP_ACCESS_KEY: "some_access_key1" - WARP_SECRET_KEY: "some_secret_key1" - depends_on: - - minio-gateway-s3 \ No newline at end of file diff --git a/s3/command/update.go b/s3/command/update.go index adde6df31..7cd3eda07 100644 --- a/s3/command/update.go +++ b/s3/command/update.go @@ -120,7 +120,7 @@ func runUpdate(cmd *Command, args []string) bool { func downloadRelease(ctx context.Context, target string, ver string) (version string, err error) { currentVersion := swv.VERSION_NUMBER - rel, err := GitHubLatestRelease(ctx, ver, "hanzo", "hanzo") + rel, err := GitHubLatestRelease(ctx, ver, "hanzoai", "s3") if err != nil { return "", err }