The v0.3.x internal-engine sqlite drops the modernc.org dependency; with
v0.6.5's replica routing this leaves zero modernc modules in the graph.
Comments updated to match.
* fix(sqlite): use hanzoai/sqlite (the one driver), drop direct modernc import
* fix(sqlite): use hanzoai/sqlite (the one driver), drop direct modernc import
---------
Co-authored-by: Hanzo AI <ai@hanzo.ai>
* replica: extract single-writer election to hanzoai/ha
Election (Owner/IsOwner/Replicas/Member) was storage-agnostic pure Go with
zero vfs-internal callers — it lived here only historically. It is now its
own coordination primitive at github.com/hanzoai/ha so any singleton (cron,
queue drain, billing sweep) gets exactly-once WITHOUT importing a SQLite
library. vfs/replica keeps what it is: SQLite replication (Replicator,
SQLiteDB, BackendStore). One concern per package.
Consumers (visor, cloud/internal/org) migrate to hanzoai/ha in lockstep;
they pin vfs v0.6.2 so this main-only removal does not disturb them until
they bump. Removes owner.go + its two election tests; doc points at ha.
* ci: enable cgo for the -race steps (pre-existing red)
go test -race requires cgo, but the self-hosted hanzo-build-linux-amd64
runner has no C compiler and defaulted CGO_ENABLED=0 (vfs is CGO-free in
production), so both race steps failed — first with 'go: -race requires
cgo', then 'C compiler gcc not found'. Every main run has been red on this
for many commits. Fix: install gcc + set CGO_ENABLED=1 on the two -race
steps only; the production build stays CGO_ENABLED=0. Now the race detector
actually runs (real coverage for the concurrent replicator/ship loops).
Unrelated to the ha extraction but folded in so this PR lands green rather
than override a known-bad check.
* replica: round-fenced writes (FencedStore) — close split-brain double-write (v0.6.3)
Election (hanzoai/ha) is coordination-free, so two replicas with divergent
membership views can each elect themselves the single writer for one org, and
the object store's unconditional Put ('overwriting is allowed') lets both
clobber the mirror. FencedStore closes this at the storage layer over a
ConditionalStore (S3 If-Match / GCS generation-match) CAS:
- one object per key carries a monotone round header
- admit a Put iff round >= recorded; reject (ErrStaleRound) when below, so a
deposed/partitioned writer is fenced once a successor advanced the round
- '>=' (not one-shot '>') because the owner re-ships at its STABLE lease round
every tick; only a LOWER round is stale
- CarryForward is the safe takeover seal: re-reads and carries the
predecessor's last landed write forward atomically via CAS, so a successor
can never clobber an acknowledged write
- round header + payload in ONE object / ONE conditional write: no window
between 'round admitted' and 'data written'
Round is a plain uint64 (the storage boundary). The coordination value
ha.Lease{Owner,Round} and the linearizable round source stay in ha and its
consumers. Pure over the ConditionalStore seam; concrete minio store lives with
the S3-client owner. Tests: deposed-fenced, same-round re-ship, handoff
lock-out, carry-forward-no-loss, and concurrent race proofs (20x race-stable).
---------
Co-authored-by: hanzo <z@hanzo.ai>
* replica: extract single-writer election to hanzoai/ha
Election (Owner/IsOwner/Replicas/Member) was storage-agnostic pure Go with
zero vfs-internal callers — it lived here only historically. It is now its
own coordination primitive at github.com/hanzoai/ha so any singleton (cron,
queue drain, billing sweep) gets exactly-once WITHOUT importing a SQLite
library. vfs/replica keeps what it is: SQLite replication (Replicator,
SQLiteDB, BackendStore). One concern per package.
Consumers (visor, cloud/internal/org) migrate to hanzoai/ha in lockstep;
they pin vfs v0.6.2 so this main-only removal does not disturb them until
they bump. Removes owner.go + its two election tests; doc points at ha.
* ci: enable cgo for the -race steps (pre-existing red)
go test -race requires cgo, but the self-hosted hanzo-build-linux-amd64
runner has no C compiler and defaulted CGO_ENABLED=0 (vfs is CGO-free in
production), so both race steps failed — first with 'go: -race requires
cgo', then 'C compiler gcc not found'. Every main run has been red on this
for many commits. Fix: install gcc + set CGO_ENABLED=1 on the two -race
steps only; the production build stays CGO_ENABLED=0. Now the race detector
actually runs (real coverage for the concurrent replicator/ship loops).
Unrelated to the ha extraction but folded in so this PR lands green rather
than override a known-bad check.
---------
Co-authored-by: hanzo <z@hanzo.ai>
xorm/GORM services can't share a handle with SQLiteDB (Restore swaps the file). Expose
SnapshotFile(path) (transient read handle, VACUUM INTO) + RestoreFile(path,data) (atomic
swap) so a service using its OWN engine adopts HA: RestoreFile before opening, SnapshotFile
in the push loop. SQLiteDB now wraps these (one and one way). Tests still green.
Completes the shared HA-SQLite lib: NewBackendStore(be) adapts any vfs backend.Backend
(file locally, s3/SeaweedFS in prod) to the Replicator Store, with a cheap content-version
sidecar (<key>.ver) so a reader answers Version() without fetching the whole DB. So a
service wires HA in one line: NewReplicator(key, NewBackendStore(be), OpenSQLite(path)).
Test: full owner->vfs-backend->reader path over the real file backend (SQLite rows
survive push/pull, incl. updates + the version-skip).
The ONE importable durability library — promoted from cloud/internal/org (proven,
but internal + unimportable + only a memDB test) into hanzoai/vfs so EVERY service
adopts the same code, never re-invents HA:
- Replicator: per-org SQLite <-> vfs object store, whole-object WAL-checkpointed
snapshots, per-org sealed. Push (owner) / Pull (reader) / PushLoop.
- owner.go: deterministic single-writer election via Rendezvous/HRW over IAM
membership — no coordinator, no lock service, no Postgres, no Redis.
- sqlite.go: the REAL SQLite DB impl the cloud replicator never had — Snapshot via
VACUUM INTO (consistent, WAL-checkpointed), Restore via atomic file swap+reopen.
CGO-free (modernc). This is what makes HA SQLite actually work on disk.
Tests: real SQLite snapshot/restore round-trip (rows preserved, usable after
restore), full owner->store->reader Push/Pull cycle + redundant-pull skip, HRW
determinism/exclusivity/spread. Adoption contract (hydrate-on-open, single-writer
gate, post-commit ship) documented for visor/iam/commerce/base rollout.
Hanzo AI
