main
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Hanzo Dev
|
6334500191 |
0.3.0: bazil.org/fuse mount + SQLite-on-FUSE e2e test (linux)
The mount layer is no longer a stub. With `-tags fuse` on Linux,
SQLite (and any POSIX consumer) opens files directly on the
mountpoint — no intermediate copy. Default builds stay
dependency-light: bazil.org/fuse + bazil.org/fuse/fs are gated
behind the build tag.
What's new
----------
- pkg/mount/fuse_unix.go (`-tags fuse && linux`): full FS/Node/Handle
bridges. Maps every POSIX op the kernel sends:
Read → File.ReadAt
Write → File.WriteAt
Fsync → File.Sync (durable through age + backend)
Flush → File.Sync (close-time flush)
Release → File.Close
Setattr.Size → File.Truncate
Lookup → FS.Lookup
ReadDirAll → FS.ReadDir
Create → FS.Create + Open
Mkdir → FS.Mkdir
Remove → FS.Remove
Open(O_TRUNC) → File.Truncate(0)
- pkg/mount/fuse_stub.go: same Mount/MountWithVFSAdapter signatures
for the default no-fuse build; returns a build-tag error.
- pkg/mount/fuse_sqlite_test.go: gated by VFS_FUSE_E2E=1 +
`-tags fuse`. Mounts in tempdir, creates `test.db`, 100 INSERTs,
unmounts, remounts, runs PRAGMA integrity_check. Verifies the DB
survives the full encrypt → backend → decrypt round trip with
zero copy.
- cmd/vfs: mount subcommand wires through MountWithVFSAdapter.
CI
--
.github/workflows/build.yml runs three test passes on Ubuntu:
1. default `go test -race ./...`
2. `go vet -tags fuse ./...`
3. apt-get fuse + `VFS_FUSE_E2E=1 go test -race -tags fuse
-run TestFUSESQLite ./pkg/mount/ -v`
Caveats
-------
- macOS: bazil.org/fuse dropped macOS support upstream (commit
65cc252). 0.3.1 will add a parallel mount_darwin.go using
github.com/jacobsa/fuse so devs on macOS get the same e2e flow.
Until then, macOS devs use the in-process API + the SQLite
byte-roundtrip test in pkg/vfs/sqlite_test.go (already shipped).
- Local NVMe disk cache (write-back tier) lands in 0.4.0; today the
cache is in-memory LRU.
|
||
|
Hanzo Dev
|
837e5bac96 |
initial: vfs scaffold (block layer + file:// + s3 backends + age PQ crypto)
Phase 0.1.0 — production-shaped scaffold for the S3-backed virtual block filesystem. Stateful services (IAM, BD, ATS, TA, KMS, onyxd, lqd snapshots) get unlimited disk via S3 with PQ-encrypted blocks; the hot tier is a configurable LRU cache. Companion to hanzo/replicate (WAL streaming) — `replicate` is file-level granularity, `vfs` is block-level (4 KiB pages, content-addressable via blake3-256). Included -------- - pkg/backend: pluggable Backend interface + Open(URL) dispatch. - pkg/backend/file: file:// backend (atomic put via temp+rename, path traversal blocked, recursive list with prefix filter). - pkg/backend/s3: AWS SDK v2 S3 backend with `?endpoint=` override for S3-compat (R2, MinIO, Hanzo Storage). NotFound → ErrNotFound. - pkg/vfs/block: blake3-256 hashing → BlockID, 2-hex shard prefix for the backend key, BlockSize=4096, zero-pad to BlockSize before encryption (block boundaries don't leak file lengths). - pkg/vfs/crypto: luxfi/age v1.5.0 wrapper. Recipients (write-side) + Identities (read-side); hybrid X25519 + ML-KEM-768 supported via the same recipient list. - pkg/vfs/cache: in-memory LRU bound by total bytes; touch-promotes on read; eviction on Put when over cap. NVMe write-back tier lands in 0.2.0. - pkg/vfs: top-level handle. PutBlock/GetBlock with cache hit-path + content-hash integrity check on read. Stats() surfaces cache + backend metadata. - pkg/mount: fuse_stub.go (default build) + fuse_unix.go (-tags fuse). Real FUSE wiring lands in 0.2.0; current stub returns a build-tag error pointing at `make build-fuse`. - cmd/vfs: cobra CLI (put/get/stats/mount). Backend registered via blank-import; --age-recipient repeatable; --age-key reads identity PEM from file or VFS_AGE_KEY env. Tests ----- - block: hash determinism, path shape, verify match/mismatch, pad. - cache: LRU eviction, touch-promotion, basic ops. - file backend: put/get/delete/stat/list, path-traversal block. - vfs roundtrip: encrypt → backend → decrypt → byte-equal plaintext, zero-pad correctness; multiple writes of identical plaintext both round-trip; stats reflect insertions. CI / build ---------- - .github/workflows/build.yml: vet + race-tests on Go 1.24, amd64 binary upload, GHCR image build + push on main / v* tags. - Dockerfile: distroless static, CGO_ENABLED=0, ARG VERSION ldflag. - Makefile: build / build-fuse / test / fmt / vet / check / image / install. VERSION-pinned. Roadmap ------- - 0.2.0: bazil.org/fuse mount, NVMe disk cache layer - 0.3.0: gcs + azureblob backends - 0.4.0: K8s sidecar mode + Helm chart - 0.5.0: WASI guest mode - 1.0.0: chunked uploads, multipart resume, GC for orphan blocks, metrics + SLO targets |