Removed /v4 suffix from module path and all internal imports.
Go module is now github.com/luxfi/zapdb (no major version suffix).
Import as: import "github.com/luxfi/zapdb"
No code changes — only module path and import rewrite.
- Rename module from github.com/dgraph-io/badger/v4 to github.com/luxfi/zapdb/v4
- Update all internal imports to use new module path
- Preserve all ZAP enhancements from badger fork
## Problem
While debugging a program which is using badger I noticed that stack
traces returned from my code (which uses errors with stack traces) are
useless. I found the reason: badger uses sentinel errors (great!) but it
creates them with stack traces at the "global init time" (e.g., `var
ErrFoo = errors.New(...)`) and not at "return from function time".
Because of that I was seeing stack traces like:
```
github.com/dgraph-io/badger/v4.init
/go/pkg/mod/github.com/dgraph-io/badger/v4@v4.2.0/compaction.go:41
runtime.doInit1
/usr/local/go/src/runtime/proc.go:6740
runtime.doInit
/usr/local/go/src/runtime/proc.go:6707
runtime.main
/usr/local/go/src/runtime/proc.go:249
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1650
```
Instead of a stack trace which would show me where the call which is
returning sentinel error is made.
## Solution
Ideally, one would create all sentinel errors without stack traces
(standard errors package works great for that) and then when the error
happens in a function, you return `errors.WithStack(ErrFoobar)` instead
of just `ErrFoobar`. This means that sentinel error is then annotated
with a stack trace at the place where the error happens. But if you do
that, then you have to check for sentinel errors with `errors.Is`
instead of just comparing errors with equality (`==`). That might be a
breaking change for people who are not using `errors.Is` as they should.
Because badger's own codebase uses equality instead of `errors.Is` I
decided to not do this in this PR, but instead:
Just make sentinel errors without stack traces. This allows me to
annotate with a stack trace inside my program (otherwise my program does
not do so because if finds an existing stack trace and assumes it is a
deeper one) and I get at least that part of the stack trace. Equality
still works as usual.
I suggest this is merged this way and in v5 of badger `errors.WithStack`
and required `errors.Is` are introduced.
## Side note
Some time ago [I made this errors package for
Go](https://gitlab.com/tozd/go/errors) which fixes various issues in
`github.com/pkg/errors` and also has direct support for sentinel (in
that package called "base") errors. It also has some features badger has
in its `y` package for dealing with errors. You could consider adopting
that package to have a comprehensive errors handling in badger.
As explained in https://discuss.dgraph.io/t/go-modules-on-badger-and-dgraph/4662,
Go Modules force an import path rename that currently breaks support for most other
dependency management systems, including dep.
It seems an effort is being made to add support for this import path renaming by
patching the go tool itself, but until the community hasn't settled and Go Modules
is an easier option to adopt, we are simply not supporting them.
For those willing to use badger v2 from Go modules it is still very simple, you might
notice that once you set the version to be v2.0.0 a +incompatible will be added to it
indicating the dependency has not opted-in into Go Modules. That's normal and not to
be a concern.
With this commit, merge operator processes the items in the order of
which they were inserted. Earlier it would be processed by the merge
function in reverse order.
For instance, if `A` , `B`, `C` were inserted, and merge function was
simple `append` operation, they would be passed to the merge function
as
```
mergeFunc(C, B) => result CB
mergeFunc(CB, A) => result CBA
```
With this change, they're passed to the merge function as
```
mergeFunc(B, C) => result BC
mergeFunc(A, BC) => result ABC
```
* Use batchSetAsync instead of transaction in merge compactions
With this commit, we use batchSetAsync API instead of the transaction
API to store the result of merge compactions. With the transaction API, we
had multiple transaction conflicts and batchSetAsync API would allow us
to perform merge compactions without transaction conflicts.
* Add functions to create, modify Entry
Currently no functions are exposed to create and modify Entry, to
be set in a transaction. Functions Set, SetWithMeta, SetWithDiscard,
SetWithTTL, all are part of Txn struct. These functions internally
create an entry and then call txn.SetEntry. All of Set functions,
set a specific property of Entry apart from key and value. This
restricts us from setting multiple property for same entry such as
meta and TTL.
This PR adds functions to set different properties on an existing
Entry. After setting desired properties, user can call txn.SetEntry
to add entry in the transaction. It also removes entry related functions
from Txn Struct except SetEntry and Set.
* Fix all compilation failure due to above change using
new Entry methods.
* Fix comments from GolangCI.
When too many entries are added to the merge operator, the compactions running
in background might accidentally delete entries before they're merged since
every entry added by the merge operator has the same key but different version.
If a user has set NumVersionsToKeep to a low value then the compactions would
delete the older values before they're merged.
With this commit, we add a new bit bitMergeEntry to all the entries inserted by
the merge operator except the entry which stores the result of merge operation.
The compactions would ignore all the entries which have bitMergeEntry set but
these entries will eventually be deleted because we set the bitDiscardEarlier
versions flag for the entry which holds the result of the merge operation.
* Support for DB-specific loggers.
This commit adds support for overriding the global logger in specific
DBs. If no DB-specific logger is given, the global logger will be used
instead.
- Do not use deletion markers during DropAll, because that causes writes at lower timestamps to be ignored (in managed DB mode).
- Instead, delete all SSTables and all log files, and reset value log and value log head to zero.
- Make DropAll open to all users of Badger, including normal mode.
Changelog:
* Do not use delete markers in DropAll. Instead, delete the entire LSM tree and all the value logs.
* Moved DropAll to db.go, so it is available to all users of Badger. Fixed a bug where we were not catching memtable flushes during DropAll.
* Make DropAll test more races. Catch another place where writes should fail due to blockedWrite.
* Self-review
Hanzo AI