StartList returns ListBuilder by value (mirrors the value-typed ObjectBuilder):
Add*/Finish keep pointer receivers, every caller binds to an addressable local so
Go auto-addresses it. Dropped the dead elemSize field (never read — stride is
implicit in which Add* is called). Byte-identical wire (golden+fuzz+all round-trips
green). NOTE: not a measured alloc win — escape analysis already stack-allocated
the &ListBuilder{} at all 9 call sites (composite stays 583ns/5allocs, the 5 being
mandatory writeEnvelopePrefix pool-egress copies). Value is code-cleanliness +
consistency.
Also fixes zap/v1 subpackage: v1.2.4's ObjectBuilder value change left v1/*.go
referencing *zap.ObjectBuilder → the subpackage didn't build (latent; only the
root package is imported downstream so it went unnoticed). Converted to the value
type. Full module (root + v1) builds + tests green.
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
Decomplect that unlocks a perf win. SetBytes deferred its data into an offsets
[]offsetEntry slice replayed in Finish — machinery that recorded intentions to
replay later, and heap-allocated a slice per object-with-a-bytes-field. Replace
it with directness: StartObject RESERVES the fixed section eagerly, so SetBytes
appends its tail + patches its own pointer on the spot; Finish just returns the
offset. Removing the offsets field makes ObjectBuilder's methods mutate only
ob.b (through the pointer) and never ob itself — so ObjectBuilder becomes a
VALUE type and StartObject stops heap-allocating &ObjectBuilder per call (it was
65% of composite-build allocs).
Byte-identical wire (golden + fuzz + pool byte-equality + nested-proof suites all
green). Real X-tx wire composite: 922ns/11allocs -> 655ns/5allocs (1.4x, <half
the allocs); vs the original byte-blob 2551ns/37allocs that's 3.9x / 7.4x fewer
allocs. Speeds EVERY zap object build node-wide, not just X.
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
SetBytes was heap-copying every payload (append([]byte(nil), v...)) at every
nesting level — the dominant cost of deep wire composition (X-chain tx build
runs ~11 nested builders). It now retains the caller's slice; the caller keeps
it valid until Finish (every in-tree constructor builds+finishes in one frame).
Byte output is unchanged: Finish writes payloads in the same SetBytes order.
GetBuilder/PutBuilder: exported sync.Pool for Builders (write-side counterpart
to the read bufpool). Reuse is unconditionally byte-safe — ensureField zero-
fills every extended span and align zero-fills padding; proven by
TestBuilderPool_ByteIdentical (dirty pooled builder emits bytes identical to a
fresh one). go.sum: re-record luxfi/pq v1.0.3 (upstream re-tag drifted content).
Micro: sample object build 354ns/9allocs -> 257ns/6allocs; the real win
compounds in utxo/wire + xvm envelope composition (next commits).
Co-authored-by: Hanzo Dev <dev@hanzo.ai>
The repeated-message wire shape: a list whose elements carry their own
variable-length tails (string/bytes/nested), which the fixed-stride inline
List cannot hold. Completes proto field-shape coverage in the runtime —
map<k,v> reduces to a repeated {key,value} entry, so this is its keystone too.
Composition, not a new wire concept: ListNested = List ∘ NestedAt. The list
slots are a contiguous stride-4 object-pointer array; each pointer is
followed exactly as a singular nested pointer to an out-of-line tailed View.
Runtime:
- builder.go: ListBuilder.AddObjectPtr(targetPos) — appends a signed 4-byte
relative object pointer (the repeated-message element kind).
- v1/list_nested.go: ListNested[N] (Len/At/All/IsZero), ListNestedAt[S,N]
(read), WriteListNested[S,N] + NestedElemSetter[N] (write). Same audited
bounds discipline as Object.Object / ListStride (RED-HIGH-1/2 inherited).
Codegen:
- elemIsTailed: a list element with a string/bytes/nested/list sub-field
auto-routes to ListNested/WriteListNested/ListNestedAt; scalar-only
elements stay in the tighter inline List/WriteList/ListAt. One Elem
declaration, the generator picks the correct encoding.
Proven end-to-end: runtime roundtrip (3 elements each with its OWN string
tail, via At and All, + empty), emit routing test (tailed->ListNested,
scalar-only->List), and a generated-code wire roundtrip (testpkg/repeatwire).
Root zap + full v1+codegen suite green; scalar listwire unchanged (no regression).
There is one perfected typed ZAP layer and it lives at github.com/luxfi/zap/v1 —
forwards-only, no backwards-compat baggage, never a breaking major. Mechanical,
behavior-preserving: v2/ -> v1/ (git mv), package zapv2 -> zapv1, import path
.../zap/v2 -> .../zap/v1 (incl. the codegen's emitted banner + import so
generated files reference /v1). Codegen golden tests pass; typed layer vets
clean. (Pre-existing CGO 'resolv' link failures in the root package are
unrelated env noise.)
Three orthogonal additions, one wire format.
v2 package — generic ad-hoc schema dispatch (~/work/lux/zap/v2):
- README + IMPOSSIBILITY + BENCH_RESULTS document the v1↔v2 split.
- builder/field/iter/list/pool/registry/schema/view + codegen package
emit v1-equivalent fast paths from declarative .zap sources.
- _compile_fail_test/ + examples/ cover negative and positive cases.
- v2_test.go exercises the generic dispatch path.
- doc.go in v1 marks v1 deprecated for *new* schema authoring; existing
v1 buffers stay parseable by both v1 and v2.
bufpool — pooled read slabs (bufpool.go + bufpool_test.go):
- Message gains optional *bufRef; Release() returns the slab to its
pool when sourced from the pooled read path, GC-managed otherwise.
- ParseHeader returns (data, rootOffset) without allocating *Message,
used by zapv2 to avoid two allocations per parse.
- bench_read_test.go + node_quic_stream_test.go cover both paths.
Transport allocator + UMA/RDMA cgo (transport/*):
- Transport.Caps()/Buffer interface: capability + slab-typed buffer
contract. Bytes() vs DevicePtr() decoupled so UMA and GPUDirect
can share the same call surface.
- DPDK PMD + GPUDirect RDMA + CUDA UMA real cgo backends gated behind
build tags; stub paths remain default on darwin / non-Linux.
- transport_test.go covers Caps reporting + Buffer lifecycle on the
default transport.
LLM.md + go.mod follow the additions.
RED-HIGH-1: Object.List length is now clamped against len(data) at parse.
Attacker-set length=0xFFFFFFFF used to flow through to consumers, where
`for i := 0; i < l.Len()` would iterate 4G times.
RED-HIGH-2: Object.Object, Object.List, and Object.Bytes now reject any
absOffset < HeaderSize. A negative relOffset can no longer alias the wire
header (Magic/Version/Flags/RootOffset/Size). The signed-cast remains so
honest builders can finalize nested items before their parent.
RED-MEDIUM-1: bump wire Version from 1 to 2 (Version1 still accepted at
Parse for backward compat). NewBuilder emits Version2 by default;
NewBuilderV1 is kept for explicit-legacy paths. Message.Version() exposes
the parsed version so v3 platformvm Wrap*Tx accessors can gate on
Version2 to reject v1-vs-v2 schema confusion (e.g. NetworkID=11 byte 0
collision with TxKindBaseFull).
RED-V18: Parse now rejects size < HeaderSize. A buffer with size=0
previously slipped through and panicked on subsequent Root()/Flags().
Regression tests pinned in-tree at zap_test.go:
TestRedRound2_HIGH1_UncappedListLength
TestRedRound2_HIGH2_BackwardListPointer
TestRedRound2_HIGH2_BackwardObjectPointer
TestRedRound2_HIGH2_BackwardBytesPointer
TestRedRound2_MEDIUM1_VersionParse
TestRedRound2_MEDIUM1_NewBuilderEmitsV2
TestRedRound2_V18_SizeZeroRejected
TestRedRound2_F1_NegativeBitPatternSweep
Originals at /tmp/red_zap_attacks/*_test.go.
ObjectBuilder.SetText/SetBytes stored deferred entries but Finish()
never wrote the actual text/bytes data or patched the relative offsets.
Now Finish() writes deferred data after the fixed section and patches
each field's relative offset correctly.
Adds 3 round-trip tests: single text, multiple text, nested objects
with text fields.
