Files
zap/node_codec.go
Hanzo AI f42cdb6618 feat(zap): v2 generic API, bufpool, transport allocator + UMA/RDMA cgo
Three orthogonal additions, one wire format.

v2 package — generic ad-hoc schema dispatch (~/work/lux/zap/v2):
- README + IMPOSSIBILITY + BENCH_RESULTS document the v1↔v2 split.
- builder/field/iter/list/pool/registry/schema/view + codegen package
  emit v1-equivalent fast paths from declarative .zap sources.
- _compile_fail_test/ + examples/ cover negative and positive cases.
- v2_test.go exercises the generic dispatch path.
- doc.go in v1 marks v1 deprecated for *new* schema authoring; existing
  v1 buffers stay parseable by both v1 and v2.

bufpool — pooled read slabs (bufpool.go + bufpool_test.go):
- Message gains optional *bufRef; Release() returns the slab to its
  pool when sourced from the pooled read path, GC-managed otherwise.
- ParseHeader returns (data, rootOffset) without allocating *Message,
  used by zapv2 to avoid two allocations per parse.
- bench_read_test.go + node_quic_stream_test.go cover both paths.

Transport allocator + UMA/RDMA cgo (transport/*):
- Transport.Caps()/Buffer interface: capability + slab-typed buffer
  contract. Bytes() vs DevicePtr() decoupled so UMA and GPUDirect
  can share the same call surface.
- DPDK PMD + GPUDirect RDMA + CUDA UMA real cgo backends gated behind
  build tags; stub paths remain default on darwin / non-Linux.
- transport_test.go covers Caps reporting + Buffer lifecycle on the
  default transport.

LLM.md + go.mod follow the additions.
2026-06-04 17:06:59 -07:00

120 lines
4.2 KiB
Go

// Copyright (C) 2025, Lux Industries Inc. All rights reserved.
// See the file LICENSE for licensing terms.
package zap
import (
"encoding/binary"
)
// node_codec.go decomplects two pieces of wire framing that previously
// appeared in three places each:
//
// 1. The NodeID handshake message (64-byte ZAP object: bytes 0..59 =
// UTF-8 nodeID, bytes 60..63 = length). Used at connection open
// by both the dialer (ConnectDirect) and the acceptor (handleConn).
// 2. The Call/response correlation header (8 bytes: reqID u32 LE,
// flag u32 LE) prepended to a wrapped ZAP message. Used by Call,
// by the incoming-Call response path in handleConn, and by the
// handleConn dispatcher to recognise the frame shape.
//
// Both have exactly one encoder and one decoder. Callers wire-encode
// and wire-decode through the same path; bugs surface in one place.
// --- NodeID handshake codec ---
// nodeIDHandshakeSize is the size of the handshake message on the
// wire — header (16) + a single 64-byte object.
const nodeIDHandshakeSize = HeaderSize + 64
// maxNodeIDLen is the largest UTF-8 nodeID we serialise; longer IDs
// are truncated by the encoder and rejected by the decoder.
const maxNodeIDLen = 60
// EncodeNodeIDHandshake builds the NodeID exchange message.
// nodeIDs longer than maxNodeIDLen are truncated; the receiver
// validates length on Decode.
func EncodeNodeIDHandshake(nodeID string) []byte {
b := NewBuilder(128)
obj := b.StartObject(64)
idBytes := []byte(nodeID)
n := len(idBytes)
if n > maxNodeIDLen {
n = maxNodeIDLen
}
for i := 0; i < n; i++ {
obj.SetUint8(i, idBytes[i])
}
obj.SetUint32(maxNodeIDLen, uint32(n))
obj.FinishAsRoot()
return b.Finish()
}
// DecodeNodeIDHandshake reads a NodeID exchange message and returns
// the peer's nodeID. An empty string (with ok=false) indicates a
// malformed or out-of-range length field.
func DecodeNodeIDHandshake(data []byte) (string, bool) {
msg, err := Parse(data)
if err != nil {
return "", false
}
root := msg.Root()
idLen := root.Uint32(maxNodeIDLen)
if idLen == 0 || idLen > maxNodeIDLen {
return "", false
}
idBytes := make([]byte, idLen)
for i := uint32(0); i < idLen; i++ {
idBytes[i] = root.Uint8(int(i))
}
return string(idBytes), true
}
// --- Call/response correlation header ---
// correlatedHeaderSize is the 8-byte ReqID + Flag preamble that
// distinguishes a Call request/response from a Send.
const correlatedHeaderSize = 8
// WrapCorrelated prepends the Call/response correlation header to
// `body`. The result is what writeMessage emits onto the wire.
func WrapCorrelated(reqID uint32, flag uint32, body []byte) []byte {
out := make([]byte, correlatedHeaderSize+len(body))
binary.LittleEndian.PutUint32(out[0:4], reqID)
binary.LittleEndian.PutUint32(out[4:8], flag)
copy(out[correlatedHeaderSize:], body)
return out
}
// UnwrapCorrelated reads the correlation header off `data` and
// returns (reqID, flag, body, ok). If `data` is shorter than the
// header or the flag isn't a recognised value, ok is false and
// the caller should treat the message as uncorrelated.
func UnwrapCorrelated(data []byte) (reqID uint32, flag uint32, body []byte, ok bool) {
if len(data) < correlatedHeaderSize {
return 0, 0, nil, false
}
reqID = binary.LittleEndian.Uint32(data[0:4])
flag = binary.LittleEndian.Uint32(data[4:8])
if flag != ReqFlagReq && flag != ReqFlagResp {
return 0, 0, nil, false
}
return reqID, flag, data[correlatedHeaderSize:], true
}
// makeCorrelatedFrame builds a single-slice correlation frame
// (header + body) for transports that take []byte (notably the QUIC
// TransportConn.Send shape). TCP callers should prefer writeCorrelated
// which avoids the body copy entirely via scatter-gather.
//
// One allocation per call. A pooled variant requires the transport
// interface to expose a "buffer consumed" callback so the slab can
// be returned safely; that's a cross-module change and is deferred.
func makeCorrelatedFrame(reqID uint32, flag uint32, body []byte) []byte {
buf := make([]byte, correlatedHeaderSize+len(body))
binary.LittleEndian.PutUint32(buf[0:4], reqID)
binary.LittleEndian.PutUint32(buf[4:8], flag)
copy(buf[correlatedHeaderSize:], body)
return buf
}