LP-107 Phase 2: math substrate — params, backend, codec, modarith,

ntt, poly, rns, sample

Eight pure-Go reference packages that own the canonical semantics of
every cryptographic-math primitive Lux protocols share. Backends
(AVX2 / NEON / cgo+C++ / CUDA / Metal / WGSL) plug in behind the
substrate; KATs gate byte-equality across them.

Packages added:

* params/    — ModulusID, NTTParamID, FHEParamID, PulsarParamID,
               HashSuiteID, BackendID; KATHeader required-fields
               schema. Stable string IDs; renaming is a breaking
               change.

* backend/   — Policy enum (PureGo / NativeCPU / GPUPreferred /
               GPURequired); Resolve(policy, registered) returns the
               BackendID per LP-107's fallback chain. GPURequired
               returns ErrUnavailable when no GPU backend is
               registered.

* codec/     — Bounded readers (closes lattice issues #2 + #4 DoS
               class permanently). Limits + LimitError + Reader.
               ReadUint{16,32,64}Slice all reject the 70T-element
               attack input before allocating; depth + frame-bytes
               caps; iterative (no recursion). Regression test
               TestReadUint64Slice_RejectsHugeLength encodes the
               original lattice issue #4 input.

* modarith/  — Modulus type with QInv, R2, Barrett constants
               (computed via math/big once per modulus). AddMod /
               SubMod / MulMod (Div64 reference path). MontMulMod /
               ToMontgomery / FromMontgomery cross-checked against
               MulMod across 100 random Pulsar-Q pairs.
               ReductionMode enum byte-equal to lattice/types.

* ntt/       — Service + Backend interface; pure-Go backend
               delegates to lattice/v7/ring.SubRing.NTT/INTT (the
               canonical Lattigo-derived Montgomery NTT body — no
               re-implementation, just dispatch). Round-trip +
               batch + determinism tests on Pulsar N=256.

* poly/      — Add / Sub / ScalarMul / PointwiseMul (NTT-domain) /
               negacyclic Mul (NTT round-trip). Composes modarith +
               ntt; no lower-level reach.

* rns/       — Basis(Moduli, Name) for FHE RNS chains. Single +
               two-prime construction validated; rejects even moduli.
               Phase 3 will add basis-extension and modulus-switching.

* sample/    — Uniform (rejection sampling, mask-and-retry); Ternary
               (density + sign byte); CenteredBinomial (popcount
               difference); DiscreteGaussianRejection (6-sigma cutoff).
               All take an io.Reader so callers can KAT-replay.

Architecture invariants enforced in package docs:

  - Go is the canonical semantic reference.
  - Backend selection MUST NOT alter transcript bytes (consensus
    paths default to PolicyPureGo / PolicyNativeCPU).
  - No unbounded codec readers anywhere near wire formats.
  - No re-implementation: where lattice/v7/ring already owns the
    canonical body, we delegate, not fork.
  - One ID space; renaming is a breaking change.

Test posture: 8/8 packages green via `GOWORK=off go test ./...`.

Phases 3-7 (queued):
  3. lattice consumes math
  4. pulsar consumes lattice + math
  5. fhe consumes math
  6. luxcpp/crypto/math native backend mirror
  7. cross-runtime KAT release gate

See SUBSTRATE.md for the full posture and migration plan; full LP at
~/work/lux/lps/LP-107-lux-math-substrate.md.
This commit is contained in:
Hanzo AI
2026-05-04 09:28:23 -07:00
parent 8d638e5218
commit e018a9c3d1
21 changed files with 2684 additions and 10 deletions
+357
View File
@@ -0,0 +1,357 @@
// Copyright (c) 2026 Lux Industries Inc.
// SPDX-License-Identifier: BSD-3-Clause
// Package codec is the bounded-decode contract for every wire format
// in luxfi/math (and downstream luxfi/lattice, luxfi/pulsar, luxfi/fhe).
//
// LP-107 §"Codec and bounded reader should be centralized" — the
// canonical motivation. The lattigo `ReadUint64Slice` recursion bug
// (issue #2) and `Vector[T].ReadFrom` OOM bug (issue #4) both stemmed
// from unbounded slice decode on untrusted wire data; this package
// fixes that class permanently.
//
// Contract:
//
// - No recursion. Slice readers are iterative; depth is bounded by
// the configured Limits.
// - No hidden growth. Every `make([]T, n)` is preceded by a `n <= cap`
// check against caller-supplied Limits.
// - No unbounded allocation. The largest cap is application-supplied
// and surfaces in error messages.
// - All readers are deterministic and reentrant; failure leaves the
// reader at the byte where the bound was exceeded.
package codec
import (
"encoding/binary"
"errors"
"fmt"
"io"
"math/bits"
)
// Limits caps the largest slice / depth a Reader will accept on a
// single decode call. Callers MUST construct Limits explicitly; there
// is no implicit default.
type Limits struct {
// MaxFrameBytes caps the total number of input bytes the Reader
// will consume in a single decode call. Used to bound peek-ahead
// buffers. 0 means unset and is treated as an error.
MaxFrameBytes int
// MaxUint16SliceLen caps the number of elements in a uint16 slice.
MaxUint16SliceLen int
// MaxUint32SliceLen caps the number of elements in a uint32 slice.
MaxUint32SliceLen int
// MaxUint64SliceLen caps the number of elements in a uint64 slice.
MaxUint64SliceLen int
// MaxDepth caps how deeply a recursively-shaped wire format may
// nest before the reader rejects (Vector[Poly] etc. are 2 levels).
MaxDepth int
}
// Validate reports whether the limits are coherent (all positive).
// Returns an error listing every zero/negative field.
func (l Limits) Validate() error {
var problems []string
if l.MaxFrameBytes <= 0 {
problems = append(problems, "MaxFrameBytes")
}
if l.MaxUint16SliceLen <= 0 {
problems = append(problems, "MaxUint16SliceLen")
}
if l.MaxUint32SliceLen <= 0 {
problems = append(problems, "MaxUint32SliceLen")
}
if l.MaxUint64SliceLen <= 0 {
problems = append(problems, "MaxUint64SliceLen")
}
if l.MaxDepth <= 0 {
problems = append(problems, "MaxDepth")
}
if len(problems) > 0 {
return fmt.Errorf("codec.Limits: zero/negative fields: %v", problems)
}
return nil
}
// DefaultLimitsLatticeWire is the conservative default for wire-format
// decoding of lattice polynomials at the canonical Pulsar parameters
// (R_q = Z_q[X]/(X^256 + 1), Q ≈ 2^48). Callers SHOULD use a
// configuration tuned to their parameter set rather than this default.
//
// MaxUint64SliceLen = 4096 matches Pulsar Vector[Poly] cap.
// MaxFrameBytes = 16 MiB allows a worst-case threshold ceremony
// transcript without truncation.
// MaxDepth = 4 Pulsar wire is 2 levels (Vector + Poly);
// 4 leaves headroom for FHE chains.
var DefaultLimitsLatticeWire = Limits{
MaxFrameBytes: 16 * 1024 * 1024,
MaxUint16SliceLen: 4096,
MaxUint32SliceLen: 4096,
MaxUint64SliceLen: 4096,
MaxDepth: 4,
}
// ErrLimitExceeded is the sentinel for any limit-bound rejection.
// errors.Is(err, ErrLimitExceeded) holds for every cap violation.
var ErrLimitExceeded = errors.New("codec: limit exceeded")
// LimitError carries the specific limit that was exceeded plus the
// observed value. Wraps ErrLimitExceeded.
type LimitError struct {
What string // human-readable name of the cap, e.g. "MaxUint64SliceLen"
Limit int
Got uint64
}
// Error implements error.
func (e *LimitError) Error() string {
return fmt.Sprintf("codec: %s exceeded: limit=%d got=%d",
e.What, e.Limit, e.Got)
}
// Unwrap implements errors.Unwrap.
func (e *LimitError) Unwrap() error { return ErrLimitExceeded }
// Reader wraps an io.Reader and a Limits config. Every slice-reading
// method on Reader is bounded by Limits.
type Reader struct {
r io.Reader
limits Limits
consumed int
depth int
}
// NewReader constructs a Reader from an io.Reader and a Limits config.
// Returns an error if Limits is invalid.
func NewReader(r io.Reader, l Limits) (*Reader, error) {
if r == nil {
return nil, fmt.Errorf("codec: nil io.Reader")
}
if err := l.Validate(); err != nil {
return nil, err
}
return &Reader{r: r, limits: l}, nil
}
// Consumed returns the number of bytes read from the underlying io.Reader.
func (r *Reader) Consumed() int { return r.consumed }
// EnterDepth bumps the nesting counter and returns an error if the
// configured MaxDepth is exceeded. Caller MUST pair with ExitDepth.
func (r *Reader) EnterDepth() error {
r.depth++
if r.depth > r.limits.MaxDepth {
return &LimitError{What: "MaxDepth", Limit: r.limits.MaxDepth, Got: uint64(r.depth)}
}
return nil
}
// ExitDepth decrements the nesting counter.
func (r *Reader) ExitDepth() {
if r.depth > 0 {
r.depth--
}
}
// readN reads exactly n bytes, bumping the consumed counter and
// validating against MaxFrameBytes.
func (r *Reader) readN(n int) ([]byte, error) {
if n < 0 {
return nil, fmt.Errorf("codec: negative read length %d", n)
}
if r.consumed+n > r.limits.MaxFrameBytes {
return nil, &LimitError{
What: "MaxFrameBytes",
Limit: r.limits.MaxFrameBytes,
Got: uint64(r.consumed + n),
}
}
buf := make([]byte, n)
if _, err := io.ReadFull(r.r, buf); err != nil {
return nil, fmt.Errorf("codec: short read: %w", err)
}
r.consumed += n
return buf, nil
}
// ReadUint16 reads a single little-endian uint16.
func (r *Reader) ReadUint16() (uint16, error) {
b, err := r.readN(2)
if err != nil {
return 0, err
}
return binary.LittleEndian.Uint16(b), nil
}
// ReadUint32 reads a single little-endian uint32.
func (r *Reader) ReadUint32() (uint32, error) {
b, err := r.readN(4)
if err != nil {
return 0, err
}
return binary.LittleEndian.Uint32(b), nil
}
// ReadUint64 reads a single little-endian uint64.
func (r *Reader) ReadUint64() (uint64, error) {
b, err := r.readN(8)
if err != nil {
return 0, err
}
return binary.LittleEndian.Uint64(b), nil
}
// ReadUint16Slice reads a length-prefixed slice of little-endian uint16.
// The length is read as a varint capped by MaxUint16SliceLen; iterative
// (no recursion).
func (r *Reader) ReadUint16Slice() ([]uint16, error) {
n, err := r.readSliceLen("uint16", r.limits.MaxUint16SliceLen)
if err != nil {
return nil, err
}
if n == 0 {
return []uint16{}, nil
}
if err := overflowMul(n, 2, r.limits.MaxFrameBytes); err != nil {
return nil, err
}
out := make([]uint16, n)
buf, err := r.readN(int(n) * 2)
if err != nil {
return nil, err
}
for i := range out {
out[i] = binary.LittleEndian.Uint16(buf[i*2:])
}
return out, nil
}
// ReadUint32Slice reads a length-prefixed slice of little-endian uint32.
func (r *Reader) ReadUint32Slice() ([]uint32, error) {
n, err := r.readSliceLen("uint32", r.limits.MaxUint32SliceLen)
if err != nil {
return nil, err
}
if n == 0 {
return []uint32{}, nil
}
if err := overflowMul(n, 4, r.limits.MaxFrameBytes); err != nil {
return nil, err
}
out := make([]uint32, n)
buf, err := r.readN(int(n) * 4)
if err != nil {
return nil, err
}
for i := range out {
out[i] = binary.LittleEndian.Uint32(buf[i*4:])
}
return out, nil
}
// ReadUint64Slice reads a length-prefixed slice of little-endian uint64.
// Bounded by MaxUint64SliceLen. The length-prefix is always read as a
// varint; values > MaxUint64SliceLen are rejected before any allocation.
//
// This is the centralized fix for both lattigo issue #2 (recursive
// `ReadUint64Slice`) and issue #4 (`Vector[T].ReadFrom` unbounded
// allocation). Callers consuming untrusted lattice wire data MUST go
// through this method, never lattigo's raw `utils/buffer.ReadUint64Slice`.
func (r *Reader) ReadUint64Slice() ([]uint64, error) {
n, err := r.readSliceLen("uint64", r.limits.MaxUint64SliceLen)
if err != nil {
return nil, err
}
if n == 0 {
return []uint64{}, nil
}
if err := overflowMul(n, 8, r.limits.MaxFrameBytes); err != nil {
return nil, err
}
out := make([]uint64, n)
buf, err := r.readN(int(n) * 8)
if err != nil {
return nil, err
}
for i := range out {
out[i] = binary.LittleEndian.Uint64(buf[i*8:])
}
return out, nil
}
// readSliceLen reads a varint length and validates against the cap.
// Returns the parsed length as uint32 (we never accept lengths that
// don't fit a 32-bit count for slice work).
func (r *Reader) readSliceLen(what string, cap int) (uint32, error) {
v, err := r.readUvarint()
if err != nil {
return 0, err
}
if v > uint64(cap) {
return 0, &LimitError{
What: fmt.Sprintf("Max%sSliceLen", capitalize(what)),
Limit: cap,
Got: v,
}
}
if v > uint64(^uint32(0)) {
return 0, &LimitError{
What: "uint32 representable",
Limit: int(^uint32(0)),
Got: v,
}
}
return uint32(v), nil
}
// readUvarint reads a varint length. Iterative; bounded to 10 bytes
// (max varint encoding of uint64).
func (r *Reader) readUvarint() (uint64, error) {
var v uint64
var shift uint
for i := 0; i < 10; i++ {
b, err := r.readN(1)
if err != nil {
return 0, err
}
c := b[0]
if c < 0x80 {
if i == 9 && c > 1 {
return 0, fmt.Errorf("codec: varint overflow")
}
v |= uint64(c) << shift
return v, nil
}
v |= uint64(c&0x7f) << shift
shift += 7
}
return 0, fmt.Errorf("codec: varint too long (>10 bytes)")
}
// overflowMul rejects n*size > frameMax even on multiplication overflow.
// Returns *LimitError on rejection.
func overflowMul(n uint32, size int, frameMax int) error {
hi, lo := bits.Mul64(uint64(n), uint64(size))
if hi != 0 || lo > uint64(frameMax) {
return &LimitError{
What: "MaxFrameBytes (slice payload)",
Limit: frameMax,
Got: lo,
}
}
return nil
}
func capitalize(s string) string {
if s == "" {
return s
}
first := s[0]
if first >= 'a' && first <= 'z' {
first -= 'a' - 'A'
}
return string(first) + s[1:]
}
+184
View File
@@ -0,0 +1,184 @@
// Copyright (c) 2026 Lux Industries Inc.
// SPDX-License-Identifier: BSD-3-Clause
package codec
import (
"bytes"
"encoding/binary"
"errors"
"testing"
)
func newReader(t *testing.T, data []byte, l Limits) *Reader {
t.Helper()
r, err := NewReader(bytes.NewReader(data), l)
if err != nil {
t.Fatalf("NewReader: %v", err)
}
return r
}
func TestLimits_Validate(t *testing.T) {
if err := DefaultLimitsLatticeWire.Validate(); err != nil {
t.Errorf("DefaultLimitsLatticeWire.Validate(): %v", err)
}
if err := (Limits{}).Validate(); err == nil {
t.Error("empty Limits.Validate() returned nil")
}
}
func TestNewReader_NilArgs(t *testing.T) {
if _, err := NewReader(nil, DefaultLimitsLatticeWire); err == nil {
t.Error("nil io.Reader: no error")
}
if _, err := NewReader(bytes.NewReader(nil), Limits{}); err == nil {
t.Error("zero Limits: no error")
}
}
func encodeUvarint(out *bytes.Buffer, v uint64) {
for v >= 0x80 {
out.WriteByte(byte(v) | 0x80)
v >>= 7
}
out.WriteByte(byte(v))
}
func TestReadUint64Slice_HappyPath(t *testing.T) {
want := []uint64{0xdeadbeef, 0xcafebabe, 0x1122334455667788}
var buf bytes.Buffer
encodeUvarint(&buf, uint64(len(want)))
for _, v := range want {
_ = binary.Write(&buf, binary.LittleEndian, v)
}
r := newReader(t, buf.Bytes(), DefaultLimitsLatticeWire)
got, err := r.ReadUint64Slice()
if err != nil {
t.Fatalf("ReadUint64Slice: %v", err)
}
if len(got) != len(want) {
t.Fatalf("len: want %d got %d", len(want), len(got))
}
for i := range want {
if got[i] != want[i] {
t.Errorf("[%d]: want %#x got %#x", i, want[i], got[i])
}
}
}
// TestReadUint64Slice_RejectsHugeLength is the regression test for
// lattigo issue #4: 9-byte input asking for 70 trillion uint64s must
// be rejected before any allocation.
func TestReadUint64Slice_RejectsHugeLength(t *testing.T) {
// 9-byte attack input from the lattice issue #4 reproducer
// produces a varint whose decoded value is way beyond MaxUint64SliceLen.
// We synthesize: varint = 70_368_955_777_453 (~70T), then expect
// a LimitError on MaxUint64SliceLen.
huge := uint64(70_368_955_777_453)
var buf bytes.Buffer
encodeUvarint(&buf, huge)
r := newReader(t, buf.Bytes(), DefaultLimitsLatticeWire)
_, err := r.ReadUint64Slice()
if err == nil {
t.Fatal("ReadUint64Slice with 70T length: returned nil error")
}
if !errors.Is(err, ErrLimitExceeded) {
t.Errorf("want ErrLimitExceeded, got %v", err)
}
var le *LimitError
if !errors.As(err, &le) {
t.Fatalf("error not a *LimitError: %T %v", err, err)
}
if le.What != "MaxUint64SliceLen" {
t.Errorf("LimitError.What = %q, want MaxUint64SliceLen", le.What)
}
if le.Got != huge {
t.Errorf("LimitError.Got = %d, want %d", le.Got, huge)
}
}
func TestReadUint32Slice_RejectsHugeLength(t *testing.T) {
huge := uint64(1_000_000_000)
var buf bytes.Buffer
encodeUvarint(&buf, huge)
r := newReader(t, buf.Bytes(), DefaultLimitsLatticeWire)
_, err := r.ReadUint32Slice()
if !errors.Is(err, ErrLimitExceeded) {
t.Errorf("want ErrLimitExceeded, got %v", err)
}
}
func TestReadUint16Slice_RejectsHugeLength(t *testing.T) {
huge := uint64(1_000_000_000)
var buf bytes.Buffer
encodeUvarint(&buf, huge)
r := newReader(t, buf.Bytes(), DefaultLimitsLatticeWire)
_, err := r.ReadUint16Slice()
if !errors.Is(err, ErrLimitExceeded) {
t.Errorf("want ErrLimitExceeded, got %v", err)
}
}
func TestReadUint16(t *testing.T) {
r := newReader(t, []byte{0xCD, 0xAB}, DefaultLimitsLatticeWire)
v, err := r.ReadUint16()
if err != nil || v != 0xABCD {
t.Errorf("ReadUint16: %v %#x", err, v)
}
}
func TestReadUint32(t *testing.T) {
r := newReader(t, []byte{0x78, 0x56, 0x34, 0x12}, DefaultLimitsLatticeWire)
v, err := r.ReadUint32()
if err != nil || v != 0x12345678 {
t.Errorf("ReadUint32: %v %#x", err, v)
}
}
func TestReadUint64(t *testing.T) {
r := newReader(t, []byte{
0x88, 0x77, 0x66, 0x55, 0x44, 0x33, 0x22, 0x11,
}, DefaultLimitsLatticeWire)
v, err := r.ReadUint64()
if err != nil || v != 0x1122334455667788 {
t.Errorf("ReadUint64: %v %#x", err, v)
}
}
func TestDepth(t *testing.T) {
limits := DefaultLimitsLatticeWire
limits.MaxDepth = 2
r := newReader(t, []byte{}, limits)
if err := r.EnterDepth(); err != nil {
t.Fatalf("depth 1: %v", err)
}
if err := r.EnterDepth(); err != nil {
t.Fatalf("depth 2: %v", err)
}
if err := r.EnterDepth(); !errors.Is(err, ErrLimitExceeded) {
t.Fatalf("depth 3: want ErrLimitExceeded, got %v", err)
}
r.ExitDepth()
r.ExitDepth()
r.ExitDepth() // safe to over-exit
}
func TestFrameBytesCap(t *testing.T) {
limits := DefaultLimitsLatticeWire
limits.MaxFrameBytes = 4
r := newReader(t, []byte{1, 2, 3, 4, 5}, limits)
if _, err := r.ReadUint32(); err != nil {
t.Fatalf("first 4 bytes: %v", err)
}
// Next byte read must hit MaxFrameBytes.
if _, err := r.ReadUint16(); !errors.Is(err, ErrLimitExceeded) {
t.Errorf("over-cap: want ErrLimitExceeded, got %v", err)
}
}