hanzo/ml

Files

T

History

carmackandHanzo Dev 44bdc61e5d hanzo-ml 0.11.91: metal(qmatmul) batched small-m matvec for quantized weights

Route m<=metal_matvec_max() (default 8) quantized matmuls through a single
batched ne11=m matvec dispatch instead of a per-column m-loop, cutting
verify-round latency in speculative decode. Dense F16/BF16/F32 weights stay on
the GEMM path: the batched matvec family indexes src0/src1 by block/element
counts and would mis-stride and mis-grid dense weights, so fwd() dtype-gates the
batched route to the quantized families only.

Adds a metal-kernels F16-GEMM-vs-CPU test and an fwd()-level F16 m=4 routing test.

2026-07-22 02:29:35 -07:00

examples

wip: checkpoint before merging origin/main (recoverable via reset --soft HEAD~1)

2026-06-03 20:33:41 -07:00

src

hanzo-ml 0.11.91: metal(qmatmul) batched small-m matvec for quantized weights

2026-07-22 02:29:35 -07:00

Cargo.toml

metal: fp16-matmul prefill fast path for Q4_K/Q6_K (metal-kernels 0.11.44, ml 0.11.85)

2026-07-21 09:33:58 -07:00

README.md

Scrub all remaining candle brand refs -> hanzo/hanzoai (docs, CI, tests, comments)

2026-06-01 11:48:33 -07:00

README.md

hanzo-metal-kernels

This crate contains Metal kernels used from hanzo-ml.