Complete training infrastructure for zen5, a 3.1T parameter model
with complexity-aware hierarchical routing across 6 diverse model
families. Only 394M params trained (0.013% of total).
- Expert extraction pipeline for dense and MoE source models
- ComplexityEstimator + AlignmentLayer + MoDERouter training
- Benchmark suite with 24 labeled routing test cases
- HuggingFace model card generation for zen5 lineup
- Full architecture specification document
