Zach Kelling 9ea55abee3 feat: zen5 MoDE — Mixture of Diverse Experts architecture ...

Complete training infrastructure for zen5, a 3.1T parameter model
with complexity-aware hierarchical routing across 6 diverse model
families. Only 394M params trained (0.013% of total).

- Expert extraction pipeline for dense and MoE source models
- ComplexityEstimator + AlignmentLayer + MoDERouter training
- Benchmark suite with 24 labeled routing test cases
- HuggingFace model card generation for zen5 lineup
- Full architecture specification document

2026-03-03 07:36:24 -08:00

..

benchmark.py

feat: zen5 MoDE — Mixture of Diverse Experts architecture

2026-03-03 07:36:24 -08:00

expert_extraction.py

feat: zen5 MoDE — Mixture of Diverse Experts architecture

2026-03-03 07:36:24 -08:00

hf_repos.py

feat: zen5 MoDE — Mixture of Diverse Experts architecture

2026-03-03 07:36:24 -08:00

router_training.py

feat: zen5 MoDE — Mixture of Diverse Experts architecture

2026-03-03 07:36:24 -08:00