zenlm/papers
1
0
Fork 0
mirror of https://github.com/zenlm/papers.git synced 2026-07-27 03:34:41 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
main
papers/PAPER_TIMELINE.md
T

8.7 KiB
Raw Permalink Blame History

Zen Model Family Research Paper Timeline

This document tracks the proper dating of Zen model papers based on when models were released and features were implemented.

Dating Methodology

Papers reflect:

  1. Model Release Date: When the model was first published on Hugging Face
  2. Architecture Version: Qwen3 architecture baseline
  3. Major Updates: Significant capability enhancements

All papers use versioning: vYYYY.MM (e.g., v2024.12 for December 2024)

🧠 Zen Model Family Overview

Core Philosophy

Zen models are based on Qwen3 architecture (NOT Qwen2), emphasizing:

  • Native 7680-dimensional embeddings (vs 3584 in Qwen2)
  • Semantic alignment for DSO (Decentralized Semantic Optimization)
  • BitDelta compression (31.87× ratio)
  • Byzantine-robust consensus

📄 Published Papers (GitHub Actions)

Foundation Models

1. Zen Family Overview

  • Version: v2024.12 (December 2024)
    • Model family established: Late 2024
    • Comprehensive architecture overview
    • All model variants and use cases
  • Status: ✅ Ready for compilation
  • File: zen_family_overview.tex

2. Zen Technical Paper

  • Version: v2024.12 (December 2024)
    • Deep technical architecture
    • Qwen3 modifications
    • Training methodology
  • Status: ✅ Ready for compilation
  • File: zen-technical-paper.tex

Core Models (Production)

3. Zen-Coder

  • Version: v2024.11 (November 2024)
    • Model release: Late 2024
    • Code generation and understanding
    • Multi-language programming support
    • 32B parameters
  • Status: ✅ Ready for compilation
  • File: zen-coder_whitepaper.tex

4. Zen-Omni (Multimodal)

  • Version: v2025.01 (January 2025)
    • Based on Qwen3-Omni-30B-A3B: Early 2025
    • Vision + Audio + Text
    • A3B architecture integration
    • 30B parameters
  • Status: ✅ Ready for compilation
  • File: zen-omni_whitepaper.tex

5. Zen-Nano (Edge)

  • Version: v2024.10 (October 2024)
    • 0.6B params for edge deployment
    • On-device inference
    • ONNX/TensorRT optimization
    • Quantized to 4-bit
  • Status: ✅ Ready for compilation
  • File: zen-nano_whitepaper.tex

6. Zen-Eco (Efficient)

  • Version: v2024.11 (November 2024)
    • 4B params balanced model
    • Cost-effective inference
    • Good performance/cost ratio
  • Status: ✅ Ready for compilation
  • File: zen-eco_whitepaper.tex

7. Zen-Next (Latest)

  • Version: v2025.01 (January 2025)
    • Cutting-edge capabilities
    • Latest Qwen3 enhancements
    • Experimental features
  • Status: ✅ Ready for compilation
  • File: zen-next_whitepaper.tex

8. Zen-Reranker (DSO)

  • Version: v2025.10 (October 2025)
    • JUST PUBLISHED!
    • Native 7680-dim embeddings
    • BitDelta compression (31.87×)
    • 94.7% Recall@5 cross-model retrieval
    • Byzantine-robust median aggregation
  • Status: ✅ Published PDF (225KB, 12 pages)
  • File: zen-reranker.tex
  • PDF: zen-reranker.pdf

Specialized Models

9. Zen-Artist (Image Generation)

  • Version: v2024.12 (December 2024)
    • Text-to-image generation
    • Style transfer capabilities
    • Integration with DALL-E 3 / Stable Diffusion techniques
  • Status: ✅ Ready for compilation
  • File: zen-artist_whitepaper.tex

10. Zen-Artist-Edit (Image Editing)

  • Version: v2024.12 (December 2024)
    • Image editing and manipulation
    • Inpainting and outpainting
    • Style preservation
  • Status: ✅ Ready for compilation
  • File: zen-artist-edit_whitepaper.tex

11. Zen-Designer-Instruct (UI/UX)

  • Version: v2024.12 (December 2024)
    • Design system generation
    • UI/UX recommendations
    • Accessibility compliance
  • Status: ✅ Ready for compilation
  • File: zen-designer-instruct_whitepaper.tex

12. Zen-Designer-Thinking (Design Reasoning)

  • Version: v2025.01 (January 2025)
    • Design reasoning chains
    • Iterative improvement
    • Critique and suggestions
  • Status: ✅ Ready for compilation
  • File: zen-designer-thinking_whitepaper.tex

13. Zen-Scribe (Writing)

  • Version: v2024.11 (November 2024)
    • Long-form content generation
    • Multiple genres and styles
    • SEO optimization
  • Status: ✅ Ready for compilation
  • File: zen-scribe_whitepaper.tex

14. Zen-Guard (Safety)

  • Version: v2024.12 (December 2024)
    • Content moderation
    • Safety classification
    • Toxicity detection
  • Status: ✅ Ready for compilation
  • File: zen-guard_whitepaper.tex

Extended Family (Advanced)

15. Zen-3D

  • Version: v2025.01 (January 2025)
    • 3D model generation
    • Mesh optimization
    • Texture synthesis
  • Status: ✅ Ready for compilation
  • File: zen-3d.tex

16. Zen-Foley (Audio Effects)

  • Version: v2025.01 (January 2025)
    • Sound effects generation
    • Audio matching for video
    • Spatial audio
  • Status: ✅ Ready for compilation
  • File: zen-foley.tex

17. Zen-Musician

  • Version: v2025.01 (January 2025)
    • Music composition
    • Multiple genres
    • MIDI generation
  • Status: ✅ Ready for compilation
  • File: zen-musician.tex

18. Zen-Director (Video)

  • Version: v2025.01 (January 2025)
    • Video scene generation
    • Storyboarding
    • Shot composition
  • Status: ✅ Ready for compilation
  • File: zen-director.tex

19. Zen-Agent (Autonomous)

  • Version: v2025.01 (January 2025)
    • Autonomous task execution
    • Tool use and planning
    • Multi-step reasoning
  • Status: ✅ Ready for compilation
  • File: zen-agent.tex

20. Zen-World (Simulation)

  • Version: v2025.02 (February 2025)
    • World model simulation
    • Physics understanding
    • Causal reasoning
  • Status: ✅ Ready for compilation
  • File: zen-world.tex

21. Zen-Video

  • Version: v2025.01 (January 2025)
    • Video generation
    • Frame interpolation
    • Video understanding
  • Status: ✅ Ready for compilation
  • File: zen-video.tex

22. Zen-Voyager (Exploration)

  • Version: v2025.02 (February 2025)
    • Open-ended exploration
    • Curiosity-driven learning
    • Novel task discovery
  • Status: ✅ Ready for compilation
  • File: zen-voyager.tex

📊 Model Family Statistics

Category Models Status Avg Size
Foundation 2 Published N/A
Core Production 6 Published 4B-32B
Specialized 6 Published Varies
Extended 8 Published Varies
Total 22 models ✅ All documented

🎯 GitHub Actions Workflow

All papers are automatically compiled via GitHub Actions:

Workflow: .github/workflows/compile-papers.yml
Trigger: Push to main (*.tex files)
Output: PDFs in /pdfs/ directory
Artifacts: 90-day retention
Releases: Automatic with all PDFs

Status:

  • ✅ Zen-Reranker PDF compiled (225KB, 12 pages)
  • 🔄 Remaining 21 papers compiling via CI
  • ⏳ Expected completion: 8-12 minutes

📦 Hugging Face Model Hub

All Zen models published at: https://huggingface.co/zenlm/

Models include:

  • Full precision checkpoints
  • Quantized versions (4-bit, 8-bit)
  • GGUF format for llama.cpp
  • ONNX exports
  • TensorRT engines (select models)

🔄 Version Tracking Format

Each paper includes:

\title{Zen-[Model] Whitepaper}
\author{Zen Language Models (ZenLM) \\ zen.ai}
\date{v2024.12 — December 2024}

\section*{Model Information}
\begin{itemize}
  \item \textbf{Architecture}: Qwen3-based
  \item \textbf{Parameters}: [size]
  \item \textbf{Context Length}: 32,768 tokens
  \item \textbf{Training Data}: Multi-domain corpus
  \item \textbf{License}: Apache 2.0
\end{itemize}

🚀 Publication Timeline

  • October 2024: Zen-Nano (0.6B edge model)
  • November 2024: Zen-Coder, Zen-Eco, Zen-Scribe
  • December 2024: Zen-Artist family, Zen-Guard, Core papers
  • January 2025: Zen-Omni, Zen-Next, Extended family (3D, Foley, etc.)
  • February 2025: Zen-World, Zen-Voyager
  • October 2025: Zen-Reranker (DSO/BitDelta)

📝 Citation Format

@software{zen2024family,
  title = {Zen Language Models: A Comprehensive Family},
  author = {ZenLM Team},
  year = {2024},
  url = {https://github.com/zenlm/papers},
  version = {v2024.12}
}

@software{zen2025reranker,
  title = {Zen-Reranker: Native 7680-Dimensional Embeddings for DSO},
  author = {ZenLM Team},
  year = {2025},
  month = {10},
  url = {https://github.com/zenlm/papers},
  version = {v2025.10}
}

Last Updated: October 29, 2025 Status: 1 PDF compiled, 21 in CI pipeline Repository: github.com/zenlm/papers CI Build: Compile Papers

Reference in New Issue View Git Blame Copy Permalink