hanzo/ml
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Scrub all remaining candle brand refs -> hanzo/hanzoai (docs, CI, tests, comments)

Browse Source 
Complete the de-brand across ALL tracked files: CHANGELOG/READMEs, .github
workflows (candle-pyo3 -> hanzo-ml-pyo3 etc.), doc comments, peripheral-crate
tests/examples (candle_onnx -> hanzo_onnx, candle_flash_attn -> hanzo_flash_attn,
candle_metal_kernels -> hanzo_metal_kernels, candle_rocm_kernels ->
hanzo_rocm_kernels), the CandleAlgo cudnn alias -> HanzoAlgo, temp-file prefixes,
URLs (huggingface/candle -> hanzoai/ml, docs.rs/candle-* -> docs.rs/hanzo-*),
EricLBuehler -> hanzoai, and the repository metadata. `git grep candle` now
returns 0. Core build verified: cargo check -p hanzo-ml --features vulkan green,
Qwen3-0.6B forward argmax matches CPU (rel 4.3e-6).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Hanzo Dev
2026-06-01 11:48:33 -07:00
co-authored by Claude Opus 4.8
parent a19e5d696a
commit b9cf30fa4e
87 changed files with 350 additions and 350 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/maturin.yml
+10 -10
View File
@@ -7,10 +7,10 @@ on:
tags:
- '*'
paths:
- candle-pyo3/**
- hanzo-ml-pyo3/**
pull_request:
paths:
- candle-pyo3/**
- hanzo-ml-pyo3/**
workflow_dispatch:
permissions:
@@ -39,12 +39,12 @@ jobs:
args: --release --out dist --find-interpreter
sccache: 'true'
manylinux: auto
working-directory: ./candle-pyo3
working-directory: ./hanzo-ml-pyo3
- name: Upload wheels
uses: actions/upload-artifact@v6
with:
name: wheels-linux-${{ matrix.target }}
path: ./candle-pyo3/dist
path: ./hanzo-ml-pyo3/dist
windows:
runs-on: windows-latest
@@ -68,12 +68,12 @@ jobs:
target: ${{ matrix.target }}
args: --release --out dist --find-interpreter ${{ env.FEATURES_FLAG }}
sccache: 'true'
working-directory: ./candle-pyo3
working-directory: ./hanzo-ml-pyo3
- name: Upload wheels
uses: actions/upload-artifact@v6
with:
name: wheels-windows-${{ matrix.target }}
path: ./candle-pyo3/dist
path: ./hanzo-ml-pyo3/dist
macos:
runs-on: macos-latest
@@ -96,12 +96,12 @@ jobs:
target: ${{ matrix.target }}
args: --release --out dist --find-interpreter ${{ env.FEATURES_FLAG }}
sccache: 'true'
working-directory: ./candle-pyo3
working-directory: ./hanzo-ml-pyo3
- name: Upload wheels
uses: actions/upload-artifact@v6
with:
name: wheels-macos-${{ matrix.target }}
path: ./candle-pyo3/dist
path: ./hanzo-ml-pyo3/dist
sdist:
runs-on: ubuntu-latest
@@ -117,10 +117,10 @@ jobs:
with:
command: sdist
args: --out dist
working-directory: ./candle-pyo3
working-directory: ./hanzo-ml-pyo3
- name: Upload sdist
uses: actions/upload-artifact@v6
with:
name: wheels-sdist
path: ./candle-pyo3/dist
path: ./hanzo-ml-pyo3/dist
.github/workflows/python.yml
+5 -5
View File
@@ -6,10 +6,10 @@ on:
branches:
- main
paths:
- candle-pyo3/**
- hanzo-ml-pyo3/**
pull_request:
paths:
- candle-pyo3/**
- hanzo-ml-pyo3/**
jobs:
build_and_test:
@@ -44,7 +44,7 @@ jobs:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: Install
working-directory: ./candle-pyo3
working-directory: ./hanzo-ml-pyo3
run: |
python -m venv .env
source .env/bin/activate
@@ -53,14 +53,14 @@ jobs:
python -m maturin develop -r --features onnx
- name: Check style
working-directory: ./candle-pyo3
working-directory: ./hanzo-ml-pyo3
run: |
source .env/bin/activate
python stub.py --check
black --check .
- name: Run tests
working-directory: ./candle-pyo3
working-directory: ./hanzo-ml-pyo3
run: |
source .env/bin/activate
python -m pytest -s -v tests
.gitignore
+8 -8
View File
@@ -31,14 +31,14 @@ flamegraph.svg
*.swo
trace-*.json
candle-wasm-examples/*/build
candle-wasm-examples/*/*.bin
candle-wasm-examples/*/*.jpeg
candle-wasm-examples/*/audios/*.wav
candle-wasm-examples/**/*.safetensors
candle-wasm-examples/**/*.gguf
candle-wasm-examples/*/package-lock.json
candle-wasm-examples/**/config*.json
hanzo-ml-wasm-examples/*/build
hanzo-ml-wasm-examples/*/*.bin
hanzo-ml-wasm-examples/*/*.jpeg
hanzo-ml-wasm-examples/*/audios/*.wav
hanzo-ml-wasm-examples/**/*.safetensors
hanzo-ml-wasm-examples/**/*.gguf
hanzo-ml-wasm-examples/*/package-lock.json
hanzo-ml-wasm-examples/**/config*.json
.DS_Store
.idea/*
__pycache__
CHANGELOG.md
+43 -43
View File
@@ -1,5 +1,5 @@
# Changelog
This documents the main changes to the `candle` crate.
This documents the main changes to the `hanzo-ml` crate.
## v0.3.1 - Unreleased
@@ -12,102 +12,102 @@ This documents the main changes to the `candle` crate.
### Added
- Added the Mistral 7b v0.1 model
[983](https://github.com/huggingface/candle/pull/983).
[983](https://github.com/hanzoai/ml/pull/983).
- Quantized version of the Mistral model
[1009](https://github.com/huggingface/candle/pull/1009).
[1009](https://github.com/hanzoai/ml/pull/1009).
- Add the gelu-erf op and activation function
[969](https://github.com/huggingface/candle/pull/969).
[969](https://github.com/hanzoai/ml/pull/969).
- Add the mixformer/phi-v1.5 model
[930](https://github.com/huggingface/candle/pull/930).
[930](https://github.com/hanzoai/ml/pull/930).
- Add the sclice-scatter op
[927](https://github.com/huggingface/candle/pull/927).
[927](https://github.com/hanzoai/ml/pull/927).
- Add the Wuerstchen diffusion model
[911](https://github.com/huggingface/candle/pull/911).
[911](https://github.com/hanzoai/ml/pull/911).
### Modified
- Support for simd128 intrinsics in some quantized vecdots
[982](https://github.com/huggingface/candle/pull/982).
[982](https://github.com/hanzoai/ml/pull/982).
- Optimize the index-select cuda kernel
[976](https://github.com/huggingface/candle/pull/976).
[976](https://github.com/hanzoai/ml/pull/976).
- Self-contained safetensor wrappers
[946](https://github.com/huggingface/candle/pull/946).
[946](https://github.com/hanzoai/ml/pull/946).
## v0.2.2 - 2023-09-18
### Added
- Support for `top_p` sampling
[819](https://github.com/huggingface/candle/pull/819).
[819](https://github.com/hanzoai/ml/pull/819).
- T5 model including decoding
[864](https://github.com/huggingface/candle/pull/864).
[864](https://github.com/hanzoai/ml/pull/864).
- 1-d upsampling
[839](https://github.com/huggingface/candle/pull/839).
[839](https://github.com/hanzoai/ml/pull/839).
### Modified
- Bugfix for conv2d
[820](https://github.com/huggingface/candle/pull/820).
[820](https://github.com/hanzoai/ml/pull/820).
- Support tensor based indexing using `.i`
[842](https://github.com/huggingface/candle/pull/842).
[842](https://github.com/hanzoai/ml/pull/842).
## v0.2.1 - 2023-09-11
### Added
- Add some RNNs (GRU and LSTM) in `candle-nn`
[674](https://github.com/huggingface/candle/pull/674),
[688](https://github.com/huggingface/candle/pull/688).
- Add some RNNs (GRU and LSTM) in `hanzo-nn`
[674](https://github.com/hanzoai/ml/pull/674),
[688](https://github.com/hanzoai/ml/pull/688).
- gguf v2 support
[725](https://github.com/huggingface/candle/pull/725).
[725](https://github.com/hanzoai/ml/pull/725).
- Quantized llama example in Python using the pyo3 api
[716](https://github.com/huggingface/candle/pull/716).
- `candle-nn` layer for conv2d-transposed
[760](https://github.com/huggingface/candle/pull/760).
[716](https://github.com/hanzoai/ml/pull/716).
- `hanzo-nn` layer for conv2d-transposed
[760](https://github.com/hanzoai/ml/pull/760).
- Add the Segment-Anything Model (SAM) as an example
[773](https://github.com/huggingface/candle/pull/773).
[773](https://github.com/hanzoai/ml/pull/773).
- TinyViT backbone for the segment anything example
[787](https://github.com/huggingface/candle/pull/787).
[787](https://github.com/hanzoai/ml/pull/787).
- Shape with holes support
[770](https://github.com/huggingface/candle/pull/770).
[770](https://github.com/hanzoai/ml/pull/770).
### Modified
- Dilations are now supported in conv-transpose2d.
[671](https://github.com/huggingface/candle/pull/671).
[671](https://github.com/hanzoai/ml/pull/671).
- Interactive mode for the quantized model
[690](https://github.com/huggingface/candle/pull/690).
[690](https://github.com/hanzoai/ml/pull/690).
- Faster softmax operation
[747](https://github.com/huggingface/candle/pull/747).
[747](https://github.com/hanzoai/ml/pull/747).
- Faster convolution operations on CPU and CUDA via im2col
[802](https://github.com/huggingface/candle/pull/802).
[802](https://github.com/hanzoai/ml/pull/802).
- Moving some models to a more central location
[796](https://github.com/huggingface/candle/pull/796).
[796](https://github.com/hanzoai/ml/pull/796).
## v0.2.0 - 2023-08-30
### Added
- Add the powf op
[664](https://github.com/huggingface/candle/pull/664).
[664](https://github.com/hanzoai/ml/pull/664).
- Stable Diffusion XL support
[647](https://github.com/huggingface/candle/pull/647).
[647](https://github.com/hanzoai/ml/pull/647).
- Add the conv-transpose2d op
[635](https://github.com/huggingface/candle/pull/635).
[635](https://github.com/hanzoai/ml/pull/635).
- Refactor the VarBuilder api
[627](https://github.com/huggingface/candle/pull/627).
[627](https://github.com/hanzoai/ml/pull/627).
- Add some quantization command
[625](https://github.com/huggingface/candle/pull/625).
[625](https://github.com/hanzoai/ml/pull/625).
- Support more quantized types, e.g. Q2K, Q4K, Q5K...
[586](https://github.com/huggingface/candle/pull/586).
[586](https://github.com/hanzoai/ml/pull/586).
- Add pose estimation to the yolo example
[589](https://github.com/huggingface/candle/pull/589).
[589](https://github.com/hanzoai/ml/pull/589).
- Api to write GGUF files
[585](https://github.com/huggingface/candle/pull/585).
[585](https://github.com/hanzoai/ml/pull/585).
- Support more quantization types
[580](https://github.com/huggingface/candle/pull/580).
[580](https://github.com/hanzoai/ml/pull/580).
- Add EfficientNet as an example Computer Vision model
[572](https://github.com/huggingface/candle/pull/572).
[572](https://github.com/hanzoai/ml/pull/572).
- Add a group parameter to convolutions
[566](https://github.com/huggingface/candle/pull/566).
[566](https://github.com/hanzoai/ml/pull/566).
- New dtype: int64
[563](https://github.com/huggingface/candle/pull/563).
[563](https://github.com/hanzoai/ml/pull/563).
- Handling of the GGUF file format.
[559](https://github.com/huggingface/candle/pull/559).
[559](https://github.com/hanzoai/ml/pull/559).
## v0.1.2 - 2023-08-21
Cargo.toml
+9 -9
View File
@@ -1,7 +1,7 @@
[workspace]
# Only the hanzo-* crates are workspace members. The legacy candle-* crates are an orphaned
# Only the hanzo-* crates are workspace members. The legacy hanzo-ml-* crates are an orphaned
# duplicate cluster (no hanzo crate depends on them); keeping them out of the build drops the
# `candle` name from the workspace and resolves the candle-pyo3 vs hanzo-ml-pyo3 `links="python"`
# `hanzo-ml` name from the workspace and resolves the hanzo-ml-pyo3 vs hanzo-ml-pyo3 `links="python"`
# conflict. The dirs remain on disk for now and can be deleted in a follow-up.
members = [
"hanzo-datasets",
@@ -15,13 +15,13 @@ members = [
"tensor-tools",
]
exclude = [
"candle-book",
"candle-flash-attn",
"candle-flash-attn-v3",
"candle-kernels",
"candle-metal-kernels",
"hanzo-ml-book",
"hanzo-flash-attn",
"hanzo-flash-attn-v3",
"hanzo-kernels",
"hanzo-metal-kernels",
"hanzo-rocm-kernels",
"candle-onnx",
"hanzo-onnx",
"hanzo-flash-attn",
"hanzo-kernels",
"hanzo-metal-kernels",
@@ -36,7 +36,7 @@ resolver = "2"
version = "0.9.2"
edition = "2021"
description = "Minimalist ML framework."
repository = "https://github.com/huggingface/candle"
repository = "https://github.com/hanzoai/ml"
keywords = ["blas", "tensor", "machine-learning"]
categories = ["science"]
license = "MIT OR Apache-2.0"
HANZO_INTEGRATION.md
+4 -4
View File
@@ -2,14 +2,14 @@
## Overview
Hanzo ML is the official ML framework for the Hanzo ecosystem, based on Candle from Hugging Face with optimizations for edge AI, multimodal workloads, and integration with Hanzo Engine.
Hanzo ML is the official ML framework for the Hanzo ecosystem, based on Hanzo from Hugging Face with optimizations for edge AI, multimodal workloads, and integration with Hanzo Engine.
## Integration with Hanzo Engine
### Repository Structure
```
~/work/hanzo/
├── ml/ # hanzoai/ml - ML framework (based on HF candle)
├── ml/ # hanzoai/ml - ML framework (based on HF hanzo-ml)
├── engine/ # hanzoai/engine - Inference engine (based on mistral-rs)
├── jin/ # Jin multimodal models
└── llm/ # LLM Gateway proxy
@@ -70,7 +70,7 @@ Both frameworks support:
```bash
cd ~/work/hanzo/ml
git fetch upstream
git merge upstream/main # Merge HF candle updates
git merge upstream/main # Merge HF hanzo-ml updates
cargo test --workspace
git push origin main
```
@@ -113,7 +113,7 @@ git push --tags
## Sync Status
### Latest Upstream Sync
- **HF Candle**: `a2029da3` (Jan 2025)
- **HF Hanzo**: `a2029da3` (Jan 2025)
- **Features Added**: SmolLM3, Qwen3 WASM, Mamba2, PaddleOCR-VL
### Engine Integration Status
LLM.md
+4 -4
View File
@@ -26,10 +26,10 @@ ml/
LLM.md
Makefile
README.md
candle-book/
candle-core/
candle-datasets/
candle-examples/
hanzo-ml-book/
hanzo-ml/
hanzo-datasets/
hanzo-ml-examples/
```
## Key Files
Makefile
+4 -4
View File
@@ -2,10 +2,10 @@
clean-ptx:
find target -name "*.ptx" -type f -delete
echo "" > candle-kernels/src/lib.rs
touch candle-kernels/build.rs
touch candle-examples/build.rs
touch candle-flash-attn/build.rs
echo "" > hanzo-kernels/src/lib.rs
touch hanzo-kernels/build.rs
touch hanzo-ml-examples/build.rs
touch hanzo-flash-attn/build.rs
clean:
cargo clean
README.md
+5 -5
View File
@@ -5,7 +5,7 @@
[![License](https://img.shields.io/badge/license-Apache%202.0-blue?style=flat-square)](https://github.com/hanzoai/ml/blob/main/LICENSE-APACHE)
Hanzo ML is a minimalist ML framework for Rust with a focus on performance (including GPU support)
and ease of use. Based on Candle from Hugging Face, Hanzo ML provides optimizations for Edge AI,
and ease of use. Based on Hanzo from Hugging Face, Hanzo ML provides optimizations for Edge AI,
quantization, and multimodal workloads.
## Key Features
@@ -177,18 +177,18 @@ And then head over to
## Useful External Resources
- [`hanzo-ml-tutorial`](https://github.com/ToluClassics/hanzo-ml-tutorial): A
very detailed tutorial showing how to convert a PyTorch model to ML.
- [`hanzo-ml-lora`](https://github.com/EricLBuehler/hanzo-ml-lora): Efficient and
- [`hanzo-ml-lora`](https://github.com/hanzoai/hanzo-ml-lora): Efficient and
ergonomic LoRA implementation for ML. `hanzo-ml-lora` has
out-of-the-box LoRA support for many models from ML, which can be found
[here](https://github.com/EricLBuehler/hanzo-ml-lora/tree/master/hanzo-ml-lora-transformers/examples).
[here](https://github.com/hanzoai/hanzo-ml-lora/tree/master/hanzo-ml-lora-transformers/examples).
- [`optimisers`](https://github.com/KGrewal1/optimisers): A collection of optimisers
including SGD with momentum, AdaGrad, AdaDelta, AdaMax, NAdam, RAdam, and RMSprop.
- [`hanzo-ml-vllm`](https://github.com/EricLBuehler/hanzo-ml-vllm): Efficient platform for inference and
- [`hanzo-ml-vllm`](https://github.com/hanzoai/hanzo-ml-vllm): Efficient platform for inference and
serving local LLMs including an OpenAI compatible API server.
- [`hanzo-ml-ext`](https://github.com/mokeyish/hanzo-ml-ext): An extension library to ML that provides PyTorch functions not currently available in ML.
- [`hanzo-ml-coursera-ml`](https://github.com/vishpat/hanzo-ml-coursera-ml): Implementation of ML algorithms from Coursera's [Machine Learning Specialization](https://www.coursera.org/specializations/machine-learning-introduction) course.
- [`kalosm`](https://github.com/floneum/floneum/tree/master/interfaces/kalosm): A multi-modal meta-framework in Rust for interfacing with local pre-trained models with support for controlled generation, custom samplers, in-memory vector databases, audio transcription, and more.
- [`hanzo-ml-sampling`](https://github.com/EricLBuehler/hanzo-ml-sampling): Sampling techniques for ML.
- [`hanzo-ml-sampling`](https://github.com/hanzoai/hanzo-ml-sampling): Sampling techniques for ML.
- [`gpt-from-scratch-rs`](https://github.com/jeroenvlek/gpt-from-scratch-rs): A port of Andrej Karpathy's _Let's build GPT_ tutorial on YouTube showcasing the ML API on a toy problem.
- [`hanzo-ml-einops`](https://github.com/tomsanbear/hanzo-ml-einops): A pure rust implementation of the python [einops](https://github.com/arogozhnikov/einops) library.
- [`atoma-infer`](https://github.com/atoma-network/atoma-infer): A Rust library for fast inference at scale, leveraging FlashAttention2 for efficient attention computation, PagedAttention for efficient KV-cache memory management, and multi-GPU support. It is OpenAI api compatible.
hanzo-datasets/README.md
+1 -1
View File
@@ -1 +1 @@
# candle-datasets
# hanzo-datasets
hanzo-datasets/src/lib.rs
+1 -1
View File
@@ -1,4 +1,4 @@
//! Datasets & Dataloaders for Candle
//! Datasets & Dataloaders for Hanzo
pub mod batcher;
pub mod hub;
pub mod nlp;
hanzo-datasets/src/vision/cifar.rs
+1 -1
View File
@@ -83,7 +83,7 @@ fn load_parquet(parquet: SerializedFileReader<std::fs::File>) -> Result<(Tensor,
}
}
}
// Reorder image-rs convention (width, height, channels) to candle/pytorch convolution convention (channels, height, width)
// Reorder image-rs convention (width, height, channels) to hanzo-ml/pytorch convolution convention (channels, height, width)
let images = (Tensor::from_vec(buffer_images, (samples, 32, 32, 3), &Device::Cpu)?
.to_dtype(DType::F32)?
.permute((0, 3, 2, 1))?
hanzo-flash-attn/README.md
+1 -1
View File
@@ -1 +1 @@
# candle-flash-attn
# hanzo-flash-attn
hanzo-flash-attn/build.rs
+1 -1
View File
@@ -2,7 +2,7 @@
// The cuda build time is very long so one can set the CANDLE_FLASH_ATTN_BUILD_DIR environment
// variable in order to cache the compiled artifacts and avoid recompiling too often.
use anyhow::{Context, Result};
use candle_flash_attn_build::{cutlass_include_arg, fetch_cutlass};
use hanzo_flash_attn_build::{cutlass_include_arg, fetch_cutlass};
use std::path::PathBuf;
const CUTLASS_COMMIT: &str = "7d49e6c7e2f8896c47f586706e67e1fb215529dc";
hanzo-flash-attn/tests/flash_attn_tests.rs
+6 -6
View File
@@ -1,5 +1,5 @@
use anyhow::Result;
use candle::{DType, Device, IndexOp, Tensor, D};
use hanzo-ml::{DType, Device, IndexOp, Tensor, D};
fn to_vec3_round(t: Tensor, digits: i32) -> Result<Vec<Vec<Vec<f32>>>> {
let b = 10f32.powi(digits);
@@ -21,7 +21,7 @@ fn fa_acausal(q: &Tensor, k: &Tensor, v: &Tensor, softmax_scale: f32) -> Result<
let k = k.to_dtype(DType::F32)?;
let v = v.to_dtype(DType::F32)?;
let att = (q.matmul(&k.t()?)? * softmax_scale as f64)?;
let att = candle_nn::ops::softmax(&att, D::Minus1)?;
let att = hanzo_nn::ops::softmax(&att, D::Minus1)?;
// Convert to contiguous as matmul doesn't support strided vs for now.
let output = att.matmul(&v.contiguous()?)?.to_dtype(in_dtype)?;
Ok(output)
@@ -35,7 +35,7 @@ fn fa_acausal_softcap(q: &Tensor, k: &Tensor, v: &Tensor, softcap: f32) -> Resul
// let att = (q.matmul(&k.t()?)? * softmax_scale as f64)?;
let att = q.matmul(&k.t()?)?;
let att = (softcap as f64 * ((att / softcap as f64)?.tanh())?)?;
let att = candle_nn::ops::softmax(&att, D::Minus1)?;
let att = hanzo_nn::ops::softmax(&att, D::Minus1)?;
// Convert to contiguous as matmul doesn't support strided vs for now.
let output = att.matmul(&v.contiguous()?)?.to_dtype(in_dtype)?;
Ok(output)
@@ -57,7 +57,7 @@ fn flash_attn_acausal() -> Result<()> {
let q = q.transpose(1, 2)?;
let k = k.transpose(1, 2)?;
let v = v.transpose(1, 2)?;
candle_flash_attn::flash_attn(&q, &k, &v, 0.5, false)?.transpose(1, 2)?
hanzo_flash_attn::flash_attn(&q, &k, &v, 0.5, false)?.transpose(1, 2)?
};
let ys2 = ys2.i(0)?.to_dtype(DType::F32)?;
let diff = ys1.sub(&ys2)?.abs()?.flatten_all()?.max(0)?;
@@ -120,7 +120,7 @@ fn flash_attn_acausal_softcap() -> Result<()> {
let q = q.transpose(1, 2)?;
let k = k.transpose(1, 2)?;
let v = v.transpose(1, 2)?;
candle_flash_attn::flash_attn_alibi_windowed_softcap(
hanzo_flash_attn::flash_attn_alibi_windowed_softcap(
&q,
&k,
&v,
@@ -158,7 +158,7 @@ fn flash_attn_varlen() -> Result<()> {
let q = q.transpose(0, 1)?;
let k = k.transpose(0, 1)?;
let v = v.transpose(0, 1)?;
candle_flash_attn::flash_attn_varlen(
hanzo_flash_attn::flash_attn_varlen(
&q, &k, &v, &seqlens_q, &seqlens_k, 32, 32, 0.5, false,
)?
.transpose(0, 1)?
hanzo-kernels/README.md
+2 -2
View File
@@ -1,4 +1,4 @@
# candle-kernels
# hanzo-kernels
This crate contains CUDA kernels used from candle. Some of these implementations
This crate contains CUDA kernels used from hanzo-ml. Some of these implementations
come from the [dfdx crate](https://github.com/coreylowman/dfdx).
hanzo-metal-kernels/README.md
+2 -2
View File
@@ -1,3 +1,3 @@
# candle-metal-kernels
# hanzo-metal-kernels
This crate contains Metal kernels used from candle.
This crate contains Metal kernels used from hanzo-ml.
hanzo-metal-kernels/examples/metal_benchmarks.rs
+3 -3
View File
@@ -1,5 +1,5 @@
use anyhow::Result;
use candle_metal_kernels::{
use hanzo_metal_kernels::{
metal::{create_command_buffer, CommandSemaphore, Device},
GemmDType, RESOURCE_OPTIONS,
};
@@ -15,7 +15,7 @@ fn run_gemm(f32: bool, n: usize) -> Result<()> {
let device = Device::system_default().unwrap();
let (b, m, n, k) = (1, n, n, n);
let kernels = candle_metal_kernels::Kernels::new();
let kernels = hanzo_metal_kernels::Kernels::new();
let command_queue = device.new_command_queue().unwrap();
let options = RESOURCE_OPTIONS;
@@ -69,7 +69,7 @@ fn run_gemm(f32: bool, n: usize) -> Result<()> {
let semaphore = Arc::new(CommandSemaphore::new());
let command_buffer = create_command_buffer(&command_queue, semaphore).unwrap();
let start_time = std::time::Instant::now();
candle_metal_kernels::call_mlx_gemm(
hanzo_metal_kernels::call_mlx_gemm(
&device,
&command_buffer,
&kernels,
hanzo-ml/build.rs
+1 -1
View File
@@ -1,4 +1,4 @@
// Build script for candle-core.
// Build script for hanzo-ml.
//
// Vulkan backend: provides every GLSL compute kernel under
// `src/vulkan/shaders/*.comp` to the backend as `$OUT_DIR/<name>.spv`, which
hanzo-ml/src/backprop.rs
+1 -1
View File
@@ -173,7 +173,7 @@ impl Tensor {
let grad = grads
.remove(node)
.expect("hanzo internal error - grad not populated");
// https://github.com/huggingface/candle/issues/1241
// https://github.com/hanzoai/ml/issues/1241
// Ideally, we would make these operations in place where possible to ensure that we
// do not have to allocate too often. Here we just call `.detach` to avoid computing
// the backprop graph of the backprop itself. This would be an issue for second order
hanzo-ml/src/cuda_backend/cudnn.rs
+20 -20
View File
@@ -35,7 +35,7 @@ pub(crate) fn launch_conv2d<
params: &crate::conv::ParamsConv2D,
dev: &crate::cuda_backend::CudaDevice,
) -> crate::Result<()> {
use crate::conv::CudnnFwdAlgo as CandleAlgo;
use crate::conv::CudnnFwdAlgo as HanzoAlgo;
use cudarc::cudnn::sys::cudnnConvolutionFwdAlgo_t as A;
let device_id = dev.id();
@@ -96,17 +96,17 @@ pub(crate) fn launch_conv2d<
};
let alg = match params.cudnn_fwd_algo {
None => conv2d.pick_algorithm()?,
Some(CandleAlgo::ImplicitGemm) => A::CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_GEMM,
Some(CandleAlgo::ImplicitPrecompGemm) => {
Some(HanzoAlgo::ImplicitGemm) => A::CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_GEMM,
Some(HanzoAlgo::ImplicitPrecompGemm) => {
A::CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_PRECOMP_GEMM
}
Some(CandleAlgo::Gemm) => A::CUDNN_CONVOLUTION_FWD_ALGO_GEMM,
Some(CandleAlgo::Direct) => A::CUDNN_CONVOLUTION_FWD_ALGO_DIRECT,
Some(CandleAlgo::Fft) => A::CUDNN_CONVOLUTION_FWD_ALGO_FFT,
Some(CandleAlgo::FftTiling) => A::CUDNN_CONVOLUTION_FWD_ALGO_FFT_TILING,
Some(CandleAlgo::Winograd) => A::CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD,
Some(CandleAlgo::WinogradNonFused) => A::CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD_NONFUSED,
Some(CandleAlgo::Count) => A::CUDNN_CONVOLUTION_FWD_ALGO_COUNT,
Some(HanzoAlgo::Gemm) => A::CUDNN_CONVOLUTION_FWD_ALGO_GEMM,
Some(HanzoAlgo::Direct) => A::CUDNN_CONVOLUTION_FWD_ALGO_DIRECT,
Some(HanzoAlgo::Fft) => A::CUDNN_CONVOLUTION_FWD_ALGO_FFT,
Some(HanzoAlgo::FftTiling) => A::CUDNN_CONVOLUTION_FWD_ALGO_FFT_TILING,
Some(HanzoAlgo::Winograd) => A::CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD,
Some(HanzoAlgo::WinogradNonFused) => A::CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD_NONFUSED,
Some(HanzoAlgo::Count) => A::CUDNN_CONVOLUTION_FWD_ALGO_COUNT,
};
let workspace_size = conv2d.get_workspace_size(alg)?;
let mut workspace = dev.cuda_stream().alloc_zeros::<u8>(workspace_size)?;
@@ -134,7 +134,7 @@ pub(crate) fn launch_conv1d<
params: &crate::conv::ParamsConv1D,
dev: &crate::cuda_backend::CudaDevice,
) -> crate::Result<()> {
use crate::conv::CudnnFwdAlgo as CandleAlgo;
use crate::conv::CudnnFwdAlgo as HanzoAlgo;
use cudarc::cudnn::sys::cudnnConvolutionFwdAlgo_t as A;
let device_id = dev.id();
@@ -197,17 +197,17 @@ pub(crate) fn launch_conv1d<
};
let alg = match params.cudnn_fwd_algo {
None => conv1d.pick_algorithm()?,
Some(CandleAlgo::ImplicitGemm) => A::CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_GEMM,
Some(CandleAlgo::ImplicitPrecompGemm) => {
Some(HanzoAlgo::ImplicitGemm) => A::CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_GEMM,
Some(HanzoAlgo::ImplicitPrecompGemm) => {
A::CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_PRECOMP_GEMM
}
Some(CandleAlgo::Gemm) => A::CUDNN_CONVOLUTION_FWD_ALGO_GEMM,
Some(CandleAlgo::Direct) => A::CUDNN_CONVOLUTION_FWD_ALGO_DIRECT,
Some(CandleAlgo::Fft) => A::CUDNN_CONVOLUTION_FWD_ALGO_FFT,
Some(CandleAlgo::FftTiling) => A::CUDNN_CONVOLUTION_FWD_ALGO_FFT_TILING,
Some(CandleAlgo::Winograd) => A::CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD,
Some(CandleAlgo::WinogradNonFused) => A::CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD_NONFUSED,
Some(CandleAlgo::Count) => A::CUDNN_CONVOLUTION_FWD_ALGO_COUNT,
Some(HanzoAlgo::Gemm) => A::CUDNN_CONVOLUTION_FWD_ALGO_GEMM,
Some(HanzoAlgo::Direct) => A::CUDNN_CONVOLUTION_FWD_ALGO_DIRECT,
Some(HanzoAlgo::Fft) => A::CUDNN_CONVOLUTION_FWD_ALGO_FFT,
Some(HanzoAlgo::FftTiling) => A::CUDNN_CONVOLUTION_FWD_ALGO_FFT_TILING,
Some(HanzoAlgo::Winograd) => A::CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD,
Some(HanzoAlgo::WinogradNonFused) => A::CUDNN_CONVOLUTION_FWD_ALGO_WINOGRAD_NONFUSED,
Some(HanzoAlgo::Count) => A::CUDNN_CONVOLUTION_FWD_ALGO_COUNT,
};
let workspace_size = conv1d.get_workspace_size(alg)?;
let mut workspace = dev.cuda_stream().alloc_zeros::<u8>(workspace_size)?;
hanzo-ml/src/cuda_backend/device.rs
+1 -1
View File
@@ -426,7 +426,7 @@ impl BackendDevice for CudaDevice {
let elem_count = shape.elem_count();
let curand = self.curand.lock().unwrap();
// curand can only generate an odd number of values.
// https://github.com/huggingface/candle/issues/734
// https://github.com/hanzoai/ml/issues/734
let elem_count_round = if elem_count % 2 == 1 {
elem_count + 1
} else {
hanzo-ml/src/error.rs
+1 -1
View File
@@ -1,4 +1,4 @@
//! Candle-specific Error and Result
//! Hanzo-specific Error and Result
use std::{convert::Infallible, fmt::Display};
use crate::{DType, DeviceLocation, Layout, MetalError, Shape};
hanzo-ml/src/lib.rs
+10 -10
View File
@@ -23,9 +23,9 @@
//!
//! ## FAQ
//!
//! - Why Candle?
//! - Why Hanzo?
//!
//! Candle stems from the need to reduce binary size in order to *enable serverless*
//! Hanzo stems from the need to reduce binary size in order to *enable serverless*
//! possible by making the whole engine smaller than PyTorch very large library volume
//!
//! And simply *removing Python* from production workloads.
@@ -35,16 +35,16 @@
//!
//! ## Other Crates
//!
//! Candle consists of a number of crates. This crate holds core the common data structures but you may wish
//! Hanzo consists of a number of crates. This crate holds core the common data structures but you may wish
//! to look at the docs for the other crates which can be found here:
//!
//! - [candle-core](https://docs.rs/candle-core/). Core Datastructures and DataTypes.
//! - [candle-nn](https://docs.rs/candle-nn/). Building blocks for Neural Nets.
//! - [candle-datasets](https://docs.rs/candle-datasets/). Rust access to commonly used Datasets like MNIST.
//! - [candle-examples](https://docs.rs/candle-examples/). Examples of Candle in Use.
//! - [candle-onnx](https://docs.rs/candle-onnx/). Loading and using ONNX models.
//! - [candle-pyo3](https://docs.rs/candle-pyo3/). Access to Candle from Python.
//! - [candle-transformers](https://docs.rs/candle-transformers/). Candle implementation of many published transformer models.
//! - [hanzo-ml](https://docs.rs/hanzo-ml/). Core Datastructures and DataTypes.
//! - [hanzo-nn](https://docs.rs/hanzo-nn/). Building blocks for Neural Nets.
//! - [hanzo-datasets](https://docs.rs/hanzo-datasets/). Rust access to commonly used Datasets like MNIST.
//! - [hanzo-ml-examples](https://docs.rs/hanzo-ml-examples/). Examples of Hanzo in Use.
//! - [hanzo-onnx](https://docs.rs/hanzo-onnx/). Loading and using ONNX models.
//! - [hanzo-ml-pyo3](https://docs.rs/hanzo-ml-pyo3/). Access to Hanzo from Python.
//! - [hanzo-transformers](https://docs.rs/hanzo-transformers/). Hanzo implementation of many published transformer models.
//!
#[cfg(feature = "accelerate")]
hanzo-ml/src/op.rs
+1 -1
View File
@@ -512,7 +512,7 @@ unary_op!(Sqr, "sqr", v, v * v, vs_sqr, vd_sqr);
unary_op!(Sqrt, "sqrt", v, v.sqrt(), vs_sqrt, vd_sqrt);
// Hardcode the value for sqrt(2/pi)
// https://github.com/huggingface/candle/issues/1982
// https://github.com/hanzoai/ml/issues/1982
#[allow(clippy::excessive_precision)]
const SQRT_TWO_OVER_PI_F32: f32 = 0.79788456080286535587989211986876373;
#[allow(clippy::excessive_precision)]
hanzo-ml/src/quantized/metal.rs
+1 -1
View File
@@ -206,7 +206,7 @@ impl QMetalStorage {
let mut dst_shape = src_shape.dims().to_vec();
// We always use a single batch dimension and stack all the tensors in the batch on the
// second dimension as the implementation in candle-metal-kernels doesn't handle batch
// second dimension as the implementation in hanzo-metal-kernels doesn't handle batch
// properly.
let m = match dst_shape.len() {
3 => dst_shape[0] * dst_shape[1],
hanzo-ml/src/quantized/mod.rs
+2 -2
View File
@@ -668,7 +668,7 @@ impl QTensor {
pub fn dequantize_f16(&self, device: &Device) -> Result<Tensor> {
// In the CUDA case, we have a specialized kernel as this can be useful for volta
// architectures. https://github.com/huggingface/candle/issues/2136
// architectures. https://github.com/hanzoai/ml/issues/2136
match &self.storage {
QStorage::Cuda(s) => {
let s = s.dequantize_f16(self.shape.elem_count())?;
@@ -716,7 +716,7 @@ impl QTensor {
_ => {
// CPU / non-CUDA fallback: per-expert quantized matmul. The packed expert bank
// [E, n, k] is sliced into equal, contiguous per-expert quantized blocks; for
// each expert that is actually selected we run candle's native quantized matmul
// each expert that is actually selected we run hanzo-ml's native quantized matmul
// on just the tokens routed to it. Nothing is dequantized, so quantized MoE runs
// on any backend (CPU, Metal, ...) at a cost proportional to the active experts.
use crate::Module; // brings QMatMul::forward into scope
hanzo-ml/src/rocm_backend/device.rs
+2 -2
View File
@@ -130,13 +130,13 @@ impl RocmDevice {
}
/// Get a reference to the underlying HIP stream.
/// This is public so that candle-nn and other crates can launch custom kernels.
/// This is public so that hanzo-nn and other crates can launch custom kernels.
pub fn stream(&self) -> &rocm_rs::hip::Stream {
&self.stream.0
}
/// Get or load a kernel function from the cache.
/// This is public so that candle-nn and other crates can launch custom kernels.
/// This is public so that hanzo-nn and other crates can launch custom kernels.
pub fn get_or_load_func(
&self,
kernel_name: &'static str,
hanzo-ml/src/rocm_backend/utils.rs
+1 -1
View File
@@ -1,4 +1,4 @@
//! Helper functions to plug ROCm kernels in candle.
//! Helper functions to plug ROCm kernels in hanzo-ml.
use crate::{Layout, Result};
use super::wrappers::SendSyncDeviceMemory;
hanzo-ml/src/rocm_backend/wrappers.rs
+1 -1
View File
@@ -14,7 +14,7 @@ use rocm_rs::rocrand::PseudoRng;
/// call on HIP — it blocks the CPU until all pending GPU work completes. Without
/// pooling, every tensor op (each allocates its output) silently serializes the
/// stream, so the GPU sits ~95% idle waiting on the CPU: the WSL-bridge ~1 tok/s
/// killer. This mirrors candle-cuda's caching allocator. Buffers are returned to
/// killer. This mirrors hanzo-ml-cuda's caching allocator. Buffers are returned to
/// the pool on Drop instead of being freed.
pub type DevicePool = Arc<Mutex<HashMap<usize, Vec<usize>>>>;
hanzo-ml/src/vulkan/shaders/cmp.comp
+1 -1
View File
@@ -1,5 +1,5 @@
#version 450
// Elementwise compare -> u32 {0,1}. op: 0=Eq 1=Ne 2=Le 3=Ge 4=Lt 5=Gt (candle CmpOp order).
// Elementwise compare -> u32 {0,1}. op: 0=Eq 1=Ne 2=Le 3=Ge 4=Lt 5=Gt (hanzo-ml CmpOp order).
layout(local_size_x = 64) in;
layout(set = 0, binding = 0) readonly buffer A { float a[]; };
layout(set = 0, binding = 1) readonly buffer B { float b[]; };
hanzo-ml/src/vulkan/shaders/gelu.comp
+1 -1
View File
@@ -1,5 +1,5 @@
#version 450
// Unary GELU (tanh approximation, matching candle's CPU Gelu):
// Unary GELU (tanh approximation, matching hanzo-ml's CPU Gelu):
// 0.5 * x * (1 + tanh(sqrt(2/pi) * (x + 0.044715 * x^3)))
layout(local_size_x = 64) in;
hanzo-ml/src/vulkan/shaders/gelu_erf.comp
+1 -1
View File
@@ -1,5 +1,5 @@
#version 450
// Exact GELU: out = 0.5 * x * (1 + erf(x / sqrt(2))). (candle "gelu_erf")
// Exact GELU: out = 0.5 * x * (1 + erf(x / sqrt(2))). (hanzo-ml "gelu_erf")
layout(local_size_x = 64) in;
layout(set = 0, binding = 0) readonly buffer In { float inp[]; };
layout(set = 0, binding = 1) writeonly buffer Out { float o[]; };
hanzo-ml/src/vulkan/shaders/rms_norm.comp
+1 -1
View File
@@ -1,6 +1,6 @@
#version 450
// RMS norm over the last dim: y = x / sqrt(mean(x^2) + eps) * alpha. One invocation per row.
// Matches candle CPU rms-norm: m = sqrt(sum(x^2)/dim + eps); y = x / m * alpha.
// Matches hanzo-ml CPU rms-norm: m = sqrt(sum(x^2)/dim + eps); y = x / m * alpha.
layout(local_size_x = 64, local_size_y = 1, local_size_z = 1) in;
layout(set = 0, binding = 0) readonly buffer X { float x[]; };
layout(set = 0, binding = 1) readonly buffer A { float alpha[]; };
hanzo-ml/src/vulkan/shaders/rope.comp
+1 -1
View File
@@ -1,5 +1,5 @@
#version 450
// GPT-NeoX style rotary embedding (candle "rotary-emb"). src is [b,h,t,d] contiguous;
// GPT-NeoX style rotary embedding (hanzo-ml "rotary-emb"). src is [b,h,t,d] contiguous;
// cos/sin are [t,d/2] (unbatched) or [b,t,d/2]. One invocation per (bh, t, d/2) triple.
// dst[i1] = src[i1]*cos - src[i2]*sin ; dst[i2] = src[i1]*sin + src[i2]*cos
// with i1 = i_t*d + i_d, i2 = i1 + d/2, cs index = i_t*(d/2)+i_d (+ b_i*t*d/2 if unbatched).
hanzo-ml/src/vulkan/shaders/scatter_set.comp
+1 -1
View File
@@ -1,7 +1,7 @@
#version 450
// scatter set along `dim`. ids/src share a shape; for src flat index g = (outer, j, inner)
// with inner < right and j < dim_src, write dst[outer*(dim_dst*right) + ids[g]*right + inner] = src[g].
// (Duplicate ids -> last writer wins, which is unspecified but matches candle's contract.)
// (Duplicate ids -> last writer wins, which is unspecified but matches hanzo-ml's contract.)
layout(local_size_x = 64) in;
layout(set = 0, binding = 0) buffer Dst { float dst[]; };
layout(set = 0, binding = 1) readonly buffer Src { float src[]; };
hanzo-ml/src/vulkan/shaders/upsample_nearest2d.comp
+1 -1
View File
@@ -1,6 +1,6 @@
#version 450
// upsample_nearest2d. input [b,c,ih,iw], out [b,c,oh,ow].
// src idx = min(in-1, (out_idx*in)/out) per axis (matches candle's floor(out*scale)).
// src idx = min(in-1, (out_idx*in)/out) per axis (matches hanzo-ml's floor(out*scale)).
layout(local_size_x = 64) in;
layout(set = 0, binding = 0) readonly buffer In { float inp[]; };
layout(set = 0, binding = 1) writeonly buffer Out { float o[]; };
hanzo-ml/src/vulkan_backend.rs
+3 -3
View File
@@ -13,7 +13,7 @@ use std::collections::HashMap;
use std::ffi::CStr;
use std::sync::{Arc, Mutex};
// --- SPIR-V kernels (compiled by candle-core build.rs from src/vulkan/shaders/*.comp) ---
// --- SPIR-V kernels (compiled by hanzo-ml build.rs from src/vulkan/shaders/*.comp) ---
macro_rules! spv {
($name:literal) => {
include_bytes!(concat!(env!("OUT_DIR"), "/", $name, ".spv"))
@@ -971,7 +971,7 @@ impl VulkanStorage {
Ok(out)
}
// rms-norm over the last dim. `self`=x, `alpha`=scale [m]. Matches candle rms-norm.
// rms-norm over the last dim. `self`=x, `alpha`=scale [m]. Matches hanzo-ml rms-norm.
pub fn rms_norm(
&self,
layout: &Layout,
@@ -1298,7 +1298,7 @@ impl BackendStorage for VulkanStorage {
return self.device.storage_from_cpu_storage(&r);
}
};
// candle pre-broadcasts both layouts to the output shape (possibly with stride-0 dims);
// hanzo-ml pre-broadcasts both layouts to the output shape (possibly with stride-0 dims);
// broadcast layouts aren't contiguous so contig_buf still materializes them, but
// same-shape contiguous operands skip the copy (one fewer dispatch each).
let mut lk = None;
hanzo-nn/src/cpu_flash_attention.rs
+1 -1
View File
@@ -336,7 +336,7 @@ fn flash_attn_cpu_single_q<T: WithDType + Sum + num_traits::real::Real>(
}
/// Main forward flash-attention CPU routine.
/// Shapes follow Candle convention: (B, S, H, D)
/// Shapes follow Hanzo convention: (B, S, H, D)
#[allow(clippy::too_many_arguments)]
fn flash_attn_cpu<T: WithDType + Sum + num_traits::real::Real>(
q_data: &[T],
hanzo-nn/src/kv_cache.rs
+1 -1
View File
@@ -569,7 +569,7 @@ impl ScatteredCacheBuilder {
}
}
// Flattening the attention mask then using Tensor::from_vec rather using Tensor::new ends
// up being almost 10x faster with candle 0.9.0. This has been fixed in candle 0.9.1.
// up being almost 10x faster with hanzo-ml 0.9.0. This has been fixed in hanzo-ml 0.9.1.
let attention_masks = attention_masks
.into_iter()
.flat_map(|m| m.into_iter().flatten())
hanzo-nn/src/lib.rs
+9 -9
View File
@@ -1,18 +1,18 @@
//! candle-nn
//! hanzo-nn
//!
//! ## Other Crates
//!
//! Candle consists of a number of crates. This crate holds structs and functions
//! Hanzo consists of a number of crates. This crate holds structs and functions
//! that allow you to build and train neural nets. You may wish
//! to look at the docs for the other crates which can be found here:
//!
//! - [candle-core](https://docs.rs/candle-core/). Core Datastructures and DataTypes.
//! - [candle-nn](https://docs.rs/candle-nn/). Building blocks for Neural Nets.
//! - [candle-datasets](https://docs.rs/candle-datasets/). Rust access to commonly used Datasets like MNIST.
//! - [candle-examples](https://docs.rs/candle-examples/). Examples of Candle in Use.
//! - [candle-onnx](https://docs.rs/candle-onnx/). Loading and using ONNX models.
//! - [candle-pyo3](https://docs.rs/candle-pyo3/). Access to Candle from Python.
//! - [candle-transformers](https://docs.rs/candle-transformers/). Candle implementation of many published transformer models.
//! - [hanzo-ml](https://docs.rs/hanzo-ml/). Core Datastructures and DataTypes.
//! - [hanzo-nn](https://docs.rs/hanzo-nn/). Building blocks for Neural Nets.
//! - [hanzo-datasets](https://docs.rs/hanzo-datasets/). Rust access to commonly used Datasets like MNIST.
//! - [hanzo-ml-examples](https://docs.rs/hanzo-ml-examples/). Examples of Hanzo in Use.
//! - [hanzo-onnx](https://docs.rs/hanzo-onnx/). Loading and using ONNX models.
//! - [hanzo-ml-pyo3](https://docs.rs/hanzo-ml-pyo3/). Access to Hanzo from Python.
//! - [hanzo-transformers](https://docs.rs/hanzo-transformers/). Hanzo implementation of many published transformer models.
//!
pub mod activation;
hanzo-onnx/README.md
+6 -6
View File
@@ -1,17 +1,17 @@
# candle-onnx
# hanzo-onnx
This crate adds ONNX support to candle
This crate adds ONNX support to hanzo-ml
## FAQ
#### Missing protoc installation when compiling candle-onnx
#### Missing protoc installation when compiling hanzo-onnx
The candle-onnx dependency prost-build no longer comes bundled with prost
The hanzo-onnx dependency prost-build no longer comes bundled with prost
binaries. This could cause the following error when attempting to compile
candle-onnx:
hanzo-onnx:
```
error: failed to run custom build command for `candle-onnx`
error: failed to run custom build command for `hanzo-onnx`
Caused by: // (...)
Could not find `protoc` installation and this build crate cannot proceed without this knowledge.
```
hanzo-onnx/src/eval.rs
+1 -1
View File
@@ -655,7 +655,7 @@ fn simple_eval_(
};
// In Pytorch or Numpy this can be done by indexing the xs tensor using the indices
// tensor directly, but candle does not support tensor indexing at the moment, so
// tensor directly, but hanzo-ml does not support tensor indexing at the moment, so
// some workarounds must be done.
let xs = match indices.dims() {
[] => {
hanzo-onnx/tests/ops.rs
+84 -84
View File
@@ -1,11 +1,11 @@
use candle::test_utils::to_vec2_round;
use candle::{DType, Device, NdArray, Result, Tensor};
use candle_onnx::onnx::attribute_proto::AttributeType;
use candle_onnx::onnx::tensor_proto::DataType;
use candle_onnx::onnx::tensor_shape_proto::{dimension, Dimension};
use candle_onnx::onnx::{type_proto, TensorProto, TensorShapeProto, TypeProto};
use candle_onnx::onnx::{AttributeProto, GraphProto, ModelProto, NodeProto, ValueInfoProto};
use candle_onnx::simple_eval;
use hanzo-ml::test_utils::to_vec2_round;
use hanzo-ml::{DType, Device, NdArray, Result, Tensor};
use hanzo_onnx::onnx::attribute_proto::AttributeType;
use hanzo_onnx::onnx::tensor_proto::DataType;
use hanzo_onnx::onnx::tensor_shape_proto::{dimension, Dimension};
use hanzo_onnx::onnx::{type_proto, TensorProto, TensorShapeProto, TypeProto};
use hanzo_onnx::onnx::{AttributeProto, GraphProto, ModelProto, NodeProto, ValueInfoProto};
use hanzo_onnx::simple_eval;
use std::collections::HashMap;
const INPUT_X: &str = "x";
@@ -33,7 +33,7 @@ fn create_model_proto_with_graph(graph: Option<GraphProto>) -> ModelProto {
fn test_evaluation_fails_without_defined_graph() -> Result<()> {
let manual_graph = create_model_proto_with_graph(None);
let inputs: HashMap<String, Tensor> = HashMap::new();
match candle_onnx::simple_eval(&manual_graph, inputs) {
match hanzo_onnx::simple_eval(&manual_graph, inputs) {
Err(err) => assert_eq!(err.to_string(), "no graph defined in proto"),
Ok(_) => panic!("Expected an error due to undefined graph"),
}
@@ -71,7 +71,7 @@ fn test_add_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), Tensor::new(&[2.], &Device::Cpu)?);
inputs.insert(INPUT_Y.to_string(), Tensor::new(&[2.], &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -111,7 +111,7 @@ fn test_sub_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), Tensor::new(&[2.], &Device::Cpu)?);
inputs.insert(INPUT_Y.to_string(), Tensor::new(&[2.], &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -151,7 +151,7 @@ fn test_mul_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), Tensor::new(&[2.], &Device::Cpu)?);
inputs.insert(INPUT_Y.to_string(), Tensor::new(&[2.], &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -191,7 +191,7 @@ fn test_div_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), Tensor::new(&[2.], &Device::Cpu)?);
inputs.insert(INPUT_Y.to_string(), Tensor::new(&[2.], &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -232,7 +232,7 @@ fn test_exp_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -277,11 +277,11 @@ fn test_equal_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), Tensor::new(&[2.], &Device::Cpu)?);
inputs.insert(INPUT_Y.to_string(), Tensor::new(&[2.], &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
let first = z.to_dtype(candle::DType::U8)?.to_vec1::<u8>()?.to_vec()[0];
let first = z.to_dtype(hanzo-ml::DType::U8)?.to_vec1::<u8>()?.to_vec()[0];
assert_eq!(first, 1);
Ok(())
@@ -317,11 +317,11 @@ fn test_not_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), Tensor::new(&[0.], &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
let first = z.to_dtype(candle::DType::U8)?.to_vec1::<u8>()?.to_vec()[0];
let first = z.to_dtype(hanzo-ml::DType::U8)?.to_vec1::<u8>()?.to_vec()[0];
assert_eq!(first, 1);
Ok(())
@@ -374,7 +374,7 @@ fn test_matmul_operation() -> Result<()> {
)?,
);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -439,7 +439,7 @@ fn test_reshape_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), x);
inputs.insert(INPUT_Y.to_string(), y);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -499,7 +499,7 @@ fn test_logsoftmax_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -562,7 +562,7 @@ fn test_softmax_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -625,7 +625,7 @@ fn test_transpose_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -684,7 +684,7 @@ fn test_dropout_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -764,7 +764,7 @@ fn test_flatten_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs.clone())?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs.clone())?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -809,7 +809,7 @@ fn test_flatten_operation() -> Result<()> {
quantization_annotation: vec![],
}));
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -958,7 +958,7 @@ fn test_constant_of_shape() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), Tensor::new(input, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval
@@ -1022,7 +1022,7 @@ fn test_unsqueeze() -> Result<()> {
let inputs = HashMap::from_iter([(INPUT_X.to_string(), x.clone()), (INPUT_Y.to_string(), y)]);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1149,7 +1149,7 @@ fn test_gather_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), Tensor::new(data, &Device::Cpu)?);
inputs.insert(INPUT_Y.to_string(), Tensor::new(indices, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1306,7 +1306,7 @@ fn test_gather_elements() -> Result<()> {
inputs.insert(INPUT_X.to_string(), Tensor::new(data, &Device::Cpu)?);
inputs.insert(INPUT_Y.to_string(), Tensor::new(indices, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1360,7 +1360,7 @@ fn test_size_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1405,7 +1405,7 @@ fn test_shape_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1467,7 +1467,7 @@ fn test_abs_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1521,7 +1521,7 @@ fn test_cos_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1569,7 +1569,7 @@ fn test_sin_operation() -> Result<()> {
let x = Tensor::from_vec(vec![0.0f32, 1.0f32, 2.0f32, 3.0f32], &[2, 2], &Device::Cpu)?;
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
assert_eq!(to_vec2_round(z, 4)?, [[0.0, 0.8415], [0.9093, 0.1411]]);
@@ -1618,7 +1618,7 @@ fn test_neg_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1675,7 +1675,7 @@ fn test_tanh_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1732,7 +1732,7 @@ fn test_sigmoid_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1789,7 +1789,7 @@ fn test_gelu_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1843,7 +1843,7 @@ fn test_relu_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -1904,7 +1904,7 @@ fn test_prelu_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), x);
inputs.insert(INPUT_Y.to_string(), y);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -2394,7 +2394,7 @@ fn test_reduce_max() -> Result<()> {
}
}
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -2913,7 +2913,7 @@ fn test_reduce_min() -> Result<()> {
}
}
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -3114,7 +3114,7 @@ fn test_reduce_mean() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), Tensor::new(data, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -3168,7 +3168,7 @@ fn test_sqrt() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), Tensor::new(data, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -3310,7 +3310,7 @@ fn test_random_uniform() -> Result<()> {
sparse_initializer: vec![],
quantization_annotation: vec![],
}));
let eval = candle_onnx::simple_eval(&manual_graph, HashMap::new())?;
let eval = hanzo_onnx::simple_eval(&manual_graph, HashMap::new())?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
let min = z
@@ -3456,7 +3456,7 @@ fn test_random_normal() -> Result<()> {
sparse_initializer: vec![],
quantization_annotation: vec![],
}));
let eval = candle_onnx::simple_eval(&manual_graph, HashMap::new())?;
let eval = hanzo_onnx::simple_eval(&manual_graph, HashMap::new())?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -3526,7 +3526,7 @@ fn test_range() -> Result<()> {
inputs.insert(INPUT_Y.to_string(), Tensor::new(limit, &Device::Cpu)?);
inputs.insert(INPUT_A.to_string(), Tensor::new(delta, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval
@@ -3587,7 +3587,7 @@ fn test_greater() -> Result<()> {
inputs.insert(INPUT_X.to_string(), Tensor::new(a, &Device::Cpu)?);
inputs.insert(INPUT_Y.to_string(), Tensor::new(b, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval
@@ -3648,7 +3648,7 @@ fn test_less() -> Result<()> {
inputs.insert(INPUT_X.to_string(), Tensor::new(a, &Device::Cpu)?);
inputs.insert(INPUT_Y.to_string(), Tensor::new(b, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval
@@ -3705,7 +3705,7 @@ fn test_log() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), Tensor::new(data, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -3770,7 +3770,7 @@ fn test_min() -> Result<()> {
inputs.insert(INPUT_Y.to_string(), Tensor::new(b, &Device::Cpu)?);
inputs.insert(INPUT_A.to_string(), Tensor::new(c, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -3848,7 +3848,7 @@ fn test_where() -> Result<()> {
inputs.insert(INPUT_Y.to_string(), Tensor::new(x, &Device::Cpu)?);
inputs.insert(INPUT_A.to_string(), Tensor::new(y, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval
@@ -3921,7 +3921,7 @@ fn test_floor() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -3997,7 +3997,7 @@ fn test_ceil() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -4189,7 +4189,7 @@ fn test_argmin() -> Result<()> {
}));
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), Tensor::new(data, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
let expected = Tensor::new(expected, &Device::Cpu)?;
@@ -4371,7 +4371,7 @@ fn test_argmax() -> Result<()> {
}));
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), Tensor::new(data, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
let expected = Tensor::new(expected, &Device::Cpu)?;
@@ -4446,7 +4446,7 @@ fn test_leakyrelu() -> Result<()> {
}));
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), Tensor::new(data, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
let expected = Tensor::new(expected, &Device::Cpu)?;
@@ -4565,10 +4565,10 @@ fn test_if() -> Result<()> {
for cond in [1u8, 0] {
let inputs =
HashMap::from_iter([("cond".to_string(), Tensor::full(cond, (1,), &Device::Cpu)?)]);
let outputs = candle_onnx::simple_eval(&manual_graph, inputs)?;
let outputs = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
let expected = if cond != 0 { &x } else { &y };
let Some(res) = outputs.get("res") else {
candle::bail!("outputs didn't contain expected key `res`: {outputs:?}");
hanzo-ml::bail!("outputs didn't contain expected key `res`: {outputs:?}");
};
assert_eq!(&res.to_vec1::<f32>()?, expected);
}
@@ -4628,9 +4628,9 @@ fn test_pad() -> Result<()> {
}));
let inputs = HashMap::from_iter([("data".to_string(), data), ("pads".to_string(), pads)]);
let res = candle_onnx::simple_eval(&model, inputs)?;
let res = hanzo_onnx::simple_eval(&model, inputs)?;
let Some(actual) = res.get("output") else {
candle::bail!("outputs didn't contain expected key `output`: {res:?}");
hanzo-ml::bail!("outputs didn't contain expected key `output`: {res:?}");
};
assert_eq!(actual.to_vec2::<f64>()?, expected.to_vec2::<f64>()?);
@@ -4685,7 +4685,7 @@ fn test_slice() -> Result<()> {
]
*/
let outputs = candle_onnx::simple_eval(
let outputs = hanzo_onnx::simple_eval(
&model,
HashMap::from_iter([
(
@@ -4749,7 +4749,7 @@ fn test_slice() -> Result<()> {
.collect(),
..GraphProto::default()
}));
let outputs = candle_onnx::simple_eval(
let outputs = hanzo_onnx::simple_eval(
&model,
HashMap::from_iter([
(
@@ -5561,7 +5561,7 @@ fn test_expand_dim_changed() -> Result<()> {
("data".to_string(), data),
("new_shape".to_string(), new_shape),
]);
let result = candle_onnx::simple_eval(&manual_graph, inputs)?;
let result = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
// Retrieve and compare the result
let expanded = result.get("expanded").expect("Output 'expanded' not found");
@@ -5631,7 +5631,7 @@ fn test_expand_dim_unchanged() -> Result<()> {
("data".to_string(), data),
("new_shape".to_string(), new_shape),
]);
let result = candle_onnx::simple_eval(&manual_graph, inputs)?;
let result = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
// Retrieve and compare the result
let expanded = result.get("expanded").expect("Output 'expanded' not found");
@@ -5664,7 +5664,7 @@ fn test_split_equal_parts_1d_opset13() -> Result<()> {
{
let manual_graph =
make_split_graph_helper(&["input"], &["output_1", "output_2", "output_3"], 0);
let eval = candle_onnx::simple_eval(&manual_graph, inputs.clone())?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs.clone())?;
assert_eq!(eval.len(), 3);
let out1 = eval.get("output_1").expect("Output 'output_1' not found");
@@ -5682,7 +5682,7 @@ fn test_split_equal_parts_1d_opset13() -> Result<()> {
let manual_graph =
make_split_graph_helper(&["input", "split"], &["output_1", "output_2"], 0);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 2);
let out1 = eval.get("output_1").expect("Output 'output_1' not found");
@@ -5739,7 +5739,7 @@ fn test_reduce_sum_default_axes_keepdims() -> Result<()> {
inputs.insert("data".to_string(), data);
// inputs.insert("axes".to_string(), axes);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let reduced = eval.get("reduced").expect("Output 'reduced' not found");
@@ -5760,7 +5760,7 @@ fn test_reduce_sum_default_axes_keepdims() -> Result<()> {
let mut inputs = HashMap::new();
inputs.insert("data".to_string(), data.clone());
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let reduced = eval.get("reduced").expect("Output 'reduced' not found");
@@ -5791,7 +5791,7 @@ fn test_reduce_sum_do_not_keep_dims() -> Result<()> {
inputs.insert("data".to_string(), data);
inputs.insert("axes".to_string(), axes);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let reduced = eval.get("reduced").expect("Output 'reduced' not found");
@@ -5820,7 +5820,7 @@ fn test_reduce_sum_do_not_keep_dims() -> Result<()> {
inputs.insert("data".to_string(), data.clone());
inputs.insert("axes".to_string(), axes);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let reduced = eval.get("reduced").expect("Output 'reduced' not found");
@@ -6170,7 +6170,7 @@ fn test_xor() -> Result<()> {
(INPUT_Y.to_string(), Tensor::new(other, &Device::Cpu)?),
]);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -6191,7 +6191,7 @@ fn test_xor() -> Result<()> {
assert_eq!(z.to_vec3::<u8>()?, expected.to_vec3::<u8>()?)
}
4 => {
// Candle has no method equivalent to `to_vec4()`
// Hanzo has no method equivalent to `to_vec4()`
// So, as a hack, we flatten it to a single dim vec to test the results
assert_eq!(
z.flatten_all()?.to_vec1::<u8>()?,
@@ -6237,11 +6237,11 @@ fn test_sign_operation() -> Result<()> {
INPUT_X.to_string(),
Tensor::new(vec![-2f32, -1., 0., 1., 2.], &Device::Cpu)?,
);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
assert_eq!(
z.to_dtype(candle::DType::I64)?.to_vec1::<i64>()?.to_vec(),
z.to_dtype(hanzo-ml::DType::I64)?.to_vec1::<i64>()?.to_vec(),
vec![-1, -1, 0, 1, 1]
);
Ok(())
@@ -6443,7 +6443,7 @@ fn test_selu_operator() -> Result<()> {
}
#[test]
fn test_hard_swish() -> candle::Result<()> {
fn test_hard_swish() -> hanzo-ml::Result<()> {
{
let manual_graph = create_model_proto_with_graph(Some(GraphProto {
node: vec![NodeProto {
@@ -6594,7 +6594,7 @@ fn test_scatternd_operation() -> Result<()> {
inputs.insert(INPUT_Y.to_string(), Tensor::new(indices, &Device::Cpu)?);
inputs.insert(INPUT_A.to_string(), Tensor::new(updates, &Device::Cpu)?);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -6656,7 +6656,7 @@ fn test_trilu_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -6722,7 +6722,7 @@ fn test_trilu_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), x);
inputs.insert(INPUT_Y.to_string(), k);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -6778,7 +6778,7 @@ fn test_trilu_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), x);
inputs.insert(INPUT_Y.to_string(), k);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -6853,7 +6853,7 @@ fn test_trilu_operation() -> Result<()> {
let mut inputs: HashMap<String, Tensor> = HashMap::new();
inputs.insert(INPUT_X.to_string(), x);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -6932,7 +6932,7 @@ fn test_trilu_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), x);
inputs.insert(INPUT_Y.to_string(), k);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
@@ -7010,7 +7010,7 @@ fn test_trilu_operation() -> Result<()> {
inputs.insert(INPUT_X.to_string(), x);
inputs.insert(INPUT_Y.to_string(), k);
let eval = candle_onnx::simple_eval(&manual_graph, inputs)?;
let eval = hanzo_onnx::simple_eval(&manual_graph, inputs)?;
assert_eq!(eval.len(), 1);
let z = eval.get(OUTPUT_Z).expect("Output 'z' not found");
hanzo-rocm-kernels/Cargo.toml
+1 -1
View File
@@ -3,7 +3,7 @@ name = "hanzo-rocm-kernels"
version = "0.10.2"
edition = "2021"
description = "ROCm/HIP kernels for Candle"
description = "ROCm/HIP kernels for Hanzo"
[dependencies]
dirs = "5"
hanzo-rocm-kernels/README.md
+10 -10
View File
@@ -1,10 +1,10 @@
# candle-rocm-kernels
# hanzo-ml-rocm-kernels
ROCm/HIP kernel support for the Candle deep learning framework.
ROCm/HIP kernel support for the Hanzo deep learning framework.
## Overview
This crate provides ROCm (AMD GPU) kernel support for Candle. Unlike CUDA which can embed PTX directly, ROCm/HIP requires ahead-of-time (AOT) compilation for specific GPU architectures.
This crate provides ROCm (AMD GPU) kernel support for Hanzo. Unlike CUDA which can embed PTX directly, ROCm/HIP requires ahead-of-time (AOT) compilation for specific GPU architectures.
## Architecture
@@ -22,12 +22,12 @@ We use an **Ahead-of-Time (AOT) compilation cache** approach:
Compiled binaries are stored at:
```
~/.cache/candle-rocm/{arch}-{rocm_version}/
~/.cache/hanzo-ml-rocm/{arch}-{rocm_version}/
```
For example:
- `~/.cache/candle-rocm/gfx908-6.1/binary_a1b2c3d4.cso`
- `~/.cache/candle-rocm/gfx942-6.2/binary_a1b2c3d4.cso`
- `~/.cache/hanzo-ml-rocm/gfx908-6.1/binary_a1b2c3d4.cso`
- `~/.cache/hanzo-ml-rocm/gfx942-6.2/binary_a1b2c3d4.cso`
Where:
- `{arch}` = GPU architecture (gfx908, gfx90a, gfx942, etc.)
@@ -50,7 +50,7 @@ Manages the AOT compilation cache:
Usage:
```rust
use candle_rocm_kernels::CacheManager;
use hanzo_rocm_kernels::CacheManager;
use rocm_rs::hip::Device;
let device = Device::new(0)?;
@@ -69,8 +69,8 @@ Higher-level manager that:
Usage:
```rust
use candle_rocm_kernels::KernelManager;
use candle_rocm_kernels::source::Source;
use hanzo_rocm_kernels::KernelManager;
use hanzo_rocm_kernels::source::Source;
let device = Device::new(0)?;
let manager = KernelManager::new(&device)?;
@@ -95,7 +95,7 @@ Currently supports:
## Building
```bash
cd candle-rocm-kernels
cd hanzo-ml-rocm-kernels
cargo build
```
hanzo-rocm-kernels/src/compile.rs
+6 -6
View File
@@ -31,7 +31,7 @@ impl KernelCache {
let rocm_version = detect_rocm_version()?;
let cache_dir = get_cache_dir()?;
// Create cache directory structure: ~/.cache/candle-rocm/{arch}-{rocm_version}/
// Create cache directory structure: ~/.cache/hanzo-ml-rocm/{arch}-{rocm_version}/
let arch_version = format!("{}-{}", arch, rocm_version);
let kernel_dir = cache_dir.join(&arch_version);
fs::create_dir_all(&kernel_dir).map_err(|e| {
@@ -242,7 +242,7 @@ fn get_cache_dir() -> Result<PathBuf, KernelError> {
.or_else(|| std::env::var("HOME").ok().map(PathBuf::from))
.ok_or_else(|| KernelError::Internal("Could not determine cache directory".to_string()))?;
Ok(home.join("candle-rocm"))
Ok(home.join("hanzo-ml-rocm"))
}
/// Compute a hash of the source code
@@ -263,10 +263,10 @@ fn compile_kernel(
) -> Result<Vec<u8>, KernelError> {
let temp_dir = std::env::temp_dir();
let source_hash = compute_source_hash(source);
let source_file = temp_dir.join(format!("candle_{}_{}.hip", name, source_hash));
let obj_file = temp_dir.join(format!("candle_{}_{}.o", name, source_hash));
let fatbin_file = temp_dir.join(format!("candle_{}_{}.fatbin", name, source_hash));
let hsaco_file = temp_dir.join(format!("candle_{}_{}.hsaco", name, source_hash));
let source_file = temp_dir.join(format!("hanzo_{}_{}.hip", name, source_hash));
let obj_file = temp_dir.join(format!("hanzo_{}_{}.o", name, source_hash));
let fatbin_file = temp_dir.join(format!("hanzo_{}_{}.fatbin", name, source_hash));
let hsaco_file = temp_dir.join(format!("hanzo_{}_{}.hsaco", name, source_hash));
// Clean up temp files on any error
let _cleanup = TempFileCleanup {
hanzo-rocm-kernels/src/kernels/sort.hip
+1 -1
View File
@@ -1,4 +1,4 @@
// ROCm/HIP single-block bitonic argsort (ported from candle/ggml argsort.cu).
// ROCm/HIP single-block bitonic argsort (ported from hanzo-ml/ggml argsort.cu).
// One block per row; shared memory holds the row's indices. Compiled at runtime.
#ifndef __HIPCC__
#define __device__
hanzo-training/README.md
+2 -2
View File
@@ -94,9 +94,9 @@ zen-agentic-dataset-private/
"code_blocks": [
{
"language": "rust",
"content": "use candle_core::*;\n\nstruct NeuralNet {\n layers: Vec<Linear>,\n}",
"content": "use hanzo_ml::*;\n\nstruct NeuralNet {\n layers: Vec<Linear>,\n}",
"file_path": "src/neural_net.rs",
"diff": "+use candle_core::*;\n+\n+struct NeuralNet {"
"diff": "+use hanzo_ml::*;\n+\n+struct NeuralNet {"
}
],
"git_context": {
hanzo-transformers/src/models/based.rs
+1 -1
View File
@@ -74,7 +74,7 @@ impl MLP {
}
// Swiglu implementation.
// Not using Activation::Swiglu because this has the gate and y arguments switched compared to the version in candle-nn/src/ops.rs
// Not using Activation::Swiglu because this has the gate and y arguments switched compared to the version in hanzo-nn/src/ops.rs
fn swiglu(xs: &Tensor) -> Result<Tensor> {
let xs = xs.chunk(2, D::Minus1)?;
&xs[1].silu()? * &xs[0]
hanzo-transformers/src/models/bert.rs
+1 -1
View File
@@ -5,7 +5,7 @@
//! - Compute similarities between a set of sentences.
//! - [Arxiv](https://arxiv.org/abs/1810.04805) "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"
//! - Upstream [GitHub repo](https://github.com/google-research/bert).
//! - See bert in [candle-examples](https://github.com/huggingface/candle/tree/main/candle-examples/) for runnable code
//! - See bert in [hanzo-ml-examples](https://github.com/hanzoai/ml/tree/main/hanzo-ml-examples/) for runnable code
//!
use super::with_tracing::{layer_norm, linear, LayerNorm, Linear};
use hanzo_ml::{DType, Device, Result, Tensor};
hanzo-transformers/src/models/blip.rs
+1 -1
View File
@@ -2,7 +2,7 @@
//!
//! The blip-image-captioning model can generate captions for an input image.
//!
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/Candle-BLIP-Image-Captioning)
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/Hanzo-BLIP-Image-Captioning)
//! - 💻 [GH Link](https://github.com/salesforce/BLIP)
//! - 🤗 [HF Link](https://huggingface.co/Salesforce/blip-image-captioning-base)
//! - 📝 [Paper](https://arxiv.org/abs/2201.12086)
hanzo-transformers/src/models/blip_text.rs
+1 -1
View File
@@ -2,7 +2,7 @@
//!
//! - 📝 [Paper](https://arxiv.org/abs/2201.12086). BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation"
//!
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/Candle-BLIP-Image-Captioning)
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/Hanzo-BLIP-Image-Captioning)
//! - 💻 [GH Link](https://github.com/salesforce/BLIP)
//! - 🤗 [HF Link](https://huggingface.co/Salesforce/blip-image-captioning-base)
//! - 📝 [Paper](https://arxiv.org/abs/2201.12086)
hanzo-transformers/src/models/convnext.rs
+1 -1
View File
@@ -1,6 +1,6 @@
//! ConvNeXt implementation.
//!
//! This candle implementation uses a pre-trained ConvNeXt network for inference. The
//! This hanzo-ml implementation uses a pre-trained ConvNeXt network for inference. The
//! classification head has been trained on the ImageNet dataset and returns the
//! probabilities for the top-5 classes.
//!
hanzo-transformers/src/models/debertav2.rs
+1 -1
View File
@@ -283,7 +283,7 @@ struct XSoftmax {}
impl XSoftmax {
pub fn apply(input: &Tensor, mask: &Tensor, dim: D, device: &Device) -> Result<Tensor> {
// NOTE: At the time of this writing, candle does not have a logical-not operator.
// NOTE: At the time of this writing, hanzo-ml does not have a logical-not operator.
let mut rmask = mask.broadcast_as(input.shape())?.to_dtype(DType::F32)?;
rmask = rmask
hanzo-transformers/src/models/dinov2.rs
+4 -4
View File
@@ -9,10 +9,10 @@
//! ```bash
//! cargo run \
//! --features cuda,depth_anything_v2 \
//! --package candle-examples \
//! --package hanzo-ml-examples \
//! --example depth_anything_v2 \
//! -- --color-map \
//! --image candle-examples/examples/yolo-v8/assets/bike.jpg
//! --image hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg
//! ```
//!
//! ## Running as an ImageNet classifier
@@ -20,14 +20,14 @@
//! The model returns the probability for the image to belong to each of the 1000 ImageNet categories.
//!
//! <div align=center>
//! <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/yolo-v8/assets/bike.jpg" alt="" width=640>
//! <img src="https://github.com/hanzoai/ml/raw/main/hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg" alt="" width=640>
//! </div>
//!
//! ```bash
//! cargo run \
//! --example dinov2 \
//! --release \
//! -- --image candle-examples/examples/yolo-v8/assets/bike.jpg
//! -- --image hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg
//!
//! > mountain bike, all-terrain bike, off-roader: 43.67%
//! > bicycle-built-for-two, tandem bicycle, tandem: 33.20%
hanzo-transformers/src/models/dinov2reg4.rs
+1 -1
View File
@@ -11,7 +11,7 @@
//!
//! ```bash
//! # Download classes names and a plant picture to identify
//! # see candle/examples/dinov2reg4 for full code.
//! # see hanzo-ml/examples/dinov2reg4 for full code.
//!
//! # Perform inference
//! cargo run \
hanzo-transformers/src/models/efficientvit.rs
+3 -3
View File
@@ -12,7 +12,7 @@
//!
//! # Example Usage
//!
//! This candle implementation uses a pre-trained EfficientViT (from Microsoft Research Asia) network for inference.
//! This hanzo-ml implementation uses a pre-trained EfficientViT (from Microsoft Research Asia) network for inference.
//! The classification head has been trained on the ImageNet dataset and returns the probabilities for the top-5 classes.
//!
//!
@@ -20,7 +20,7 @@
//! cargo run
//! --example efficientvit \
//! --release -- \
//! --image candle-examples/examples/yolo-v8/assets/bike.jpg --which m1
//! --image hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg --which m1
//!
//! > loaded image Tensor[dims 3, 224, 224; f32]
//! > model built
@@ -32,7 +32,7 @@
//! ```
//!
//! <div align=center>
//! <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/yolo-v8/assets/bike.jpg" alt="" width=640>
//! <img src="https://github.com/hanzoai/ml/raw/main/hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg" alt="" width=640>
//! </div>
//!
use hanzo_ml::{Result, Tensor, D};
hanzo-transformers/src/models/encodec.rs
+1 -1
View File
@@ -387,7 +387,7 @@ impl EncodecLSTM {
impl Module for EncodecLSTM {
fn forward(&self, xs: &Tensor) -> Result<Tensor> {
use hanzo_nn::RNN;
// This is different from the Python transformers version as candle LSTM is batch first.
// This is different from the Python transformers version as hanzo-ml LSTM is batch first.
let xs = xs.t()?;
let residual = &xs;
let mut xs = xs.clone();
hanzo-transformers/src/models/eva2.rs
+2 -2
View File
@@ -13,7 +13,7 @@
//! cargo run \
//! --example eva2 \
//! --release -- \
//! --image candle-examples/examples/yolo-v8/assets/bike.jpg
//! --image hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg
//!
//! > mountain bike, all-terrain bike, off-roader: 37.09%
//! > maillot : 8.30%
@@ -23,7 +23,7 @@
//! ```
//!
//! <div align=center>
//! <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/yolo-v8/assets/bike.jpg" alt="" width=640>
//! <img src="https://github.com/hanzoai/ml/raw/main/hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg" alt="" width=640>
//! </div>
//!
use hanzo_ml::{IndexOp, Result, Tensor, D};
hanzo-transformers/src/models/flux/mod.rs
+1 -1
View File
@@ -17,7 +17,7 @@
//! ```
//!
//! <div align=center>
//! <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/flux/assets/flux-robot.jpg" alt="" width=320>
//! <img src="https://github.com/hanzoai/ml/raw/main/hanzo-ml-examples/examples/flux/assets/flux-robot.jpg" alt="" width=320>
//! </div>
//!
hanzo-transformers/src/models/llama2_c.rs
+1 -1
View File
@@ -2,7 +2,7 @@
//!
//! See ["LLaMA 2: Open Foundation and Fine-Tuned Chat Models"](https://arxiv.org/abs/2307.09288)
//!
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/lmz/candle-llama2)
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/lmz/hanzo-ml-llama2)
//! - 💻 llama2.c [GH Link](https://github.com/karpathy/llama2.c)
//!
hanzo-transformers/src/models/llava/mod.rs
+2 -2
View File
@@ -329,7 +329,7 @@ impl LLaVA {
} else {
bail!("Unexpected mm_patch_merge_type: {mm_patch_merge_type}")
};
// can easily be replaced by nonzero if it is implemented in candle
// can easily be replaced by nonzero if it is implemented in hanzo-ml
let input_ids_vec = input_ids.squeeze(0)?.to_vec1::<i64>()?;
let mut image_indices = {
let mut image_indices = vec![0_i64];
@@ -367,7 +367,7 @@ impl LLaVA {
image_indices.push((input_ids_noim_len) as i64);
let input_ids_noim = Tensor::from_vec(input_ids_noim, input_ids_noim_len, &self.device)?;
let cur_input_embeds = self.llama.embed(&input_ids_noim)?;
// can be replace by split if it is implemented in candle
// can be replace by split if it is implemented in hanzo-ml
let input_embed_no_ims = {
let mut input_embeds = Vec::new();
for i in 0..image_indices.len() - 1 {
hanzo-transformers/src/models/metavoice.rs
+1 -1
View File
@@ -161,7 +161,7 @@ pub mod speaker_encoder {
fn forward(&self, xs: &Tensor) -> Result<Tensor> {
use hanzo_nn::RNN;
// This is different from the Python transformers version as candle LSTM is batch first.
// This is different from the Python transformers version as hanzo-ml LSTM is batch first.
let xs = xs.t()?;
let mut xs = xs.clone();
for layer in self.lstms.iter() {
hanzo-transformers/src/models/mimi/mod.rs
+1 -1
View File
@@ -2,7 +2,7 @@
//!
//! [Mimi](https://huggingface.co/kyutai/mimi) is a state of the art audio
//! compression model using an encoder/decoder architecture with residual vector
//! quantization. The candle implementation supports streaming meaning that it's
//! quantization. The hanzo-ml implementation supports streaming meaning that it's
//! possible to encode or decode a stream of audio tokens on the flight to provide
//! low latency interaction with an audio model.
//!
hanzo-transformers/src/models/mmdit/mod.rs
+1 -1
View File
@@ -7,7 +7,7 @@
//! - 💻 ComfyUI [reference implementation](https://github.com/comfyanonymous/ComfyUI/blob/78e133d0415784924cd2674e2ee48f3eeca8a2aa/comfy/ldm/modules/diffusionmodules/mmdit.py)
//! - 💻 Stability-AI [MMDiT-X implementation](https://github.com/Stability-AI/sd3.5/blob/4e484e05308d83fb77ae6f680028e6c313f9da54/mmditx.py)
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/Candle-BLIP-Image-Captioning)
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/Hanzo-BLIP-Image-Captioning)
//! - 💻 [GH Link](https://github.com/salesforce/BLIP)
//! - 🤗 [HF Link](https://huggingface.co/Salesforce/blip-image-captioning-base)
//! - 📝 [Paper](https://arxiv.org/abs/2201.12086)
hanzo-transformers/src/models/mobileone.rs
+1 -1
View File
@@ -1,6 +1,6 @@
//! # MobileOne
//!
//! MobileOne inference implementation based on timm and candle-repvgg
//! MobileOne inference implementation based on timm and hanzo-ml-repvgg
//!
//! See ["MobileOne: An Improved One millisecond Mobile Backbone"](https://arxiv.org/abs/2206.04040)
hanzo-transformers/src/models/mod.rs
+1 -1
View File
@@ -1,4 +1,4 @@
//! Candle implementations for various deep learning models
//! Hanzo implementations for various deep learning models
//!
//! This crate provides implementations of popular machine learning models and architectures for different modalities.
//!
hanzo-transformers/src/models/modernbert.rs
+1 -1
View File
@@ -3,7 +3,7 @@
//! ModernBERT is a modernized bidirectional encoder-only Transformer model.
//! - [Arxiv](https://arxiv.org/abs/2412.13663) "Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference"
//! - Upstream [GitHub repo](https://github.com/AnswerDotAI/ModernBERT).
//! - See modernbert in [candle-examples](https://github.com/huggingface/candle/tree/main/candle-examples/) for runnable code
//! - See modernbert in [hanzo-ml-examples](https://github.com/hanzoai/ml/tree/main/hanzo-ml-examples/) for runnable code
//!
use hanzo_ml::{DType, Device, IndexOp, Result, Tensor, D};
hanzo-transformers/src/models/moondream.rs
+1 -1
View File
@@ -18,7 +18,7 @@
//! # download an example image
//! wget https://raw.githubusercontent.com/vikhyat/moondream/main/assets/demo-1.jpg
//!
//! # Now you can run Moondream from the `candle-examples` crate:
//! # Now you can run Moondream from the `hanzo-ml-examples` crate:
//! cargo run --example moondream \
//! --release -- \
//! --prompt "What is the girl eating?"
hanzo-transformers/src/models/nvembed_v2/model.rs
+2 -2
View File
@@ -6,7 +6,7 @@ use crate::models::{
use hanzo_ml::{DType, Device, Result, Tensor, D};
use hanzo_nn::{ops::softmax_last_dim, LayerNormConfig, Module, VarBuilder};
// Geglu and feedforward from candle-transformers/src/models/stable_diffusion/attention.rs
// Geglu and feedforward from hanzo-transformers/src/models/stable_diffusion/attention.rs
#[derive(Debug)]
struct GeGlu {
proj: Linear,
@@ -60,7 +60,7 @@ impl Module for FeedForward {
}
}
// CrossAttention from candle-transformers/src/models/stable_diffusion/attention.rs
// CrossAttention from hanzo-transformers/src/models/stable_diffusion/attention.rs
#[derive(Debug)]
struct CrossAttention {
to_q: Linear,
hanzo-transformers/src/models/phi.rs
+1 -1
View File
@@ -8,7 +8,7 @@
//! - Layer normalization
//! - QK normalization
//!
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/Candle-phi1-phi2-wasm-demo)
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/Hanzo-phi1-phi2-wasm-demo)
//! - 🤗 [HF Link](https://huggingface.co/microsoft/phi-2)
//!
hanzo-transformers/src/models/pixtral/mod.rs
+2 -2
View File
@@ -11,14 +11,14 @@
//! # Example
//!
//! <div align=center>
//! <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/flux/assets/flux-robot.jpg" alt="" width=320>
//! <img src="https://github.com/hanzoai/ml/raw/main/hanzo-ml-examples/examples/flux/assets/flux-robot.jpg" alt="" width=320>
//! </div>
//!
//! ```bash
//! cargo run --profile=release-with-debug \
//! --features cuda \
//! --example pixtral -- \
//! --image candle-examples/examples/flux/assets/flux-robot.jpg
//! --image hanzo-ml-examples/examples/flux/assets/flux-robot.jpg
//! ```
//!
//! ```txt
hanzo-transformers/src/models/quantized_llama.rs
+1 -1
View File
@@ -13,7 +13,7 @@
//! - 💻 [GH Link](https://github.com/facebookresearch/llama)
//! - 📝 [Paper](https://arxiv.org/abs/2302.13971)
//!
//! ![](https://raw.githubusercontent.com/huggingface/candle/main/candle-examples/examples/quantized/assets/aoc.gif)
//! ![](https://raw.githubusercontent.com/hanzoai/ml/main/hanzo-ml-examples/examples/quantized/assets/aoc.gif)
//!
use std::collections::HashMap;
hanzo-transformers/src/models/rwkv_v5.rs
+1 -1
View File
@@ -2,7 +2,7 @@
//!
//! The [RWKV model](https://wiki.rwkv.com/) is a recurrent neural network model
//! with performance on par with transformer architectures. Several variants are
//! available, candle implements the v5 and v6 versions and can be used with
//! available, hanzo-ml implements the v5 and v6 versions and can be used with
//! Eagle 7B([blog post](https://blog.rwkv.com/p/eagle-7b-soaring-past-transformers)).
//!
//! Key characteristics:
hanzo-transformers/src/models/rwkv_v6.rs
+1 -1
View File
@@ -2,7 +2,7 @@
//!
//! The [RWKV model](https://wiki.rwkv.com/) is a recurrent neural network model
//! with performance on par with transformer architectures. Several variants are
//! available, candle implements the v5 and v6 versions and can be used with
//! available, hanzo-ml implements the v5 and v6 versions and can be used with
//! Eagle 7B([blog post](https://blog.rwkv.com/p/eagle-7b-soaring-past-transformers)).
//!
//! Key characteristics:
hanzo-transformers/src/models/segment_anything/image_encoder.rs
+1 -1
View File
@@ -39,7 +39,7 @@ impl Module for PatchEmbed {
// (attn.reshape((b, q_h, q_w, k_h, k_w))?
// + rel_h.unsqueeze(4)?.broadcast_add(&rel_w.unsqueeze(3)?)?)?
// .reshape((b, q_h * q_w, k_h * k_w))
// Ideally we would perform this operation in place but this is not supported in candle at the
// Ideally we would perform this operation in place but this is not supported in hanzo-ml at the
// moment. We should also investigate using f16 rather than f32.
struct Add3(usize, usize, usize, usize, usize);
impl hanzo_ml::CustomOp3 for Add3 {
hanzo-transformers/src/models/segment_anything/mod.rs
+5 -5
View File
@@ -6,7 +6,7 @@
//! points to be part of the background so _not_ in the target mask, specifying some
//! bounding box).
//!
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/candle-segment-anything-wasm)
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/hanzo-ml-segment-anything-wasm)
//! - 💻 [GH Link](https://github.com/facebookresearch/segment-anything)
//! - 📝 [Paper](https://arxiv.org/abs/2304.02643)
//! - 💡 The default backbone can be replaced by the smaller and faster TinyViT model based on [MobileSAM](https://github.com/ChaoningZhang/MobileSAM).
@@ -16,14 +16,14 @@
//!
//! ```bash
//! cargo run --example segment-anything --release -- \
//! --image candle-examples/examples/yolo-v8/assets/bike.jpg
//! --image hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg
//! --use-tiny --point 0.6,0.6 --point 0.6,0.55
//! ```
//!
//! <div align=center style="display: flex; justify-content: center; gap: 10px;">
//! <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/yolo-v8/assets/bike.jpg" alt="" width="30%">
//! <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/segment-anything/assets/single_pt_prompt.jpg" alt="" width="30%">
//! <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/segment-anything/assets/two_pt_prompt.jpg" alt="" width="30%">
//! <img src="https://github.com/hanzoai/ml/raw/main/hanzo-ml-examples/examples/yolo-v8/assets/bike.jpg" alt="" width="30%">
//! <img src="https://github.com/hanzoai/ml/raw/main/hanzo-ml-examples/examples/segment-anything/assets/single_pt_prompt.jpg" alt="" width="30%">
//! <img src="https://github.com/hanzoai/ml/raw/main/hanzo-ml-examples/examples/segment-anything/assets/two_pt_prompt.jpg" alt="" width="30%">
//! </div>
//!
//!
hanzo-transformers/src/models/smol/README.md
+12 -12
View File
@@ -170,8 +170,8 @@ The quantization work enables running SmolLM3 efficiently on consumer hardware w
## Implementation Credits
### This Candle Implementation
**Implemented for**: Candle ML Framework
### This Hanzo Implementation
**Implemented for**: Hanzo ML Framework
**Implementation Date**: Nov 2025
**Features**:
- Full precision model (F32/F16/BF16)
@@ -185,8 +185,8 @@ The quantization work enables running SmolLM3 efficiently on consumer hardware w
### Related Tools & Frameworks
**Candle**: Minimalist ML framework in Rust by HuggingFace
- GitHub: https://github.com/huggingface/candle
**Hanzo**: Minimalist ML framework in Rust by HuggingFace
- GitHub: https://github.com/hanzoai/ml
**llama.cpp**: Efficient LLM inference in C/C++
- GitHub: https://github.com/ggerganov/llama.cpp
@@ -203,7 +203,7 @@ Special thanks to:
1. **HuggingFace Team** - For developing SmolLM3 and making it openly available under Apache 2.0 license
2. **NoPE Researchers** - For advancing the field with novel positional encoding approaches
3. **Unsloth** - For providing optimized quantized versions
4. **Candle Contributors** - For building an excellent ML framework in Rust
4. **Hanzo Contributors** - For building an excellent ML framework in Rust
5. **Open Source Community** - For tools like llama.cpp that enable verification and benchmarking
## Citation
@@ -231,28 +231,28 @@ If you use SmolLM3 in your research or applications, please cite:
}
```
### Candle Framework
### Hanzo Framework
```bibtex
@software{candle,
title={Candle: Minimalist ML Framework},
@software{hanzo-ml,
title={Hanzo: Minimalist ML Framework},
author={HuggingFace},
year={2024},
url={https://github.com/huggingface/candle}
url={https://github.com/hanzoai/ml}
}
```
## License
- **SmolLM3 Model**: Apache 2.0
- **This Implementation**: Follows Candle framework license
- **Candle Framework**: Apache 2.0 and MIT dual-licensed
- **This Implementation**: Follows Hanzo framework license
- **Hanzo Framework**: Apache 2.0 and MIT dual-licensed
## Further Reading
- **SmolLM Blog Series**: https://huggingface.co/blog/smollm and https://huggingface.co/blog/smollm3
- **Model Card Details**: https://huggingface.co/HuggingFaceTB/SmolLM3-3B
- **NoPE Paper**: https://arxiv.org/abs/2410.01926
- **Candle Documentation**: https://huggingface.github.io/candle/
- **Hanzo Documentation**: https://huggingface.github.io/hanzo-ml/
---
hanzo-transformers/src/models/smol/quantized_smollm3.rs
+1 -1
View File
@@ -336,7 +336,7 @@ impl QuantizedAttention {
(q, k)
};
// can remove this continguous call if using ConcatKV-Cache https://github.com/huggingface/candle/pull/3143
// can remove this continguous call if using ConcatKV-Cache https://github.com/hanzoai/ml/pull/3143
let (k, v) = self.kv_cache.append(&k.contiguous()?, &v.contiguous()?)?;
let k = repeat_kv(k, self.num_kv_groups)?;
hanzo-transformers/src/models/stable_diffusion/mod.rs
+3 -3
View File
@@ -11,14 +11,14 @@
//! # Example
//!
//! <div align=center>
//! <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/stable-diffusion/assets/stable-diffusion-xl.jpg" alt="rusty robot holding a candle" width=320>
//! <img src="https://github.com/hanzoai/ml/raw/main/hanzo-ml-examples/examples/stable-diffusion/assets/stable-diffusion-xl.jpg" alt="rusty robot holding a hanzo-ml" width=320>
//! </div>
//!
//! _"A rusty robot holding a fire torch in its hand."_ Generated by Stable Diffusion XL using Rust and [candle](https://github.com/huggingface/candle).
//! _"A rusty robot holding a fire torch in its hand."_ Generated by Stable Diffusion XL using Rust and [hanzo-ml](https://github.com/hanzoai/ml).
//!
//! ```bash
//! # example running with cuda
//! # see the candle-examples/examples/stable-diffusion for all options
//! # see the hanzo-ml-examples/examples/stable-diffusion for all options
//! cargo run --example stable-diffusion --release --features=cuda,cudnn \
//! -- --prompt "a cosmonaut on a horse (hd, realistic, high-def)"
//!
hanzo-transformers/src/models/t5.rs
+1 -1
View File
@@ -11,7 +11,7 @@
//! - Support for sequence-to-sequence tasks
//!
//! References:
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/Candle-T5-Generation-Wasm)
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/radames/Hanzo-T5-Generation-Wasm)
//! - 💻[GH Model](https://github.com/huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py)
//! - 🤗 [HF Link](https://huggingface.co/docs/transformers/model_doc/t5)
//! - 📝 [T5 Paper](https://arxiv.org/abs/1910.10683)
hanzo-transformers/src/models/whisper/mod.rs
+1 -1
View File
@@ -5,7 +5,7 @@
//! convert audio files (in the `.wav` format) to text. Supported features include
//! language detection as well as multilingual speech recognition.
//!
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/lmz/candle-whisper)
//! - ⚡ [Interactive Wasm Example](https://huggingface.co/spaces/lmz/hanzo-ml-whisper)
//! - 💻 [GH Link](https://github.com/openai/whisper)
//! - 💻 Transformers Python [reference implementation](https://github.com/huggingface/transformers/blob/main/src/transformers/models/whisper/modeling_whisper.py)
//!
hanzo-transformers/src/models/wuerstchen/mod.rs
+1 -1
View File
@@ -10,7 +10,7 @@
//! ## Example
//!
//! <div align=center>
//! <img src="https://github.com/huggingface/candle/raw/main/candle-examples/examples/wuerstchen/assets/cat.jpg" alt="" width=320>
//! <img src="https://github.com/hanzoai/ml/raw/main/hanzo-ml-examples/examples/wuerstchen/assets/cat.jpg" alt="" width=320>
//! <p>"Anthropomorphic cat dressed as a fire fighter"</p>
//! </div>
hanzo-transformers/src/models/yi.rs
+1 -1
View File
@@ -1,6 +1,6 @@
//! Yi model implementation.
//!
//! This candle implementation uses a pre-trained Yi decoder-only large language model for inference.
//! This hanzo-ml implementation uses a pre-trained Yi decoder-only large language model for inference.
//! The model was trained by 01.AI and follows a standard transformer architecture similar to LLaMA.
//!
//! Original code:
hanzo-transformers/src/utils.rs
+1 -1
View File
@@ -32,7 +32,7 @@ pub fn repeat_kv(xs: Tensor, n_rep: usize) -> Result<Tensor> {
let (b_sz, n_kv_head, seq_len, head_dim) = xs.dims4()?;
// Using cat is faster than a broadcast as it avoids going through a potentially
// strided copy.
// https://github.com/huggingface/candle/pull/2043
// https://github.com/hanzoai/ml/pull/2043
Tensor::cat(&vec![&xs; n_rep], 2)?.reshape((b_sz, n_kv_head * n_rep, seq_len, head_dim))
}
}
scripts/ultimate_setup.sh
+3 -3
View File
@@ -1,18 +1,18 @@
#!/bin/bash
# Hanzo ML Setup Script
# Updates from upstream Candle, checks compilation, sets up training
# Updates from upstream Hanzo, checks compilation, sets up training
set -e
HANZO_ML_DIR="/Users/z/work/hanzo/ml"
UPSTREAM_CANDLE="/Users/z/work/hf/candle"
UPSTREAM_CANDLE="/Users/z/work/hf/hanzo-ml"
ZEN_DATASET="/Users/z/work/zen/zen-agentic-dataset"
HANZO_ENGINE="/Users/z/work/hanzo/engine"
cd "$HANZO_ML_DIR"
# Pull latest upstream changes
echo "Pulling latest Candle upstream changes..."
echo "Pulling latest Hanzo upstream changes..."
if [ -d "$UPSTREAM_CANDLE" ]; then
cd "$UPSTREAM_CANDLE"
git pull origin main || true