Compare commits

..
Author SHA1 Message Date
hanzo-dev dad46e4918 ci: add opt-in mode: delegate — hand the build to platform.hanzo.ai
The default `mode: buildx` path is unchanged: buildx → test → deploy ON the
arc runner. `mode: delegate` instead POSTs each hanzo.yml image to platform's
direct build webhook (POST /v1/arcd/enqueue, bearer PLATFORM_BUILD_CALLBACK_TOKEN,
body {repo, sha, image, ref, branch, dockerfile, context, os, arch}) and exits
in seconds — platform builds in-cluster with BuildKit on its own pool, pushes to
the registry, and rolls the operator Service CR. Same downstream as platform's
GitHub-App webhook (one build path, two front doors); no runner buildx, no KMS,
no runner-side deploy.

- new `mode` workflow_call input (default `buildx`) — delegation is opt-in, so
  existing repos are untouched.
- new "Delegate build to platform" step (mode == delegate): parses hanzo.yml,
  enqueues one build per (image, platform) with the buildx tag shape the deploy
  path expects (sha-<short>-<arch>[-<suffix>]); fails loud on a non-202.
- buildx/GHCR/KMS/test/deploy steps gated `mode != 'delegate'`; checkout + parse
  toolchain still run (needed to read hanzo.yml).
- endpoint override via the PLATFORM_ENQUEUE_URL repo/org var.
- README documents the delegate opt-in.
2026-07-04 21:10:48 -07:00
zandGitHub c06948b467 Merge pull request #7 from hanzoai/fix/buildx-ghpat-fallback
fix(ci): fall back buildx gh_token to GH_PAT when KMS token is empty
2026-07-04 21:09:07 -07:00
hanzo-dev 2ef5d47f94 fix(ci): fall back buildx gh_token to GH_PAT when KMS token is empty
The buildx `gh_token` secret (for reading private cross-org Go modules like
github.com/hanzoai/cloud during `go mod tidy`) was mounted ONLY from the
KMS-fetched GIT_TOKEN. When the KMS deploy-cred login fails (or KMS is
unconfigured), GIT_TOKEN is empty, the --secret is omitted, and the buildx
stage's `go mod tidy` hits the private repo unauthenticated -> git exit 128 ->
image build fails. This broke every repo with a private cross-org dep
(hanzoai/commerce, hanzoai/ai) while the KMS login was down.

Fall back to the existing valid GH_PAT org secret — the SAME BuildKit gh_token
cloud/release.yml already uses successfully. Guarded + exported; no-op for
public-only builds when both are empty. One credential path, proven working.
2026-07-04 21:08:47 -07:00
3a9e055c45 feat(ci): opt-in multi-arch builds (amd64+arm64) via hanzo.yml platforms (#5)
DOKS has no arm64 nodes. Add per-image opt-in `platforms:` in hanzo.yml:
default stays [linux/amd64] (every existing repo's -amd64 tag shape UNCHANGED,
zero behavior change). Set [linux/amd64, linux/arm64] → buildx emits a
multi-arch manifest list (one digest, both arches); binfmt/QEMU installed for
arm64 emulation, and pure-Go CGO_ENABLED=0 Dockerfiles honoring $TARGETARCH
cross-compile natively (fast). For native-speed arm64, register a bare-metal
arm64 host (spark/GB10) as the hanzo-build-linux-arm64 runner (values exist).

Co-authored-by: hanzo-dev <dev@hanzo.ai>
2026-07-04 19:47:09 -07:00
hanzo-dev 8ae1c46625 fix(ci): KMS step must set +e (GitHub wraps run in bash -e)
The step is documented BEST-EFFORT (GHCR push uses the workflow token;
deploy creds are optional), and its 'set -uo pipefail' deliberately omits
-e. But GitHub runs 'run:' under 'bash -eo pipefail', so errexit is active
regardless — an unguarded curl (a KMS secret 404 at the caller's org/path)
aborted the step (exit 22) and failed the whole build. Explicitly 'set +e'
so KMS degrades gracefully (missing kubeconfig -> deploy simply skipped).
2026-07-03 15:42:25 -07:00
hanzo-dev 9d18d4a6ba fix(ci): provision jq + PyYAML on the runner before parsing hanzo.yml
The stock arc runner image (ghcr.io/actions/actions-runner:latest) is
minimal and ships neither jq nor python3-yaml, but the build step parses
hanzo.yml with python3+PyYAML under 'set -euo pipefail' and fails:
  ModuleNotFoundError: No module named 'yaml'
(The KMS step swallowed the same error via '|| true'.)

Add a guarded setup step (apt-get python3-yaml jq) right after checkout —
idempotent, a no-op once a runner image bakes them in. Keeps the reusable
self-contained so any org can import it onto a bare runner.
2026-07-03 15:34:24 -07:00
ecc3da75f0 fix(ci): build.yml runner defaults to online ARC pool, not offline evo (same disease as .github#11) (#3)
Co-authored-by: Hanzo AI <ai@hanzo.ai>
2026-07-02 23:38:52 -07:00
2 changed files with 152 additions and 5 deletions
+125 -5
View File
@@ -16,6 +16,18 @@ on:
of your own self-hosted arc runners instead.
type: string
default: '["hanzo-build-linux-amd64"]'
mode:
description: >-
Build execution mode. `buildx` (default) runs the full buildx →
test → deploy pipeline ON the arc runner. `delegate` instead POSTs the
build to platform.hanzo.ai (`/v1/arcd/enqueue`) — platform builds
in-cluster with BuildKit and rolls the operator Service CR itself, so
the GitHub job finishes in seconds with no runner buildx. A repo opts
in by passing `with: { mode: delegate }`; everything else is unchanged.
Requires the `PLATFORM_BUILD_CALLBACK_TOKEN` secret (via secrets:
inherit).
type: string
default: buildx
permissions:
contents: read
@@ -27,7 +39,76 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Provision parse toolchain (jq + PyYAML)
# This reusable parses the caller's hanzo.yml with python3 + PyYAML and
# slices JSON with jq. The stock arc runner image
# (ghcr.io/actions/actions-runner:latest) is minimal and ships NEITHER,
# so provision them here. Guarded (a no-op the moment a runner image bakes
# them in) — this keeps the reusable self-contained: any org can import it
# onto a bare runner and it just works.
run: |
set -e
need=0
python3 -c 'import yaml' 2>/dev/null || need=1
command -v jq >/dev/null 2>&1 || need=1
if [ "$need" = 1 ]; then
sudo apt-get update -qq
sudo apt-get install -y -qq python3-yaml jq
fi
python3 -c 'import yaml; print("PyYAML", yaml.__version__)'
jq --version
- name: Delegate build to platform (mode=delegate)
# The GHA-escape fast path: instead of running buildx on this runner, POST
# each image in hanzo.yml to platform.hanzo.ai's direct-enqueue webhook
# (`/v1/arcd/enqueue`). Platform creates a build_job row, launches an
# in-cluster BuildKit Job on its own pool, pushes to the registry, and —
# for a system service — patches the operator Service CR to roll it. The
# downstream is IDENTICAL to the platform GitHub-App webhook path (one
# build path, two front doors), so a delegated build behaves exactly like
# a platform-native one. This job then exits in seconds — no buildx, no
# KMS, no runner-side deploy.
if: inputs.mode == 'delegate'
env:
ENQUEUE_URL: ${{ vars.PLATFORM_ENQUEUE_URL || 'https://platform.hanzo.ai/v1/arcd/enqueue' }}
ENQUEUE_TOKEN: ${{ secrets.PLATFORM_BUILD_CALLBACK_TOKEN }}
run: |
set -euo pipefail
if [ -z "${ENQUEUE_TOKEN:-}" ]; then
echo "::error::mode=delegate needs the PLATFORM_BUILD_CALLBACK_TOKEN secret (secrets: inherit)"; exit 1
fi
REPO="${{ github.repository }}"
SHA="${{ github.sha }}"
SHORT=$(echo "$SHA" | cut -c1-7)
REF="${{ github.ref }}"
BRANCH="${{ github.ref_name }}"
# One enqueue per (image, platform), mirroring the buildx tag shape the
# deploy path expects (`sha-<short>-<arch>[-<suffix>]`). Default arch is
# amd64 (single-arch), so an existing repo's tag shape is unchanged.
python3 -c "import yaml,json;print(json.dumps(yaml.safe_load(open('hanzo.yml'))['images']))" | jq -c '.[]' | while read -r img; do
name=$(echo "$img"|jq -r .name); repo=$(echo "$img"|jq -r .repo)
ctx=$(echo "$img"|jq -r .context); df=$(echo "$img"|jq -r '.dockerfile // (.context+"/Dockerfile")')
sfx=$(echo "$img"|jq -r '."tag-suffix" // ""')
echo "$img" | jq -r '(.platforms // ["linux/amd64"])[]' | while read -r plat; do
arch="${plat##*/}"
image="${repo}:sha-${SHORT}-${arch}${sfx:+-$sfx}"
body=$(jq -nc \
--arg repo "$REPO" --arg sha "$SHA" --arg image "$image" \
--arg ref "$REF" --arg branch "$BRANCH" \
--arg dockerfile "$df" --arg context "$ctx" --arg arch "$arch" \
'{repo:$repo,sha:$sha,image:$image,ref:$ref,branch:$branch,dockerfile:$dockerfile,context:$context,os:"linux",arch:$arch}')
echo "::group::delegate $name → $image"
code=$(curl -sS -o /tmp/enqueue.out -w '%{http_code}' -X POST "$ENQUEUE_URL" \
-H "Authorization: Bearer $ENQUEUE_TOKEN" -H 'Content-Type: application/json' -d "$body")
cat /tmp/enqueue.out; echo
# 202 Accepted = queued; 409 = no live runner for the pool (surface it loud).
if [ "$code" != "202" ]; then echo "::error::enqueue $image failed (HTTP $code)"; exit 1; fi
echo "::endgroup::"
done
done
- name: Authenticated git for go modules (rate-limit + any private repo)
if: inputs.mode != 'delegate'
# luxfi/hanzoai/zooai Go modules are PUBLIC, so `go` resolves them through
# the default public proxy (proxy.golang.org) + checksum db (sum.golang.org)
# — canonical, IMMUTABLE hashes that a force-moved tag can no longer break.
@@ -64,6 +145,7 @@ jobs:
|| echo "::warning::GH_PAT set but repo probe failed"
- name: Log in to GHCR (automatic workflow token)
if: inputs.mode != 'delegate'
# GHCR push needs no stored credential: GitHub injects a per-job
# GITHUB_TOKEN scoped to THIS repo, and `permissions: packages: write`
# (above) lets it push the repo's own package (e.g. zooai/node →
@@ -73,12 +155,19 @@ jobs:
- name: Fetch deploy credentials from KMS
id: kms
if: inputs.mode != 'delegate'
env:
KMS_CLIENT_ID: ${{ secrets.KMS_CLIENT_ID }}
KMS_CLIENT_SECRET: ${{ secrets.KMS_CLIENT_SECRET }}
KMS_ENDPOINT: ${{ vars.KMS_ENDPOINT || 'https://kms.hanzo.ai' }}
KMS_ORG: ${{ vars.KMS_ORG }}
run: |
# This step is BEST-EFFORT (see below): GHCR push already works via the
# workflow token, and deploy creds are optional. GitHub wraps `run:` in
# `bash -eo pipefail`, so we MUST explicitly `set +e` — otherwise an
# unguarded curl (e.g. a KMS secret 404 at this org/path) aborts the
# step and fails the whole build. Keep -u/-o pipefail; drop -e.
set +e
set -uo pipefail
# Canonical luxfi/kms surface — /v1/kms (the Infisical /api/* surface was
# removed when KMS migrated to luxfi/kms, MPC-rooted). Auth = an IAM
@@ -113,8 +202,20 @@ jobs:
if [ -n "$KUBECONFIG_B64" ]; then echo "$KUBECONFIG_B64" | base64 -d > "$RUNNER_TEMP/kubeconfig"; echo "kubeconfig=$RUNNER_TEMP/kubeconfig" >> "$GITHUB_OUTPUT"; fi
- name: Build & push images (per hanzo.yml)
if: inputs.mode != 'delegate'
env:
GH_PAT: ${{ secrets.GH_PAT }}
run: |
set -euo pipefail
# Build-time private cross-org Go module read (the buildx `gh_token`
# secret): prefer the KMS-fetched GIT_TOKEN, else fall back to the org
# GH_PAT — the SAME BuildKit gh_token cloud's release.yml uses (proven
# working). Keeps image builds green when the KMS deploy-cred fetch is
# unavailable (a repo with a private cross-org dep like hanzoai/cloud
# otherwise fails `go mod tidy` with git exit 128 in the buildx stage).
# No-op for public-only builds when both are empty. Exported so the
# `--secret id=gh_token,env=GIT_TOKEN` below reads it from the env.
export GIT_TOKEN="${GIT_TOKEN:-${GH_PAT:-}}"
SHORT=$(echo "${{ github.sha }}" | cut -c1-7)
IS_TAG=$([ "${{ github.ref_type }}" = "tag" ] && echo 1 || echo 0)
VER="${{ github.ref_name }}"; VER="${VER#v}"
@@ -125,17 +226,36 @@ jobs:
# (e.g. "ce"/"ee") it qualifies every tag; when absent the tags are
# clean (no trailing dash). Build + deploy must agree on this shape.
sfx=$(echo "$img"|jq -r '."tag-suffix" // ""')
TAGS="-t $repo:sha-${SHORT}-amd64${sfx:+-$sfx} -t $repo:${sfx:+$sfx-}latest"
[ "$IS_TAG" = 1 ] && TAGS="$TAGS -t $repo:${VER}-amd64${sfx:+-$sfx}"
echo "::group::build $name → $repo (${sfx})"
# platforms is OPT-IN per image in hanzo.yml (default: amd64 only, so
# every existing repo's tag shape "-amd64" is UNCHANGED). Set e.g.
# platforms: [linux/amd64, linux/arm64]
# to emit a multi-arch MANIFEST LIST — one digest serving both arches.
# DOKS has no arm64 nodes, so arm64 builds via buildx QEMU emulation
# (binfmt set up below); pure-Go (CGO_ENABLED=0) Dockerfiles that honor
# $TARGETARCH cross-compile natively (fast, no emulation). For true
# native-speed arm64, register a bare-metal arm64 host (spark/GB10) as
# the hanzo-build-linux-arm64 self-hosted runner (values-build-arm64.yaml).
plats=$(echo "$img"|jq -r '(.platforms // ["linux/amd64"]) | join(",")')
if [ "$plats" = "linux/amd64" ]; then
# single-arch: keep the exact legacy tag shape (-amd64) deploys expect.
TAGS="-t $repo:sha-${SHORT}-amd64${sfx:+-$sfx} -t $repo:${sfx:+$sfx-}latest"
[ "$IS_TAG" = 1 ] && TAGS="$TAGS -t $repo:${VER}-amd64${sfx:+-$sfx}"
else
# multi-arch: one arch-neutral manifest-list tag (no -amd64 suffix).
docker run --privileged --rm tonistiigi/binfmt --install arm64 >/dev/null 2>&1 || true
TAGS="-t $repo:sha-${SHORT}${sfx:+-$sfx} -t $repo:${sfx:+$sfx-}latest"
[ "$IS_TAG" = 1 ] && TAGS="$TAGS -t $repo:${VER}${sfx:+-$sfx}"
fi
echo "::group::build $name → $repo (${sfx}) [$plats]"
# GIT_TOKEN (from KMS, via GITHUB_ENV) is passed as the `gh_token`
# BuildKit secret so Dockerfiles can clone private Go modules; omitted
# cleanly when absent (public-only builds unaffected).
docker buildx build --platform linux/amd64 ${GIT_TOKEN:+--secret id=gh_token,env=GIT_TOKEN} --push $TAGS -f "$df" "$ctx"
docker buildx build --platform "$plats" ${GIT_TOKEN:+--secret id=gh_token,env=GIT_TOKEN} --push $TAGS -f "$df" "$ctx"
echo "::endgroup::"
done
- name: Test (per hanzo.yml)
if: inputs.mode != 'delegate'
run: |
set -euo pipefail
python3 -c "import yaml,json;print(json.dumps(yaml.safe_load(open('hanzo.yml')).get('test') or []))" | jq -c '.[]' | while read -r t; do
@@ -144,7 +264,7 @@ jobs:
done
- name: Deploy (per hanzo.yml)
if: github.event_name != 'pull_request' && steps.kms.outputs.kubeconfig != ''
if: inputs.mode != 'delegate' && github.event_name != 'pull_request' && steps.kms.outputs.kubeconfig != ''
env:
KUBECONFIG: ${{ steps.kms.outputs.kubeconfig }}
run: |
+27
View File
@@ -52,6 +52,33 @@ build minutes). To run on **your own** self-hosted arc runners, pass their label
secrets: inherit
```
## Delegate to platform (skip runner buildx)
By default the build runs buildx **on** the arc runner. To instead hand the build
to **platform.hanzo.ai** — which builds in-cluster with BuildKit and rolls the
service itself — pass `mode: delegate`:
```yaml
uses: hanzoai/ci/.github/workflows/build.yml@v1
with:
mode: delegate
secrets: inherit
```
The GitHub job then just POSTs each image in `hanzo.yml` to platform's direct
build webhook (`/v1/arcd/enqueue`) and exits in **seconds** — no runner buildx,
no KMS, no runner-side deploy. Platform creates the build job, launches an
in-cluster BuildKit Job on its own pool, pushes to the registry, and patches the
operator `Service` CR to roll it. It's the same build path as the platform
GitHub-App webhook — one build path, two front doors.
Requires one extra secret, `PLATFORM_BUILD_CALLBACK_TOKEN` (org- or repo-level,
picked up via `secrets: inherit`). Override the endpoint with the
`PLATFORM_ENQUEUE_URL` repo/org variable (default `https://platform.hanzo.ai/v1/arcd/enqueue`).
`mode: buildx` (the default) is unchanged — existing repos keep running buildx on
arc, so delegation is strictly opt-in.
## Credentials
The only GitHub secrets a repo sets are `KMS_CLIENT_ID` / `KMS_CLIENT_SECRET`