release: platform-native build+publish via hanzo.yml (retire GHA)

ONE declarative flow for both lux release artifacts, on platform.hanzo.ai +
self-hosted arcd — NO GitHub Actions:

- hanzo.yml: node image build (matrix linux/{amd64,arm64}, native long-poll
  dispatch onto lux-build-* pools, tag-pattern {{git.branch}} -> :vX.Y.Z).
  Validated against the platform parsePlatformConfig.
- scripts/publish_plugin_set.sh: extracts the 12 baked VM plugins from the
  node image + SHA256SUMS, uploads to s3://lux-plugins-<env>/<pluginset>/
  (operator pluginSource), round-trip sha-verified. DRY — no second compile.
- RELEASE.md: the ONE canonical build+publish runbook; lists the .github
  workflows to retire (docker.yml, release.yml, build*.yml, ci.yml, ...).
- LLM.md: points at RELEASE.md as the canonical release path.

Proof (no GitHub): node:v1.30.40 provenance == 44b67b99a0 (reused, not
rebuilt); evm@v1.99.37 + dexvm@v1.5.15 rebuilt natively on the spark fleet
(go1.26.4, CGO=0) and the full 12-plugin set published to
s3://lux-plugins-staging/ (round-trip verified). neo's lux-plugins-devnet
prefix untouched.
This commit is contained in:
zeekay
2026-06-23 09:01:48 -07:00
parent 60a6c4b2ad
commit d440fda72a
4 changed files with 375 additions and 1 deletions
+11 -1
View File
@@ -137,7 +137,17 @@ charge less.
## Essential Commands ## Essential Commands
### Building ### Release & build (canonical) — via platform.hanzo.ai, NOT GitHub Actions
The ONE way to build + publish releases is **[`RELEASE.md`](./RELEASE.md)**:
platform.hanzo.ai reads [`hanzo.yml`](./hanzo.yml) on a `v*` tag push and
schedules the image build onto self-hosted **arcd** pools (`lux-build-linux-*`)
over the native long-poll fabric — no GitHub-Actions hop. ONE `Dockerfile`
build yields BOTH artifacts: the node image (`ghcr.io/luxfi/node:vX.Y.Z`, luxd
+ 12 baked VM plugins) and, via [`scripts/publish_plugin_set.sh`](./scripts/publish_plugin_set.sh),
the plugin set to `s3://lux-plugins-<env>/<pluginset>/` (operator `pluginSource`).
The `.github/workflows/*` build/release workflows are retired (RELEASE.md §Retire).
### Building (local dev only)
```bash ```bash
# Build node binary # Build node binary
./scripts/run_task.sh build ./scripts/run_task.sh build
+187
View File
@@ -0,0 +1,187 @@
# Lux release — build + publish via platform.hanzo.ai (the ONE canonical way)
This is the single, repeatable way to build and publish the Lux release
artifacts. It runs entirely on our own infrastructure — **the PaaS
(platform.hanzo.ai) + self-hosted arcd runners + DOKS/fleet**. There is **no
GitHub Actions build path** (the `.github/workflows/*` build/release workflows
are retired — see [§Retire](#retire-the-github-actions-build-workflows)).
## What a release produces
ONE `Dockerfile` multi-stage build (this repo) is the single source of truth.
It compiles `luxd` + all 12 VM plugins (CGO_ENABLED=0) and yields TWO
distribution surfaces:
| # | Artifact | Destination | Consumed by |
|---|----------|-------------|-------------|
| 1 | node image (luxd + 12 plugins baked at `/luxd/build/plugins/`) | `ghcr.io/luxfi/node:vX.Y.Z` | operator pod image; `startup.sh cp /luxd/build/plugins/*` |
| 2 | plugin set (the 12 VM-ID binaries + `SHA256SUMS`) | `s3://lux-plugins-<env>/<pluginset>/` | operator `plugin-fetch` init container (LuxNetwork CR `pluginSource`) |
Artifact 2 is **extracted from** artifact 1 — the plugins are never compiled
twice. One build, two surfaces (DRY, orthogonal).
The plugin versions are pinned as Dockerfile build-args, kept in lockstep with
this repo's `go.mod`:
- `EVM_VERSION` (luxfi/evm — C-Chain EVM, the `0x9999` settlement surface)
- `CHAINS_REF` (luxfi/chains — the 10 non-DEX VMs incl. bridgevm)
- `DEX_REF` (luxfi/dex `cmd/dchain` — the native D-Chain DEX VM)
## The machinery
```
git tag vX.Y.Z (push)
│ GitHub App webhook ─▶ https://platform.hanzo.ai/v1/github-webhook
platform BuildScheduler ── reads hanzo.yml @ tag, validates, enqueues
│ one build_job per matrix entry
native long-poll fabric (build-queue.ts) NO GitHub Actions hop
│ arcd runner POST /v1/arcd/poll (HMAC)
arcd runner on pool lux-build-linux-<arch>
│ git checkout @ tag → docker build -f Dockerfile . → docker push
ghcr.io/luxfi/node:vX.Y.Z (artifact 1)
│ POST /v1/arcd/complete (status, image_digest)
build_job (DB, system-of-record)
```
- **PaaS**: `platform.hanzo.ai` (`~/work/hanzo/platform`,
`pkg/platform/src/services/ci/`). Owns the schema, the scheduler, the durable
`build_job` record, and the native long-poll dispatch.
- **Build muscle**: self-hosted **arcd** runner pools, `lux-build-linux-amd64`
and `lux-build-linux-arm64` (one daemon per fleet host; `spark` = linux/arm64,
amd64 via buildx). NO GitHub-hosted runners; NO GitHub Actions orchestration
on the native path.
- **Contract**: `~/work/hanzo/platform/docs/PLATFORM_CI.md`.
## Build — the one command
A release is a semver tag push. The declarative entrypoint is this repo's
[`hanzo.yml`](./hanzo.yml); the trigger is one of:
**(a) Tag push (normal release).** Cutting the tag IS the release:
```bash
git tag v1.30.41 && git push origin v1.30.41
```
The webhook maps `refs/tags/v1.30.41``branch=v1.30.41`; `hanzo.yml`'s
`tag-pattern: "{{git.branch}}"` yields the image tag `v1.30.41`. Platform
schedules the amd64 + arm64 builds onto the live `lux-build-*` arcd pools and
pushes the multi-arch image to GHCR.
**(b) On-demand (re-release / backfill).** The platform `buildJob.trigger`
tRPC mutation schedules the same build for an explicit ref, no push required:
```
buildJob.trigger({
installationId: "<luxfi GitHub App installation id>",
repo: "luxfi/node",
sha: "<commit at the tag>",
ref: "refs/tags/v1.30.41",
branch: "v1.30.41" // → image tag via {{git.branch}}
})
```
Track it: `buildJob.list` / `buildJob.one` / `buildJob.logs` (org-scoped).
> A pool goes **native** the moment an arcd runner self-registers for it
> (`arcd_runner.lastSeen` within 90s); until then platform transparently falls
> back to `workflow_dispatch` so a build is never stranded. To run a release
> fully GitHub-free, ensure a `lux-build-linux-{amd64,arm64}` runner is live
> (`tRPC arcd` / the `arcd_runner` table). Set platform env
> `WORKFLOW_DISPATCH_FALLBACK=false` to forbid the legacy hop.
### What the runner runs (identical on a fleet host, for manual/DR builds)
The native path runs exactly the repo's `Dockerfile`. To reproduce on a fleet
host directly (e.g. `spark`), with no platform and no GitHub:
```bash
# on spark (linux/arm64; amd64 via buildx)
git clone --branch v1.30.41 git@github.com:luxfi/node.git && cd node
docker buildx build --platform linux/amd64 \
--build-arg CGO_ENABLED=0 \
-t ghcr.io/luxfi/node:v1.30.41 -f Dockerfile --push .
```
## Publish the plugin set — step 2
After the image exists, publish artifact 2 from it (one command, idempotent,
no second compile). Run on any fleet host or a DOKS Job that has `crane`/docker
+ `mc`; typically the same arcd runner that just built the image:
```bash
scripts/publish_plugin_set.sh \
ghcr.io/luxfi/node:v1.30.41 \
lux-plugins-<env>/<pluginset> \
lux # mc alias for the target MinIO/S3
# e.g. lux-plugins-testnet/v1.3.5
```
It extracts the 12 plugin binaries from the image, writes `SHA256SUMS`, uploads
all to `s3://lux-plugins-<env>/<pluginset>/`, and verifies remote==local sha.
S3 is the in-cluster MinIO (`s3.lux-system.svc.cluster.local:9000`, external
`s3.lux.network`). Configure the `mc` alias once with the `hanzo-s3-secret`
credentials:
```bash
mc alias set lux <endpoint> hanzo "$(kubectl -n lux-system get secret \
hanzo-s3-secret -o jsonpath='{.data.password}' | base64 -d)" --api s3v4
```
A pluginset prefix is **immutable** — bump `<pluginset>` for a new release,
never overwrite a prefix a live network points at.
## Deploy — step 3 (operator, not this repo)
luxd rollout is owned by the **lux operator** (`~/work/lux/operator`,
`LuxNetwork` CR). Update the CR's `image.tag` (artifact 1) and, when the
network fetches plugins from S3, the `pluginSource.bucket` + per-plugin
`sha256` (artifact 2, from the `SHA256SUMS` you just published). The operator's
`plugin-fetch` init container verifies each sha256 fail-closed. This is
deliberately decoupled from build: `hanzo.yml` has **no `deploy:` block**.
## Reproducibility
- The build is **functionally reproducible**: same source tags + same toolchain
(Go 1.26.4) + `CGO_ENABLED=0` ⇒ functionally identical plugins, provable by a
fleet rebuild (verified: `spark` rebuilt evm@v1.99.37 + dexvm@v1.5.15 from the
same tags). It is **not bit-identical by construction**: the Dockerfile plugin
stages omit `-trimpath` and use `-mod=mod` with a first-party `go.sum` strip
(re-resolves luxfi/* deps), so embedded paths + re-tagged module content can
shift the bytes (Go `BuildID` differs; binary ~16 KB larger). The published
image is the canonical artifact; verify against ITS baked sha (what
`publish_plugin_set.sh` records), not a separate fleet build.
- To make releases bit-reproducible (future hardening, patch-only): add
`-trimpath` to every plugin `go build` and pin `go.sum` (drop the strip +
`-mod=mod`). Tracked as a follow-up; not required for correctness.
## Retire the GitHub Actions build workflows
These `.github/workflows/*` build/release/CI workflows are superseded by this
flow and must be removed/disabled (platform owns build; the native long-poll
owns dispatch). Delete them once a `lux-build-*` arcd runner is live:
| Workflow | Replaced by |
|----------|-------------|
| `docker.yml` (built `ghcr.io/luxfi/node` on the `lux-build` ARC pool) | `hanzo.yml` (artifact 1) — native long-poll, NO GitHub Actions |
| `release.yml` | the tag-push trigger above + `scripts/publish_plugin_set.sh` |
| `build.yml`, `ci.yml` | platform CI test step (runner runs `go test` pre-build) |
| `build-linux-binaries.yml` | `Dockerfile` builder stage (luxd binary) |
| `build-ubuntu-amd64-release.yml`, `build-ubuntu-arm64-release.yml` | `Dockerfile` + buildx multi-arch |
| `build-macos-release.yml`, `build-win-release.yml`, `build-and-test-mac-windows.yml` | arcd `lux-build-{macos,windows}-*` pools (matrix in `hanzo.yml` when desired) |
| `build-deb-pkg.sh`, `build-tgz-pkg.sh` (under `.github/workflows/`) | packaging step on the arcd runner (post-build), not GitHub Actions |
| `codeql-analysis.yml`, `fuzz.yml`, `fuzz_merkledb.yml`, `test-database-replay.yml` | scheduled jobs on arcd / DOKS (not a build dependency) |
| `buf-lint.yml`, `buf-push.yml`, `labels.yml`, `stale.yml` | repo-hygiene; migrate to arcd cron or drop |
The same retirement applies to the equivalent build/release workflows in the
plugin-source repos (`luxfi/evm`, `luxfi/chains`, `luxfi/dex`): their artifacts
are built from source by THIS repo's `Dockerfile` at the pinned refs, so those
repos need no independent image/release CI — only their tags. Migrate each by
adding a `hanzo.yml` (if it ships its own image) or deleting its build CI (if it
is consumed only as a Go module / plugin source here).
+39
View File
@@ -0,0 +1,39 @@
# Platform-native CI/CD — luxfi/node (the GitHub-Actions escape)
#
# Build is owned by platform.hanzo.ai (NOT GitHub Actions). On a tag push,
# platform reads this file, schedules ONE build job per matrix entry onto the
# self-hosted arcd long-poll fabric, and an arcd runner on the `lux-build-*`
# pool builds + pushes the image to GHCR. There is NO GitHub-Actions build hop:
# the legacy .github/workflows/docker.yml is retired (see RELEASE.md §retire).
#
# This ONE Dockerfile build produces BOTH lux release artifacts:
# 1. the node image -> ghcr.io/luxfi/node:<tag> (luxd + all 12 VM plugins
# baked at /luxd/build/plugins/)
# 2. the plugin SET -> s3://lux-plugins-<env>/<pluginset>/ (published in a
# second, decoupled step from the image's baked plugins — see
# scripts/publish_plugin_set.sh; consumed by the operator plugin-fetch
# init container via the LuxNetwork CR pluginSource).
#
# Schema reference: ~/work/hanzo/platform/docs/PLATFORM_CI.md.
# Release runbook (the ONE canonical way): ./RELEASE.md.
#
# Pool resolution: <org>-build-<os>-<arch>. org=luxfi -> brand=lux, so this
# repo's pools are `lux-build-linux-amd64` and `lux-build-linux-arm64`. These
# MUST match the live arcd scale-set names exactly.
#
# Tagging: a release is a `v*` git-tag push. The webhook decoder maps the tag
# ref to `branch=<tagname>`, so `tag-pattern: "{{git.branch}}"` yields the
# image tag `vX.Y.Z` (immutable, semver-only — no :latest, no floating tags).
build:
matrix:
- { os: linux, arch: amd64 }
- { os: linux, arch: arm64 }
dockerfile: ./Dockerfile
context: .
image: ghcr.io/luxfi/node
tag-pattern: "{{git.branch}}"
push: true
dispatch: native
# No `deploy:` block. luxd rollout is owned by the lux operator (LuxNetwork CR,
# ~/work/lux/operator) which the hanzo operator Service-CR rollout does not
# model. Deploy is a separate, operator-driven step (RELEASE.md §deploy).
+138
View File
@@ -0,0 +1,138 @@
#!/usr/bin/env bash
#
# publish_plugin_set.sh — publish the VM plugin SET to S3 from a node image.
#
# The ONE canonical way to produce lux release artifact #2 (the plugin set the
# operator's plugin-fetch init container downloads per the LuxNetwork CR
# pluginSource). It is decoupled from, and DRY with, artifact #1 (the node
# image): the plugins are ALREADY built + baked into the image by the single
# Dockerfile multi-stage build, so this step EXTRACTS them — it never compiles
# them a second time. One source of truth, two distribution surfaces.
#
# NO GitHub. Runs on any fleet host or DOKS Job that has `crane` (or docker) +
# the MinIO client `mc`. Typically invoked by the same arcd runner that just
# built + pushed the image (it has the image locally), or on demand from a
# fleet host against the published image.
#
# Usage:
# publish_plugin_set.sh <image-ref> <s3-dest> [mc-alias]
#
# <image-ref> Fully-qualified node image, e.g. ghcr.io/luxfi/node:v1.30.41
# (digest-pinned forms are accepted and preferred for releases).
# <s3-dest> Bucket/prefix WITHOUT scheme, e.g.
# lux-plugins-testnet/v1.3.5 (matches the LuxNetwork CR
# pluginSource.bucket = s3://lux-plugins-testnet/v1.3.5/).
# [mc-alias] Configured `mc` alias for the target MinIO/S3 (default: lux).
# Configure once: `mc alias set lux <endpoint> <key> <secret>`.
#
# The plugin set is the 12 VM-ID files under /luxd/build/plugins/ in the image.
# A SHA256SUMS manifest (objectKey<two-spaces>sha256, one per line) is generated
# and uploaded alongside — the operator plugin-fetch init container verifies
# each plugin's sha256 against the CR (fail-closed on mismatch), so the manifest
# is also the source for the CR's pluginSource.vmPlugins[].sha256 fields.
#
# Idempotent: re-running with the same image + dest re-uploads byte-identical
# objects. A pluginset version is immutable by convention — bump <s3-dest> for a
# new release, never overwrite a published prefix that a live network points at.
set -euo pipefail
IMAGE_REF="${1:-}"
S3_DEST="${2:-}"
MC_ALIAS="${3:-lux}"
if [[ -z "${IMAGE_REF}" || -z "${S3_DEST}" ]]; then
echo "usage: $0 <image-ref> <s3-dest> [mc-alias]" >&2
echo " e.g. $0 ghcr.io/luxfi/node:v1.30.41 lux-plugins-testnet/v1.3.5 lux" >&2
exit 2
fi
# Canonical plugin set: the 12 VM-ID filenames the Dockerfile writes to
# /luxd/build/plugins/. Kept in lockstep with the Dockerfile plugin stages.
PLUGINS=(
mgj786NP7uDwBCcq6YwThhaN8FLyybkCa4zBWTQbNgmK6k9A6 # evm (C-Chain EVM, 0x9999)
mDVT5EWMumBp3LCqvKwuyZQeY1VXr1jvjGNAt8nL4UFiXvqXr # dexvm (native D-Chain DEX)
juFxSrbCM4wszxddKepj1GWwmrn9YgN1g4n3VUWPpRo9JjERA # aivm
kMhHABHM8j4bH94MCc4rsTNdo5E9En37MMyiujk4WdNxgXFsY # bridgevm
nZQm4Dmg1rjX18rb8maL9gamYyXPf1xCvF7ymWzxp6a1nSQTt # graphvm
oR6tnZHezwogyf9fRnomNXC9ojwCEBAU6jdUzpgy2PB1tD7fM # identityvm
pJJCSV7hHYVY6TUZwR8qUPAfuhX8JLb2C1AzNSezrYNbgau8M # keyvm
r5m1ujrmXxVcQetG3CQfuDLHp2RHKh6vCDaFgBRQfUcTZh7eS # oraclevm
ry9Sg8rZdT26iEKvJDmC2wkESs4SDKgZEhk5BgLSwg1EpcNug # quantumvm
sP6dLqrrBR9w3soP18fbJ3YzZecZdD7DDdfH2cFhhLq7Hy9bz # relayvm
tGVBwRxpmD2aFdg3iYjgRvrCe8Jcmq9UNKxyHMus2NZ8WcD8t # thresholdvm
vv3qPfyTVXZ5ArRZA9Jh4hbYDTBe43f7sgQg4CHfNg1rnnvX9 # zkvm
)
WORK="$(mktemp -d)"
trap 'rm -rf "${WORK}"' EXIT
OUT="${WORK}/plugins"
mkdir -p "${OUT}"
echo "==> extracting plugin set from ${IMAGE_REF}"
# Build the tar member list (paths inside the image rootfs).
members=()
for id in "${PLUGINS[@]}"; do
members+=("luxd/build/plugins/${id}")
done
# Prefer crane (no docker daemon needed). Fall back to docker create+cp.
if command -v crane >/dev/null 2>&1; then
crane export "${IMAGE_REF}" - | tar -x -C "${WORK}" "${members[@]}"
elif command -v docker >/dev/null 2>&1; then
cid="$(docker create "${IMAGE_REF}")"
for id in "${PLUGINS[@]}"; do
docker cp "${cid}:/luxd/build/plugins/${id}" "${OUT}/${id}"
done
docker rm -f "${cid}" >/dev/null
# normalize layout to match crane's export path
mkdir -p "${WORK}/luxd/build/plugins"
mv "${OUT}"/* "${WORK}/luxd/build/plugins/" 2>/dev/null || true
OUT="${WORK}/luxd/build/plugins"
else
echo "FATAL: neither crane nor docker is available to extract the image" >&2
exit 1
fi
# crane export wrote to ${WORK}/luxd/build/plugins; converge OUT to it.
[[ -d "${WORK}/luxd/build/plugins" ]] && OUT="${WORK}/luxd/build/plugins"
echo "==> generating SHA256SUMS manifest"
( cd "${OUT}"
: > SHA256SUMS
for id in "${PLUGINS[@]}"; do
[[ -s "${id}" ]] || { echo "FATAL: plugin ${id} missing from image ${IMAGE_REF}" >&2; exit 1; }
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "${id}" >> SHA256SUMS
else
printf '%s %s\n' "$(shasum -a 256 "${id}" | awk '{print $1}')" "${id}" >> SHA256SUMS
fi
done
)
echo "----- SHA256SUMS -----"
cat "${OUT}/SHA256SUMS"
echo "----------------------"
echo "==> uploading plugin set + manifest to s3://${S3_DEST}/ (alias ${MC_ALIAS})"
# Ensure the bucket exists (no-op if present); never deletes/overwrites siblings.
bucket="${S3_DEST%%/*}"
mc mb --ignore-existing "${MC_ALIAS}/${bucket}" >/dev/null 2>&1 || true
for id in "${PLUGINS[@]}"; do
mc cp "${OUT}/${id}" "${MC_ALIAS}/${S3_DEST}/"
done
mc cp "${OUT}/SHA256SUMS" "${MC_ALIAS}/${S3_DEST}/"
echo "==> verifying round-trip integrity (remote sha == local sha)"
fail=0
while read -r want id; do
got="$(mc cat "${MC_ALIAS}/${S3_DEST}/${id}" | { sha256sum 2>/dev/null || shasum -a 256; } | awk '{print $1}')"
if [[ "${got}" != "${want}" ]]; then
echo "MISMATCH ${id}: local ${want} != remote ${got}" >&2
fail=1
else
echo "ok ${id} ${got}"
fi
done < "${OUT}/SHA256SUMS"
[[ "${fail}" -eq 0 ]] || { echo "FATAL: upload integrity check failed" >&2; exit 1; }
echo "==> published plugin set to s3://${S3_DEST}/ (${#PLUGINS[@]} plugins + SHA256SUMS)"