rob thijssen
a9f18c9951
The first real run took 100 minutes. Roughly 60 of those were spent doing things that either did not need doing or did not need doing under emulation. dracut ran three times. The kernel's %posttrans runs kernel-install, which builds an initramfs, and dracut-config-rescue makes it build a second, rescue one — both before this build has written /etc/dracut.conf.d/10-c630.conf and before /proc is bind-mounted, so both are wrong as well as expensive. stage2 then builds the real one. Setting initrd_generator=none in the install root for the duration of the transaction suppresses both: 50-dracut.install and 51-dracut-rescue.install each bail when KERNEL_INSTALL_INITRD_GENERATOR is not "dracut". Excluding dracut-config-rescue also spares the laptop a rescue initramfs on every future kernel update, which on this hardware is not cheap. That file must not ship. With it in place the machine would boot fine and then fail to come back after its next kernel update — a bug that surfaces weeks later looking nothing like an image problem. It is removed from the working copy, and asserted absent again immediately before the root filesystem is built. Compression was running as an aarch64 binary under qemu-user for no reason; zstd does not care what architecture it runs on. It now runs on the host when the host has zstd, falling back to in-container otherwise so the build never depends on it. Measured ~20 minutes against ~2. Add BASE_RECIPE to the staged-base stamp. Excluding a weak dependency changes what the base contains without changing the package list it hashes, so without this the next build would happily reuse a stale base. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XWRjNJMistCy6ngXH5aJLS
3.7 KiB
Gitea runner setup
The build runs on runners labelled metal, executing directly on the host
rather than in a runner-provided container, because it needs to drive podman
itself.
One-time host preparation
On each runner host:
sudo dnf install -y podman qemu-user-static-aarch64
sudo systemctl restart systemd-binfmt
Verify:
cat /proc/sys/fs/binfmt_misc/qemu-aarch64
The flags: line must contain F. That flag makes the kernel open the
interpreter at registration time and keep the reference, so qemu-aarch64-static
resolves even inside a container that does not have it. Without it, every
aarch64 binary inside the build container fails with exec format error.
qemu-user-static-aarch64 registers it correctly; plain qemu-user does not.
The workflow attempts this install itself, but that only works if the runner
account has passwordless sudo. Run probe-runner.yaml to find out.
Checking what a runner can do
The probe runner workflow is manual-dispatch only and reports architecture,
user, sudo, podman, emulation status and free disk. Run it once per runner
before debugging a build failure.
Disk space
The build keeps persistent state in /var/tmp/c630-build on each runner:
| Path | Contents | Rough size |
|---|---|---|
/var/tmp/c630-build/work/<variant>/base |
Staged post-dnf root filesystem | 2 GiB minimal, 8 GiB workstation |
/var/tmp/c630-build/dnf |
Downloaded rpms and repo metadata | 2 GiB minimal, 6 GiB workstation |
Plus, transiently in the job workspace, the working rootfs copy, three
filesystem images and the final disk image. Budget roughly 20 GiB for
minimal and 45 GiB for workstation per runner.
The workflow prunes cached rpms untouched for 30 days. The staged base is not
pruned — it is invalidated by content hash, not age, so a stale one is
harmless. Delete /var/tmp/c630-build to reclaim the space.
Runtime
Every aarch64 binary runs under qemu-user emulation, and rpm scriptlets are the
worst case. A cold minimal build is roughly 40 minutes; workstation is
several hours. timeout-minutes is set to 600 accordingly.
The first measured run took 100 minutes, and about 60 of those were avoidable:
| Cost | Cause | Fix |
|---|---|---|
| ~40 min | The kernel's %posttrans ran dracut twice (normal + rescue) before the C630 dracut config existed, producing an initramfs immediately thrown away |
initrd_generator=none in the install root during the transaction; dracut-config-rescue excluded |
| ~20 min | zstd running under emulation |
Compression moved to the host |
Both are worth remembering if either ever regresses: the symptom is a build that appears to hang with no output, because rpm scriptlet output is buffered until the transaction ends.
A warm build — one where the package set has not changed since that runner last built — skips the dnf transaction entirely and finishes in minutes. Because the state is per-runner and jobs are scheduled across all nine, expect the first build on each runner to be cold. Pinning this workflow to a single runner with a dedicated label would make warm builds the norm at the cost of parallelism.
If this becomes tiresome, the fix is a native aarch64 runner. Register one with
an aarch64 label and change runs-on: metal to runs-on: aarch64 in
.gitea/workflows/build-image.yaml; build/build-image.sh already skips the
emulation check when the host is already the target architecture.
Changing which runners are used
All nine current runners carry metal and podman. runs-on: metal therefore
matches any of them. To pin the build to a subset, give those runners a distinct
label and use it — Gitea requires a runner to carry every label listed in
runs-on.