rob thijssen
a9f18c9951
Some checks failed
build image / build (push) Failing after 44m21s
The first real run took 100 minutes. Roughly 60 of those were spent doing things that either did not need doing or did not need doing under emulation. dracut ran three times. The kernel's %posttrans runs kernel-install, which builds an initramfs, and dracut-config-rescue makes it build a second, rescue one — both before this build has written /etc/dracut.conf.d/10-c630.conf and before /proc is bind-mounted, so both are wrong as well as expensive. stage2 then builds the real one. Setting initrd_generator=none in the install root for the duration of the transaction suppresses both: 50-dracut.install and 51-dracut-rescue.install each bail when KERNEL_INSTALL_INITRD_GENERATOR is not "dracut". Excluding dracut-config-rescue also spares the laptop a rescue initramfs on every future kernel update, which on this hardware is not cheap. That file must not ship. With it in place the machine would boot fine and then fail to come back after its next kernel update — a bug that surfaces weeks later looking nothing like an image problem. It is removed from the working copy, and asserted absent again immediately before the root filesystem is built. Compression was running as an aarch64 binary under qemu-user for no reason; zstd does not care what architecture it runs on. It now runs on the host when the host has zstd, falling back to in-container otherwise so the build never depends on it. Measured ~20 minutes against ~2. Add BASE_RECIPE to the staged-base stamp. Excluding a weak dependency changes what the base contains without changing the package list it hashes, so without this the next build would happily reuse a stale base. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XWRjNJMistCy6ngXH5aJLS
88 lines
3.7 KiB
Markdown
88 lines
3.7 KiB
Markdown
# Gitea runner setup
|
|
|
|
The build runs on runners labelled `metal`, executing directly on the host
|
|
rather than in a runner-provided container, because it needs to drive podman
|
|
itself.
|
|
|
|
## One-time host preparation
|
|
|
|
On each runner host:
|
|
|
|
```sh
|
|
sudo dnf install -y podman qemu-user-static-aarch64
|
|
sudo systemctl restart systemd-binfmt
|
|
```
|
|
|
|
Verify:
|
|
|
|
```sh
|
|
cat /proc/sys/fs/binfmt_misc/qemu-aarch64
|
|
```
|
|
|
|
The `flags:` line must contain `F`. That flag makes the kernel open the
|
|
interpreter at registration time and keep the reference, so `qemu-aarch64-static`
|
|
resolves even inside a container that does not have it. Without it, every
|
|
aarch64 binary inside the build container fails with `exec format error`.
|
|
`qemu-user-static-aarch64` registers it correctly; plain `qemu-user` does not.
|
|
|
|
The workflow attempts this install itself, but that only works if the runner
|
|
account has passwordless sudo. Run `probe-runner.yaml` to find out.
|
|
|
|
## Checking what a runner can do
|
|
|
|
The `probe runner` workflow is manual-dispatch only and reports architecture,
|
|
user, sudo, podman, emulation status and free disk. Run it once per runner
|
|
before debugging a build failure.
|
|
|
|
## Disk space
|
|
|
|
The build keeps persistent state in `/var/tmp/c630-build` on each runner:
|
|
|
|
| Path | Contents | Rough size |
|
|
|---|---|---|
|
|
| `/var/tmp/c630-build/work/<variant>/base` | Staged post-dnf root filesystem | 2 GiB minimal, 8 GiB workstation |
|
|
| `/var/tmp/c630-build/dnf` | Downloaded rpms and repo metadata | 2 GiB minimal, 6 GiB workstation |
|
|
|
|
Plus, transiently in the job workspace, the working rootfs copy, three
|
|
filesystem images and the final disk image. Budget roughly 20 GiB for
|
|
`minimal` and 45 GiB for `workstation` per runner.
|
|
|
|
The workflow prunes cached rpms untouched for 30 days. The staged base is not
|
|
pruned — it is invalidated by content hash, not age, so a stale one is
|
|
harmless. Delete `/var/tmp/c630-build` to reclaim the space.
|
|
|
|
## Runtime
|
|
|
|
Every aarch64 binary runs under qemu-user emulation, and rpm scriptlets are the
|
|
worst case. A cold `minimal` build is roughly 40 minutes; `workstation` is
|
|
several hours. `timeout-minutes` is set to 600 accordingly.
|
|
|
|
The first measured run took 100 minutes, and about 60 of those were avoidable:
|
|
|
|
| Cost | Cause | Fix |
|
|
|---|---|---|
|
|
| ~40 min | The kernel's `%posttrans` ran dracut twice (normal + rescue) before the C630 dracut config existed, producing an initramfs immediately thrown away | `initrd_generator=none` in the install root during the transaction; `dracut-config-rescue` excluded |
|
|
| ~20 min | `zstd` running under emulation | Compression moved to the host |
|
|
|
|
Both are worth remembering if either ever regresses: the symptom is a build
|
|
that appears to hang with no output, because rpm scriptlet output is buffered
|
|
until the transaction ends.
|
|
|
|
A warm build — one where the package set has not changed since that runner last
|
|
built — skips the dnf transaction entirely and finishes in minutes. Because the
|
|
state is per-runner and jobs are scheduled across all nine, expect the first
|
|
build on each runner to be cold. Pinning this workflow to a single runner with
|
|
a dedicated label would make warm builds the norm at the cost of parallelism.
|
|
|
|
If this becomes tiresome, the fix is a native aarch64 runner. Register one with
|
|
an `aarch64` label and change `runs-on: metal` to `runs-on: aarch64` in
|
|
`.gitea/workflows/build-image.yaml`; `build/build-image.sh` already skips the
|
|
emulation check when the host is already the target architecture.
|
|
|
|
## Changing which runners are used
|
|
|
|
All nine current runners carry `metal` and `podman`. `runs-on: metal` therefore
|
|
matches any of them. To pin the build to a subset, give those runners a distinct
|
|
label and use it — Gitea requires a runner to carry every label listed in
|
|
`runs-on`.
|