diff --git a/readme.md b/readme.md new file mode 100644 index 0000000..6d6ee28 --- /dev/null +++ b/readme.md @@ -0,0 +1,121 @@ +# mistralrs-package + +RPM packaging pipeline for [mistral.rs](https://github.com/EricLBuehler/mistral.rs) on Fedora 43 / x86_64 with CUDA support. + +This repo does not contain the mistral.rs source. It clones upstream at a given release tag, cross-compiles with CUDA, and produces signed RPMs published to a dnf repo at `rpm.lair.cafe`. + +## How it works + +Two Gitea Actions workflows drive the pipeline: + +1. **poll-upstream** runs every 15 minutes, checks GitHub for the latest mistral.rs release tag, and triggers a build if the corresponding RPM doesn't already exist on `rpm.lair.cafe`. +2. **build-release** runs in three stages: + - **build** — clones upstream at the tag and compiles `mistralrs-server` with flavour-specific CUDA features on a `cuda-13.0` runner. + - **package** — builds an RPM from the compiled binary using `rpmbuild`. + - **publish** — GPG-signs the RPMs, rsyncs them to `rpm.lair.cafe`, and updates the repo metadata with `createrepo_c`. + +### Flavours + +Build flavours are defined in the workflow matrix. Each flavour specifies a name, CUDA home path, cargo features, and compute capabilities. The RPM spec uses `update-alternatives` so multiple flavours can coexist, with priority: base=10, fa=20, nccl=30. + +Currently defined: + +| Flavour | Features | Compute cap | +|----------|-------------------------------|-------------| +| cuda13 | cuda, cudnn, flash-attn, nccl | sm_120 | + +### Systemd integration + +Each RPM installs a templated systemd unit (`mistralrs-@.service`). Instances are configured via environment files in `/etc/mistralrs/`: + +```bash +# copy the example config +sudo cp /etc/mistralrs/cuda13.conf.example /etc/mistralrs/mymodel.conf +# edit MISTRALRS_ARGS, HF_TOKEN, etc. +sudo systemctl start mistralrs-cuda13@mymodel +``` + +## Infrastructure setup + +The RPM repo is hosted on `oolon` (oolon.kosherinata.internal) behind nginx with TLS via Let's Encrypt. The setup scripts in `script/setup/` are run once from a dev workstation with SSH access to oolon. + +### 1. DNS + +```bash +./script/setup/dns.sh +``` + +Creates a CNAME record for `rpm.lair.cafe` via the Cloudflare API. Requires a Cloudflare API token in `~/.cloudflare/lair.cafe`. + +### 2. TLS certificate + +```bash +./script/setup/cert.sh +``` + +Obtains a Let's Encrypt certificate for `rpm.lair.cafe` using the Cloudflare DNS challenge. Run on oolon. + +### 3. Nginx and repo directory + +```bash +./script/setup/nginx.sh +``` + +Syncs the nginx config to oolon, creates the `gitea_ci` system user with SSH access for CI publishing, sets up the RPM repo directory at `/var/www/rpm/fedora/43/x86_64`, and reloads nginx. Requires the `gitea_ci` SSH public key at `~/.ssh/id_gitea_ci.pub`. + +### 4. GPG signing key + +```bash +./script/setup/gpg.sh +``` + +Manages the RPM signing key in a dedicated keyring at `~/.gnupg/lair`: + +- Creates a certify-only ed25519 master key (no expiry) for `rpm@lair.cafe` if one doesn't exist. +- Adds a signing subkey with 1-year expiry. +- Cross-signs the key with your personal keys from the default keyring. +- Exports the public key and syncs it to `oolon:/var/www/rpm/.gpg`. + +After running the script, add two secrets to the Gitea repo: + +| Secret | Value | +|---------------------|-----------------------------------------------------------------------------------------| +| `RPM_SIGNING_KEY` | Output of `gpg --homedir ~/.gnupg/lair --armor --export-secret-subkeys !` | +| `RPM_SIGNING_KEY_ID`| `rpm@lair.cafe` | + +The trailing `!` in the export command restricts the export to that specific subkey. Only the signing subkey is shared with CI; the master key stays on the workstation. + +#### Rotating the signing subkey + +```bash +gpg --homedir ~/.gnupg/lair --quick-add-key ed25519 sign 1y +``` + +Then update the `RPM_SIGNING_KEY` secret in Gitea with the new subkey. The public key served to users doesn't change since it's anchored to the master key. + +## Client setup + +```bash +sudo rpm --import https://rpm.lair.cafe/.gpg +sudo dnf config-manager addrepo --from-repofile=/dev/stdin <.gpg +EOF +sudo dnf install mistralrs-server-cuda13 +``` + +## CI secrets + +The build-release workflow requires the following secrets: + +| Secret | Purpose | +|------------------|----------------------------------------------| +| `DISPATCH_TOKEN` | Gitea API token for triggering builds | +| `RPM_SIGNING_KEY`| ASCII-armored GPG signing subkey | +| `RPM_SIGNING_KEY_ID` | GPG key UID (`rpm@lair.cafe`) | +| `RSYNC_TARGET` | SSH target for rsync (e.g. `gitea_ci@oolon`) | +| `RSYNC_SSH_KEY` | SSH private key for the `gitea_ci` user |