rob thijssen
cacdbebbf7
- Set RPM_REPO_HOST=oolon.kosherinata.internal as a plain env var instead of treating the hostname as a secret via RSYNC_TARGET - Add explicit SSH connectivity test step using StrictHostKeyChecking=accept-new - Remove ssh-keyscan in favour of accept-new which provides meaningful errors - Remove RSYNC_TARGET secret dependency Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
121 lines
4.8 KiB
Markdown
121 lines
4.8 KiB
Markdown
# mistralrs-package
|
|
|
|
RPM packaging pipeline for [mistral.rs](https://github.com/EricLBuehler/mistral.rs) on Fedora 43 / x86_64 with CUDA support.
|
|
|
|
This repo does not contain the mistral.rs source. It clones upstream at a given release tag, cross-compiles with CUDA, and produces signed RPMs published to a dnf repo at `rpm.lair.cafe`.
|
|
|
|
## How it works
|
|
|
|
Two Gitea Actions workflows drive the pipeline:
|
|
|
|
1. **poll-upstream** runs every 15 minutes, checks GitHub for the latest mistral.rs release tag, and triggers a build if the corresponding RPM doesn't already exist on `rpm.lair.cafe`.
|
|
2. **build-release** runs in three stages:
|
|
- **build** — clones upstream at the tag and compiles `mistralrs-server` with flavour-specific CUDA features on a `cuda-13.0` runner.
|
|
- **package** — builds an RPM from the compiled binary using `rpmbuild`.
|
|
- **publish** — GPG-signs the RPMs, rsyncs them to `rpm.lair.cafe`, and updates the repo metadata with `createrepo_c`.
|
|
|
|
### Flavours
|
|
|
|
Build flavours are defined in the workflow matrix. Each flavour specifies a name, CUDA home path, cargo features, and compute capabilities. The RPM spec uses `update-alternatives` so multiple flavours can coexist, with priority: base=10, fa=20, nccl=30.
|
|
|
|
Currently defined:
|
|
|
|
| Flavour | Features | Compute cap |
|
|
|----------|-------------------------------|-------------|
|
|
| cuda13 | cuda, cudnn, flash-attn, nccl | sm_120 |
|
|
|
|
### Systemd integration
|
|
|
|
Each RPM installs a templated systemd unit (`mistralrs-<flavour>@.service`). Instances are configured via environment files in `/etc/mistralrs/`:
|
|
|
|
```bash
|
|
# copy the example config
|
|
sudo cp /etc/mistralrs/cuda13.conf.example /etc/mistralrs/mymodel.conf
|
|
# edit MISTRALRS_ARGS, HF_TOKEN, etc.
|
|
sudo systemctl start mistralrs-cuda13@mymodel
|
|
```
|
|
|
|
## Infrastructure setup
|
|
|
|
The RPM repo is hosted on `oolon` (oolon.kosherinata.internal) behind nginx with TLS via Let's Encrypt. The setup scripts in `script/setup/` are run once from a dev workstation with SSH access to oolon.
|
|
|
|
### 1. DNS
|
|
|
|
```bash
|
|
./script/setup/dns.sh
|
|
```
|
|
|
|
Creates a CNAME record for `rpm.lair.cafe` via the Cloudflare API. Requires a Cloudflare API token in `~/.cloudflare/lair.cafe`.
|
|
|
|
### 2. TLS certificate
|
|
|
|
```bash
|
|
./script/setup/cert.sh
|
|
```
|
|
|
|
Obtains a Let's Encrypt certificate for `rpm.lair.cafe` using the Cloudflare DNS challenge. Run on oolon.
|
|
|
|
### 3. Nginx and repo directory
|
|
|
|
```bash
|
|
./script/setup/nginx.sh
|
|
```
|
|
|
|
Syncs the nginx config to oolon, creates the `gitea_ci` system user with SSH access for CI publishing, sets up the RPM repo directory at `/var/www/rpm/fedora/43/x86_64`, and reloads nginx. Requires the `gitea_ci` SSH public key at `~/.ssh/id_gitea_ci.pub`.
|
|
|
|
### 4. GPG signing key
|
|
|
|
```bash
|
|
./script/setup/gpg.sh
|
|
```
|
|
|
|
Manages the RPM signing key in a dedicated keyring at `~/.gnupg/lair`:
|
|
|
|
- Creates a certify-only ed25519 master key (no expiry) for `rpm@lair.cafe` if one doesn't exist.
|
|
- Adds a signing subkey with 1-year expiry.
|
|
- Cross-signs the key with your personal keys from the default keyring.
|
|
- Exports the public key and syncs it to `oolon:/var/www/rpm/<short-id>.gpg`.
|
|
|
|
After running the script, add two secrets to the Gitea repo:
|
|
|
|
| Secret | Value |
|
|
|---------------------|-----------------------------------------------------------------------------------------|
|
|
| `RPM_SIGNING_KEY` | Output of `gpg --homedir ~/.gnupg/lair --armor --export-secret-subkeys <subkey-fpr>!` |
|
|
| `RPM_SIGNING_KEY_ID`| `rpm@lair.cafe` |
|
|
|
|
The trailing `!` in the export command restricts the export to that specific subkey. Only the signing subkey is shared with CI; the master key stays on the workstation.
|
|
|
|
#### Rotating the signing subkey
|
|
|
|
```bash
|
|
gpg --homedir ~/.gnupg/lair --quick-add-key <master-fpr> ed25519 sign 1y
|
|
```
|
|
|
|
Then update the `RPM_SIGNING_KEY` secret in Gitea with the new subkey. The public key served to users doesn't change since it's anchored to the master key.
|
|
|
|
## Client setup
|
|
|
|
```bash
|
|
sudo rpm --import https://rpm.lair.cafe/<short-id>.gpg
|
|
sudo dnf config-manager addrepo --from-repofile=/dev/stdin <<EOF
|
|
[lair-rpm]
|
|
name=lair.cafe RPM repo
|
|
baseurl=https://rpm.lair.cafe/fedora/43/x86_64
|
|
enabled=1
|
|
gpgcheck=1
|
|
gpgkey=https://rpm.lair.cafe/<short-id>.gpg
|
|
EOF
|
|
sudo dnf install mistralrs-server-cuda13
|
|
```
|
|
|
|
## CI secrets
|
|
|
|
The build-release workflow requires the following secrets:
|
|
|
|
| Secret | Purpose |
|
|
|------------------|----------------------------------------------|
|
|
| `DISPATCH_TOKEN` | Gitea API token for triggering builds |
|
|
| `RPM_SIGNING_KEY`| ASCII-armored GPG signing subkey |
|
|
| `RPM_SIGNING_KEY_ID` | GPG key UID (`rpm@lair.cafe`) |
|
|
| `RSYNC_SSH_KEY` | SSH private key for the `gitea_ci` user |
|