mistralrs-package/readme.md

# mistralrs-package

RPM packaging pipeline for [mistral.rs](https://github.com/EricLBuehler/mistral.rs) on Fedora 43 / x86_64 with CUDA support.

This repo does not contain the mistral.rs source. It clones upstream at a given release tag, cross-compiles with CUDA, and produces signed RPMs published to a dnf repo at `rpm.lair.cafe`.

## How it works

Two Gitea Actions workflows drive the pipeline:

1. **poll-upstream** runs every 15 minutes, checks GitHub for the latest mistral.rs release tag, and triggers a build if the corresponding RPM doesn't already exist on `rpm.lair.cafe`.
2. **build-release** runs in three stages:
   - **build** — clones upstream at the tag and compiles `mistralrs` with flavour-specific CUDA features on a `cuda-13.0` runner.
   - **package** — builds an RPM from the compiled binary using `rpmbuild`.
   - **publish** — GPG-signs the RPMs, rsyncs them to `rpm.lair.cafe`, and updates the repo metadata with `createrepo_c`.

### Flavours

Build flavours are defined in the workflow matrix. Each flavour specifies a name, CUDA home path, cargo features, and compute capabilities. The RPM spec uses `update-alternatives` so multiple flavours can coexist, with priority: base=10, fa=20, nccl=30.

Currently defined:

| Flavour  | Features                      | Compute cap |
|----------|-------------------------------|-------------|
| cuda13   | cuda, cudnn, flash-attn, nccl | sm_120      |

### Systemd integration

Each RPM installs a templated systemd unit (`mistralrs-<flavour>@.service`). Instances are configured via environment files in `/etc/mistralrs/`:

```bash
# copy the example config
sudo cp /etc/mistralrs/cuda13.conf.example /etc/mistralrs/mymodel.conf
# edit MISTRALRS_ARGS, HF_TOKEN, etc.
sudo systemctl start mistralrs-cuda13@mymodel
```

## Infrastructure setup

The RPM repo is hosted on `oolon` (oolon.kosherinata.internal) behind nginx with TLS via Let's Encrypt. The setup scripts in `script/setup/` are run once from a dev workstation with SSH access to oolon.

### 1. DNS

```bash
./script/setup/dns.sh
```

Creates a CNAME record for `rpm.lair.cafe` via the Cloudflare API. Requires a Cloudflare API token in `~/.cloudflare/lair.cafe`.

### 2. TLS certificate

```bash
./script/setup/cert.sh
```

Obtains a Let's Encrypt certificate for `rpm.lair.cafe` using the Cloudflare DNS challenge. Run on oolon.

### 3. Nginx and repo directory

```bash
./script/setup/nginx.sh
```

Syncs the nginx config to oolon, creates the `gitea_ci` system user with SSH access for CI publishing, sets up the RPM repo directory at `/var/www/rpm/fedora/43/x86_64`, and reloads nginx. Requires the `gitea_ci` SSH public key at `~/.ssh/id_gitea_ci.pub`.

### 4. GPG signing key

```bash
./script/setup/gpg.sh
```

Manages the RPM signing key in a dedicated keyring at `~/.gnupg/lair`:

- Creates a certify-only ed25519 master key (no expiry) for `rpm@lair.cafe` if one doesn't exist.
- Adds a signing subkey with 1-year expiry.
- Cross-signs the key with your personal keys from the default keyring.
- Exports the public key and syncs it to `oolon:/var/www/rpm/<short-id>.gpg`.

After running the script, add two secrets to the Gitea repo:

| Secret              | Value                                                                                   |
|---------------------|-----------------------------------------------------------------------------------------|
| `RPM_SIGNING_KEY`   | Output of `gpg --homedir ~/.gnupg/lair --armor --export-secret-subkeys <subkey-fpr>!`   |
| `RPM_SIGNING_KEY_ID`| `rpm@lair.cafe`                                                                         |

The trailing `!` in the export command restricts the export to that specific subkey. Only the signing subkey is shared with CI; the master key stays on the workstation.

#### Rotating the signing subkey

```bash
gpg --homedir ~/.gnupg/lair --quick-add-key <master-fpr> ed25519 sign 1y
```

Then update the `RPM_SIGNING_KEY` secret in Gitea with the new subkey. The public key served to users doesn't change since it's anchored to the master key.

### 5. Runner prerequisites

#### nvm (for UI builds)

Runners that build the UI need [nvm](https://github.com/nvm-sh/nvm) installed for the `gitea_runner` user and an `nvm` label in their runner config:

```bash
sudo -u gitea_runner bash -c 'curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash'
```

Then add `nvm` to the labels in `/etc/act_runner/config.yml`:

```yaml
runner:
  labels:
    - "fedora-43:host"
    - "nvm"
```

Restart the runner after changing labels. The `deploy-ui` workflow uses `runs-on: [fedora-43, nvm]` to select runners with Node.js capability.

#### sequoia-sq (for RPM signing)

Runners that run the publish job need `sequoia-sq` installed:

```bash
sudo dnf install sequoia-sq
```

## Client setup

```bash
sudo rpm --import https://rpm.lair.cafe/<short-id>.gpg
sudo tee /etc/yum.repos.d/lair-cafe.repo > /dev/null <<'EOF'
[lair-cafe]
name=lair.cafe RPM Repository
baseurl=https://rpm.lair.cafe/fedora/$releasever/$basearch/
enabled=1
gpgcheck=1
gpgkey=https://rpm.lair.cafe/<short-id>.gpg
EOF
sudo dnf install mistralrs-cuda13
```

## Forcing a rebuild

To force a rebuild of an already-published RPM (e.g. after a packaging change), remove the RPM from the repo server and update the index:

```bash
ssh oolon "
    sudo rm /var/www/rpm/fedora/43/x86_64/mistralrs-cuda13-<version>-1.fc43.x86_64.rpm \
    && cd /var/www/rpm/fedora/43/x86_64 \
    && sudo createrepo_c --update .;
"
```

The next poll-upstream cycle (every 15 minutes) will detect the missing package and trigger a full rebuild. You can also trigger poll-upstream manually from the Gitea Actions UI to avoid waiting.

Do not delete the RPM without running `createrepo_c --update` afterwards — this leaves the repo index referencing a missing file, which causes errors for dnf clients.

## CI secrets

The build-release workflow requires the following secrets:

| Secret           | Purpose                                      |
|------------------|----------------------------------------------|
| `DISPATCH_TOKEN` | Gitea API token for triggering builds         |
| `RPM_SIGNING_KEY`| ASCII-armored GPG signing subkey              |
| `RPM_SIGNING_KEY_ID` | GPG key UID (`rpm@lair.cafe`)            |
| `RSYNC_SSH_KEY`  | SSH private key for the `gitea_ci` user       |