mistralrs-package

RPM packaging pipeline for mistral.rs on Fedora 43 / x86_64 with CUDA support.

This repo does not contain the mistral.rs source. It clones upstream at a given release tag, cross-compiles with CUDA, and produces signed RPMs published to a dnf repo at rpm.lair.cafe.

How it works

Two Gitea Actions workflows drive the pipeline:

1. poll-upstream runs every 15 minutes, checks GitHub for the latest mistral.rs release tag, and triggers a build if the corresponding RPM doesn't already exist on rpm.lair.cafe.
2. build-release runs in three stages:
   • build — clones upstream at the tag and compiles mistralrs-server with flavour-specific CUDA features on a cuda-13.0 runner.
   • package — builds an RPM from the compiled binary using rpmbuild.
   • publish — GPG-signs the RPMs, rsyncs them to rpm.lair.cafe, and updates the repo metadata with createrepo_c.

Flavours

Build flavours are defined in the workflow matrix. Each flavour specifies a name, CUDA home path, cargo features, and compute capabilities. The RPM spec uses update-alternatives so multiple flavours can coexist, with priority: base=10, fa=20, nccl=30.

Currently defined:

Flavour	Features	Compute cap
cuda13	cuda, cudnn, flash-attn, nccl	sm_120

Systemd integration

Each RPM installs a templated systemd unit (mistralrs-<flavour>@.service). Instances are configured via environment files in /etc/mistralrs/:

# copy the example config
sudo cp /etc/mistralrs/cuda13.conf.example /etc/mistralrs/mymodel.conf
# edit MISTRALRS_ARGS, HF_TOKEN, etc.
sudo systemctl start mistralrs-cuda13@mymodel

Infrastructure setup

The RPM repo is hosted on oolon (oolon.kosherinata.internal) behind nginx with TLS via Let's Encrypt. The setup scripts in script/setup/ are run once from a dev workstation with SSH access to oolon.

1. DNS

./script/setup/dns.sh

Creates a CNAME record for rpm.lair.cafe via the Cloudflare API. Requires a Cloudflare API token in ~/.cloudflare/lair.cafe.

2. TLS certificate

./script/setup/cert.sh

Obtains a Let's Encrypt certificate for rpm.lair.cafe using the Cloudflare DNS challenge. Run on oolon.

3. Nginx and repo directory

./script/setup/nginx.sh

Syncs the nginx config to oolon, creates the gitea_ci system user with SSH access for CI publishing, sets up the RPM repo directory at /var/www/rpm/fedora/43/x86_64, and reloads nginx. Requires the gitea_ci SSH public key at ~/.ssh/id_gitea_ci.pub.

4. GPG signing key

./script/setup/gpg.sh

Manages the RPM signing key in a dedicated keyring at ~/.gnupg/lair:

• Creates a certify-only ed25519 master key (no expiry) for rpm@lair.cafe if one doesn't exist.
• Adds a signing subkey with 1-year expiry.
• Cross-signs the key with your personal keys from the default keyring.
• Exports the public key and syncs it to oolon:/var/www/rpm/<short-id>.gpg.

After running the script, add two secrets to the Gitea repo:

Secret	Value
RPM_SIGNING_KEY	Output of gpg --homedir ~/.gnupg/lair --armor --export-secret-subkeys <subkey-fpr>!
RPM_SIGNING_KEY_ID	rpm@lair.cafe

The trailing ! in the export command restricts the export to that specific subkey. Only the signing subkey is shared with CI; the master key stays on the workstation.

Rotating the signing subkey

gpg --homedir ~/.gnupg/lair --quick-add-key <master-fpr> ed25519 sign 1y

Then update the RPM_SIGNING_KEY secret in Gitea with the new subkey. The public key served to users doesn't change since it's anchored to the master key.

Client setup

sudo rpm --import https://rpm.lair.cafe/<short-id>.gpg
sudo dnf config-manager addrepo --from-repofile=/dev/stdin <<EOF
[lair-rpm]
name=lair.cafe RPM repo
baseurl=https://rpm.lair.cafe/fedora/43/x86_64
enabled=1
gpgcheck=1
gpgkey=https://rpm.lair.cafe/<short-id>.gpg
EOF
sudo dnf install mistralrs-server-cuda13

CI secrets

The build-release workflow requires the following secrets:

Secret	Purpose
DISPATCH_TOKEN	Gitea API token for triggering builds
RPM_SIGNING_KEY	ASCII-armored GPG signing subkey
RPM_SIGNING_KEY_ID	GPG key UID (rpm@lair.cafe)
RSYNC_TARGET	SSH target for rsync (e.g. gitea_ci@oolon)
RSYNC_SSH_KEY	SSH private key for the gitea_ci user