9 Commits

Author SHA1 Message Date
ada76b0153 fix(rpm): add missing native build dependencies
All checks were successful
CI / Format, lint, build, test (push) Successful in 4m34s
CI / Build neuron SRPM (push) Successful in 1m49s
CI / Build cortex SRPM (push) Successful in 44s
CI / Publish cortex to COPR (push) Successful in 7m14s
CI / Publish neuron to COPR (push) Successful in 5m43s
CI / Bump version in source (push) Successful in 52s
COPR build failed on openssl-sys because openssl headers were not
available in the mock chroot. Adding:

- pkgconfig(openssl): fixes the immediate openssl-sys failure.
  Kept as a build dep because we plan to add optional mTLS between
  cortex and neuron, which requires native-tls/openssl at build time.
- cmake, gcc-c++: aws-lc-sys (pulled via rustls) compiles libcrypto
  via cmake and includes C++ sources. Would be the next failure after
  openssl.
- perl-interpreter: catchall for -sys crate build scripts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 10:49:20 +03:00
15ded3a5bd ci: cache target/, disable incremental, drop redundant build
Three complementary tweaks to close the gap sccache alone can't:

- CARGO_INCREMENTAL=0: reclaims the 17 incremental-mode cache misses
  per run and prevents cargo from writing incremental fingerprints
  that defeat sccache. Incremental mode is useless in CI anyway since
  each run starts from scratch.
- actions/cache for ~/.cargo and target/: sidesteps sccache's
  structural limits (proc-macro non-cacheables, clippy-vs-rustc
  separate namespaces) by caching the whole build output keyed on
  Cargo.lock. Also caches ~/.cargo/bin so the installed sccache
  binary survives between runs.
- Drop the separate 'cargo build' step: 'cargo test --workspace'
  builds everything anyway, so the standalone build was a full
  redundant workspace compile pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 09:44:45 +03:00
7befa882d5 fix: yaml syntax
Some checks failed
CI / Format, lint, build, test (push) Successful in 1m42s
CI / Build neuron SRPM (push) Successful in 42s
CI / Build cortex SRPM (push) Successful in 1m40s
CI / Publish neuron to COPR (push) Failing after 4m11s
CI / Publish cortex to COPR (push) Failing after 3m16s
CI / Bump version in source (push) Has been skipped
2026-04-16 09:25:02 +03:00
d03fae960a fix(ci): unset RUSTC_WRAPPER during sccache install
All checks were successful
CI / Format, lint, build, test (push) Successful in 2m40s
CI / Build cortex SRPM (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
The workflow-level env set RUSTC_WRAPPER=sccache for every step,
including the install step itself. cargo install sccache then
tried to invoke `sccache rustc -vV` to detect the toolchain before
sccache existed on PATH, failing with "No such file or directory".
Override RUSTC_WRAPPER to empty on the install step so cargo uses
rustc directly; subsequent steps still inherit the wrapper.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 08:31:26 +03:00
7b2235d56b fix(ci): install sccache with S3 feature if missing
Some checks failed
CI / Format, lint, build, test (push) Failing after 4s
CI / Build cortex SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
The distro sccache package lacks S3 support. Install from cargo
with --features s3 if the existing binary can't connect to the
S3 backend. Skips install if already present and working.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 17:44:21 +03:00
54f9f3dc36 ci: add sccache with MinIO backend for build caching
Some checks failed
CI / Format, lint, build, test (push) Failing after 3s
CI / Build cortex SRPM (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
All Rust compilation steps now use sccache backed by MinIO S3
at caveman.kosherinata.internal:9000. Credentials via repo secrets
SCCACHE_S3_ACCESS_KEY and SCCACHE_S3_SECRET_KEY. Cache is shared
across all bare metal runners.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 17:38:13 +03:00
caee8bba11 fix(ci): use GITEA_TOKEN env var for push, not checkout
Some checks failed
CI / Format, lint, build, test (push) Successful in 2m40s
CI / Build neuron SRPM (push) Successful in 47s
CI / Build cortex SRPM (push) Successful in 48s
CI / Publish cortex to COPR (push) Failing after 7s
CI / Publish neuron to COPR (push) Failing after 3s
CI / Bump version in source (push) Has been skipped
Token is only needed for the authenticated push, not the public
checkout. Set remote URL with token inline before pushing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 16:31:13 +03:00
324dfa05c5 ci: add RPM packaging for cortex and neuron
- cortex.spec: gateway binary, cortex.service systemd unit,
  cortex.toml + models.toml config files
- neuron.spec: neuron binary, neuron.service systemd unit,
  neuron.toml config file
- Parallel CI: srpm-cortex and srpm-neuron jobs build SRPMs
  concurrently, then publish to separate COPR repos
  (helexa/cortex and helexa/neuron)
- bump-version job: after both COPR publishes succeed, stamps
  tag version into Cargo.toml, specs, Cargo.lock and pushes
  to main via GITEA_TOKEN
- Shared cortex user/group across both packages
- Example configs: cortex.example.toml, neuron.example.toml,
  models.example.toml

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 16:28:31 +03:00
c85d50066e ci: add RPM packaging for cortex and neuron
- cortex.spec: gateway binary, cortex.service systemd unit,
  cortex.toml + models.toml config files
- neuron.spec: neuron binary, neuron.service systemd unit,
  neuron.toml config file
- Parallel CI: srpm-cortex and srpm-neuron jobs build SRPMs
  concurrently, then publish to separate COPR repos
  (helexa/cortex and helexa/neuron)
- Shared cortex user/group across both packages
- Example configs: cortex.example.toml, neuron.example.toml,
  models.example.toml

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 16:09:04 +03:00
8 changed files with 338 additions and 44 deletions

View File

@@ -2,11 +2,21 @@ name: CI
on: on:
push: push:
branches: ['**'] branches: ["**"]
tags: ['v*'] tags: ["v*"]
pull_request: pull_request:
branches: [main] branches: [main]
env:
CARGO_INCREMENTAL: "0"
RUSTC_WRAPPER: sccache
SCCACHE_BUCKET: sccache
SCCACHE_ENDPOINT: http://caveman.kosherinata.internal:9000
SCCACHE_REGION: auto
SCCACHE_S3_USE_SSL: "false"
AWS_ACCESS_KEY_ID: ${{ secrets.SCCACHE_S3_ACCESS_KEY }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.SCCACHE_S3_SECRET_KEY }}
jobs: jobs:
check: check:
name: Format, lint, build, test name: Format, lint, build, test
@@ -14,20 +24,43 @@ jobs:
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Cache cargo registry and target
uses: actions/cache@v4
with:
path: |
~/.cargo/bin
~/.cargo/registry/index
~/.cargo/registry/cache
~/.cargo/git/db
target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- name: Ensure sccache with S3 support
env:
RUSTC_WRAPPER: ""
run: |
if sccache --version 2>/dev/null && sccache --show-stats 2>/dev/null; then
echo "sccache with S3 support already installed"
else
cargo install sccache --features s3 --locked
fi
- name: Check formatting - name: Check formatting
run: cargo fmt --check --all run: cargo fmt --check --all
- name: Clippy - name: Clippy
run: cargo clippy --workspace -- -D warnings run: cargo clippy --workspace -- -D warnings
- name: Build
run: cargo build --workspace
- name: Test - name: Test
run: cargo test --workspace run: cargo test --workspace
rpm: - name: Show sccache stats
name: Build SRPM run: sccache --show-stats
srpm-cortex:
name: Build cortex SRPM
runs-on: fedora runs-on: fedora
needs: check needs: check
if: startsWith(github.ref, 'refs/tags/v') if: startsWith(github.ref, 'refs/tags/v')
@@ -39,14 +72,12 @@ jobs:
run: | run: |
VERSION="${GITHUB_REF#refs/tags/v}" VERSION="${GITHUB_REF#refs/tags/v}"
echo "VERSION=${VERSION}" >> "$GITHUB_OUTPUT" echo "VERSION=${VERSION}" >> "$GITHUB_OUTPUT"
echo "Building version: ${VERSION}"
- name: Stamp version into spec - name: Stamp version
run: | run: |
VERSION="${{ steps.version.outputs.VERSION }}" VERSION="${{ steps.version.outputs.VERSION }}"
sed -i '/\[workspace\.package\]/,/\[/{ s/^version = ".*"/version = "'"${VERSION}"'"/ }' Cargo.toml sed -i '/\[workspace\.package\]/,/\[/{ s/^version = ".*"/version = "'"${VERSION}"'"/ }' Cargo.toml
sed -i "s/^Version:.*/Version: ${VERSION}/" cortex.spec sed -i "s/^Version:.*/Version: ${VERSION}/" cortex.spec
echo "Stamped version ${VERSION}"
- name: Generate source tarball - name: Generate source tarball
run: | run: |
@@ -77,19 +108,70 @@ jobs:
- name: Upload SRPM artifact - name: Upload SRPM artifact
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3
with: with:
name: srpm name: srpm-cortex
path: '*.src.rpm' path: "*.src.rpm"
copr: srpm-neuron:
name: Publish to COPR name: Build neuron SRPM
runs-on: fedora runs-on: fedora
needs: rpm needs: check
if: startsWith(github.ref, 'refs/tags/v') if: startsWith(github.ref, 'refs/tags/v')
steps:
- uses: actions/checkout@v4
- name: Determine version
id: version
run: |
VERSION="${GITHUB_REF#refs/tags/v}"
echo "VERSION=${VERSION}" >> "$GITHUB_OUTPUT"
- name: Stamp version
run: |
VERSION="${{ steps.version.outputs.VERSION }}"
sed -i '/\[workspace\.package\]/,/\[/{ s/^version = ".*"/version = "'"${VERSION}"'"/ }' Cargo.toml
sed -i "s/^Version:.*/Version: ${VERSION}/" neuron.spec
- name: Generate source tarball
run: |
set -ex
VERSION="${{ steps.version.outputs.VERSION }}"
tar czf /tmp/neuron-${VERSION}.tar.gz \
--transform "s,^\.,neuron-${VERSION}," \
--exclude='./target' \
--exclude='./.git' \
--exclude='*.tar.gz' \
--exclude='*.src.rpm' \
.
mv /tmp/neuron-${VERSION}.tar.gz .
- name: Vendor Rust dependencies
run: |
VERSION="${{ steps.version.outputs.VERSION }}"
cargo vendor vendor/
tar czf neuron-${VERSION}-vendor.tar.gz vendor/
rm -rf vendor/
- name: Build SRPM
run: |
rpmbuild -bs neuron.spec \
--define "_sourcedir $(pwd)" \
--define "_srcrpmdir $(pwd)"
- name: Upload SRPM artifact
uses: actions/upload-artifact@v3
with:
name: srpm-neuron
path: "*.src.rpm"
copr-cortex:
name: Publish cortex to COPR
runs-on: fedora
needs: srpm-cortex
steps: steps:
- name: Download SRPM - name: Download SRPM
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
with: with:
name: srpm name: srpm-cortex
- name: Configure copr-cli - name: Configure copr-cli
run: | run: |
@@ -97,4 +179,49 @@ jobs:
echo "${{ secrets.COPR_CONFIG }}" > ~/.config/copr echo "${{ secrets.COPR_CONFIG }}" > ~/.config/copr
- name: Submit build to COPR - name: Submit build to COPR
run: copr-cli build cortex *.src.rpm run: copr-cli build helexa/cortex *.src.rpm
copr-neuron:
name: Publish neuron to COPR
runs-on: fedora
needs: srpm-neuron
steps:
- name: Download SRPM
uses: actions/download-artifact@v3
with:
name: srpm-neuron
- name: Configure copr-cli
run: |
mkdir -p ~/.config
echo "${{ secrets.COPR_CONFIG }}" > ~/.config/copr
- name: Submit build to COPR
run: copr-cli build helexa/neuron *.src.rpm
bump-version:
name: Bump version in source
runs-on: fedora
needs: [copr-cortex, copr-neuron]
steps:
- uses: actions/checkout@v4
- name: Stamp version and push
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
run: |
VERSION="${GITHUB_REF#refs/tags/v}"
sed -i '/\[workspace\.package\]/,/\[/{ s/^version = ".*"/version = "'"${VERSION}"'"/ }' Cargo.toml
sed -i "s/^Version:.*/Version: ${VERSION}/" cortex.spec
sed -i "s/^Version:.*/Version: ${VERSION}/" neuron.spec
cargo check --workspace 2>/dev/null || true
git config user.name "Gitea Actions"
git config user.email "actions@git.lair.cafe"
git add Cargo.toml Cargo.lock cortex.spec neuron.spec
if git diff --cached --quiet; then
echo "Version already at ${VERSION}"
else
git commit -m "chore: bump version to ${VERSION}"
git remote set-url origin "https://gitea-actions:${GITEA_TOKEN}@git.lair.cafe/helexa/cortex.git"
git push origin HEAD:main
fi

View File

@@ -590,31 +590,23 @@ Topology-aware placement (min_devices, min_device_vram_mb) deferred —
the router currently routes based on polled model status. Catalogue the router currently routes based on polled model status. Catalogue
placement matching can be added incrementally. placement matching can be added incrementally.
### Phase 10: neuron packaging (RPM) ### Phase 10: RPM packaging
**Goal:** `neuron` and `cortex` are installable via `dnf` from the Completed. Both packages have RPM specs, systemd units, and example configs.
grenade COPR repo. CI builds parallel SRPMs on tag push and publishes to separate COPR repos.
**Steps:** - `cortex.spec``helexa/cortex` COPR: binary, systemd unit, config files
1. `neuron.spec` — RPM spec file for the neuron binary. Install to - `neuron.spec` `helexa/neuron` COPR: binary, systemd unit, config
`/usr/libexec/cortex/neuron`. Systemd unit - `data/cortex.service`, `data/neuron.service` — systemd units
`neuron.service`. Config at `/etc/cortex/neuron.toml`. - `cortex.example.toml`, `neuron.example.toml`, `models.example.toml`
2. Update `cortex.spec` — ensure the cortex binary, config, and - CI: parallel `srpm-cortex` + `srpm-neuron` jobs, then parallel COPR publish
`models.toml` are packaged correctly.
3. Gitea Actions CI job: on tag push, build SRPM, submit to COPR. Install:
4. Document the install path:
```sh ```sh
dnf copr enable grenade/cortex dnf copr enable helexa/cortex && dnf install cortex # gateway host
# on the gateway host: dnf copr enable helexa/neuron && dnf install neuron # GPU nodes
dnf install cortex
# on each GPU node:
dnf install neuron
``` ```
**Done when:** `dnf install neuron` on a Fedora 43 host drops the
binary, config, and systemd unit. `systemctl start neuron` runs
discovery and serves `/discovery`.
### Phase 11: llama.cpp harness stub ### Phase 11: llama.cpp harness stub
**Goal:** Prove the harness abstraction works with a second engine. **Goal:** Prove the harness abstraction works with a second engine.

View File

@@ -1,7 +1,7 @@
Name: cortex Name: cortex
Version: 0.1.0 Version: 0.1.0
Release: 1%{?dist} Release: 1%{?dist}
Summary: Inference gateway for multi-node mistral.rs clusters Summary: Inference gateway for multi-node GPU clusters
License: GPL-3.0-or-later License: GPL-3.0-or-later
URL: https://git.lair.cafe/helexa/cortex URL: https://git.lair.cafe/helexa/cortex
@@ -13,13 +13,19 @@ ExclusiveArch: x86_64
BuildRequires: rust >= 1.85 BuildRequires: rust >= 1.85
BuildRequires: cargo BuildRequires: cargo
BuildRequires: gcc BuildRequires: gcc
BuildRequires: gcc-c++
BuildRequires: cmake
BuildRequires: perl-interpreter
BuildRequires: pkgconfig(openssl)
BuildRequires: systemd-rpm-macros BuildRequires: systemd-rpm-macros
Requires(pre): shadow-utils
%description %description
Cortex is a Rust reverse-proxy that sits in front of multiple mistral.rs Cortex is a Rust reverse-proxy that sits in front of multiple inference
inference nodes and presents a unified OpenAI and Anthropic compatible nodes (via neuron daemons) and presents a unified OpenAI and Anthropic
API surface. It handles model routing, lifecycle management, request compatible API surface. It handles model routing, lifecycle management,
translation, and metrics collection. request translation, and metrics collection.
%prep %prep
%autosetup %autosetup
@@ -38,12 +44,33 @@ cargo build --release -p cortex-cli
%install %install
install -Dm755 target/release/cortex %{buildroot}%{_bindir}/cortex install -Dm755 target/release/cortex %{buildroot}%{_bindir}/cortex
install -Dm644 data/cortex.service %{buildroot}%{_unitdir}/cortex.service
install -dm750 %{buildroot}%{_sysconfdir}/cortex
install -Dm640 cortex.example.toml %{buildroot}%{_sysconfdir}/cortex/cortex.toml
install -Dm640 models.example.toml %{buildroot}%{_sysconfdir}/cortex/models.toml
%pre
getent group cortex >/dev/null || groupadd -r cortex
getent passwd cortex >/dev/null || useradd -r -g cortex -d /var/lib/cortex -s /sbin/nologin cortex
%post
%systemd_post cortex.service
%preun
%systemd_preun cortex.service
%postun
%systemd_postun_with_restart cortex.service
%files %files
%license LICENSE %license LICENSE
%doc README.md %doc README.md
%{_bindir}/cortex %{_bindir}/cortex
%{_unitdir}/cortex.service
%dir %attr(750,root,cortex) %{_sysconfdir}/cortex
%config(noreplace) %attr(640,root,cortex) %{_sysconfdir}/cortex/cortex.toml
%config(noreplace) %attr(640,root,cortex) %{_sysconfdir}/cortex/models.toml
%changelog %changelog
* Mon Apr 14 2026 Rob Thijssen <grenade@rob.tn> - 0.1.0-1 * Tue Apr 15 2026 Rob Thijssen <grenade@rob.tn> - 0.1.0-1
- Initial package - Initial package

15
data/cortex.service Normal file
View File

@@ -0,0 +1,15 @@
[Unit]
Description=Cortex — inference gateway for multi-node GPU clusters
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/cortex serve --config /etc/cortex/cortex.toml
Restart=on-failure
RestartSec=5
User=cortex
Group=cortex
[Install]
WantedBy=multi-user.target

15
data/neuron.service Normal file
View File

@@ -0,0 +1,15 @@
[Unit]
Description=Neuron — per-node GPU discovery and harness daemon for cortex
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/neuron --config /etc/cortex/neuron.toml
Restart=on-failure
RestartSec=5
User=cortex
Group=cortex
[Install]
WantedBy=multi-user.target

29
models.example.toml Normal file
View File

@@ -0,0 +1,29 @@
# models.example.toml — model catalogue
#
# Copy to /etc/cortex/models.toml and adjust for your environment.
# Describes how to serve each model. Cortex matches these profiles
# against discovered neuron topologies for placement decisions.
[[models]]
id = "your-org/large-model"
harness = "mistralrs"
quant = "Q4_K_M"
vram_mb = 19000
min_devices = 2
min_device_vram_mb = 10000
pinned_on = ["gpu-large"]
[[models]]
id = "your-org/medium-model"
harness = "mistralrs"
quant = "Q6_K"
vram_mb = 12000
min_devices = 1
pinned_on = ["gpu-medium"]
[[models]]
id = "your-org/embedding-model"
harness = "mistralrs"
quant = "Q8_0"
vram_mb = 8000
min_devices = 1

16
neuron.example.toml Normal file
View File

@@ -0,0 +1,16 @@
# neuron.example.toml — example configuration
#
# Copy to /etc/cortex/neuron.toml and adjust for your environment.
#
# Environment variable overrides use NEURON_ prefix with __ separators:
# NEURON_PORT=9090
port = 9090
# -- Harnesses ---------------------------------------------------------------
# Each [[harnesses]] entry declares an inference engine managed by neuron.
[[harnesses]]
name = "mistralrs"
endpoint = "http://localhost:8080"
systemd_unit = "mistralrs.service"

73
neuron.spec Normal file
View File

@@ -0,0 +1,73 @@
Name: neuron
Version: 0.1.0
Release: 1%{?dist}
Summary: Per-node GPU discovery and harness management daemon for cortex
License: GPL-3.0-or-later
URL: https://git.lair.cafe/helexa/cortex
Source0: %{name}-%{version}.tar.gz
Source1: %{name}-%{version}-vendor.tar.gz
ExclusiveArch: x86_64
BuildRequires: rust >= 1.85
BuildRequires: cargo
BuildRequires: gcc
BuildRequires: gcc-c++
BuildRequires: cmake
BuildRequires: perl-interpreter
BuildRequires: pkgconfig(openssl)
BuildRequires: systemd-rpm-macros
Requires(pre): shadow-utils
%description
Neuron is a per-node daemon for cortex inference clusters. It discovers
local GPU hardware via nvidia-smi, manages inference harnesses (mistral.rs,
llama.cpp), and exposes an HTTP API for model lifecycle management.
%prep
%autosetup
tar xf %{SOURCE1}
mkdir -p .cargo
cat > .cargo/config.toml << 'EOF'
[source.crates-io]
replace-with = "vendored-sources"
[source.vendored-sources]
directory = "vendor"
EOF
%build
cargo build --release -p neuron
%install
install -Dm755 target/release/neuron %{buildroot}%{_bindir}/neuron
install -Dm644 data/neuron.service %{buildroot}%{_unitdir}/neuron.service
install -dm750 %{buildroot}%{_sysconfdir}/cortex
install -Dm640 neuron.example.toml %{buildroot}%{_sysconfdir}/cortex/neuron.toml
%pre
getent group cortex >/dev/null || groupadd -r cortex
getent passwd cortex >/dev/null || useradd -r -g cortex -d /var/lib/cortex -s /sbin/nologin cortex
%post
%systemd_post neuron.service
%preun
%systemd_preun neuron.service
%postun
%systemd_postun_with_restart neuron.service
%files
%license LICENSE
%doc README.md
%{_bindir}/neuron
%{_unitdir}/neuron.service
%dir %attr(750,root,cortex) %{_sysconfdir}/cortex
%config(noreplace) %attr(640,root,cortex) %{_sysconfdir}/cortex/neuron.toml
%changelog
* Tue Apr 15 2026 Rob Thijssen <grenade@rob.tn> - 0.1.0-1
- Initial package