9 Commits

Author SHA1 Message Date
ada76b0153 fix(rpm): add missing native build dependencies
All checks were successful
CI / Format, lint, build, test (push) Successful in 4m34s
CI / Build neuron SRPM (push) Successful in 1m49s
CI / Build cortex SRPM (push) Successful in 44s
CI / Publish cortex to COPR (push) Successful in 7m14s
CI / Publish neuron to COPR (push) Successful in 5m43s
CI / Bump version in source (push) Successful in 52s
COPR build failed on openssl-sys because openssl headers were not
available in the mock chroot. Adding:

- pkgconfig(openssl): fixes the immediate openssl-sys failure.
  Kept as a build dep because we plan to add optional mTLS between
  cortex and neuron, which requires native-tls/openssl at build time.
- cmake, gcc-c++: aws-lc-sys (pulled via rustls) compiles libcrypto
  via cmake and includes C++ sources. Would be the next failure after
  openssl.
- perl-interpreter: catchall for -sys crate build scripts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 10:49:20 +03:00
15ded3a5bd ci: cache target/, disable incremental, drop redundant build
Three complementary tweaks to close the gap sccache alone can't:

- CARGO_INCREMENTAL=0: reclaims the 17 incremental-mode cache misses
  per run and prevents cargo from writing incremental fingerprints
  that defeat sccache. Incremental mode is useless in CI anyway since
  each run starts from scratch.
- actions/cache for ~/.cargo and target/: sidesteps sccache's
  structural limits (proc-macro non-cacheables, clippy-vs-rustc
  separate namespaces) by caching the whole build output keyed on
  Cargo.lock. Also caches ~/.cargo/bin so the installed sccache
  binary survives between runs.
- Drop the separate 'cargo build' step: 'cargo test --workspace'
  builds everything anyway, so the standalone build was a full
  redundant workspace compile pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 09:44:45 +03:00
7befa882d5 fix: yaml syntax
Some checks failed
CI / Format, lint, build, test (push) Successful in 1m42s
CI / Build neuron SRPM (push) Successful in 42s
CI / Build cortex SRPM (push) Successful in 1m40s
CI / Publish neuron to COPR (push) Failing after 4m11s
CI / Publish cortex to COPR (push) Failing after 3m16s
CI / Bump version in source (push) Has been skipped
2026-04-16 09:25:02 +03:00
d03fae960a fix(ci): unset RUSTC_WRAPPER during sccache install
All checks were successful
CI / Format, lint, build, test (push) Successful in 2m40s
CI / Build cortex SRPM (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
The workflow-level env set RUSTC_WRAPPER=sccache for every step,
including the install step itself. cargo install sccache then
tried to invoke `sccache rustc -vV` to detect the toolchain before
sccache existed on PATH, failing with "No such file or directory".
Override RUSTC_WRAPPER to empty on the install step so cargo uses
rustc directly; subsequent steps still inherit the wrapper.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 08:31:26 +03:00
7b2235d56b fix(ci): install sccache with S3 feature if missing
Some checks failed
CI / Format, lint, build, test (push) Failing after 4s
CI / Build cortex SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
The distro sccache package lacks S3 support. Install from cargo
with --features s3 if the existing binary can't connect to the
S3 backend. Skips install if already present and working.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 17:44:21 +03:00
54f9f3dc36 ci: add sccache with MinIO backend for build caching
Some checks failed
CI / Format, lint, build, test (push) Failing after 3s
CI / Build cortex SRPM (push) Has been skipped
CI / Build neuron SRPM (push) Has been skipped
CI / Publish cortex to COPR (push) Has been skipped
CI / Publish neuron to COPR (push) Has been skipped
CI / Bump version in source (push) Has been skipped
All Rust compilation steps now use sccache backed by MinIO S3
at caveman.kosherinata.internal:9000. Credentials via repo secrets
SCCACHE_S3_ACCESS_KEY and SCCACHE_S3_SECRET_KEY. Cache is shared
across all bare metal runners.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 17:38:13 +03:00
caee8bba11 fix(ci): use GITEA_TOKEN env var for push, not checkout
Some checks failed
CI / Format, lint, build, test (push) Successful in 2m40s
CI / Build neuron SRPM (push) Successful in 47s
CI / Build cortex SRPM (push) Successful in 48s
CI / Publish cortex to COPR (push) Failing after 7s
CI / Publish neuron to COPR (push) Failing after 3s
CI / Bump version in source (push) Has been skipped
Token is only needed for the authenticated push, not the public
checkout. Set remote URL with token inline before pushing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 16:31:13 +03:00
324dfa05c5 ci: add RPM packaging for cortex and neuron
- cortex.spec: gateway binary, cortex.service systemd unit,
  cortex.toml + models.toml config files
- neuron.spec: neuron binary, neuron.service systemd unit,
  neuron.toml config file
- Parallel CI: srpm-cortex and srpm-neuron jobs build SRPMs
  concurrently, then publish to separate COPR repos
  (helexa/cortex and helexa/neuron)
- bump-version job: after both COPR publishes succeed, stamps
  tag version into Cargo.toml, specs, Cargo.lock and pushes
  to main via GITEA_TOKEN
- Shared cortex user/group across both packages
- Example configs: cortex.example.toml, neuron.example.toml,
  models.example.toml

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 16:28:31 +03:00
c85d50066e ci: add RPM packaging for cortex and neuron
- cortex.spec: gateway binary, cortex.service systemd unit,
  cortex.toml + models.toml config files
- neuron.spec: neuron binary, neuron.service systemd unit,
  neuron.toml config file
- Parallel CI: srpm-cortex and srpm-neuron jobs build SRPMs
  concurrently, then publish to separate COPR repos
  (helexa/cortex and helexa/neuron)
- Shared cortex user/group across both packages
- Example configs: cortex.example.toml, neuron.example.toml,
  models.example.toml

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 16:09:04 +03:00
8 changed files with 338 additions and 44 deletions

View File

@@ -2,11 +2,21 @@ name: CI
on:
push:
branches: ['**']
tags: ['v*']
branches: ["**"]
tags: ["v*"]
pull_request:
branches: [main]
env:
CARGO_INCREMENTAL: "0"
RUSTC_WRAPPER: sccache
SCCACHE_BUCKET: sccache
SCCACHE_ENDPOINT: http://caveman.kosherinata.internal:9000
SCCACHE_REGION: auto
SCCACHE_S3_USE_SSL: "false"
AWS_ACCESS_KEY_ID: ${{ secrets.SCCACHE_S3_ACCESS_KEY }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.SCCACHE_S3_SECRET_KEY }}
jobs:
check:
name: Format, lint, build, test
@@ -14,20 +24,43 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Cache cargo registry and target
uses: actions/cache@v4
with:
path: |
~/.cargo/bin
~/.cargo/registry/index
~/.cargo/registry/cache
~/.cargo/git/db
target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- name: Ensure sccache with S3 support
env:
RUSTC_WRAPPER: ""
run: |
if sccache --version 2>/dev/null && sccache --show-stats 2>/dev/null; then
echo "sccache with S3 support already installed"
else
cargo install sccache --features s3 --locked
fi
- name: Check formatting
run: cargo fmt --check --all
- name: Clippy
run: cargo clippy --workspace -- -D warnings
- name: Build
run: cargo build --workspace
- name: Test
run: cargo test --workspace
rpm:
name: Build SRPM
- name: Show sccache stats
run: sccache --show-stats
srpm-cortex:
name: Build cortex SRPM
runs-on: fedora
needs: check
if: startsWith(github.ref, 'refs/tags/v')
@@ -39,14 +72,12 @@ jobs:
run: |
VERSION="${GITHUB_REF#refs/tags/v}"
echo "VERSION=${VERSION}" >> "$GITHUB_OUTPUT"
echo "Building version: ${VERSION}"
- name: Stamp version into spec
- name: Stamp version
run: |
VERSION="${{ steps.version.outputs.VERSION }}"
sed -i '/\[workspace\.package\]/,/\[/{ s/^version = ".*"/version = "'"${VERSION}"'"/ }' Cargo.toml
sed -i "s/^Version:.*/Version: ${VERSION}/" cortex.spec
echo "Stamped version ${VERSION}"
- name: Generate source tarball
run: |
@@ -77,19 +108,70 @@ jobs:
- name: Upload SRPM artifact
uses: actions/upload-artifact@v3
with:
name: srpm
path: '*.src.rpm'
name: srpm-cortex
path: "*.src.rpm"
copr:
name: Publish to COPR
srpm-neuron:
name: Build neuron SRPM
runs-on: fedora
needs: rpm
needs: check
if: startsWith(github.ref, 'refs/tags/v')
steps:
- uses: actions/checkout@v4
- name: Determine version
id: version
run: |
VERSION="${GITHUB_REF#refs/tags/v}"
echo "VERSION=${VERSION}" >> "$GITHUB_OUTPUT"
- name: Stamp version
run: |
VERSION="${{ steps.version.outputs.VERSION }}"
sed -i '/\[workspace\.package\]/,/\[/{ s/^version = ".*"/version = "'"${VERSION}"'"/ }' Cargo.toml
sed -i "s/^Version:.*/Version: ${VERSION}/" neuron.spec
- name: Generate source tarball
run: |
set -ex
VERSION="${{ steps.version.outputs.VERSION }}"
tar czf /tmp/neuron-${VERSION}.tar.gz \
--transform "s,^\.,neuron-${VERSION}," \
--exclude='./target' \
--exclude='./.git' \
--exclude='*.tar.gz' \
--exclude='*.src.rpm' \
.
mv /tmp/neuron-${VERSION}.tar.gz .
- name: Vendor Rust dependencies
run: |
VERSION="${{ steps.version.outputs.VERSION }}"
cargo vendor vendor/
tar czf neuron-${VERSION}-vendor.tar.gz vendor/
rm -rf vendor/
- name: Build SRPM
run: |
rpmbuild -bs neuron.spec \
--define "_sourcedir $(pwd)" \
--define "_srcrpmdir $(pwd)"
- name: Upload SRPM artifact
uses: actions/upload-artifact@v3
with:
name: srpm-neuron
path: "*.src.rpm"
copr-cortex:
name: Publish cortex to COPR
runs-on: fedora
needs: srpm-cortex
steps:
- name: Download SRPM
uses: actions/download-artifact@v3
with:
name: srpm
name: srpm-cortex
- name: Configure copr-cli
run: |
@@ -97,4 +179,49 @@ jobs:
echo "${{ secrets.COPR_CONFIG }}" > ~/.config/copr
- name: Submit build to COPR
run: copr-cli build cortex *.src.rpm
run: copr-cli build helexa/cortex *.src.rpm
copr-neuron:
name: Publish neuron to COPR
runs-on: fedora
needs: srpm-neuron
steps:
- name: Download SRPM
uses: actions/download-artifact@v3
with:
name: srpm-neuron
- name: Configure copr-cli
run: |
mkdir -p ~/.config
echo "${{ secrets.COPR_CONFIG }}" > ~/.config/copr
- name: Submit build to COPR
run: copr-cli build helexa/neuron *.src.rpm
bump-version:
name: Bump version in source
runs-on: fedora
needs: [copr-cortex, copr-neuron]
steps:
- uses: actions/checkout@v4
- name: Stamp version and push
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
run: |
VERSION="${GITHUB_REF#refs/tags/v}"
sed -i '/\[workspace\.package\]/,/\[/{ s/^version = ".*"/version = "'"${VERSION}"'"/ }' Cargo.toml
sed -i "s/^Version:.*/Version: ${VERSION}/" cortex.spec
sed -i "s/^Version:.*/Version: ${VERSION}/" neuron.spec
cargo check --workspace 2>/dev/null || true
git config user.name "Gitea Actions"
git config user.email "actions@git.lair.cafe"
git add Cargo.toml Cargo.lock cortex.spec neuron.spec
if git diff --cached --quiet; then
echo "Version already at ${VERSION}"
else
git commit -m "chore: bump version to ${VERSION}"
git remote set-url origin "https://gitea-actions:${GITEA_TOKEN}@git.lair.cafe/helexa/cortex.git"
git push origin HEAD:main
fi

View File

@@ -590,30 +590,22 @@ Topology-aware placement (min_devices, min_device_vram_mb) deferred —
the router currently routes based on polled model status. Catalogue
placement matching can be added incrementally.
### Phase 10: neuron packaging (RPM)
### Phase 10: RPM packaging
**Goal:** `neuron` and `cortex` are installable via `dnf` from the
grenade COPR repo.
Completed. Both packages have RPM specs, systemd units, and example configs.
CI builds parallel SRPMs on tag push and publishes to separate COPR repos.
**Steps:**
1. `neuron.spec` — RPM spec file for the neuron binary. Install to
`/usr/libexec/cortex/neuron`. Systemd unit
`neuron.service`. Config at `/etc/cortex/neuron.toml`.
2. Update `cortex.spec` — ensure the cortex binary, config, and
`models.toml` are packaged correctly.
3. Gitea Actions CI job: on tag push, build SRPM, submit to COPR.
4. Document the install path:
```sh
dnf copr enable grenade/cortex
# on the gateway host:
dnf install cortex
# on each GPU node:
dnf install neuron
```
- `cortex.spec``helexa/cortex` COPR: binary, systemd unit, config files
- `neuron.spec` `helexa/neuron` COPR: binary, systemd unit, config
- `data/cortex.service`, `data/neuron.service` — systemd units
- `cortex.example.toml`, `neuron.example.toml`, `models.example.toml`
- CI: parallel `srpm-cortex` + `srpm-neuron` jobs, then parallel COPR publish
**Done when:** `dnf install neuron` on a Fedora 43 host drops the
binary, config, and systemd unit. `systemctl start neuron` runs
discovery and serves `/discovery`.
Install:
```sh
dnf copr enable helexa/cortex && dnf install cortex # gateway host
dnf copr enable helexa/neuron && dnf install neuron # GPU nodes
```
### Phase 11: llama.cpp harness stub

View File

@@ -1,7 +1,7 @@
Name: cortex
Version: 0.1.0
Release: 1%{?dist}
Summary: Inference gateway for multi-node mistral.rs clusters
Summary: Inference gateway for multi-node GPU clusters
License: GPL-3.0-or-later
URL: https://git.lair.cafe/helexa/cortex
@@ -13,13 +13,19 @@ ExclusiveArch: x86_64
BuildRequires: rust >= 1.85
BuildRequires: cargo
BuildRequires: gcc
BuildRequires: gcc-c++
BuildRequires: cmake
BuildRequires: perl-interpreter
BuildRequires: pkgconfig(openssl)
BuildRequires: systemd-rpm-macros
Requires(pre): shadow-utils
%description
Cortex is a Rust reverse-proxy that sits in front of multiple mistral.rs
inference nodes and presents a unified OpenAI and Anthropic compatible
API surface. It handles model routing, lifecycle management, request
translation, and metrics collection.
Cortex is a Rust reverse-proxy that sits in front of multiple inference
nodes (via neuron daemons) and presents a unified OpenAI and Anthropic
compatible API surface. It handles model routing, lifecycle management,
request translation, and metrics collection.
%prep
%autosetup
@@ -38,12 +44,33 @@ cargo build --release -p cortex-cli
%install
install -Dm755 target/release/cortex %{buildroot}%{_bindir}/cortex
install -Dm644 data/cortex.service %{buildroot}%{_unitdir}/cortex.service
install -dm750 %{buildroot}%{_sysconfdir}/cortex
install -Dm640 cortex.example.toml %{buildroot}%{_sysconfdir}/cortex/cortex.toml
install -Dm640 models.example.toml %{buildroot}%{_sysconfdir}/cortex/models.toml
%pre
getent group cortex >/dev/null || groupadd -r cortex
getent passwd cortex >/dev/null || useradd -r -g cortex -d /var/lib/cortex -s /sbin/nologin cortex
%post
%systemd_post cortex.service
%preun
%systemd_preun cortex.service
%postun
%systemd_postun_with_restart cortex.service
%files
%license LICENSE
%doc README.md
%{_bindir}/cortex
%{_unitdir}/cortex.service
%dir %attr(750,root,cortex) %{_sysconfdir}/cortex
%config(noreplace) %attr(640,root,cortex) %{_sysconfdir}/cortex/cortex.toml
%config(noreplace) %attr(640,root,cortex) %{_sysconfdir}/cortex/models.toml
%changelog
* Mon Apr 14 2026 Rob Thijssen <grenade@rob.tn> - 0.1.0-1
* Tue Apr 15 2026 Rob Thijssen <grenade@rob.tn> - 0.1.0-1
- Initial package

15
data/cortex.service Normal file
View File

@@ -0,0 +1,15 @@
[Unit]
Description=Cortex — inference gateway for multi-node GPU clusters
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/cortex serve --config /etc/cortex/cortex.toml
Restart=on-failure
RestartSec=5
User=cortex
Group=cortex
[Install]
WantedBy=multi-user.target

15
data/neuron.service Normal file
View File

@@ -0,0 +1,15 @@
[Unit]
Description=Neuron — per-node GPU discovery and harness daemon for cortex
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/neuron --config /etc/cortex/neuron.toml
Restart=on-failure
RestartSec=5
User=cortex
Group=cortex
[Install]
WantedBy=multi-user.target

29
models.example.toml Normal file
View File

@@ -0,0 +1,29 @@
# models.example.toml — model catalogue
#
# Copy to /etc/cortex/models.toml and adjust for your environment.
# Describes how to serve each model. Cortex matches these profiles
# against discovered neuron topologies for placement decisions.
[[models]]
id = "your-org/large-model"
harness = "mistralrs"
quant = "Q4_K_M"
vram_mb = 19000
min_devices = 2
min_device_vram_mb = 10000
pinned_on = ["gpu-large"]
[[models]]
id = "your-org/medium-model"
harness = "mistralrs"
quant = "Q6_K"
vram_mb = 12000
min_devices = 1
pinned_on = ["gpu-medium"]
[[models]]
id = "your-org/embedding-model"
harness = "mistralrs"
quant = "Q8_0"
vram_mb = 8000
min_devices = 1

16
neuron.example.toml Normal file
View File

@@ -0,0 +1,16 @@
# neuron.example.toml — example configuration
#
# Copy to /etc/cortex/neuron.toml and adjust for your environment.
#
# Environment variable overrides use NEURON_ prefix with __ separators:
# NEURON_PORT=9090
port = 9090
# -- Harnesses ---------------------------------------------------------------
# Each [[harnesses]] entry declares an inference engine managed by neuron.
[[harnesses]]
name = "mistralrs"
endpoint = "http://localhost:8080"
systemd_unit = "mistralrs.service"

73
neuron.spec Normal file
View File

@@ -0,0 +1,73 @@
Name: neuron
Version: 0.1.0
Release: 1%{?dist}
Summary: Per-node GPU discovery and harness management daemon for cortex
License: GPL-3.0-or-later
URL: https://git.lair.cafe/helexa/cortex
Source0: %{name}-%{version}.tar.gz
Source1: %{name}-%{version}-vendor.tar.gz
ExclusiveArch: x86_64
BuildRequires: rust >= 1.85
BuildRequires: cargo
BuildRequires: gcc
BuildRequires: gcc-c++
BuildRequires: cmake
BuildRequires: perl-interpreter
BuildRequires: pkgconfig(openssl)
BuildRequires: systemd-rpm-macros
Requires(pre): shadow-utils
%description
Neuron is a per-node daemon for cortex inference clusters. It discovers
local GPU hardware via nvidia-smi, manages inference harnesses (mistral.rs,
llama.cpp), and exposes an HTTP API for model lifecycle management.
%prep
%autosetup
tar xf %{SOURCE1}
mkdir -p .cargo
cat > .cargo/config.toml << 'EOF'
[source.crates-io]
replace-with = "vendored-sources"
[source.vendored-sources]
directory = "vendor"
EOF
%build
cargo build --release -p neuron
%install
install -Dm755 target/release/neuron %{buildroot}%{_bindir}/neuron
install -Dm644 data/neuron.service %{buildroot}%{_unitdir}/neuron.service
install -dm750 %{buildroot}%{_sysconfdir}/cortex
install -Dm640 neuron.example.toml %{buildroot}%{_sysconfdir}/cortex/neuron.toml
%pre
getent group cortex >/dev/null || groupadd -r cortex
getent passwd cortex >/dev/null || useradd -r -g cortex -d /var/lib/cortex -s /sbin/nologin cortex
%post
%systemd_post neuron.service
%preun
%systemd_preun neuron.service
%postun
%systemd_postun_with_restart neuron.service
%files
%license LICENSE
%doc README.md
%{_bindir}/neuron
%{_unitdir}/neuron.service
%dir %attr(750,root,cortex) %{_sysconfdir}/cortex
%config(noreplace) %attr(640,root,cortex) %{_sysconfdir}/cortex/neuron.toml
%changelog
* Tue Apr 15 2026 Rob Thijssen <grenade@rob.tn> - 0.1.0-1
- Initial package