diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..98373f3 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,50 @@ +# rustingface — notes for agents + +Read [`doc/spec.md`](doc/spec.md) first; it is the contract, not a sketch. Then +read `~/git/architecture/generic.md` for the conventions this repo inherits +(workspace layout, deployment, systemd, firewalld, SELinux, commits) and +`port-allocations.md` for why the port is `20482`. + +## What must not regress + +The five guarantees in `doc/spec.md` §1 are contractual. In particular: + +- **Never normalise a replayed header.** `ETag`, `X-Repo-Commit`, + `X-Linked-Etag` and `X-Linked-Size` are recorded as upstream sent them and + emitted unchanged, quotes included. Client cache layouts are keyed on these. + Do not compute a git blob SHA-1, derive an etag from content, or decide what + a commit "should" be. +- **Never let a redirect eat the Hub's headers.** The Hub answers a resolve + request with a redirect and puts the replayable metadata on *that* response; + the CDN it points at has none of it. `rustingface-data`'s upstream client + follows redirects by hand for this reason. If you ever re-enable reqwest's + own redirect policy, every recorded etag silently becomes wrong. +- **Never set `Content-Encoding` on the resolve path.** The client sends + `Accept-Encoding: identity` precisely so the sizes it learns are real. +- **Never write a manifest entry before its blob upload has completed.** The + ordering in `rustingface-core/src/fetch.rs` is deliberate: a crash may leak + an orphan blob, which `gc` reclaims, but a manifest entry pointing at + incomplete bytes is a silent correctness failure. +- **Never complete a client response before the manifest write lands.** The tee + runs one chunk behind for this reason. A file that was served but not + recorded is invisible the moment the instance is sealed. + +## Testing + +`cargo test --workspace` covers it, including `tests/sovereignty.rs`, which +runs the real service against a fake Hub speaking the real header protocol. +Add a case there for any behaviour a client depends on. + +The conformance suite (`test/conformance/`) needs network and a pinned +`huggingface_hub`. Run it before claiming a change is safe for clients — most +of the real bugs in this repo's history were found by it and not by the unit +tests. It has also been run against the live MinIO on caveman; do that for +anything touching `rustingface-data`, because the local filesystem store hides +whole classes of problem (it has no multipart, no conditional update, and no +presigning). + +## Where things live + +Types in `entities`, logic in `core`, I/O in `data`, HTTP in `api`, wiring in +the binary. `core` defines the `Store` and `Upstream` ports and must not gain a +direct dependency on `object_store` or `reqwest`. diff --git a/readme.md b/readme.md new file mode 100644 index 0000000..ea339f3 --- /dev/null +++ b/readme.md @@ -0,0 +1,198 @@ +# rustingface + +**A sovereign, HuggingFace-compatible model registry.** + +Single Rust binary. One config file. S3-compatible storage. Anything a client +fetches through it lives in your bucket until you explicitly remove it. + +Set `HF_ENDPOINT` and nothing else changes: + +```sh +export HF_ENDPOINT=https://rf.internal +python -c "from transformers import AutoModel; AutoModel.from_pretrained('Qwen/Qwen3-0.6B')" +``` + +The design is in [`doc/spec.md`](doc/spec.md). Architectural conventions this +project inherits are in [`~/git/architecture`](https://git.lair.cafe/grenade/architecture) +— workspace layout, deployment, systemd hardening, firewalld, SELinux, port +allocation. + +--- + +## What it guarantees + +1. **Fetch-once retention.** Any blob served through rustingface is durably + stored before the response completes. +2. **No eviction.** No TTL, no LRU, no background cleaner. Storage growth is + the operator's problem, by design. +3. **Offline sufficiency.** Once a `(repo, revision, file)` triple has been + served, it is servable forever with no network path to any upstream. +4. **Reference stability.** `main` resolves to the same commit on every later + request until an operator repins it. +5. **State portability.** The bucket is the entire system state. + +It does not reimplement the Hub's semantics. It records the exact HTTP metadata +upstream returned at first fetch — `ETag`, `X-Repo-Commit`, `X-Linked-Etag`, +`X-Linked-Size` — and replays it byte-identically, so a `~/.cache/huggingface` +populated through rustingface is interchangeable with one populated directly +from the Hub. + +--- + +## Layout + +``` +crates/ + rustingface-entities/ manifest, ref and repo schemas; bucket key layout; + configuration; the X-Error-Code taxonomy. No I/O. + rustingface-core/ resolver, freeze pinning, single-flight, the + streaming tee, policy, gc/verify/refresh. Defines the + Store and Upstream ports. + rustingface-data/ adapters: object_store (S3 and local) and reqwest. + rustingface-api/ the axum surface and the header contract. + rustingface/ the binary: serve, plus the admin subcommands. +asset/ systemd, firewalld, nginx, config template +script/infra-setup.sh one-time host provisioning +test/conformance/ the suite that drives a real huggingface_hub +``` + +Two departures from `architecture/generic.md`, both deliberate: + +- **One binary, not `-api` plus `-cli`.** `doc/spec.md` §9 specifies a single + binary with subcommands so the CLI and the service cannot drift. The library + crate split is unchanged. +- **`rustingface-api` is a library**, consumed by the binary, rather than being + the binary itself. Same reason. + +--- + +## Build and run + +```sh +cargo build --release +cargo test --workspace +cargo clippy --workspace --all-targets --all-features -- -D warnings +``` + +Locally, against a directory instead of a bucket: + +```sh +cargo run -- --config asset/config/config.toml.example serve +HF_ENDPOINT=http://127.0.0.1:20482 hf download Qwen/Qwen3-0.6B config.json +``` + +`storage.local_path` forfeits guarantee 5 — a directory is not a bucket you can +re-point a new host at — and the service says so at startup. It exists for +tests and trials. + +### Configuration + +One TOML file, `/etc/rustingface/config.toml` by default. Every key is +overridable by environment (`RUSTINGFACE__SERVER__LISTEN`). Secrets are named +by path, never inline, and a path that does not exist is retried under +`$CREDENTIALS_DIRECTORY` so one file works both under systemd's +`LoadCredential=` and when run by hand. See +[`asset/config/config.toml.tmpl`](asset/config/config.toml.tmpl). + +The two settings worth understanding: + +- **`policy.ref_resolution`** — `freeze` (default) records the first resolution + of a mutable ref and reuses it indefinitely. `follow` re-resolves upstream, + which breaks guarantee 4 and warns at startup. +- **`upstream.enabled`** — `false` seals the registry. No upstream client is + constructed at all, so there is no code path that could reach the network. + +--- + +## Admin CLI + +Same binary, subcommands. + +``` +rustingface serve # default +rustingface fetch [@rev] [--files …] # warm the bucket ahead of need +rustingface pin # set a ref explicitly +rustingface refresh [] # re-resolve upstream and repin +rustingface list [--repo-type models] # stored repos, revisions, sizes +rustingface show [@rev] # manifest contents +rustingface rm [@rev] --yes # remove refs/manifests (NOT blobs) +rustingface gc [--dry-run] # delete blobs no manifest references +rustingface verify [] # re-read blobs, check digests +rustingface doctor # config, S3 reachability, permissions +``` + +`rm` and `gc` are separate on purpose. Removal detaches metadata; reclamation +is a second, explicit, dry-runnable step. Nothing deletes bytes without an +operator typing `gc`. + +--- + +## Deployment + +| Concern | Where | +| --- | --- | +| Service | `bob.hanzalova.internal:20482` | +| Object storage | MinIO on `caveman.kosherinata.internal:9000`, bucket `rustingface` | +| Ingress | `rf.internal`, via the hanzalova edge proxy (mesh-only, no public record) | +| Port | `20482`, derived from the service name per `architecture/port-allocations.md` §3 | + +Deployment is CI-driven (`.gitea/workflows/deploy.yml`): push to `main`, the +workflow runs the lint/test gate, builds a static musl binary, ships it as +`gitea_ci` over scoped sudo, and health-checks with `/healthz` followed by +`rustingface doctor` — which proves the bucket is actually writable rather than +merely that a socket opened. + +One-time host provisioning is `script/infra-setup.sh`, run by an operator from +a workstation with full sudo. It is idempotent and skips unreachable hosts. + +Required Gitea repo secrets: `RSYNC_SSH_KEY`, `S3_ACCESS_KEY_ID`, +`S3_SECRET_ACCESS_KEY`, `HF_TOKEN`. + +The service holds no durable state, which is what makes the systemd hardening +easy: `ProtectSystem=strict` with no `ReadWritePaths` and no `StateDirectory` +at all. + +--- + +## Testing + +```sh +cargo test --workspace # unit + the sovereignty suite +python3 -m venv .venv && .venv/bin/pip install -r test/conformance/requirements.txt +.venv/bin/python test/conformance/conformance.py --binary target/release/rustingface +``` + +`crates/rustingface-api/tests/sovereignty.rs` runs the whole service against a +fake Hub that speaks the real header protocol and counts what it was asked for: +cold fetch, sealed replay, single-flight (eight clients, one upstream fetch), +client disconnect mid-transfer, range resume, freeze stability across an +upstream commit move, `gc` after `rm`, digest mismatch, deduplication, policy, +bearer auth, and state portability. + +`test/conformance/conformance.py` drives a **pinned, unmodified +`huggingface_hub`** through rustingface against the real Hub, then seals it and +re-runs. It also re-checks the `X-Error-Code` strings against the installed +client, because those are client-internal constants rather than a stable API — +a client upgrade is a spec-review trigger. + +Run the real offline test by hand before trusting a deployment: null-route +`huggingface.co` at the site router and verify there, not by trusting the +service. + +--- + +## Xet + +The Hub is migrating to Xet-backed storage. rustingface implements none of it: +it never advertises Xet capability downstream and never negotiates it upstream, +so files arrive whole through the Git LFS bridge. The cost is fetch-time +bandwidth on first pull. The benefit is that the on-disk format stays "files +with names" and no future maintainer can strand your data behind a +chunk-reconstruction format. `upstream.disable_xet = false` is refused at +startup. + +--- + +## Licence + +GPL-3.0-or-later.