docs: readme and agent notes
Covers what the guarantees mean in practice, the two deliberate departures from architecture/generic.md §1, how to run it locally, the deployment topology, and how to run both test suites. CLAUDE.md records the invariants that are easy to break silently -- header replay, redirect handling, write ordering, and completing a response before its manifest write lands -- since each of those was a real bug rather than a hypothetical one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XZG2i4AmfSqE97EJGBVb64
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,50 @@
|
||||
# rustingface — notes for agents
|
||||
|
||||
Read [`doc/spec.md`](doc/spec.md) first; it is the contract, not a sketch. Then
|
||||
read `~/git/architecture/generic.md` for the conventions this repo inherits
|
||||
(workspace layout, deployment, systemd, firewalld, SELinux, commits) and
|
||||
`port-allocations.md` for why the port is `20482`.
|
||||
|
||||
## What must not regress
|
||||
|
||||
The five guarantees in `doc/spec.md` §1 are contractual. In particular:
|
||||
|
||||
- **Never normalise a replayed header.** `ETag`, `X-Repo-Commit`,
|
||||
`X-Linked-Etag` and `X-Linked-Size` are recorded as upstream sent them and
|
||||
emitted unchanged, quotes included. Client cache layouts are keyed on these.
|
||||
Do not compute a git blob SHA-1, derive an etag from content, or decide what
|
||||
a commit "should" be.
|
||||
- **Never let a redirect eat the Hub's headers.** The Hub answers a resolve
|
||||
request with a redirect and puts the replayable metadata on *that* response;
|
||||
the CDN it points at has none of it. `rustingface-data`'s upstream client
|
||||
follows redirects by hand for this reason. If you ever re-enable reqwest's
|
||||
own redirect policy, every recorded etag silently becomes wrong.
|
||||
- **Never set `Content-Encoding` on the resolve path.** The client sends
|
||||
`Accept-Encoding: identity` precisely so the sizes it learns are real.
|
||||
- **Never write a manifest entry before its blob upload has completed.** The
|
||||
ordering in `rustingface-core/src/fetch.rs` is deliberate: a crash may leak
|
||||
an orphan blob, which `gc` reclaims, but a manifest entry pointing at
|
||||
incomplete bytes is a silent correctness failure.
|
||||
- **Never complete a client response before the manifest write lands.** The tee
|
||||
runs one chunk behind for this reason. A file that was served but not
|
||||
recorded is invisible the moment the instance is sealed.
|
||||
|
||||
## Testing
|
||||
|
||||
`cargo test --workspace` covers it, including `tests/sovereignty.rs`, which
|
||||
runs the real service against a fake Hub speaking the real header protocol.
|
||||
Add a case there for any behaviour a client depends on.
|
||||
|
||||
The conformance suite (`test/conformance/`) needs network and a pinned
|
||||
`huggingface_hub`. Run it before claiming a change is safe for clients — most
|
||||
of the real bugs in this repo's history were found by it and not by the unit
|
||||
tests. It has also been run against the live MinIO on caveman; do that for
|
||||
anything touching `rustingface-data`, because the local filesystem store hides
|
||||
whole classes of problem (it has no multipart, no conditional update, and no
|
||||
presigning).
|
||||
|
||||
## Where things live
|
||||
|
||||
Types in `entities`, logic in `core`, I/O in `data`, HTTP in `api`, wiring in
|
||||
the binary. `core` defines the `Store` and `Upstream` ports and must not gain a
|
||||
direct dependency on `object_store` or `reqwest`.
|
||||
@@ -0,0 +1,198 @@
|
||||
# rustingface
|
||||
|
||||
**A sovereign, HuggingFace-compatible model registry.**
|
||||
|
||||
Single Rust binary. One config file. S3-compatible storage. Anything a client
|
||||
fetches through it lives in your bucket until you explicitly remove it.
|
||||
|
||||
Set `HF_ENDPOINT` and nothing else changes:
|
||||
|
||||
```sh
|
||||
export HF_ENDPOINT=https://rf.internal
|
||||
python -c "from transformers import AutoModel; AutoModel.from_pretrained('Qwen/Qwen3-0.6B')"
|
||||
```
|
||||
|
||||
The design is in [`doc/spec.md`](doc/spec.md). Architectural conventions this
|
||||
project inherits are in [`~/git/architecture`](https://git.lair.cafe/grenade/architecture)
|
||||
— workspace layout, deployment, systemd hardening, firewalld, SELinux, port
|
||||
allocation.
|
||||
|
||||
---
|
||||
|
||||
## What it guarantees
|
||||
|
||||
1. **Fetch-once retention.** Any blob served through rustingface is durably
|
||||
stored before the response completes.
|
||||
2. **No eviction.** No TTL, no LRU, no background cleaner. Storage growth is
|
||||
the operator's problem, by design.
|
||||
3. **Offline sufficiency.** Once a `(repo, revision, file)` triple has been
|
||||
served, it is servable forever with no network path to any upstream.
|
||||
4. **Reference stability.** `main` resolves to the same commit on every later
|
||||
request until an operator repins it.
|
||||
5. **State portability.** The bucket is the entire system state.
|
||||
|
||||
It does not reimplement the Hub's semantics. It records the exact HTTP metadata
|
||||
upstream returned at first fetch — `ETag`, `X-Repo-Commit`, `X-Linked-Etag`,
|
||||
`X-Linked-Size` — and replays it byte-identically, so a `~/.cache/huggingface`
|
||||
populated through rustingface is interchangeable with one populated directly
|
||||
from the Hub.
|
||||
|
||||
---
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
crates/
|
||||
rustingface-entities/ manifest, ref and repo schemas; bucket key layout;
|
||||
configuration; the X-Error-Code taxonomy. No I/O.
|
||||
rustingface-core/ resolver, freeze pinning, single-flight, the
|
||||
streaming tee, policy, gc/verify/refresh. Defines the
|
||||
Store and Upstream ports.
|
||||
rustingface-data/ adapters: object_store (S3 and local) and reqwest.
|
||||
rustingface-api/ the axum surface and the header contract.
|
||||
rustingface/ the binary: serve, plus the admin subcommands.
|
||||
asset/ systemd, firewalld, nginx, config template
|
||||
script/infra-setup.sh one-time host provisioning
|
||||
test/conformance/ the suite that drives a real huggingface_hub
|
||||
```
|
||||
|
||||
Two departures from `architecture/generic.md`, both deliberate:
|
||||
|
||||
- **One binary, not `-api` plus `-cli`.** `doc/spec.md` §9 specifies a single
|
||||
binary with subcommands so the CLI and the service cannot drift. The library
|
||||
crate split is unchanged.
|
||||
- **`rustingface-api` is a library**, consumed by the binary, rather than being
|
||||
the binary itself. Same reason.
|
||||
|
||||
---
|
||||
|
||||
## Build and run
|
||||
|
||||
```sh
|
||||
cargo build --release
|
||||
cargo test --workspace
|
||||
cargo clippy --workspace --all-targets --all-features -- -D warnings
|
||||
```
|
||||
|
||||
Locally, against a directory instead of a bucket:
|
||||
|
||||
```sh
|
||||
cargo run -- --config asset/config/config.toml.example serve
|
||||
HF_ENDPOINT=http://127.0.0.1:20482 hf download Qwen/Qwen3-0.6B config.json
|
||||
```
|
||||
|
||||
`storage.local_path` forfeits guarantee 5 — a directory is not a bucket you can
|
||||
re-point a new host at — and the service says so at startup. It exists for
|
||||
tests and trials.
|
||||
|
||||
### Configuration
|
||||
|
||||
One TOML file, `/etc/rustingface/config.toml` by default. Every key is
|
||||
overridable by environment (`RUSTINGFACE__SERVER__LISTEN`). Secrets are named
|
||||
by path, never inline, and a path that does not exist is retried under
|
||||
`$CREDENTIALS_DIRECTORY` so one file works both under systemd's
|
||||
`LoadCredential=` and when run by hand. See
|
||||
[`asset/config/config.toml.tmpl`](asset/config/config.toml.tmpl).
|
||||
|
||||
The two settings worth understanding:
|
||||
|
||||
- **`policy.ref_resolution`** — `freeze` (default) records the first resolution
|
||||
of a mutable ref and reuses it indefinitely. `follow` re-resolves upstream,
|
||||
which breaks guarantee 4 and warns at startup.
|
||||
- **`upstream.enabled`** — `false` seals the registry. No upstream client is
|
||||
constructed at all, so there is no code path that could reach the network.
|
||||
|
||||
---
|
||||
|
||||
## Admin CLI
|
||||
|
||||
Same binary, subcommands.
|
||||
|
||||
```
|
||||
rustingface serve # default
|
||||
rustingface fetch <repo>[@rev] [--files …] # warm the bucket ahead of need
|
||||
rustingface pin <repo> <ref> <commit> # set a ref explicitly
|
||||
rustingface refresh <repo> [<ref>] # re-resolve upstream and repin
|
||||
rustingface list [--repo-type models] # stored repos, revisions, sizes
|
||||
rustingface show <repo>[@rev] # manifest contents
|
||||
rustingface rm <repo>[@rev] --yes # remove refs/manifests (NOT blobs)
|
||||
rustingface gc [--dry-run] # delete blobs no manifest references
|
||||
rustingface verify [<repo>] # re-read blobs, check digests
|
||||
rustingface doctor # config, S3 reachability, permissions
|
||||
```
|
||||
|
||||
`rm` and `gc` are separate on purpose. Removal detaches metadata; reclamation
|
||||
is a second, explicit, dry-runnable step. Nothing deletes bytes without an
|
||||
operator typing `gc`.
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
| Concern | Where |
|
||||
| --- | --- |
|
||||
| Service | `bob.hanzalova.internal:20482` |
|
||||
| Object storage | MinIO on `caveman.kosherinata.internal:9000`, bucket `rustingface` |
|
||||
| Ingress | `rf.internal`, via the hanzalova edge proxy (mesh-only, no public record) |
|
||||
| Port | `20482`, derived from the service name per `architecture/port-allocations.md` §3 |
|
||||
|
||||
Deployment is CI-driven (`.gitea/workflows/deploy.yml`): push to `main`, the
|
||||
workflow runs the lint/test gate, builds a static musl binary, ships it as
|
||||
`gitea_ci` over scoped sudo, and health-checks with `/healthz` followed by
|
||||
`rustingface doctor` — which proves the bucket is actually writable rather than
|
||||
merely that a socket opened.
|
||||
|
||||
One-time host provisioning is `script/infra-setup.sh`, run by an operator from
|
||||
a workstation with full sudo. It is idempotent and skips unreachable hosts.
|
||||
|
||||
Required Gitea repo secrets: `RSYNC_SSH_KEY`, `S3_ACCESS_KEY_ID`,
|
||||
`S3_SECRET_ACCESS_KEY`, `HF_TOKEN`.
|
||||
|
||||
The service holds no durable state, which is what makes the systemd hardening
|
||||
easy: `ProtectSystem=strict` with no `ReadWritePaths` and no `StateDirectory`
|
||||
at all.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
```sh
|
||||
cargo test --workspace # unit + the sovereignty suite
|
||||
python3 -m venv .venv && .venv/bin/pip install -r test/conformance/requirements.txt
|
||||
.venv/bin/python test/conformance/conformance.py --binary target/release/rustingface
|
||||
```
|
||||
|
||||
`crates/rustingface-api/tests/sovereignty.rs` runs the whole service against a
|
||||
fake Hub that speaks the real header protocol and counts what it was asked for:
|
||||
cold fetch, sealed replay, single-flight (eight clients, one upstream fetch),
|
||||
client disconnect mid-transfer, range resume, freeze stability across an
|
||||
upstream commit move, `gc` after `rm`, digest mismatch, deduplication, policy,
|
||||
bearer auth, and state portability.
|
||||
|
||||
`test/conformance/conformance.py` drives a **pinned, unmodified
|
||||
`huggingface_hub`** through rustingface against the real Hub, then seals it and
|
||||
re-runs. It also re-checks the `X-Error-Code` strings against the installed
|
||||
client, because those are client-internal constants rather than a stable API —
|
||||
a client upgrade is a spec-review trigger.
|
||||
|
||||
Run the real offline test by hand before trusting a deployment: null-route
|
||||
`huggingface.co` at the site router and verify there, not by trusting the
|
||||
service.
|
||||
|
||||
---
|
||||
|
||||
## Xet
|
||||
|
||||
The Hub is migrating to Xet-backed storage. rustingface implements none of it:
|
||||
it never advertises Xet capability downstream and never negotiates it upstream,
|
||||
so files arrive whole through the Git LFS bridge. The cost is fetch-time
|
||||
bandwidth on first pull. The benefit is that the on-disk format stays "files
|
||||
with names" and no future maintainer can strand your data behind a
|
||||
chunk-reconstruction format. `upstream.disable_xet = false` is refused at
|
||||
startup.
|
||||
|
||||
---
|
||||
|
||||
## Licence
|
||||
|
||||
GPL-3.0-or-later.
|
||||
Reference in New Issue
Block a user