# rustingface **A sovereign, HuggingFace-compatible model registry.** Single Rust binary. One config file. S3-compatible storage. Anything a client fetches through it lives in your bucket until you explicitly remove it. --- ## 1. Purpose Inference operators build stacks pinned to specific model revisions, then depend on `huggingface.co` remaining reachable, unchanged, and willing to serve them for the life of that stack. rustingface removes that dependency without requiring any client-side change beyond setting `HF_ENDPOINT`. ### Guarantees The following are contractual. Everything else in this document is implementation detail in service of them. 1. **Fetch-once retention.** Any blob served through rustingface is durably stored in the configured bucket before the response completes. It is removed only by an explicit operator action. 2. **No eviction.** There is no TTL, no LRU, no size-triggered sweep, no background cleaner. Storage growth is the operator's problem, by design. 3. **Offline sufficiency.** Once a `(repo, revision, file)` triple has been served, it is servable again with no network path to any upstream, forever. 4. **Reference stability.** A mutable ref (`main`) resolves to the same commit SHA on every subsequent request, until the operator explicitly repins it. 5. **State portability.** The bucket is the entire system state. Lose the host, reinstall the binary, point it at the bucket, and service resumes. There is no second datastore to back up in sync. ### Non-goals - Multi-tenancy. Running this as a service for third parties would relocate the dependency rather than remove it, and would make the operator a redistributor of weights under licences that frequently forbid it. rustingface is software you run, not a service you consume. - Write/upload API, repo creation, commits, PRs, discussions. - Spaces, Inference API, model search, trending, likes. - Git or Git-LFS server protocol. - Xet client or CAS implementation (see §8). - A web UI. `hf` CLI and the admin CLI are the interfaces. --- ## 2. Core principle: record and replay rustingface does **not** reimplement the Hub's semantics. It records the exact HTTP metadata upstream returned at first fetch, and replays it verbatim thereafter. Concretely: `ETag`, `X-Repo-Commit`, `X-Linked-Etag` and `X-Linked-Size` are persisted as opaque strings in the manifest and echoed back byte-identically on every later request. rustingface never computes a git blob SHA-1, never derives an etag from content, never decides what a commit hash "should" be. This matters because client cache layouts are keyed on etag. Replaying upstream values verbatim means a `~/.cache/huggingface` populated via rustingface is interchangeable with one populated directly from the Hub, and stays correct across upstream changes to how those values are computed. The corollary: rustingface cannot serve a `(repo, revision, file)` triple it has never seen upstream. That is the intended failure mode, not a limitation to engineer around. --- ## 3. Architecture ``` client (transformers / vLLM / hf CLI) │ HF_ENDPOINT=https://models.internal ▼ ┌──────────────────────────────────────┐ │ rustingface │ │ │ │ axum router ──► resolver │ │ │ │ │ ├─► metadata (S3) │ │ ├─► blobs (S3) │ │ └─► upstream │ │ (miss only) │ └──────────────────────────────────────┘ │ │ ▼ ▼ S3-compatible huggingface.co (MinIO/Ceph/…) (optional, miss-only) ``` Single process. No database, no queue, no cache server, no sidecar. ### Request lifecycle (resolve) ``` 1. parse (repo_type, repo_id, revision, filename) 2. revision is a ref name? ──► look up refs/… ──► commit sha │ miss + upstream enabled └──► resolve upstream, freeze, persist 3. load manifests/…/.json 4. entry present + blob exists? ──► serve (proxy or 302 presign) 5. otherwise, if upstream enabled: single-flight upstream fetch tee: client stream ‖ S3 multipart upload on completion: write manifest 6. otherwise: 404 with the appropriate X-Error-Code ``` --- ## 4. Storage layout Everything lives in one bucket. Prefixes are stable and human-inspectable. ``` blobs/sha256/// manifests////.json refs////.json repos////repo.json ``` - `repo_type` ∈ `models` | `datasets`. - Two-level hex fanout on blobs to avoid hot key prefixes on backends that shard by key range. - Blob keys are content-addressed by SHA-256 of the file bytes. Deduplication across repos and revisions is automatic. - Manifests are immutable once written. A commit SHA fully determines content. - Refs are the only mutable objects, and only under explicit operator action. ### Manifest schema ```jsonc { "schema": 1, "repo_type": "models", "repo_id": "Qwen/Qwen3-32B", "commit": "8f2a...c31", "recorded_at": "2026-08-31T09:14:02Z", "upstream": "https://huggingface.co", "repo_meta": { // verbatim subset of the upstream /api/models response, sufficient to // answer model-info requests offline "id": "Qwen/Qwen3-32B", "sha": "8f2a...c31", "lastModified": "2026-06-11T00:00:00.000Z", "private": false, "gated": false, "tags": ["text-generation", "safetensors"], "siblings": [{ "rfilename": "config.json" }, { "rfilename": "model-00001-of-00017.safetensors" }] }, "entries": [ { "path": "config.json", "size": 1204, "digest": "sha256:9c1e...ff", // rustingface blob key "etag": "\"a3f9c2...\"", // upstream ETag, verbatim, incl. quotes "linked_etag": null, // upstream X-Linked-Etag, verbatim "linked_size": null, // upstream X-Linked-Size, verbatim "lfs": false }, { "path": "model-00001-of-00017.safetensors", "size": 4831838208, "digest": "sha256:71ab...09", "etag": "\"71ab...09\"", "linked_etag": "\"71ab...09\"", "linked_size": 4831838208, "lfs": true } ] } ``` `entries` is append-only within a commit. A partially-populated manifest is valid and expected: clients typically fetch a subset of a repo's files, and rustingface records what it has actually seen. A file absent from `entries` is a miss, not a 404, when upstream is available. ### Ref schema ```jsonc { "schema": 1, "ref": "main", "commit": "8f2a...c31", "pinned_at": "2026-08-31T09:14:02Z", "pinned_by": "auto-freeze" // or "operator" after an explicit repin } ``` ### Write ordering and crash safety Strict two-phase, always in this order: 1. Complete the S3 multipart upload of the blob. The blob is not referenced by anything yet; a crash here leaks an orphan, recoverable by `gc`. 2. Write or update the manifest referencing it. A manifest must never reference a blob whose upload has not completed. This ordering means the only possible inconsistency after a crash is an unreferenced blob, which is safe and reclaimable. The inverse (a dangling manifest entry) would be a silent correctness failure and must be impossible by construction. Concurrent manifest writes for the same commit are resolved by read-merge-write with a bounded retry. Entries are deterministic for a given commit, so merges cannot conflict semantically; last-writer-wins on the merged set is correct. Ref creation should use a conditional put (`If-None-Match: *`) where the backend supports it, so first-write-wins freezing is atomic. Where unsupported, fall back to read-then-write; the race window produces two identical values in the common case and is corrected by the next explicit repin otherwise. --- ## 5. Configuration Single TOML file, default `/etc/rustingface/config.toml`. Overridable per-key by environment (`RUSTINGFACE__SERVER__LISTEN`). Secrets are read from files, never inline, so they can be managed by `pass`/systemd credentials and kept out of the config. ```toml [server] listen = "127.0.0.1:8080" external_url = "https://models.internal.example" # used to build redirect URLs client_stall_timeout = "60s" # idle guard on the tee; see below shutdown_grace = "15s" # bounded drain on SIGTERM; see below spool_dir = "/tmp/rustingface-spool" # in-flight bytes, readable by followers flight_wait_timeout = "5s" # how long a follower waits for a spool [storage] endpoint = "https://s3.internal.example:9000" bucket = "rustingface" region = "us-east-1" path_style = true access_key_id_file = "/etc/rustingface/s3-access-key" secret_access_key_file = "/etc/rustingface/s3-secret-key" blob_delivery = "proxy" # proxy | redirect connect_timeout = "10s" read_timeout = "60s" # idle guard; see below presign_ttl = "15m" # redirect mode only multipart_part_size = "16MiB" [upstream] enabled = true # false = hard-sealed, serve only what is stored endpoint = "https://huggingface.co" token_file = "/etc/rustingface/hf-token" # optional connect_timeout = "10s" read_timeout = "60s" disable_xet = true # see §8; do not change [policy] ref_resolution = "freeze" # freeze | follow allow_new_repos = true # false = only repos already in the bucket allowlist = [] # optional glob list, e.g. ["Qwen/*", "meta-llama/*"] [auth] mode = "none" # none | bearer token_file = "/etc/rustingface/client-tokens" [observability] metrics = true # /metrics, Prometheus text format log_format = "json" # json | text ``` ### Timeouts Every timeout in this configuration is an **idle** timeout: it bounds silence, never progress. There is deliberately no ceiling on how long a request may take. A blob response lasts `size / bandwidth`. A total-duration cap on it is therefore not a timeout but a maximum servable file size, and one expressed in the wrong units: the same client asking for the same file succeeds over a fast link and fails on every retry over a slow one, with no diagnostic difference between the two. A retry loop cannot converge against it, because nothing about the next attempt is any faster. - `server.client_stall_timeout` — how long a client may stop reading before the tee detaches it and finishes the transfer without it. - `storage.read_timeout` — how long the object store may go silent mid-response. It resets on every chunk, and also bounds the wait for response headers, so a hung backend still fails a small metadata request promptly. - `upstream.read_timeout` — the same guard on the Hub side. `server.shutdown_grace` is the one bound that is not idle-based, and deliberately so. It caps how long in-flight responses may drain on SIGTERM before the process stops regardless. A blob response is an in-flight request lasting `size / bandwidth`, so an unbounded drain does not converge: systemd's `TimeoutStopSec` expires first and escalates to `SIGABRT`. Cutting a download deliberately is the better failure — the client retries, and nothing in the bucket depends on the response, because a manifest entry is only ever written once its blob is durable. This applies to the object-storage client in particular. `object_store` defaults to a 30-second cap on the whole request, body included, and then hides its own consequences: it catches the resulting body errors and silently retries with a resumed range, so a large read reconnects every 30 seconds until the 180-second `retry_timeout` is exhausted, at which point the body ends with a 200 already on the wire and nothing logged. The adapter must disable that cap explicitly, or every proxied blob over `180s x bandwidth` fails deterministically and forever -- at 8MB/s, anything over about 1.4GB. A ranged request on a file the bucket does not hold is subject to the same principle from the other side. The whole file is still fetched and stored, since a fragment must never be stored, but the client is sent its slice as those bytes pass rather than being made to wait in silence for the transfer to end. A resumed `hf download` asks for exactly that shape on every retry, and silence there is indistinguishable to it from a dead server. That holds only for a range running to the end of the file. A range that stops short of it cannot be *completed* from the transfer at all: its last chunk is withheld until the manifest write lands, and that waits on every remaining byte, because the digest is verified only once the whole file has passed. Such a request is answered `503` with `Retry-After` while the fetch runs on, rather than being handed its bytes and then stalled a few hundred bytes from the end. ### `policy.ref_resolution` - `freeze` (default) — the first resolution of a mutable ref is recorded and reused indefinitely. Upstream is never consulted for that ref again. - `follow` — mutable refs are re-resolved upstream when reachable, falling back to the stored pin when not. `follow` breaks guarantee 4 and is offered only for development. It should log a warning at startup. In `freeze` mode, moving to a newer revision is an operator action (`rustingface refresh`), which is the entire point: upstream cannot change what your fleet resolves to. --- ## 6. HTTP surface All paths accept an optional `/api/models/…` or `/api/datasets/…` repo type prefix as the upstream does. `{repo_id}` is `namespace/name` or a bare `name`. ### Metadata | Method | Path | Notes | |---|---|---| | `GET` | `/api/models/{repo_id}` | Serves `repo_meta` from the manifest of the current pinned ref | | `GET` | `/api/models/{repo_id}/revision/{revision}` | As above, for a specific revision | | `GET` | `/api/models/{repo_id}/refs` | `{"branches":[{"name","ref","targetCommit"}],"tags":[]}` from stored refs | | `GET` | `/api/models/{repo_id}/tree/{revision}/{path}` | Supports `recursive`, `expand`; built from manifest `entries` | | `GET` | `/api/datasets/...` | Same four shapes | | `GET` | `/api/whoami-v2` | Static identity so `hf auth login` succeeds | ### Blobs | Method | Path | |---|---| | `HEAD` | `/{repo_id}/resolve/{revision}/{filename}` | | `GET` | `/{repo_id}/resolve/{revision}/{filename}` | | `HEAD`/`GET` | `/datasets/{repo_id}/resolve/{revision}/{filename}` | ### Response header contract This is the part clients actually depend on. On both `HEAD` and `GET`: | Header | Value | |---|---| | `X-Repo-Commit` | Resolved commit SHA, replayed from the manifest | | `ETag` | Upstream `ETag`, verbatim, quotes included | | `X-Linked-Etag` | Upstream value, verbatim; omitted if upstream omitted it | | `X-Linked-Size` | Upstream value, verbatim; omitted if upstream omitted it | | `Content-Length` | True byte length | | `Accept-Ranges` | `bytes` | Rules: - **Never** set `Content-Encoding`. The client sends `Accept-Encoding: identity` precisely to learn real sizes; compressing this path corrupts size accounting. - The client prefers `X-Linked-Etag` and falls back to `ETag`. Emit both exactly as upstream did rather than normalising. - Range requests must be honoured on the proxy path for resumable downloads. - In `redirect` mode, respond `302` with `Location` set to a presigned URL and the metadata headers still present on the `HEAD` response. ### Error semantics The client maps a custom header to typed exceptions. Emit it. | Condition | Status | `X-Error-Code` | |---|---|---| | Repo not stored, upstream disabled or unreachable | `404` | `RepoNotFound` | | Revision unknown and unresolvable | `404` | `RevisionNotFound` | | File not in manifest and unfetchable | `404` | `EntryNotFound` | | Upstream returned gated/403 | `403` | `GatedRepo` | | Upstream auth failed | `401` | — | > Verify the exact `X-Error-Code` strings against the `huggingface_hub` version > you pin in the conformance suite (§11); they are client-internal constants and > should be treated as version-coupled rather than stable API. Failures must be loud. A missing blob with no upstream is an error response, not an empty file or a partial stream. Recent `huggingface_hub` raises `IncompleteSnapshotError` rather than returning a partial directory when the Hub is unreachable and files are missing, which is the behaviour you want to preserve rather than paper over. --- ## 7. Fetch path ### Single-flight Concurrent requests for the same uncached `(repo_type, repo_id, commit, path)` must produce exactly one upstream fetch. Key an in-flight map on that tuple; subsequent callers subscribe to the same broadcast rather than issuing their own request. Without this, a fleet-wide rollout of a new model produces N concurrent 40GB pulls. Subscribing is what the spool is for. A follower cannot be handed the leader's live stream — the bytes that already passed are gone from it, and the multipart upload is not readable until it completes — so without one it can only wait for the entire transfer, with nothing on the wire, which a client's read timeout ends long before. The leader therefore writes every chunk to `server.spool_dir` as it passes and publishes how much is safe to read; a follower opens that file and streams it from the start, following it as it grows. One upstream fetch, every caller served, nobody in silence. The spool is not state. Losing it loses nothing, because a blob is not recorded until it is durable in the bucket, so a crash mid-transfer leaves an unreferenced blob and the next request refetches. It is the role the in-memory buffer already plays for small files, on disk so it can hold a large one. With `spool_dir` unset there is nothing for a follower to read and it is answered `503` instead. ### Streaming tee The upstream response body is consumed once and written to two sinks concurrently: - the client response stream, and - an S3 multipart upload, buffered at `multipart_part_size`. Digest is computed incrementally over the same stream. On completion, verify the digest against upstream's LFS oid where one was provided; mismatch aborts the upload and fails the request. **If the client disconnects mid-transfer, the upload continues to completion in a detached task.** Discarding 30GB of transferred bytes because a client hit Ctrl-C is the single most annoying possible behaviour, and the retention guarantee is better served by finishing. ### Upstream auth `upstream.token_file` is used as `Authorization: Bearer` on upstream requests only. It is never echoed downstream and never logged. Gated repos work if the operator's token has accepted the terms; the resulting stored copy is for that operator's own use, which is the licensing posture this design depends on. --- ## 8. Xet policy The Hub is migrating to Xet-backed storage, where a Xet-aware client receives file reconstruction information from a CAS rather than a URL. rustingface implements none of it. - **Downstream:** never advertise Xet capability. Clients then take the ordinary resolve-and-redirect path. This is the half that has to keep working as the Hub migrates, so the conformance suite runs a *Xet-capable* client — `hf_xet` installs as a `huggingface_hub` dependency whether or not a user asks for it — and asserts it still chooses HTTP. Testing it with Xet disabled client-side would prove nothing about the configuration people actually run. - **Upstream:** set `HF_HUB_DISABLE_XET=1` semantics on outbound fetches so rustingface receives whole-file URLs via the Git LFS bridge, which reconstructs the file and returns a single resource URL for legacy clients. Storing whole blobs keeps the bucket independently readable, which is guarantee 5. The cost is fetch-time bandwidth efficiency on first pull. The benefit is that the on-disk format stays "files with names", and a future maintainer of this project cannot strand anyone's data behind a chunk-reconstruction format. That trade is the right way round for a sovereignty tool. --- ## 9. Admin CLI Same binary, subcommands. ``` rustingface serve # default; run the HTTP service rustingface fetch [@rev] [--files …] # warm the bucket ahead of need rustingface pin # set a ref explicitly rustingface refresh [] # re-resolve upstream and repin rustingface list [--repo-type models] # stored repos, revisions, sizes rustingface show [@rev] # manifest contents rustingface rm [@rev] # remove refs/manifests (NOT blobs) rustingface gc [--dry-run] # delete blobs no manifest references rustingface verify [] # re-read blobs, check digests rustingface doctor # config, S3 reachability, permissions ``` `rm` and `gc` are deliberately separate. Removal detaches metadata; reclamation is a second, explicit, dry-runnable step. Nothing deletes bytes without an operator typing `gc`. --- ## 10. Deployment ### Crates | Concern | Crate | |---|---| | HTTP server | `axum` + `tower-http` | | Object storage | `object_store` (S3/local/Azure behind one API, multipart handled) | | Upstream client | `reqwest` with `rustls-tls` | | Runtime | `tokio` | | Config | `serde` + `toml` | | Hashing | `sha2` | | CLI | `clap` | | Single-flight | `dashmap` + `tokio::sync::broadcast` | | Telemetry | `tracing`, `tracing-subscriber`, `metrics-exporter-prometheus` | ### TLS Terminate with `rustls` using the `aws-lc-rs` provider and a hybrid post-quantum key exchange group (`X25519MLKEM768`), with certificates issued by the internal step-ca. Alternatively terminate at a reverse proxy and run rustingface on loopback; `server.external_url` exists so redirect URLs remain correct either way. ### systemd unit ```ini [Unit] Description=rustingface sovereign model registry After=network-online.target Wants=network-online.target [Service] Type=exec ExecStart=/usr/local/bin/rustingface serve --config /etc/rustingface/config.toml DynamicUser=yes LoadCredential=s3-key:/etc/rustingface/s3-secret-key LoadCredential=hf-token:/etc/rustingface/hf-token NoNewPrivileges=yes ProtectSystem=strict ProtectHome=yes PrivateTmp=yes PrivateDevices=yes ProtectKernelTunables=yes ProtectKernelModules=yes ProtectControlGroups=yes RestrictAddressFamilies=AF_INET AF_INET6 RestrictNamespaces=yes LockPersonality=yes MemoryDenyWriteExecute=yes SystemCallFilter=@system-service SystemCallArchitectures=native CapabilityBoundingSet= AmbientCapabilities= UMask=0077 Restart=on-failure RestartSec=5s [Install] WantedBy=multi-user.target ``` There is no local state directory to protect, which is what makes the hardening this easy: the process holds no durable data. `StateDirectory=` is deliberately absent. A Podman quadlet variant should mount only the config and credentials, with `--read-only` and SELinux labels intact. ### Sizing Memory is bounded by concurrent transfers × `multipart_part_size` × 2 (tee buffers), plus a small metadata cache. A 16MiB part size with 32 concurrent transfers is roughly 1GiB worst case. There is no reason for this service to need more than 2GiB. --- ## 11. Conformance testing The spec is only as good as the client compatibility, so test against real clients rather than against this document. **Matrix:** `huggingface_hub` (`hf_hub_download`, `snapshot_download`, `hf` CLI), `transformers.from_pretrained`, `diffusers`, vLLM `--model`, and a `safetensors`-only path. Pin exact client versions in CI and treat a client upgrade as a spec-review trigger. **The test that matters:** 1. Configure a fresh bucket. Set `HF_ENDPOINT`. 2. Pull a model. Confirm bytes landed in the bucket. 3. **Null-route `huggingface.co` at the site router.** 4. Wipe every client-side cache. 5. Pull the same model again. It must succeed, identically, with no upstream traffic — verify at the router, not by trusting the service. 6. Restart rustingface, wipe its host entirely, reinstall, re-point at the bucket, repeat step 5. Anything less than step 6 tests a cache, not sovereignty. **Additional cases:** concurrent cold fetch of the same file from 8 clients (exactly one upstream request); client disconnect mid-transfer (upload still completes); resumed/ranged download; gated repo with and without a valid token; `main` pinning stability across an upstream commit; `gc` after `rm` reclaiming exactly the right blobs and no others. --- ## 12. Phasing | Phase | Scope | |---|---| | 0 | Resolve + model info + tree, proxy delivery, freeze pinning, single-flight, tee upload. Models only. | | 1 | Datasets, redirect delivery with presigned URLs, admin CLI (`fetch`/`list`/`show`), metrics. | | 2 | `gc`, `verify`, `refresh`, allowlist policy, bearer auth for clients. | | 3 | Sealed mode (`upstream.enabled = false`) as a first-class deployment target; conformance suite in CI. | | Later | Optional OCI/ModelPack export as a second read path over the same content-addressed blobs, hedging the possibility that the ecosystem standardises away from the HF protocol. | --- ## 13. Open questions - **Repo metadata completeness.** `repo_meta` is recorded from whatever upstream returned at first fetch. If a client later requests a field that was not captured, is that a miss (fetch upstream) or a partial answer? Leaning toward recording the full upstream JSON response verbatim and filtering on read, which makes the question moot at the cost of some stored bytes. - **`siblings` vs `entries` divergence.** `repo_meta.siblings` lists every file in the repo upstream; `entries` lists only what has been fetched. `tree` and model-info responses will therefore advertise files that are not stored. Correct in `upstream.enabled = true`, and misleading in sealed mode. Sealed mode should probably filter `siblings` down to `entries`. - **Symlink/alias handling** for repos renamed upstream. Probably out of scope: a rename produces a new repo id, and the old manifest stays valid forever, which is arguably the desired behaviour.