rustingface/doc/spec.md

# rustingface

**A sovereign, HuggingFace-compatible model registry.**

Single Rust binary. One config file. S3-compatible storage. Anything a client
fetches through it lives in your bucket until you explicitly remove it.

---

## 1. Purpose

Inference operators build stacks pinned to specific model revisions, then
depend on `huggingface.co` remaining reachable, unchanged, and willing to serve
them for the life of that stack. rustingface removes that dependency without
requiring any client-side change beyond setting `HF_ENDPOINT`.

### Guarantees

The following are contractual. Everything else in this document is
implementation detail in service of them.

1. **Fetch-once retention.** Any blob served through rustingface is durably
   stored in the configured bucket before the response completes. It is removed
   only by an explicit operator action.
2. **No eviction.** There is no TTL, no LRU, no size-triggered sweep, no
   background cleaner. Storage growth is the operator's problem, by design.
3. **Offline sufficiency.** Once a `(repo, revision, file)` triple has been
   served, it is servable again with no network path to any upstream, forever.
4. **Reference stability.** A mutable ref (`main`) resolves to the same commit
   SHA on every subsequent request, until the operator explicitly repins it.
5. **State portability.** The bucket is the entire system state. Lose the host,
   reinstall the binary, point it at the bucket, and service resumes. There is
   no second datastore to back up in sync.

### Non-goals

- Multi-tenancy. Running this as a service for third parties would relocate the
  dependency rather than remove it, and would make the operator a redistributor
  of weights under licences that frequently forbid it. rustingface is software
  you run, not a service you consume.
- Write/upload API, repo creation, commits, PRs, discussions.
- Spaces, Inference API, model search, trending, likes.
- Git or Git-LFS server protocol.
- Xet client or CAS implementation (see §8).
- A web UI. `hf` CLI and the admin CLI are the interfaces.

---

## 2. Core principle: record and replay

rustingface does **not** reimplement the Hub's semantics. It records the exact
HTTP metadata upstream returned at first fetch, and replays it verbatim
thereafter.

Concretely: `ETag`, `X-Repo-Commit`, `X-Linked-Etag` and `X-Linked-Size` are
persisted as opaque strings in the manifest and echoed back byte-identically on
every later request. rustingface never computes a git blob SHA-1, never derives
an etag from content, never decides what a commit hash "should" be.

This matters because client cache layouts are keyed on etag. Replaying upstream
values verbatim means a `~/.cache/huggingface` populated via rustingface is
interchangeable with one populated directly from the Hub, and stays correct
across upstream changes to how those values are computed.

The corollary: rustingface cannot serve a `(repo, revision, file)` triple it has
never seen upstream. That is the intended failure mode, not a limitation to
engineer around.

---

## 3. Architecture

```
   client (transformers / vLLM / hf CLI)
        │  HF_ENDPOINT=https://models.internal
        ▼
  ┌──────────────────────────────────────┐
  │            rustingface               │
  │                                      │
  │  axum router ──► resolver            │
  │                    │                 │
  │                    ├─► metadata (S3) │
  │                    ├─► blobs (S3)    │
  │                    └─► upstream      │
  │                        (miss only)   │
  └──────────────────────────────────────┘
        │                        │
        ▼                        ▼
   S3-compatible            huggingface.co
   (MinIO/Ceph/…)           (optional, miss-only)
```

Single process. No database, no queue, no cache server, no sidecar.

### Request lifecycle (resolve)

```
1. parse (repo_type, repo_id, revision, filename)
2. revision is a ref name?  ──► look up refs/…  ──► commit sha
                                     │ miss + upstream enabled
                                     └──► resolve upstream, freeze, persist
3. load manifests/…/<commit>.json
4. entry present + blob exists?  ──► serve (proxy or 302 presign)
5. otherwise, if upstream enabled:
      single-flight upstream fetch
      tee: client stream  ‖  S3 multipart upload
      on completion: write manifest
6. otherwise: 404 with the appropriate X-Error-Code
```

---

## 4. Storage layout

Everything lives in one bucket. Prefixes are stable and human-inspectable.

```
blobs/sha256/<aa>/<bb>/<full-64-hex>
manifests/<repo_type>/<namespace>/<name>/<commit-sha>.json
refs/<repo_type>/<namespace>/<name>/<ref-name>.json
repos/<repo_type>/<namespace>/<name>/repo.json
```

- `repo_type` ∈ `models` | `datasets`.
- Two-level hex fanout on blobs to avoid hot key prefixes on backends that
  shard by key range.
- Blob keys are content-addressed by SHA-256 of the file bytes. Deduplication
  across repos and revisions is automatic.
- Manifests are immutable once written. A commit SHA fully determines content.
- Refs are the only mutable objects, and only under explicit operator action.

### Manifest schema

```jsonc
{
  "schema": 1,
  "repo_type": "models",
  "repo_id": "Qwen/Qwen3-32B",
  "commit": "8f2a...c31",
  "recorded_at": "2026-08-31T09:14:02Z",
  "upstream": "https://huggingface.co",
  "repo_meta": {
    // verbatim subset of the upstream /api/models response, sufficient to
    // answer model-info requests offline
    "id": "Qwen/Qwen3-32B",
    "sha": "8f2a...c31",
    "lastModified": "2026-06-11T00:00:00.000Z",
    "private": false,
    "gated": false,
    "tags": ["text-generation", "safetensors"],
    "siblings": [{ "rfilename": "config.json" }, { "rfilename": "model-00001-of-00017.safetensors" }]
  },
  "entries": [
    {
      "path": "config.json",
      "size": 1204,
      "digest": "sha256:9c1e...ff",     // rustingface blob key
      "etag": "\"a3f9c2...\"",           // upstream ETag, verbatim, incl. quotes
      "linked_etag": null,                // upstream X-Linked-Etag, verbatim
      "linked_size": null,                // upstream X-Linked-Size, verbatim
      "lfs": false
    },
    {
      "path": "model-00001-of-00017.safetensors",
      "size": 4831838208,
      "digest": "sha256:71ab...09",
      "etag": "\"71ab...09\"",
      "linked_etag": "\"71ab...09\"",
      "linked_size": 4831838208,
      "lfs": true
    }
  ]
}
```

`entries` is append-only within a commit. A partially-populated manifest is
valid and expected: clients typically fetch a subset of a repo's files, and
rustingface records what it has actually seen. A file absent from `entries` is
a miss, not a 404, when upstream is available.

### Ref schema

```jsonc
{
  "schema": 1,
  "ref": "main",
  "commit": "8f2a...c31",
  "pinned_at": "2026-08-31T09:14:02Z",
  "pinned_by": "auto-freeze"    // or "operator" after an explicit repin
}
```

### Write ordering and crash safety

Strict two-phase, always in this order:

1. Complete the S3 multipart upload of the blob. The blob is not referenced by
   anything yet; a crash here leaks an orphan, recoverable by `gc`.
2. Write or update the manifest referencing it.

A manifest must never reference a blob whose upload has not completed. This
ordering means the only possible inconsistency after a crash is an unreferenced
blob, which is safe and reclaimable. The inverse (a dangling manifest entry)
would be a silent correctness failure and must be impossible by construction.

Concurrent manifest writes for the same commit are resolved by read-merge-write
with a bounded retry. Entries are deterministic for a given commit, so merges
cannot conflict semantically; last-writer-wins on the merged set is correct.

Ref creation should use a conditional put (`If-None-Match: *`) where the backend
supports it, so first-write-wins freezing is atomic. Where unsupported, fall
back to read-then-write; the race window produces two identical values in the
common case and is corrected by the next explicit repin otherwise.

---

## 5. Configuration

Single TOML file, default `/etc/rustingface/config.toml`. Overridable per-key by
environment (`RUSTINGFACE__SERVER__LISTEN`). Secrets are read from files, never
inline, so they can be managed by `pass`/systemd credentials and kept out of the
config.

```toml
[server]
listen         = "127.0.0.1:8080"
external_url   = "https://models.internal.example"   # used to build redirect URLs
client_stall_timeout = "60s"     # idle guard on the tee; see below

[storage]
endpoint               = "https://s3.internal.example:9000"
bucket                 = "rustingface"
region                 = "us-east-1"
path_style             = true
access_key_id_file     = "/etc/rustingface/s3-access-key"
secret_access_key_file = "/etc/rustingface/s3-secret-key"
blob_delivery          = "proxy"        # proxy | redirect
connect_timeout        = "10s"
read_timeout           = "60s"          # idle guard; see below
presign_ttl            = "15m"          # redirect mode only
multipart_part_size    = "16MiB"

[upstream]
enabled         = true                  # false = hard-sealed, serve only what is stored
endpoint        = "https://huggingface.co"
token_file      = "/etc/rustingface/hf-token"   # optional
connect_timeout = "10s"
read_timeout    = "60s"
disable_xet     = true                  # see §8; do not change

[policy]
ref_resolution = "freeze"               # freeze | follow
allow_new_repos = true                  # false = only repos already in the bucket
allowlist = []                          # optional glob list, e.g. ["Qwen/*", "meta-llama/*"]

[auth]
mode       = "none"                     # none | bearer
token_file = "/etc/rustingface/client-tokens"

[observability]
metrics = true                          # /metrics, Prometheus text format
log_format = "json"                     # json | text
```

### Timeouts

Every timeout in this configuration is an **idle** timeout: it bounds silence,
never progress. There is deliberately no ceiling on how long a request may
take.

A blob response lasts `size / bandwidth`. A total-duration cap on it is
therefore not a timeout but a maximum servable file size, and one expressed in
the wrong units: the same client asking for the same file succeeds over a fast
link and fails on every retry over a slow one, with no diagnostic difference
between the two. A retry loop cannot converge against it, because nothing about
the next attempt is any faster.

- `server.client_stall_timeout` — how long a client may stop reading before the
  tee detaches it and finishes the transfer without it.
- `storage.read_timeout` — how long the object store may go silent mid-response.
  It resets on every chunk, and also bounds the wait for response headers, so a
  hung backend still fails a small metadata request promptly.
- `upstream.read_timeout` — the same guard on the Hub side.

This applies to the object-storage client in particular. `object_store`
defaults to a 30-second cap on the whole request, body included, and then hides
its own consequences: it catches the resulting body errors and silently retries
with a resumed range, so a large read reconnects every 30 seconds until the
180-second `retry_timeout` is exhausted, at which point the body ends with a 200
already on the wire and nothing logged. The adapter must disable that cap
explicitly, or every proxied blob over `180s x bandwidth` fails deterministically
and forever -- at 8MB/s, anything over about 1.4GB.

A ranged request on a file the bucket does not hold is subject to the same
principle from the other side. The whole file is still fetched and stored, since
a fragment must never be stored, but the client is sent its slice as those bytes
pass rather than being made to wait in silence for the transfer to end. A
resumed `hf download` asks for exactly that shape on every retry, and silence
there is indistinguishable to it from a dead server.

### `policy.ref_resolution`

- `freeze` (default) — the first resolution of a mutable ref is recorded and
  reused indefinitely. Upstream is never consulted for that ref again.
- `follow` — mutable refs are re-resolved upstream when reachable, falling back
  to the stored pin when not.

`follow` breaks guarantee 4 and is offered only for development. It should log a
warning at startup. In `freeze` mode, moving to a newer revision is an operator
action (`rustingface refresh`), which is the entire point: upstream cannot
change what your fleet resolves to.

---

## 6. HTTP surface

All paths accept an optional `/api/models/…` or `/api/datasets/…` repo type
prefix as the upstream does. `{repo_id}` is `namespace/name` or a bare `name`.

### Metadata

| Method | Path | Notes |
|---|---|---|
| `GET` | `/api/models/{repo_id}` | Serves `repo_meta` from the manifest of the current pinned ref |
| `GET` | `/api/models/{repo_id}/revision/{revision}` | As above, for a specific revision |
| `GET` | `/api/models/{repo_id}/refs` | `{"branches":[{"name","ref","targetCommit"}],"tags":[]}` from stored refs |
| `GET` | `/api/models/{repo_id}/tree/{revision}/{path}` | Supports `recursive`, `expand`; built from manifest `entries` |
| `GET` | `/api/datasets/...` | Same four shapes |
| `GET` | `/api/whoami-v2` | Static identity so `hf auth login` succeeds |

### Blobs

| Method | Path |
|---|---|
| `HEAD` | `/{repo_id}/resolve/{revision}/{filename}` |
| `GET` | `/{repo_id}/resolve/{revision}/{filename}` |
| `HEAD`/`GET` | `/datasets/{repo_id}/resolve/{revision}/{filename}` |

### Response header contract

This is the part clients actually depend on. On both `HEAD` and `GET`:

| Header | Value |
|---|---|
| `X-Repo-Commit` | Resolved commit SHA, replayed from the manifest |
| `ETag` | Upstream `ETag`, verbatim, quotes included |
| `X-Linked-Etag` | Upstream value, verbatim; omitted if upstream omitted it |
| `X-Linked-Size` | Upstream value, verbatim; omitted if upstream omitted it |
| `Content-Length` | True byte length |
| `Accept-Ranges` | `bytes` |

Rules:

- **Never** set `Content-Encoding`. The client sends `Accept-Encoding: identity`
  precisely to learn real sizes; compressing this path corrupts size accounting.
- The client prefers `X-Linked-Etag` and falls back to `ETag`. Emit both exactly
  as upstream did rather than normalising.
- Range requests must be honoured on the proxy path for resumable downloads.
- In `redirect` mode, respond `302` with `Location` set to a presigned URL and
  the metadata headers still present on the `HEAD` response.

### Error semantics

The client maps a custom header to typed exceptions. Emit it.

| Condition | Status | `X-Error-Code` |
|---|---|---|
| Repo not stored, upstream disabled or unreachable | `404` | `RepoNotFound` |
| Revision unknown and unresolvable | `404` | `RevisionNotFound` |
| File not in manifest and unfetchable | `404` | `EntryNotFound` |
| Upstream returned gated/403 | `403` | `GatedRepo` |
| Upstream auth failed | `401` | — |

> Verify the exact `X-Error-Code` strings against the `huggingface_hub` version
> you pin in the conformance suite (§11); they are client-internal constants and
> should be treated as version-coupled rather than stable API.

Failures must be loud. A missing blob with no upstream is an error response, not
an empty file or a partial stream. Recent `huggingface_hub` raises
`IncompleteSnapshotError` rather than returning a partial directory when the Hub
is unreachable and files are missing, which is the behaviour you want to
preserve rather than paper over.

---

## 7. Fetch path

### Single-flight

Concurrent requests for the same uncached `(repo_type, repo_id, commit, path)`
must produce exactly one upstream fetch. Key an in-flight map on that tuple;
subsequent callers subscribe to the same broadcast rather than issuing their own
request. Without this, a fleet-wide rollout of a new model produces N concurrent
40GB pulls.

### Streaming tee

The upstream response body is consumed once and written to two sinks
concurrently:

- the client response stream, and
- an S3 multipart upload, buffered at `multipart_part_size`.

Digest is computed incrementally over the same stream. On completion, verify the
digest against upstream's LFS oid where one was provided; mismatch aborts the
upload and fails the request.

**If the client disconnects mid-transfer, the upload continues to completion in
a detached task.** Discarding 30GB of transferred bytes because a client hit
Ctrl-C is the single most annoying possible behaviour, and the retention
guarantee is better served by finishing.

### Upstream auth

`upstream.token_file` is used as `Authorization: Bearer` on upstream requests
only. It is never echoed downstream and never logged. Gated repos work if the
operator's token has accepted the terms; the resulting stored copy is for that
operator's own use, which is the licensing posture this design depends on.

---

## 8. Xet policy

The Hub is migrating to Xet-backed storage, where a Xet-aware client receives
file reconstruction information from a CAS rather than a URL. rustingface
implements none of it.

- **Downstream:** never advertise Xet capability. Clients then take the ordinary
  resolve-and-redirect path.
- **Upstream:** set `HF_HUB_DISABLE_XET=1` semantics on outbound fetches so
  rustingface receives whole-file URLs via the Git LFS bridge, which reconstructs
  the file and returns a single resource URL for legacy clients. Storing whole
  blobs keeps the bucket independently readable, which is guarantee 5.

The cost is fetch-time bandwidth efficiency on first pull. The benefit is that
the on-disk format stays "files with names", and a future maintainer of this
project cannot strand anyone's data behind a chunk-reconstruction format. That
trade is the right way round for a sovereignty tool.

---

## 9. Admin CLI

Same binary, subcommands.

```
rustingface serve                          # default; run the HTTP service
rustingface fetch <repo>[@rev] [--files …] # warm the bucket ahead of need
rustingface pin <repo> <ref> <commit>      # set a ref explicitly
rustingface refresh <repo> [<ref>]         # re-resolve upstream and repin
rustingface list [--repo-type models]      # stored repos, revisions, sizes
rustingface show <repo>[@rev]              # manifest contents
rustingface rm <repo>[@rev]                # remove refs/manifests (NOT blobs)
rustingface gc [--dry-run]                 # delete blobs no manifest references
rustingface verify [<repo>]                # re-read blobs, check digests
rustingface doctor                         # config, S3 reachability, permissions
```

`rm` and `gc` are deliberately separate. Removal detaches metadata; reclamation
is a second, explicit, dry-runnable step. Nothing deletes bytes without an
operator typing `gc`.

---

## 10. Deployment

### Crates

| Concern | Crate |
|---|---|
| HTTP server | `axum` + `tower-http` |
| Object storage | `object_store` (S3/local/Azure behind one API, multipart handled) |
| Upstream client | `reqwest` with `rustls-tls` |
| Runtime | `tokio` |
| Config | `serde` + `toml` |
| Hashing | `sha2` |
| CLI | `clap` |
| Single-flight | `dashmap` + `tokio::sync::broadcast` |
| Telemetry | `tracing`, `tracing-subscriber`, `metrics-exporter-prometheus` |

### TLS

Terminate with `rustls` using the `aws-lc-rs` provider and a hybrid post-quantum
key exchange group (`X25519MLKEM768`), with certificates issued by the internal
step-ca. Alternatively terminate at a reverse proxy and run rustingface on
loopback; `server.external_url` exists so redirect URLs remain correct either
way.

### systemd unit

```ini
[Unit]
Description=rustingface sovereign model registry
After=network-online.target
Wants=network-online.target

[Service]
Type=exec
ExecStart=/usr/local/bin/rustingface serve --config /etc/rustingface/config.toml
DynamicUser=yes
LoadCredential=s3-key:/etc/rustingface/s3-secret-key
LoadCredential=hf-token:/etc/rustingface/hf-token
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictAddressFamilies=AF_INET AF_INET6
RestrictNamespaces=yes
LockPersonality=yes
MemoryDenyWriteExecute=yes
SystemCallFilter=@system-service
SystemCallArchitectures=native
CapabilityBoundingSet=
AmbientCapabilities=
UMask=0077
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target
```

There is no local state directory to protect, which is what makes the hardening
this easy: the process holds no durable data. `StateDirectory=` is deliberately
absent.

A Podman quadlet variant should mount only the config and credentials, with
`--read-only` and SELinux labels intact.

### Sizing

Memory is bounded by concurrent transfers × `multipart_part_size` × 2 (tee
buffers), plus a small metadata cache. A 16MiB part size with 32 concurrent
transfers is roughly 1GiB worst case. There is no reason for this service to
need more than 2GiB.

---

## 11. Conformance testing

The spec is only as good as the client compatibility, so test against real
clients rather than against this document.

**Matrix:** `huggingface_hub` (`hf_hub_download`, `snapshot_download`, `hf` CLI),
`transformers.from_pretrained`, `diffusers`, vLLM `--model`, and a
`safetensors`-only path. Pin exact client versions in CI and treat a client
upgrade as a spec-review trigger.

**The test that matters:**

1. Configure a fresh bucket. Set `HF_ENDPOINT`.
2. Pull a model. Confirm bytes landed in the bucket.
3. **Null-route `huggingface.co` at the site router.**
4. Wipe every client-side cache.
5. Pull the same model again. It must succeed, identically, with no upstream
   traffic — verify at the router, not by trusting the service.
6. Restart rustingface, wipe its host entirely, reinstall, re-point at the
   bucket, repeat step 5.

Anything less than step 6 tests a cache, not sovereignty.

**Additional cases:** concurrent cold fetch of the same file from 8 clients
(exactly one upstream request); client disconnect mid-transfer (upload still
completes); resumed/ranged download; gated repo with and without a valid token;
`main` pinning stability across an upstream commit; `gc` after `rm` reclaiming
exactly the right blobs and no others.

---

## 12. Phasing

| Phase | Scope |
|---|---|
| 0 | Resolve + model info + tree, proxy delivery, freeze pinning, single-flight, tee upload. Models only. |
| 1 | Datasets, redirect delivery with presigned URLs, admin CLI (`fetch`/`list`/`show`), metrics. |
| 2 | `gc`, `verify`, `refresh`, allowlist policy, bearer auth for clients. |
| 3 | Sealed mode (`upstream.enabled = false`) as a first-class deployment target; conformance suite in CI. |
| Later | Optional OCI/ModelPack export as a second read path over the same content-addressed blobs, hedging the possibility that the ecosystem standardises away from the HF protocol. |

---

## 13. Open questions

- **Repo metadata completeness.** `repo_meta` is recorded from whatever upstream
  returned at first fetch. If a client later requests a field that was not
  captured, is that a miss (fetch upstream) or a partial answer? Leaning toward
  recording the full upstream JSON response verbatim and filtering on read,
  which makes the question moot at the cost of some stored bytes.
- **`siblings` vs `entries` divergence.** `repo_meta.siblings` lists every file
  in the repo upstream; `entries` lists only what has been fetched. `tree` and
  model-info responses will therefore advertise files that are not stored.
  Correct in `upstream.enabled = true`, and misleading in sealed mode. Sealed
  mode should probably filter `siblings` down to `entries`.
- **Symlink/alias handling** for repos renamed upstream. Probably out of scope:
  a rename produces a new repo id, and the old manifest stays valid forever,
  which is arguably the desired behaviour.