feat(deploy): mirror on demand for authenticated clients

Sets allow_new_repos true in the deployed configuration, so an
authenticated request for a repository the bucket does not hold fetches
it from upstream and stores it on the way past.

This is what makes the registry useful as a drop-in HF_ENDPOINT for a
consumer: helexa's neurons resolve eight catalogued models, none of
which were in the bucket, and pre-seeding each one by hand before it can
be served defeats the purpose of putting a mirror in front of the Hub at
all. With the flag off the service answered exactly as designed --
"forbidden by policy: Qwen/Qwen3-1.7B is not already stored and
policy.allow_new_repos is false" -- which is correct behaviour and the
wrong deployment for this use.

The trade is real and now stated where it will be read rather than
buried: the bucket becomes bounded by auth rather than by policy, so
every issued client token is a licence to store arbitrary upstream
repositories. policy.allowlist is the middle setting that keeps
on-demand mirroring while confining it to expected namespaces, and it
stays empty here only because the consumer's catalogue is expected to
grow.

The agent notes previously asserted that the deployment relied on the
flag being false; that claim would have been quietly wrong from this
commit onward, so it is rewritten rather than left to mislead.

.gitea/workflows/deploy.yml

@@ -21,7 +21,7 @@ env:
   UPSTREAM_ENABLED: "true"
   AUTH_MODE: bearer
   AUTH_ANONYMOUS: catalog
-  ALLOW_NEW_REPOS: "false"
+  ALLOW_NEW_REPOS: "true"
   PROXY_HOST: hanzalova.internal
   WEBROOT: /var/www/rustingface
 

CLAUDE.md

@@ -72,8 +72,19 @@ A client's token is never forwarded upstream, and rustingface's token is never
 shown to a client. Do not let these merge.
 
 `auth.mode = "bearer"` controls *who*, not *what* — a token holder can still
-cause any repository to be fetched. **`policy.allow_new_repos = false` is what
-bounds the bucket**, and the deployed configuration relies on it.
+cause any repository to be fetched. `policy.allow_new_repos` is what decides
+whether that matters, and **the deployment now sets it `true`**: an
+authenticated request for a repository the bucket does not hold fetches it from
+upstream and stores it on the way past. That is deliberate — it is how a
+consumer pointed here by `HF_ENDPOINT` mirrors what it uses without an operator
+pre-seeding anything.
+
+The consequence is that **the bucket is bounded by auth, not by policy**. Every
+issued client token is a licence to store arbitrary upstream repositories, so
+treat `client-tokens` as a capability list rather than a convenience. If that
+bound is ever needed back, `policy.allowlist` is the setting that keeps
+on-demand mirroring while confining it to expected namespaces; turning
+`allow_new_repos` off again makes growth an operator action instead.
 
 `auth.anonymous = "catalog"` opens `/v1/` to unauthenticated requests. That is
 only safe while every handler under `/v1/` reads the bucket and never consults

asset/config/config.toml.tmpl

@@ -45,10 +45,19 @@ disable_xet     = true
 # freeze: the first resolution of a mutable ref is recorded and reused
 # indefinitely. Moving to a newer revision is `rustingface refresh`.
 ref_resolution  = "freeze"
-# false: only repositories already in the bucket may be served, so no client
-# -- authenticated or not -- can cause an arbitrary model to be fetched and
-# stored. Growing the registry is an operator action: `rustingface fetch`.
-# This, not auth, is what bounds how large the bucket gets.
+# true: an authenticated client asking for a repository the bucket does not
+# hold causes it to be fetched from upstream and stored on the way past. That
+# is the point of the deployment -- a consumer configured with HF_ENDPOINT
+# pointing here mirrors what it uses, without an operator pre-seeding anything.
+#
+# The cost is that the bucket is then bounded by *auth* rather than by policy:
+# any token holder can cause any upstream repository to be stored. `false`
+# moves that bound back into policy, at the price of making growth an operator
+# action (`rustingface fetch`).
+#
+# `allowlist` is the middle setting and is deliberately empty here: a glob list
+# such as ["Qwen/*"] would keep on-demand mirroring while confining it to
+# namespaces the operator expects. Empty means no restriction.
 allow_new_repos = {{ALLOW_NEW_REPOS}}
 allowlist       = []
 

readme.md

@@ -165,6 +165,12 @@ The two settings worth understanding:
   otherwise cause an arbitrary model to be fetched. Growing the registry
   becomes an operator action (`rustingface fetch`).
 
+  The reference deployment sets it **`true`**, because the point of that
+  deployment is that a client pointed here by `HF_ENDPOINT` mirrors what it
+  uses on first request. There the bucket is bounded by who holds a token, and
+  `policy.allowlist` is available to confine on-demand mirroring to expected
+  namespaces without switching it off.
+
 ---
 
 ## Admin CLI