Merge branch 'feat/allow-on-demand-mirroring'

.gitea/workflows/deploy.yml

@@ -21,7 +21,7 @@ env:
   UPSTREAM_ENABLED: "true"
   AUTH_MODE: bearer
   AUTH_ANONYMOUS: catalog
-  ALLOW_NEW_REPOS: "false"
+  ALLOW_NEW_REPOS: "true"
   PROXY_HOST: hanzalova.internal
   WEBROOT: /var/www/rustingface
 

CLAUDE.md

@@ -72,8 +72,19 @@ A client's token is never forwarded upstream, and rustingface's token is never
 shown to a client. Do not let these merge.
 
 `auth.mode = "bearer"` controls *who*, not *what* — a token holder can still
-cause any repository to be fetched. **`policy.allow_new_repos = false` is what
-bounds the bucket**, and the deployed configuration relies on it.
+cause any repository to be fetched. `policy.allow_new_repos` is what decides
+whether that matters, and **the deployment now sets it `true`**: an
+authenticated request for a repository the bucket does not hold fetches it from
+upstream and stores it on the way past. That is deliberate — it is how a
+consumer pointed here by `HF_ENDPOINT` mirrors what it uses without an operator
+pre-seeding anything.
+
+The consequence is that **the bucket is bounded by auth, not by policy**. Every
+issued client token is a licence to store arbitrary upstream repositories, so
+treat `client-tokens` as a capability list rather than a convenience. If that
+bound is ever needed back, `policy.allowlist` is the setting that keeps
+on-demand mirroring while confining it to expected namespaces; turning
+`allow_new_repos` off again makes growth an operator action instead.
 
 `auth.anonymous = "catalog"` opens `/v1/` to unauthenticated requests. That is
 only safe while every handler under `/v1/` reads the bucket and never consults

asset/config/config.toml.tmpl

@@ -45,10 +45,19 @@ disable_xet     = true
 # freeze: the first resolution of a mutable ref is recorded and reused
 # indefinitely. Moving to a newer revision is `rustingface refresh`.
 ref_resolution  = "freeze"
-# false: only repositories already in the bucket may be served, so no client
-# -- authenticated or not -- can cause an arbitrary model to be fetched and
-# stored. Growing the registry is an operator action: `rustingface fetch`.
-# This, not auth, is what bounds how large the bucket gets.
+# true: an authenticated client asking for a repository the bucket does not
+# hold causes it to be fetched from upstream and stored on the way past. That
+# is the point of the deployment -- a consumer configured with HF_ENDPOINT
+# pointing here mirrors what it uses, without an operator pre-seeding anything.
+#
+# The cost is that the bucket is then bounded by *auth* rather than by policy:
+# any token holder can cause any upstream repository to be stored. `false`
+# moves that bound back into policy, at the price of making growth an operator
+# action (`rustingface fetch`).
+#
+# `allowlist` is the middle setting and is deliberately empty here: a glob list
+# such as ["Qwen/*"] would keep on-demand mirroring while confining it to
+# namespaces the operator expects. Empty means no restriction.
 allow_new_repos = {{ALLOW_NEW_REPOS}}
 allowlist       = []
 

readme.md

@@ -165,6 +165,12 @@ The two settings worth understanding:
   otherwise cause an arbitrary model to be fetched. Growing the registry
   becomes an operator action (`rustingface fetch`).
 
+  The reference deployment sets it **`true`**, because the point of that
+  deployment is that a client pointed here by `HF_ENDPOINT` mirrors what it
+  uses on first request. There the bucket is bounded by who holds a token, and
+  `policy.allowlist` is available to confine on-demand mirroring to expected
+  namespaces without switching it off.
+
 ---
 
 ## Admin CLI