A cold prefix range delivers its bytes at once, then stalls on the holdback #4

New Issue

grenade · 2026-09-02T11:13:55Z

grenade commented

2026-09-02 11:13:55 +00:00

Summary

Since #1, a ranged GET on a file the bucket does not hold streams the
client its slice as the bytes pass instead of waiting for the whole
transfer. That works, but the last chunk of the slice is still withheld
until the manifest write lands — and the manifest write only happens when
the whole file has been fetched and stored.

For an open-ended bytes=N- (what a resumed hf download sends) this is
invisible: the slice ends where the file ends, so the holdback is released
moments after the last byte would have arrived anyway.

For a prefix range it is not. Measured against the live service, asking
for the first 1MiB of a 3.3GB shard:

http=206 ttfb=1.040851s total=120.001638s bytes=1048169

1,048,169 of 1,048,576 bytes arrive in about a second, and then the
connection sits idle until the remaining ~3.3GB has transferred. The client
sees a stall a few hundred bytes from the end.

Why the holdback exists

It is correct, and #1 did not introduce it. The rule (spec §1, and the
"never complete a client response before the manifest write lands" note in
CLAUDE.md) exists so a client cannot consider a response complete over
bytes nothing has recorded. Releasing the tail early would mean a resumed
download could believe a file is finished while the manifest has no entry
for it, which is exactly the silent-corruption case the design forbids.

So this is not a request to drop the holdback.

The actual problem

The holdback's duration is currently tied to the whole transfer rather
than to the client's slice. A client asking for a small prefix is made to
wait on bytes it did not ask for and will never receive.

Suggested direction

Ideas, roughly in order of how much they change:

Serve a cold prefix range from the bucket after the fact instead of from
the tee — i.e. only narrow the tee for ranges that extend to EOF, and let
a prefix range wait for the store and then read back. Simple, but it
reinstates a long silence for that shape, which is what #1 removed.
Complete the client's slice as soon as the bytes of that slice are
durable, rather than when the whole file is. This needs a durability
point earlier than the manifest write, which the current
blob-then-manifest ordering does not provide.
Accept it and document it, on the grounds that no client rustingface
serves actually asks for a small prefix of a large blob — hf download
resumes open-ended.

Worth measuring what the pinned client actually sends before choosing;
if nothing real hits this shape, the last option may be the honest one.

## Summary Since #1, a ranged GET on a file the bucket does not hold streams the client its slice as the bytes pass instead of waiting for the whole transfer. That works, but the *last chunk* of the slice is still withheld until the manifest write lands — and the manifest write only happens when the whole file has been fetched and stored. For an open-ended `bytes=N-` (what a resumed `hf download` sends) this is invisible: the slice ends where the file ends, so the holdback is released moments after the last byte would have arrived anyway. For a **prefix** range it is not. Measured against the live service, asking for the first 1MiB of a 3.3GB shard: ``` http=206 ttfb=1.040851s total=120.001638s bytes=1048169 ``` 1,048,169 of 1,048,576 bytes arrive in about a second, and then the connection sits idle until the remaining ~3.3GB has transferred. The client sees a stall a few hundred bytes from the end. ## Why the holdback exists It is correct, and #1 did not introduce it. The rule (spec §1, and the "never complete a client response before the manifest write lands" note in CLAUDE.md) exists so a client cannot consider a response complete over bytes nothing has recorded. Releasing the tail early would mean a resumed download could believe a file is finished while the manifest has no entry for it, which is exactly the silent-corruption case the design forbids. So this is not a request to drop the holdback. ## The actual problem The holdback's *duration* is currently tied to the whole transfer rather than to the client's slice. A client asking for a small prefix is made to wait on bytes it did not ask for and will never receive. ## Suggested direction Ideas, roughly in order of how much they change: - Serve a cold prefix range from the bucket after the fact instead of from the tee — i.e. only narrow the tee for ranges that extend to EOF, and let a prefix range wait for the store and then read back. Simple, but it reinstates a long silence for that shape, which is what #1 removed. - Complete the client's slice as soon as the *bytes of that slice* are durable, rather than when the whole file is. This needs a durability point earlier than the manifest write, which the current blob-then-manifest ordering does not provide. - Accept it and document it, on the grounds that no client rustingface serves actually asks for a small prefix of a large blob — `hf download` resumes open-ended. Worth measuring what the pinned client actually sends before choosing; if nothing real hits this shape, the last option may be the honest one.

grenade referenced this issue from a commit

2026-09-02 11:24:55 +00:00

fix(core): a cold range that stops short of the end no longer stalls

grenade closed this issue

2026-09-02 11:34:19 +00:00

grenade referenced this issue from a commit

2026-09-02 11:34:20 +00:00

Merge branch 'fix/bounded-cold-range-does-not-stall'

Sign in to join this conversation.