Since #1 a ranged GET on a file the bucket does not hold streams its slice from the tee instead of waiting for the whole transfer. For a range that runs to the end of the file -- what a resumed `hf download` sends -- that is exactly right, and it stays. For a range that stops short of the end it is not enough. The slice arrives quickly and then the *last chunk* is withheld until the manifest write lands, which waits on every remaining byte, because the digest is verified only once the whole file has passed. Measured against the live service, asking for the first 1MiB of a 3.3GB shard: http=206 ttfb=1.040851s total=120.001638s bytes=1048169 1,048,169 of 1,048,576 bytes in about a second, then an idle connection for the rest of the transfer. The client's read timeout ends it long before, so in practice the request fails anyway -- slowly and with no explanation. The holdback is not the thing to change: releasing that tail early would signal "durable and recorded" over bytes that are neither, which is precisely what it exists to prevent. So answer 503 with Retry-After instead, and let the fetch run on detached; the retry is served from the bucket. `hf_transfer` splits every download into bounded ranged chunks, so this shape is not hypothetical wherever it is enabled. The existing ranged test now asserts the two-step behaviour and keeps its guarantee that the bucket holds the whole file and never a fragment. A new test pins that a resume to EOF is still served from the tee, so the fix for #1 cannot be undone by this one. Closes #4 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PqNtYNhov3fukx46KS9R7L
130 lines
5.0 KiB
Rust
130 lines
5.0 KiB
Rust
//! Turning an [`Error`] into the response a Hub client expects.
|
|
//!
|
|
//! `huggingface_hub` branches on the `X-Error-Code` header to raise typed
|
|
//! exceptions, so emitting it is not cosmetic. The status alone would collapse
|
|
//! "no such repo", "no such revision" and "no such file" into one 404 that the
|
|
//! client cannot tell apart.
|
|
|
|
use axum::response::{IntoResponse, Response};
|
|
use http::{StatusCode, header};
|
|
use rustingface_entities::error::Error;
|
|
|
|
/// Header name the client reads to classify a failure.
|
|
pub const X_ERROR_CODE: &str = "x-error-code";
|
|
/// Header the Hub uses to carry a human-readable reason.
|
|
pub const X_ERROR_MESSAGE: &str = "x-error-message";
|
|
|
|
/// The exact `X-Error-Message` the client matches to raise `DisabledRepoError`.
|
|
///
|
|
/// Verified against the pinned `huggingface_hub` by the conformance suite:
|
|
/// unlike the others, a disabled repo is dispatched on the *message*, not on
|
|
/// `X-Error-Code`, so emitting only the code would have the client raise a
|
|
/// generic `HfHubHTTPError` instead.
|
|
pub const DISABLED_REPO_MESSAGE: &str = "Access to this resource is disabled.";
|
|
|
|
/// Wrapper giving [`Error`] an axum response.
|
|
pub struct ApiError(pub Error);
|
|
|
|
impl From<Error> for ApiError {
|
|
fn from(err: Error) -> Self {
|
|
Self(err)
|
|
}
|
|
}
|
|
|
|
impl IntoResponse for ApiError {
|
|
fn into_response(self) -> Response {
|
|
let ApiError(err) = self;
|
|
let status =
|
|
StatusCode::from_u16(err.status()).unwrap_or(StatusCode::INTERNAL_SERVER_ERROR);
|
|
let message = err.to_string();
|
|
// A disabled repo is dispatched by the client on this exact message
|
|
// rather than on X-Error-Code; everything else carries its own reason.
|
|
let wire_message = match &err {
|
|
Error::DisabledRepo(_) => DISABLED_REPO_MESSAGE.to_owned(),
|
|
_ => sanitise(&message),
|
|
};
|
|
|
|
let mut response =
|
|
(status, axum::Json(serde_json::json!({ "error": message }))).into_response();
|
|
|
|
if let Some(code) = err.code()
|
|
&& let Ok(value) = header::HeaderValue::from_str(code.as_str())
|
|
{
|
|
response.headers_mut().insert(X_ERROR_CODE, value);
|
|
}
|
|
|
|
// Sent for correctness rather than in hope: `huggingface_hub` ignores
|
|
// it and sleeps its own exponential backoff. A client that does read
|
|
// it should not hammer a transfer that takes minutes.
|
|
if matches!(err, Error::FetchInProgress(_)) {
|
|
response
|
|
.headers_mut()
|
|
.insert(header::RETRY_AFTER, header::HeaderValue::from_static("5"));
|
|
}
|
|
// Header values must be visible ASCII; a path or upstream detail could
|
|
// carry anything, so a message that will not encode is simply dropped
|
|
// rather than replacing the response with a 500.
|
|
if let Ok(value) = header::HeaderValue::from_str(&wire_message) {
|
|
response.headers_mut().insert(X_ERROR_MESSAGE, value);
|
|
}
|
|
|
|
if err.status() >= 500 {
|
|
tracing::error!(%err, "request failed");
|
|
} else {
|
|
tracing::debug!(%err, "request refused");
|
|
}
|
|
response
|
|
}
|
|
}
|
|
|
|
/// Reduce a message to characters a header value can carry.
|
|
fn sanitise(message: &str) -> String {
|
|
message
|
|
.chars()
|
|
.map(|c| if (' '..='~').contains(&c) { c } else { ' ' })
|
|
.take(512)
|
|
.collect()
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn a_missing_entry_carries_its_code_and_a_404() {
|
|
let response = ApiError(Error::EntryNotFound("config.json".into())).into_response();
|
|
assert_eq!(response.status(), StatusCode::NOT_FOUND);
|
|
assert_eq!(response.headers()[X_ERROR_CODE], "EntryNotFound");
|
|
}
|
|
|
|
#[test]
|
|
fn a_gated_repo_is_a_403_with_its_own_code() {
|
|
let response = ApiError(Error::GatedRepo("terms".into())).into_response();
|
|
assert_eq!(response.status(), StatusCode::FORBIDDEN);
|
|
assert_eq!(response.headers()[X_ERROR_CODE], "GatedRepo");
|
|
}
|
|
|
|
#[test]
|
|
fn an_upstream_auth_failure_is_a_401_with_no_code() {
|
|
let response = ApiError(Error::UpstreamUnauthorized("bad token".into())).into_response();
|
|
assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
|
|
assert!(!response.headers().contains_key(X_ERROR_CODE));
|
|
}
|
|
|
|
#[test]
|
|
fn a_disabled_repo_carries_the_exact_message_the_client_matches_on() {
|
|
// The client dispatches DisabledRepoError on this string rather than
|
|
// on X-Error-Code, so it must go out verbatim.
|
|
let response = ApiError(Error::DisabledRepo("upstream said so".into())).into_response();
|
|
assert_eq!(response.status(), StatusCode::FORBIDDEN);
|
|
assert_eq!(response.headers()[X_ERROR_MESSAGE], DISABLED_REPO_MESSAGE);
|
|
}
|
|
|
|
#[test]
|
|
fn a_message_with_a_newline_still_produces_a_valid_header() {
|
|
let response = ApiError(Error::EntryNotFound("a\nb\u{1f600}".into())).into_response();
|
|
let value = response.headers()[X_ERROR_MESSAGE].to_str().unwrap();
|
|
assert!(!value.contains('\n'), "{value:?}");
|
|
}
|
|
}
|