Treat incomplete responses as terminal errors (#53451)

## Why

Incomplete responses such as `max_output_tokens` were classified as stream errors, allowing retries and compaction model fallback after the response had hit its output limit.

## What changed

Introduce `ResponseIncomplete` errors that preserve the server-provided reason and stop stream processing immediately. Make these errors non-retryable and stop compaction without falling back to the selected model. Record incomplete responses separately in sampler failure metrics.

## Testing

Add regression tests that verify incomplete responses surface a terminal error without retries over SSE and WebSocket, and that compaction stops without retrying or switching models for both local and remote compaction.

GitOrigin-RevId: 6e107d441b9bf03fa47ea47b88eeb94b71a9f729
This commit is contained in:
jif
2026-10-12 18:53:49 +00:00
committed by copyberry
parent c308e127e0
commit bc9cad3473
12 changed files with 209 additions and 4 deletions
@@ -78,6 +78,7 @@ pub fn telemetry_api_error_message(error: &ApiError) -> String {
ApiError::ContentFilter => {
"Incomplete response returned, reason: content_filter".to_string()
}
ApiError::ResponseIncomplete(_) => "incomplete response".to_string(),
ApiError::ContextWindowExceeded => "context window exceeded".to_string(),
ApiError::QuotaExceeded => "quota exceeded".to_string(),
ApiError::UsageNotIncluded => "usage not included".to_string(),