Account for token usage from incomplete responses (#53670)

## Why

Terminal `response.incomplete` errors discard reported token usage. Counting that usage as a new context baseline can also double-count input and trigger unnecessary compaction on the next turn.

## What changed

- Carry the response ID and valid reported usage with incomplete-response errors.
- Record usage for sampling and foreground compaction in cumulative totals, rollout records, budgets, and token-usage contributors without replacing the context baseline. Preserve the original terminal error, including when accounting exhausts the budget or cancellation races the error.
- Charge incomplete background compaction requests to the rollout budget without publishing foreground token usage.
- Ignore missing or malformed usage rather than estimating it.

## Testing

Extend coverage for SSE and WebSocket incomplete responses, rollout usage records, compaction accounting, deferred token-count events, and token-usage contributors. Add regressions for unnecessary compaction on the next turn and background compaction publishing foreground usage.

GitOrigin-RevId: bf1ee6df3912052ab7c19e1c6af8ad3dc4b07675
This commit is contained in:
jif
2026-10-13 13:36:24 +00:00
committed by copyberry
parent a68a7a6ab0
commit 816fa702ec
12 changed files with 383 additions and 29 deletions
+3 -2
View File
@@ -1,6 +1,7 @@
use crate::rate_limits::RateLimitError;
use codex_client::TransportError;
use codex_http_client::RetryAfter;
use codex_protocol::error::ResponseIncompleteError;
use codex_protocol::protocol::MisalignmentErrorDetails;
use http::StatusCode;
use serde_json::Value;
@@ -16,8 +17,8 @@ pub enum ApiError {
Stream(String),
#[error("stream error: Incomplete response returned, reason: content_filter")]
ContentFilter,
#[error("Incomplete response returned, reason: {0}")]
ResponseIncomplete(String),
#[error(transparent)]
ResponseIncomplete(Box<ResponseIncompleteError>),
#[error("context window exceeded")]
ContextWindowExceeded,
#[error("quota exceeded")]