Optimize remote compaction history handling (#34431)

## Why

Remote compaction can process large histories. Repeatedly estimating and replacing the full history, and cloning it when tracing is disabled, adds avoidable CPU and memory overhead.

## What changed

- Estimate each history item's token count once, update the total as trailing tool outputs are rewritten, and replace history only after all rewrites are selected.
- Preserve unclamped token totals while calculating removed tokens so saturated estimates do not hide overflow.
- Snapshot compaction input history only when rollout tracing is enabled.
- Reuse the v2 request input instead of cloning it before adding the compaction trigger.

## Testing

- Cover enabled and disabled compaction trace contexts through `is_enabled()` assertions.

GitOrigin-RevId: 8de9c9704ba29532fba05430eb204bbae2e8bf83
This commit is contained in:
Charlie Marsh
2026-07-20 23:30:30 +00:00
committed by copyberry
parent 56c11cf658
commit fd3c1dc13d
9 changed files with 72 additions and 40 deletions
+5 -1
View File
@@ -490,7 +490,11 @@ fn estimate_encrypted_function_output_length(encoded_len: usize) -> usize {
encoded_len.saturating_mul(9).div_ceil(16)
}
fn estimate_item_token_count(item: &ResponseItem) -> i64 {
/// Returns the same coarse, model-visible token estimate used for full history estimates.
///
/// Ordinary items are JSON-serialized, so callers estimating many items should reuse these
/// results instead of repeatedly estimating the full history.
pub(crate) fn estimate_item_token_count(item: &ResponseItem) -> i64 {
let model_visible_bytes = estimate_response_item_model_visible_bytes(item);
approx_tokens_from_byte_count_i64(model_visible_bytes)
}
+1
View File
@@ -3,5 +3,6 @@ mod normalize;
pub(crate) mod updates;
pub(crate) use history::ContextManager;
pub(crate) use history::estimate_item_token_count;
pub(crate) use history::is_user_turn_boundary;
pub(crate) use history::truncate_function_output_payload;