Optimize remote compaction history handling (#34431)

## Why

Remote compaction can process large histories. Repeatedly estimating and replacing the full history, and cloning it when tracing is disabled, adds avoidable CPU and memory overhead.

## What changed

- Estimate each history item's token count once, update the total as trailing tool outputs are rewritten, and replace history only after all rewrites are selected.
- Preserve unclamped token totals while calculating removed tokens so saturated estimates do not hide overflow.
- Snapshot compaction input history only when rollout tracing is enabled.
- Reuse the v2 request input instead of cloning it before adding the compaction trigger.

## Testing

- Cover enabled and disabled compaction trace contexts through `is_enabled()` assertions.

GitOrigin-RevId: 8de9c9704ba29532fba05430eb204bbae2e8bf83
This commit is contained in:
Charlie Marsh
2026-07-20 23:30:30 +00:00
committed by copyberry
parent 56c11cf658
commit fd3c1dc13d
9 changed files with 72 additions and 40 deletions
+4 -2
View File
@@ -19,7 +19,7 @@ use tracing::info;
pub(super) struct RemoteCompactAttempt {
pub(super) new_history: Vec<ResponseItem>,
pub(super) trace_input_history: Vec<ResponseItem>,
pub(super) trace_input_history: Option<Vec<ResponseItem>>,
}
pub(super) async fn run_remote_compact_attempt(
@@ -57,7 +57,9 @@ pub(super) async fn run_remote_compact_attempt(
.saturating_sub(estimated_deleted_tokens.min(max_local_deleted_tokens))
});
}
let trace_input_history = history.raw_items().to_vec();
let trace_input_history = compaction_trace
.is_enabled()
.then(|| history.raw_items().to_vec());
let prompt_input = history.for_prompt(&turn_context.model_info.input_modalities);
let tool_router = built_tools(
sess.as_ref(),