Track inference tool changes in turn analytics (#50964)

## What changed

Add `tools_change_count` to turn profiles and turn analytics events. Compare the full model-visible tool list before each sampling request, retaining the previous list across turns and connection resets. The first list establishes a baseline; subsequent changes increment the current turn's count.

## Testing

Add coverage for schema changes, tool additions, ordering changes, empty lists, and tracking across sessions and transport fallback. Extend the turn event serialization test to include `tools_change_count`.

GitOrigin-RevId: 44a15c42dc7d0b35c166666faeb9627b718b33dd
This commit is contained in:
Ahmed Ibrahim
2026-10-04 21:19:05 +00:00
committed by copyberry
parent 335c7f8eca
commit 4ad985e2ca
10 changed files with 91 additions and 0 deletions
+3
View File
@@ -2568,6 +2568,9 @@ async fn try_run_sampling_request(
turn_context.provider.info().name.as_str(),
);
let sampling_timing_guard = turn_context.turn_timing_state.begin_sampling();
if client_session.inference_tools_changed(&step_context.tool_router.model_visible_specs()) {
turn_context.turn_timing_state.record_tools_change();
}
// Do not enter this span: overlapping tools must not retain it past sampling.
let sampling_span = trace_span!(
"codex.sampling",