mirror of
https://github.com/openai/codex.git
synced 2026-11-21 22:50:37 +00:00
Attribute turn metrics to the models used during the turn (#44656)
## Why Turn completion metrics used session-level telemetry, which could label usage with the wrong model after a model switch. A turn can also include compaction and responses from different models, so a single token total cannot attribute usage correctly. ## What changed - Accumulate token usage by model using each response's resolved step settings, and emit one histogram sample per model and token type per turn. - Preserve zero-valued samples when no usage is reported, without adding a sample for an unused selected model when only compaction ran. - Use turn telemetry for tool-call, memory, network-proxy, and running-process metrics. ## Testing Add regression tests for model switches, step-level model changes, compaction followed by a response or an aborted turn, and turns without responses. Verify repeated requests aggregate into one sample per model and earlier session usage is excluded. GitOrigin-RevId: 03ab1dcde7da0b16b3a500a9a05e4c86583ee585
This commit is contained in:
committed by
copyberry
parent
8570091e14
commit
c8a8295e79
@@ -2811,7 +2811,11 @@ async fn try_run_sampling_request(
|
||||
)
|
||||
.await;
|
||||
let budget_result = sess
|
||||
.record_token_usage_info(&turn_context, token_usage.as_ref())
|
||||
.record_token_usage_info(
|
||||
&turn_context,
|
||||
&step_context.settings,
|
||||
token_usage.as_ref(),
|
||||
)
|
||||
.await;
|
||||
should_emit_token_count = true;
|
||||
should_emit_turn_diff = true;
|
||||
|
||||
Reference in New Issue
Block a user