## What changed
Emit `codex.skill_invocation` logs for explicit skill injection and detected implicit use, including skill-tool reads. Include skill name, invocation type, conversation, turn, model, and client metadata, plus scope, plugin ID, and ChatGPT user and account IDs when available.
Exclude skill contents, resource paths, email addresses, prompts, and tool payloads. Use the existing OTEL log exporter without an additional opt-in; `otel.exporter = "none"` continues to export no logs.
Document the event fields, exporter configuration, and detection semantics in `codex-rs/otel/README.md`.
GitOrigin-RevId: 0f9c492c3f224455f3de3660be89692d776c1951
## What changed
- Give tool fragment sizes and namespace counts logarithmic boundaries through 32,768.
- Give enabled and kept skill counts integer boundaries from 0 through 512, removed description characters logarithmic boundaries through 131,072, and skill truncation flags boundaries at 0 and 1.
- Record these metrics with explicit boundaries, separating zero values and retaining overflow buckets above each range.
## Testing
Extend tool and skill metric tests to assert explicit boundaries. Add an export test covering zero, one, the upper boundary, and overflow, including bucket counts, ordering, and sample totals.
GitOrigin-RevId: 9fbf3acbe352d7c74f3ad11f0b690148c083a6ef
## What changed
- Record successful spawn phase durations in `codex.multi_agent.spawn.phase.duration_ms`, covering residency reservation and fork context when applicable, child creation, durability waits, input admission, and total latency.
- Count spawn failures from the V1 and V2 tool handlers in `codex.multi_agent.spawn.failure`, grouped by bounded error categories.
- Use bounded labels for fork mode, multi-agent version, product SKU, and successful-spawn history mode. Emit phase timings only after a successful spawn.
## Testing
Add in-memory metrics tests for phase durations and labels, plus a tool-handler test verifying that a thread-limit failure emits one counter increment with the expected labels.
GitOrigin-RevId: 04cba9042b9b34acdf57376ace5288c3f741f3b2
## What changed
- Add `StorageTelemetry` to record operation counts, durations, secure-store outcomes, and file fallback reasons using bounded labels. Emit fallback warnings without credential payloads or error text.
- Add task-local client attribution with explicit propagation across task boundaries.
- Preserve typed keyring error causes through `std::io::Error` and the local secrets backend, classifying supported native failures without parsing error messages.
- Remove service names, account names, value lengths, and error text from keyring trace logs.
## Testing
Add tests for buffered metric emission, fallback outcomes, exclusion of sensitive error contents, client attribution isolation, and typed native error classification.
GitOrigin-RevId: 18e7201566a3b09061a6014fd6bd90eae96e3fc2
## What changed
Record `codex.thread.tools.namespaces_total` and `codex.thread.tools.fragment_bytes` histograms through session telemetry. Tag samples by `stage` (`before` or `after` the 4 KiB cap) and `kind` (`snapshot` or `delta`). Measure bytes after description normalization and XML escaping, including the `<tools>` wrapper, with explicit fragment-size buckets.
Delta counts include only changed entries, and unchanged state emits no samples.
## Testing
Extend rendering tests to verify snapshot and delta metrics, omission counts, byte sizes with Unicode and XML escaping, and no samples for unchanged state.
GitOrigin-RevId: 26543585e2bea54bff6d63263faa9a6ca5dca379
## What changed
Add `otel.log_guardian_assessments`, disabled by default and requiring an OTLP HTTP or gRPC log exporter. Emit `codex.guardian_assessment` when a synchronous Guardian review finishes, including review identifiers, status, risk level, user authorization, and rationale. Include `allow` or `deny` outcomes only for completed model assessments, not review errors, timeouts, or cancellations.
Cap exported rationales at 64 KiB on a UTF-8 character boundary and record their original byte length and truncation status. Use the log-only route to exclude assessment payloads from trace export, local state logs, and bug-report log buffers. Cached V2 approvals are not included.
## Testing
Add coverage for configuration gating, enabled and disabled denial logging, terminal statuses and outcomes, UTF-8 rationale truncation, export routing, and bug-report log buffer exclusion.
GitOrigin-RevId: dcbb5b429c8726a5a5c80ff82f9d625e81796ede
## What changed
Use `KNOWN_PRODUCT_SKUS` in `SessionTelemetry::with_product_sku`, preserving the existing mapping: `codex` stays `codex`, missing or empty values omit the label, and all other values become `other`.
Remove the registry test `dispatch_uses_the_invoking_turn_product_sku` and its unused imports.
GitOrigin-RevId: b4ad6cbc6205aa35fb2866d72f7205814f5b3566
## Why
Tool telemetry needs to distinguish product SKUs without turning arbitrary configuration values into metric labels.
## What changed
- Read `apps_mcp_product_sku` from the invoking turn when dispatching tools, replacing any stale SKU in the telemetry metadata.
- Add `product_sku` to tool call count and duration metrics and tool result log and trace events.
- Preserve `codex`, map other nonempty values to `other`, and omit the attribute for empty or missing values.
## Testing
Add coverage for SKU separation on a shared metrics client, normalization, and successful and failing dispatches with stale telemetry metadata. Extend export routing tests to check SKU attributes in logs and matching traces.
GitOrigin-RevId: 3fc58c9f7245fd5b65f7cba122e08d7ea838b2d6
## What changed
Add `otel.log_agent_responses`, disabled by default and requiring an OTLP HTTP or gRPC log exporter. Emit `codex.agent_response` for completed `final_answer` messages from main agents and spawned task agents, with session, turn, item, and available parent/root attribution.
Strip memory citations from exported text, preserve the stored answer, and cap responses at 64 KiB on a UTF-8 character boundary. Include the original byte length and a truncation flag, and suppress duplicate item completions within each turn. Exclude commentary and non-task agents.
## Testing
Extend configuration tests for default-off and explicit opt-in behavior, the agent execution test for main/subagent response attribution, and the log-buffer test for response-event exclusion.
GitOrigin-RevId: eacda1150de2e6bac2c667d71b57214b0a086f1a
## What changed
Add `record_global_operation` to record an operation count and duration in milliseconds, buffering up to 256 pairs until a metrics provider is installed. Validate both metric names and tags before accepting a pair, limit each operation to 16 tags and 1024 bytes of metadata, and drop whole pairs when the buffer is full.
Drain pending operations on installation and serialize recording with provider lifecycle changes so a pair stays with one installation. Resume buffering when the active provider shuts down without disturbing a newer installation. Disabling metrics discards pending operations and suppresses recording until metrics are enabled again.
## Testing
Add unit and integration coverage for buffer limits, validation, concurrent installation and replacement, redirected global handles, shutdown buffering, and accepted or rejected opt-out configuration changes.
GitOrigin-RevId: 3b094e69bf975016ff1fb8f99b3611c2ccddccf3
## Why
AWS credential discovery, analytics, and telemetry used clients that did not share the application's network policy. These requests need to honor destination restrictions and permission revocation.
## What changed
- Route AWS credential and region HTTP requests through the shared HTTP client, and check the signing destination before loading credentials. Skip discovery for static access keys with an explicit region.
- Apply account-scoped policy to Bedrock authentication. Require unrestricted policy for credential exporters and reauthentication commands, and cancel active work when permission is revoked. Document that AWS profile `credential_process` network traffic remains outside the application's HTTP policy.
- Send analytics through the authenticated account's HTTP client factory.
- Guard OTLP log, trace, and metric exports with revocable permits, disable them under destination restrictions, and make managed HTTP exports cancellable. Suppress global Statsig settings while managed policy is active.
## Testing
Add coverage for metadata credential request cancellation, allowed and denied SigV4 destinations, AWS credential precedence and endpoint configuration, and blocked credential exporter recovery. Update telemetry tests for account transitions and managed-policy Statsig suppression.
GitOrigin-RevId: d8a2018bfbbe971ec430699b0d3f86a7a9e1e072
## Why
OS discovery can launch subprocesses on Linux. Repeating it for requests, analytics events, and metrics provider creation adds unnecessary work for stable metadata.
## What changed
Cache `os_info::get()` with a separate `LazyLock` in user-agent generation, analytics runtime metadata, and metrics OS resource attributes. Continue reading the user-agent originator and suffix on each call so updates remain effective.
## Testing
Add an isolated Linux regression test that races initial user-agent lookups and repeats calls, verifies `lsb_release` and `getconf` each run once, and checks that subsequent originator and suffix updates appear in the user agent.
GitOrigin-RevId: 337e45ff88b863d2a4aa387a8e7d2e291c045906
## Why
Per-socket metrics label resends after reconnect as initial requests, obscuring why full input was sent. Client telemetry can preserve that reason across reconnect failures and turn boundaries.
## What changed
- Add `codex.websocket.continuation` to count `response.create` send attempts, including failed sends, with `mode`, `reason`, and `phase` tags.
- Distinguish incremental continuations, fresh requests, restored history, closed connections, and other full-input sends. Track warmup and generation separately.
- Preserve the first continuation reset reason until the next send and identify resumed or forked history.
- Allow session telemetry to use a `MetricsClient` supplied through thread extensions and document the counter's semantics.
## Testing
Add metric assertions for incremental reuse, warmup, changed requests, account switches, reconnects, and resumed or forked history. Add a unit test for preserving reset reasons across repeated resets and turn boundaries.
GitOrigin-RevId: 29324f5e9d5677e0c6068f89b2256a92a5a0d157
## What changed
- Accept `fileId` alongside the existing `url` form for app-server image inputs, and forward file references to the Responses API as `file_id`. Update generated schemas and client types.
- Preserve file references, image detail hints, and mixed inline/file image ordering through user-message events, thread history, and rollout migration. Retain file images when truncating tool output.
- Pass file references through image preparation without resolving them, while keeping resize-notice numbering correct. Omit them from unsupported TUI display and Guardian image context.
- Reject image-edit requests whose recent-image window includes a file reference, preventing selection of an older inline image instead.
## Testing
Add coverage for serialization, request and rollout preservation, mixed-image history ordering, incomplete ordering metadata, tool-output truncation, and rejection of unsupported image-edit selections.
GitOrigin-RevId: 6ca20a8577155cc934b720803c3b7b3bffdf972a
## Why
Guardian request and section cost distributions need shared bucket boundaries across synchronous and asynchronous reviewers so their measurements align.
## What changed
- Add `histogram_with_boundaries` to session telemetry and extension metrics while preserving session attribution.
- Use shared request-token buckets up to 2,000,000 tokens and section-cost buckets up to 16,777,216 for both review paths.
## Testing
Extend telemetry tests to verify explicit bucket boundaries and sample counts, and Guardian integration coverage to check exported request and section metric bounds for both synchronous and asynchronous reviews.
GitOrigin-RevId: e34f6972b3418bac3b061939b62eeeccaec5a299
## What changed
- Record `exec_server_client_requests_total` for every client RPC call attempt,
labeled by protocol method.
- Count attempts before local admission so rejected, timed-out, cancelled, and
transport-failed calls are included, while notifications and responses are not.
- Export the counter to configured OpenTelemetry collectors while excluding it
from the built-in Statsig metrics set.
## Testing
- Cover every RPC call entry point, successful responses, local and transport
failures, timeouts, cancellation, notifications, and disabled metrics.
- Verify that the OTLP HTTP exporter retains the counter.
GitOrigin-RevId: d266e840d5871aa2c34bed8ce44f3345e2b4650f
## What changed
- Request and parse server-issued Guardian tickets for eligible Codex backend responses.
- Scope each ticket to the response that produced a Guardian review or classifier request, including HTTP and WebSocket retries, without adding it to model-visible or persisted context.
- Validate ticket shape, reserve its metadata keys, redact it from debug and request-body logging, and attach it only at the Guardian transport boundary.
## Testing
- Cover ticket parsing, endpoint and authentication routing, retry scoping, classifier retries, and redaction for compressed and uncompressed requests.
GitOrigin-RevId: 6219dd15c9f19ed3387de516aac1b8e7c9cdb0e3
## What changed
Add `install_global_metrics` as the public entry point for installing an
externally managed, non-Statsig `MetricsClient` during single-threaded startup.
The returned handle retains ownership of the installed exporter so callers can
flush and shut it down.
GitOrigin-RevId: 1108a0468588046e5cc8266d2af6143f14456d82
## Why
Exec-server process and network-policy events need stable attribution to the
launching tool call and executor without recording process payloads.
## What changed
- Add optional `ExecMetadata` to `ExecParams` and propagate the thread and tool
call IDs from unified exec.
- Emit bounded OpenTelemetry events for process start, spawn failure, sandbox
denial, and exit. Correlate them with the launch trace and registry-issued
executor identity while excluding arguments, paths, environment values,
output, and error text.
- Preserve launch attribution across long-running processes and reconnects, and
attach the same metadata to network-policy audit events.
- Keep the protocol backward compatible when metadata is omitted, and prevent
invalid trace headers from inheriting an unrelated active span.
## Testing
Add coverage for metadata serialization and propagation, lifecycle event
fields, trace relationships, reconnect behavior, spawn failures, sandbox
denials, and network-policy attribution.
GitOrigin-RevId: 7aa480a7289c73cb95e2c124c35500bb6f0d5084
## Why
Reasoning configuration changes need to retain their position and trusted provenance when model history is persisted and replayed. Client-injected history must not be able to forge these controls.
## What changed
- Add a typed `configuration_update` response item carrying reasoning effort, including custom model-defined values.
- Persist harness-authored updates with provenance and preserve them across history reconstruction, thread resume, raw response notifications, and agent forks.
- Exclude untrusted configuration updates from model history, strip client-supplied provenance metadata, and reject configuration updates supplied as turn input.
- Export the new item through the JSON and TypeScript app-server schemas and classify it in telemetry and persistence metrics.
## Testing
- Cover serialization, provenance persistence, history filtering and rollback, resume reconstruction, and injection attempts before and after restart.
GitOrigin-RevId: eb5559d2b52b7a931621e7c9812f009ff9fb8939
## Why
SQLite log persistence needs visibility into batch size, write latency,
failures, and entries dropped before they reach the writer. Exporter
diagnostics must not feed back into the SQLite log sink and keep metric
exports active indefinitely.
## What changed
- Record write count, duration, batch bytes, entry count, largest-entry size,
and queue drops with bounded outcome tags.
- Add explicit byte histogram boundaries for log batch metrics.
- Filter OTLP transport noise and the benign unsolicited HTTP/2 PING warning
from SQLite log persistence to prevent export cycles while retaining other
HTTP/2 warnings.
## Testing
- Cover metric values, tags, queue-drop reasons, and histogram boundaries.
- Verify successful and failed OTLP HTTP and gRPC exports become idle instead
of generating recurring SQLite log-write metrics.
GitOrigin-RevId: a63224ae15945fe84adc977a8b7d55bb47a4de53
## Why
Telemetry transport and storage can emit their own diagnostics, causing exports
or SQLite writes to recursively generate more telemetry.
## What changed
- Exclude `h2` spans from trace export so OTLP transport activity cannot trigger
additional OTLP exports.
- Exclude all `sqlx`, `sqlx_core`, and `sqlx_sqlite` diagnostics from the SQLite
log sink, including when no external subscriber filter is installed.
- Keep similarly named targets such as `sqlx_application` eligible for storage.
## Testing
- Extend the SQLite log-filter test to cover the broader SQLx exclusion and the
unrelated-target case.
GitOrigin-RevId: 13a832e1f228e9564a31cc08f00add286342bac1
## What changed
- Emit `codex.turn.cost_microusd` as a counter with turn, conversation,
interruption, speed, and reasoning-effort attributes. Convert the estimated
USD string to microdollars, rounding to the nearest microdollar and skipping
values that cannot be represented safely.
- Start turn-cost collection when an OTLP metrics exporter is configured, even
when the OTEL log exporter is disabled.
## Testing
- Verify cost conversion, rounding, and metric attributes with an in-memory
metrics snapshot.
- Verify that a metrics-only OTLP configuration starts the turn-cost worker.
GitOrigin-RevId: e60587079f097f10cc1b6d1bf889d3f7a0dc3d5f
## What changed
- Add the experimental `amazonBedrockAccessKeys` app-server login flow, persist its credentials in the configured auth store, and use them for SigV4-signed Bedrock requests.
- Report `bedrockAccessKeys` as a distinct auth mode and document access-key login alongside AWS profile and environment credential setup.
- Clear the selected Bedrock provider, AWS settings, and Bedrock-specific model on logout while leaving external AWS credentials and `.env` untouched.
- Reject managed access-key login when higher-precedence configuration would prevent its region or profile settings from taking effect.
## Testing
- Cover access-key login, storage, request signing, configuration conflicts, status reporting, and logout cleanup.
GitOrigin-RevId: bbf297887c6503d3651ceb2aad85c4d626e32723
## What changed
- Add `otel.tool_result.max_bytes`, defaulting to 2048 bytes, to control the
text included in `codex.tool_result` logs independently of model-visible
output limits.
- Apply truncation centrally when emitting telemetry, preserve UTF-8
boundaries, and report whether the logged output was truncated.
- Record typed tool names and namespaces, agent names, and process-local result
sequence numbers while keeping tool arguments and output out of trace events.
## Testing
- Cover configuration propagation, UTF-8-safe truncation, raw tool-output
logging, and log/trace event routing and fields.
GitOrigin-RevId: 89ca76e55f1a09b729e7770ce01f2e4d98f9a22d
## Why
Creating the telemetry shutdown thread during shutdown can fail under resource
pressure, including when the native thread guard page cannot be allocated.
## What changed
- Start and verify a dedicated shutdown worker when `OtelProvider` is created.
- Send the provider to the prepared worker for bounded shutdown, while preserving
timeout behavior and avoiding a potentially blocking destructor if worker
preparation failed.
- Rename the fallible provider constructor to `try_new`.
## Testing
Add Unix regression coverage that injects guard-page allocation failures, plus
coverage for worker preparation failure, successful shutdown, and timeouts.
GitOrigin-RevId: 3656298078a800a7fa392437c2ee4a68753092e3
## What changed
- Track app-server turn lifecycle events for OpenAI API-key sessions when OTLP telemetry export is enabled.
- Query the turn-cost endpoint after completed or aborted turns and emit `codex.turn_cost` with the estimated USD cost, interruption state, model, speed, and reasoning effort.
- Preserve provider organization and project headers, wait for all observed responses to be priced, and retry pending or transiently failed queries with bounded tracking.
- Re-probe availability after authentication changes and shut the worker down with other app-server background tasks.
## Testing
- Cover late API-key login, transient probe failures, telemetry after thread removal, and turns with multiple responses.
- Verify turn-cost requests use API-key authentication and provider scope headers and deserialize priced response data.
GitOrigin-RevId: a303162a54798b554fcb2e9e2b372e1cf9f49f39
## What changed
- Represent blocked network requests as approval actions so permission hooks,
automatic review, and user review use the common approval flow.
- Route network requests using the active turn's review settings, including for
background terminals started by an earlier turn.
- Record the final applied network decision in tool telemetry without exposing
the destination or assigning an approval source.
- Persist deny amendments and keep the blocked request denied.
## Testing
Added coverage for strict automatic review, cross-turn background network
requests, deny amendment persistence, and destination-safe telemetry.
GitOrigin-RevId: a2a9d106962f407ed93f4d198f40ced15f090b8e
## What changed
- Recognize validated create and edit marker commands from the trusted presentations, documents, spreadsheets, and PDF plugins.
- Emit a `codex_artifact_operation` analytics event plus started-count and expected-output-count metrics when a recognized marker starts through unified exec.
- Attribute remote executor commands only when their plugin cache path and script contents match a trusted local plugin, including primary-runtime plugins.
## Testing
- Cover supported marker combinations and reject mismatched plugins, paths, arguments, counts, and output formats.
- Cover local, relocated remote, and Windows-style plugin attribution, including rejection of modified remote scripts.
GitOrigin-RevId: 475cc1689aa7f371a24dec0a8bdb6f7fd5a2e922
## What changed
- Exclude conversation turn count, response-engine latency, and turn token usage metrics from the built-in Statsig exporter, alongside the existing API and tool-call exclusions.
- Apply the Statsig exclusion list consistently to counters, histograms, gauges, observable gauges, and duration histograms.
- Preserve all excluded metrics for custom OTLP exporters and extend the loopback coverage to verify they are exported.
GitOrigin-RevId: e49f9aa1433126a3de8dbed76a4d2688f2831af8
## What changed
Add the `codex.turn.unified_exec.running_processes` counter and increment it
by the number of background terminal processes remaining when a turn finishes.
GitOrigin-RevId: 82a33b89533960d86cef95360fa122836c8aa84d
## Why
An account change can select different OpenTelemetry collector settings. The
long-running app server needs to stop exporting telemetry with the previous
account's configuration.
## What changed
- Rebuild and install the log, trace, and metrics exporters when authentication
changes, and shut down the previous provider without blocking the app server.
- Make cached global metrics clients and the tracing layer follow the replacement
provider so existing callers continue exporting through the current account's
configuration.
- Clear the cached cloud configuration loader after login and logout so telemetry
reloads from the new account context.
## Testing
- Add an app-server integration test that switches accounts and verifies logs,
traces, and metrics reach the new collector while trace context is preserved.
- Add a metrics test covering cached clients after provider replacement.
GitOrigin-RevId: a74359777a35edbee5bbe0b2baccc155074cb172
## Why
A stalled telemetry exporter can block shutdown indefinitely when the interactive TUI exits.
## What changed
- Add `OtelProvider::shutdown_with_timeout` to run exporter shutdown on a detached thread and report completion, worker failure, or timeout.
- Give interactive TUI telemetry shutdown a 500 ms time budget and warn if cleanup does not finish successfully.
## Testing
- Cover successful shutdown on current-thread and multi-thread Tokio runtimes, worker creation failure, and timeout behavior.
- Verify that OTLP HTTP shutdown returns on time when a collector stalls.
GitOrigin-RevId: 5b5f31a841fbc5788715cb3065d522c6dafa994f
## What changed
- Treat `codex.api_request` and `codex.api_request.duration_ms` as runtime-only metrics for Statsig exporters, matching the existing tool-call metric behavior.
- Keep both metrics available through other OTLP exporters.
## Testing
- Extend the Statsig filtering test to cover both API request metrics.
- Verify the OTLP HTTP exporter still sends the API request count and duration.
GitOrigin-RevId: f0f800ade7979e4da72ae59641bf42aa8a3ebd5a
## What changed
- Treat `codex.tool.call` and `codex.tool.call.duration_ms` as runtime-only metrics when using the built-in Statsig exporter.
- Continue exporting both metrics through explicitly configured OTLP exporters.
## Testing
- Verify the Statsig configuration omits the tool-call metrics while retaining unrelated metrics.
- Extend the OTLP HTTP loopback test to cover the tool-call counter and duration histogram.
GitOrigin-RevId: 46dd2642063b34e9a4c69b67d58b45721609fcf2
## What changed
- Guard `OtelProvider::shutdown` so exporters shut down at most once, including when an explicit shutdown is followed by `Drop`.
- Let tracer provider shutdown handle flushing without a separate `force_flush` call.
- Add regression tests for explicit shutdown and drop-only shutdown.
GitOrigin-RevId: f3c8769146ee57102477eb9ee7fd948f688efb79
## What changed
- Add an optional `ExtensionMetrics` capability to thread, turn-input, and world-state contributors, backed by session telemetry in the host.
- Record skill counts, omissions, and truncated description characters for extension-rendered catalogs, tagged by the catalog surface.
- Preserve host session attribution on extension metrics, including the effective model for each turn, and prevent extension tags from overriding host metadata.
- Avoid duplicate host-catalog samples by recording world-state metrics only when the section is published or changes.
## Testing
- Cover metric values and surface tags, session metadata preservation, turn-level model changes, and host world-state publication behavior.
GitOrigin-RevId: 9059e521943a8f81df25a38b03692eddbc1d990d
## Why
Requests longer than 10 seconds were grouped into the overflow bucket, limiting visibility into long-running request latency.
## What changed
Add second-based duration histogram boundaries at 12, 15, 20, 30, 60, and 120 seconds.
## Testing
Extend the duration histogram test to cover every new boundary and a 121-second overflow value.
GitOrigin-RevId: 063a5d397dbd37d3b3940c7a1632b7b479e874a4
## What changed
- Parse `cache_write_tokens` from response input token details and carry it through token usage aggregation.
- Expose `cache_write_input_tokens` in protocol, app-server, exec, and TypeScript SDK usage events, defaulting it to zero for compatibility with older payloads.
- Report cache-write usage in analytics, metrics, traces, and rollout data alongside existing cache-read usage.
## Testing
- Add coverage for response parsing, usage event output, aggregation, analytics, metrics, and telemetry fields.
GitOrigin-RevId: 634e4fbe3086a8923c0ee13b622573e7d9d221bd
## What changed
Add finer-grained buckets from 1 to 10 seconds and extend millisecond duration histograms with buckets up to 120 seconds.
GitOrigin-RevId: dee492de08db058444926d22281ef7bc4ba1cdeb
## What changed
- Emit `responsesapi.websocket_timing` payloads as opt-in trace events with request context, while excluding them from diagnostic uploads and persisted logs by default.
- Preserve fractional-millisecond TBT values in telemetry histograms and runtime summaries, rounding only when formatting the TUI label.
## Testing
- Cover timing-log filtering, fractional TBT collection and display, and existing whole-millisecond duration behavior.
GitOrigin-RevId: 5f0a1c60c237abed17cc8b2d569ca77fbbc6ec41
## Summary
When enabled for the OpenAI provider, Codex sends
`stream_options.reasoning_summary_delivery = "sequential_cutoff"` on
HTTP and
WebSocket requests, including prewarm, and renders completed summary
sections
from `reasoning_summary_text.done`. Flag-off and non-OpenAI behavior is
unchanged.
## Expected rollout
```text
reasoning 0 added
summary 0 done
summary 1 done
summary 2 starts
summary 2 cancelled / incomplete
reasoning 0 done <-- cancel summary 2 work and mark it incomplete
message 1 added
message 1 text streams
message 1 completed
```
Depends on
[openai/openai#1096660](https://github.com/openai/openai/pull/1096660).
## Why
Codex telemetry pipeline needs a per-request TTFT value. The existing
`codex.turn_ttft` is recorded once per turn, so it cannot represent
later inference requests in the same turn and can miss the beginning of
hidden reasoning.
This restores the low-volume per-request signal proposed in
https://github.com/bk-nvidia/codex/pull/3 without bringing back
per-WebSocket-event TRACE logging.
## What changed
- start a timer when each mapped Responses stream begins
- latch the timer on the first `response.output_item.added`, including
an empty hidden-reasoning item
- attach `ttft_ms` to the existing `codex.sse_event` /
`response.completed` telemetry record
- cover the new completion field with an integration test
## Semantics
The value is per inference request, not per turn. It measures
mapped-stream-to-first-output-item latency, matching the
customer-proposed metric. For HTTP, the stream is already established
before timing begins, so request setup and response-header latency are
excluded.
`response.output_item.added` is a client-visible proxy for the start of
hidden reasoning; this does not claim access to the server's internal
first raw-token timestamp.
## Validation
- `just test -p codex-otel` (47 passed)
- `just test -p codex-core process_sse_emits_completed_telemetry` (1
passed after the final timer-placement change)
- attempted `just test -p codex-core`: 2,855 passed and 53 failed
because of unrelated local-environment failures (missing
`test_stdio_server` fixture binary, shell startup noise, and
timing-sensitive tests); the focused telemetry test passed in that run
as well
Fixes distributed trace continuity across exec-server JSON-RPC HTTP
egress by adding an executor client span and injecting its W3C context
through a reusable `codex-otel` helper.
This preserves the caller trace across core/tool → executor →
provider/MCP instead of dropping parentage at raw reqwest.
Note that this doesn't include the websocket path, which is needed to
really get the full story but at least we cover the basic http path with
this change.
## Summary
- Record bounded connection, request, and process lifecycle metrics.
- Report active gauges from callbacks on every collection, including
delta exports.
- Serialize active-count updates so concurrent starts and finishes
cannot publish stale values.
- Serialize process exit, explicit termination, and shutdown through the
process registry so exactly one completion result wins.
- Keep the implementation small with single-owner RAII guards and one
real OTLP/HTTP integration test using the existing `wiremock`
dependency.
## Root cause
Process exit and session shutdown previously used cloned completion
state. That avoided duplicate emission, but it duplicated lifecycle
ownership and made the ordering harder to reason about. The process
registry mutex already defines the lifecycle ordering, so the final
implementation stores the metric guard and termination flag directly on
the process entry. Whichever path claims the entry first owns the
completion result.
Production metric export uses delta temporality. Event-only synchronous
gauge recordings disappear after the next collection when no count
changes, so active counts now use observable callbacks that report
current state on every collection.
The cleanup also removes the constant `result="accepted"` connection
tag, redundant route and response assertions, a custom HTTP collector,
and fallback initialization machinery that did not add behavior.
## Stack
Review and land this stack in order:
1. #27466 — trace exec-server JSON-RPC requests
2. #27467 — record bounded connection, request, and process lifecycle
metrics **(this PR)**
3. #27470 — observe remote registration and Noise rendezvous lifecycle
## Validation
- `just test -p codex-exec-server --lib` (158 passed)
- `just test -p codex-cli --test exec_server` (3 passed)
- `just test -p codex-otel
observable_gauge_is_collected_on_every_delta_snapshot` (1 passed)
- `CARGO_BUILD_JOBS=1 just fix -p codex-otel -p codex-exec-server`
- `just fmt`
- `git diff --check`
## Stack
This is PR 3 of the simplified HAI single-run-task stack:
- [#19047](https://github.com/openai/codex/pull/19047) Agent Identity
assertion and task-registration primitives, including the shared
run-task helper used by existing Agent Identity JWT auth.
- [#19049](https://github.com/openai/codex/pull/19049)
Disabled-by-default ChatGPT auth opt-in that provisions/reuses persisted
Agent Identity runtime auth and its single run task.
- [#19051](https://github.com/openai/codex/pull/19051) Run-scoped
provider auth that uses one backend-owned task id for first-party
inference and compaction requests.
[#19054](https://github.com/openai/codex/pull/19054) collapsed out of
the active stack because the simplified design no longer needs a
separate background/control-plane task helper.
## Summary
This PR moves Agent Identity usage into provider auth resolution. That
keeps `AgentAssertion` auth tied to first-party OpenAI provider requests
instead of applying a late session-wide override that could affect
local, custom, Bedrock, API-key, or external-bearer providers.
What changed:
- adds a small `ProviderAuthScope` struct carrying the run auth policy
and session source needed by provider-scoped auth resolution
- lets `Session` opt the existing `ModelClient` into `ChatGptAuth`
policy when `use_agent_identity` is enabled, without adding a second
model-client constructor
- resolves Agent Identity only for first-party OpenAI provider auth
paths
- uses the persisted run task id from the `AgentIdentityAuth` record to
build `AgentAssertion` auth for Responses requests
- routes shared request setup through scoped provider auth so unary
compact requests use the same run-task assertion path as inference turns
- keeps local/custom/Bedrock/env-key/external-bearer provider auth
unchanged
- lets missing run-task state surface through the existing model-request
error path instead of silently falling back to bearer auth
This PR intentionally does not create thread-scoped, target-scoped, or
background-scoped task identities. The run task is the only task Codex
registers in this POC shape.
## Testing
- `just test -p codex-model-provider`
- `just test -p codex-core client::tests::provider_auth_scope_uses`
- `just test -p codex-core remote_compact_uses_agent_identity_assertion`
## Why
Authentication mode is a domain concept used by login, model selection,
telemetry, and transports. Keeping the canonical type in app-server
protocol forces those lower-level crates to depend on an unrelated wire
API.
## What changed
- Added canonical `codex_protocol::auth::AuthMode` domain values.
- Kept the app-server wire DTO unchanged and added an explicit app-side
conversion.
- Removed production app-server-protocol dependencies from login,
model-provider-info, models-manager, and otel call paths.
## Stack
This is PR 2 of 6, stacked on [PR
#29714](https://github.com/openai/codex/pull/29714). Review only the
delta from `codex/split-json-rpc-protocols`. Next: [PR
#29722](https://github.com/openai/codex/pull/29722).
## Validation
- Auth and login coverage passed in the focused protocol/domain test
run.
- App-server account and auth conversion coverage passed.
When using Responses Lite, we should all use `additional_tools` and a
developer item instead of the top level tools array & instructions
field. This keeps things 1-to-1.
Forced namespacing for _all_ tools will land in a following PR after
some coordination & fixes in Responses API (around collisions & return
items).
The goal is to eventually expand the scope of this to _all_ requests
from codex, but that will require larger coordination across providers &
slower rollout.
## Summary
NVIDIA asked to measure Fast mode usage and reasoning effort from Codex
CLI OTEL logs. Add the finalized `service_tier` and
`model_reasoning_effort` to the existing `codex.sse_event`
`response.completed` record.
This intentionally reuses the existing completion event and leaves
transport APIs and shared telemetry plumbing unchanged.
## Testing
- `cargo build -p codex-cli --bin codex`
- `just test -p codex-core responses_api_emits_api_request_event`
- End-to-end with the built CLI and a local OTLP/HTTP collector:
- Fast/high emitted `service_tier=priority` and
`model_reasoning_effort=high` with token usage.
- Standard/low omitted `service_tier` and emitted
`model_reasoning_effort=low` with token usage.
jif