## What changed
- Request and parse server-issued Guardian tickets for eligible Codex backend responses.
- Scope each ticket to the response that produced a Guardian review or classifier request, including HTTP and WebSocket retries, without adding it to model-visible or persisted context.
- Validate ticket shape, reserve its metadata keys, redact it from debug and request-body logging, and attach it only at the Guardian transport boundary.
## Testing
- Cover ticket parsing, endpoint and authentication routing, retry scoping, classifier retries, and redaction for compressed and uncompressed requests.
GitOrigin-RevId: 6219dd15c9f19ed3387de516aac1b8e7c9cdb0e3
## Why
Reasoning configuration changes need to retain their position and trusted provenance when model history is persisted and replayed. Client-injected history must not be able to forge these controls.
## What changed
- Add a typed `configuration_update` response item carrying reasoning effort, including custom model-defined values.
- Persist harness-authored updates with provenance and preserve them across history reconstruction, thread resume, raw response notifications, and agent forks.
- Exclude untrusted configuration updates from model history, strip client-supplied provenance metadata, and reject configuration updates supplied as turn input.
- Export the new item through the JSON and TypeScript app-server schemas and classify it in telemetry and persistence metrics.
## Testing
- Cover serialization, provenance persistence, history filtering and rollback, resume reconstruction, and injection attempts before and after restart.
GitOrigin-RevId: eb5559d2b52b7a931621e7c9812f009ff9fb8939
## What changed
- Emit `codex.turn.cost_microusd` as a counter with turn, conversation,
interruption, speed, and reasoning-effort attributes. Convert the estimated
USD string to microdollars, rounding to the nearest microdollar and skipping
values that cannot be represented safely.
- Start turn-cost collection when an OTLP metrics exporter is configured, even
when the OTEL log exporter is disabled.
## Testing
- Verify cost conversion, rounding, and metric attributes with an in-memory
metrics snapshot.
- Verify that a metrics-only OTLP configuration starts the turn-cost worker.
GitOrigin-RevId: e60587079f097f10cc1b6d1bf889d3f7a0dc3d5f
## What changed
- Add `otel.tool_result.max_bytes`, defaulting to 2048 bytes, to control the
text included in `codex.tool_result` logs independently of model-visible
output limits.
- Apply truncation centrally when emitting telemetry, preserve UTF-8
boundaries, and report whether the logged output was truncated.
- Record typed tool names and namespaces, agent names, and process-local result
sequence numbers while keeping tool arguments and output out of trace events.
## Testing
- Cover configuration propagation, UTF-8-safe truncation, raw tool-output
logging, and log/trace event routing and fields.
GitOrigin-RevId: 89ca76e55f1a09b729e7770ce01f2e4d98f9a22d
## What changed
- Track app-server turn lifecycle events for OpenAI API-key sessions when OTLP telemetry export is enabled.
- Query the turn-cost endpoint after completed or aborted turns and emit `codex.turn_cost` with the estimated USD cost, interruption state, model, speed, and reasoning effort.
- Preserve provider organization and project headers, wait for all observed responses to be priced, and retry pending or transiently failed queries with bounded tracking.
- Re-probe availability after authentication changes and shut the worker down with other app-server background tasks.
## Testing
- Cover late API-key login, transient probe failures, telemetry after thread removal, and turns with multiple responses.
- Verify turn-cost requests use API-key authentication and provider scope headers and deserialize priced response data.
GitOrigin-RevId: a303162a54798b554fcb2e9e2b372e1cf9f49f39
## What changed
- Represent blocked network requests as approval actions so permission hooks,
automatic review, and user review use the common approval flow.
- Route network requests using the active turn's review settings, including for
background terminals started by an earlier turn.
- Record the final applied network decision in tool telemetry without exposing
the destination or assigning an approval source.
- Persist deny amendments and keep the blocked request denied.
## Testing
Added coverage for strict automatic review, cross-turn background network
requests, deny amendment persistence, and destination-safe telemetry.
GitOrigin-RevId: a2a9d106962f407ed93f4d198f40ced15f090b8e
## What changed
- Add an optional `ExtensionMetrics` capability to thread, turn-input, and world-state contributors, backed by session telemetry in the host.
- Record skill counts, omissions, and truncated description characters for extension-rendered catalogs, tagged by the catalog surface.
- Preserve host session attribution on extension metrics, including the effective model for each turn, and prevent extension tags from overriding host metadata.
- Avoid duplicate host-catalog samples by recording world-state metrics only when the section is published or changes.
## Testing
- Cover metric values and surface tags, session metadata preservation, turn-level model changes, and host world-state publication behavior.
GitOrigin-RevId: 9059e521943a8f81df25a38b03692eddbc1d990d
## What changed
- Parse `cache_write_tokens` from response input token details and carry it through token usage aggregation.
- Expose `cache_write_input_tokens` in protocol, app-server, exec, and TypeScript SDK usage events, defaulting it to zero for compatibility with older payloads.
- Report cache-write usage in analytics, metrics, traces, and rollout data alongside existing cache-read usage.
## Testing
- Add coverage for response parsing, usage event output, aggregation, analytics, metrics, and telemetry fields.
GitOrigin-RevId: 634e4fbe3086a8923c0ee13b622573e7d9d221bd
## What changed
- Emit `responsesapi.websocket_timing` payloads as opt-in trace events with request context, while excluding them from diagnostic uploads and persisted logs by default.
- Preserve fractional-millisecond TBT values in telemetry histograms and runtime summaries, rounding only when formatting the TUI label.
## Testing
- Cover timing-log filtering, fractional TBT collection and display, and existing whole-millisecond duration behavior.
GitOrigin-RevId: 5f0a1c60c237abed17cc8b2d569ca77fbbc6ec41
## Summary
When enabled for the OpenAI provider, Codex sends
`stream_options.reasoning_summary_delivery = "sequential_cutoff"` on
HTTP and
WebSocket requests, including prewarm, and renders completed summary
sections
from `reasoning_summary_text.done`. Flag-off and non-OpenAI behavior is
unchanged.
## Expected rollout
```text
reasoning 0 added
summary 0 done
summary 1 done
summary 2 starts
summary 2 cancelled / incomplete
reasoning 0 done <-- cancel summary 2 work and mark it incomplete
message 1 added
message 1 text streams
message 1 completed
```
Depends on
[openai/openai#1096660](https://github.com/openai/openai/pull/1096660).
## Why
Codex telemetry pipeline needs a per-request TTFT value. The existing
`codex.turn_ttft` is recorded once per turn, so it cannot represent
later inference requests in the same turn and can miss the beginning of
hidden reasoning.
This restores the low-volume per-request signal proposed in
https://github.com/bk-nvidia/codex/pull/3 without bringing back
per-WebSocket-event TRACE logging.
## What changed
- start a timer when each mapped Responses stream begins
- latch the timer on the first `response.output_item.added`, including
an empty hidden-reasoning item
- attach `ttft_ms` to the existing `codex.sse_event` /
`response.completed` telemetry record
- cover the new completion field with an integration test
## Semantics
The value is per inference request, not per turn. It measures
mapped-stream-to-first-output-item latency, matching the
customer-proposed metric. For HTTP, the stream is already established
before timing begins, so request setup and response-header latency are
excluded.
`response.output_item.added` is a client-visible proxy for the start of
hidden reasoning; this does not claim access to the server's internal
first raw-token timestamp.
## Validation
- `just test -p codex-otel` (47 passed)
- `just test -p codex-core process_sse_emits_completed_telemetry` (1
passed after the final timer-placement change)
- attempted `just test -p codex-core`: 2,855 passed and 53 failed
because of unrelated local-environment failures (missing
`test_stdio_server` fixture binary, shell startup noise, and
timing-sensitive tests); the focused telemetry test passed in that run
as well
## Stack
This is PR 3 of the simplified HAI single-run-task stack:
- [#19047](https://github.com/openai/codex/pull/19047) Agent Identity
assertion and task-registration primitives, including the shared
run-task helper used by existing Agent Identity JWT auth.
- [#19049](https://github.com/openai/codex/pull/19049)
Disabled-by-default ChatGPT auth opt-in that provisions/reuses persisted
Agent Identity runtime auth and its single run task.
- [#19051](https://github.com/openai/codex/pull/19051) Run-scoped
provider auth that uses one backend-owned task id for first-party
inference and compaction requests.
[#19054](https://github.com/openai/codex/pull/19054) collapsed out of
the active stack because the simplified design no longer needs a
separate background/control-plane task helper.
## Summary
This PR moves Agent Identity usage into provider auth resolution. That
keeps `AgentAssertion` auth tied to first-party OpenAI provider requests
instead of applying a late session-wide override that could affect
local, custom, Bedrock, API-key, or external-bearer providers.
What changed:
- adds a small `ProviderAuthScope` struct carrying the run auth policy
and session source needed by provider-scoped auth resolution
- lets `Session` opt the existing `ModelClient` into `ChatGptAuth`
policy when `use_agent_identity` is enabled, without adding a second
model-client constructor
- resolves Agent Identity only for first-party OpenAI provider auth
paths
- uses the persisted run task id from the `AgentIdentityAuth` record to
build `AgentAssertion` auth for Responses requests
- routes shared request setup through scoped provider auth so unary
compact requests use the same run-task assertion path as inference turns
- keeps local/custom/Bedrock/env-key/external-bearer provider auth
unchanged
- lets missing run-task state surface through the existing model-request
error path instead of silently falling back to bearer auth
This PR intentionally does not create thread-scoped, target-scoped, or
background-scoped task identities. The run task is the only task Codex
registers in this POC shape.
## Testing
- `just test -p codex-model-provider`
- `just test -p codex-core client::tests::provider_auth_scope_uses`
- `just test -p codex-core remote_compact_uses_agent_identity_assertion`
When using Responses Lite, we should all use `additional_tools` and a
developer item instead of the top level tools array & instructions
field. This keeps things 1-to-1.
Forced namespacing for _all_ tools will land in a following PR after
some coordination & fixes in Responses API (around collisions & return
items).
The goal is to eventually expand the scope of this to _all_ requests
from codex, but that will require larger coordination across providers &
slower rollout.
## Summary
NVIDIA asked to measure Fast mode usage and reasoning effort from Codex
CLI OTEL logs. Add the finalized `service_tier` and
`model_reasoning_effort` to the existing `codex.sse_event`
`response.completed` record.
This intentionally reuses the existing completion event and leaves
transport APIs and shared telemetry plumbing unchanged.
## Testing
- `cargo build -p codex-cli --bin codex`
- `just test -p codex-core responses_api_emits_api_request_event`
- End-to-end with the built CLI and a local OTLP/HTTP collector:
- Fast/high emitted `service_tier=priority` and
`model_reasoning_effort=high` with token usage.
- Standard/low omitted `service_tier` and emitted
`model_reasoning_effort=low` with token usage.
## Why
Every successful Responses WebSocket event currently produces three
local log records: the full payload at TRACE, an OpenTelemetry log
event, and an OpenTelemetry trace event.
On busy threads these records fill the 1,000-row log partition in
seconds and cause continuous SQLite insert-and-prune churn.
Related to
https://openai.slack.com/archives/C095U48JNL9/p1782128972644209
## What changed
- Stop logging each successful Responses WebSocket payload at TRACE.
- Stop emitting `codex.websocket_event` as OpenTelemetry log and trace
events.
- Keep WebSocket event counters, duration metrics, response timing
metrics, parsing, and error handling.
Responses API safety buffering metadata currently stops at the transport
boundary, so app-server clients cannot render the in-progress safety
review state.
This change:
- decodes and deduplicates `safety_buffering` metadata from Responses
API SSE and WebSocket events without suppressing the original response
event
- emits a typed core event containing the requested model plus backend
use cases and reasons
- forwards that event as `turn/safetyBuffering/updated` through
app-server v2 and updates generated protocol schemas
- keeps the side-channel event out of persisted rollouts and turn timing
This supports the Codex Apps buffering UX and depends on the Responses
API backend work in https://github.com/openai/openai/pull/1044569 and
https://github.com/openai/openai/pull/1044571.
Validation:
- focused `codex-core` safety-buffering integration test passes
- `cargo check -p codex-core -p codex-app-server -p
codex-app-server-protocol`
- `just fix -p codex-api -p codex-protocol -p codex-core -p
codex-app-server-protocol -p codex-app-server -p codex-rollout -p
codex-rollout-trace -p codex-otel`
- `just fmt`
- broad package test run: 4,430/4,492 passed; 62 unrelated
local-environment/concurrency failures involved unavailable test
binaries, MCP subprocess setup, and app-server timeouts
## Description
This PR adds an optional `metadata` field to `ResponseItem` for
Responses API calls. Only mechanical plumbing, no actual values
populated and sent yet. Turns out just adding a new field to
`ResponseItem` has quite a large blast radius already.
This change is backwards compatible because `metadata` is optional and
omitted when absent, so existing response items and rollout history
without it still deserialize and requests that do not set it keep the
same wire shape. For provider compatibility, we strip out `metadata`
before non-OpenAI Responses requests so Azure and AWS Bedrock never see
this field.
My followup PR here will actually make use of it to start storing and
passing along `turn_id`: https://github.com/openai/codex/pull/28360
## What changed
- Added `ResponseItemMetadata` with optional `turn_id`, plus optional
`metadata` on Responses API item variants and inter-agent communication.
- Preserved item metadata through response-item rewrites such as
truncation, missing tool-output synthesis, compaction history
rebuilding, visible-history conversion, rollout/resume, and generated
app-server schemas/types.
- Strip item metadata from non-OpenAI Responses requests while
preserving it for OpenAI-shaped requests.
- Updated the mechanical fixture/test construction churn required by the
new optional field.
## Why
First-party backends can supply turn-scoped moderation metadata that
app-server clients need for client-side presentation. Exposing this as
an experimental typed notification lets opted-in clients consume it
without interpreting raw Responses API events.
## What changed
- forward `response.metadata.openai_chatgpt_moderation_metadata` from
Responses API SSE and WebSocket streams as turn-scoped moderation
metadata
- emit the experimental app-server v2 `turn/moderationMetadata`
notification with `{ threadId, turnId, metadata }`
- add app-server integration coverage for the typed moderation metadata
notification
## Testing
- `just test -p codex-core
build_ws_client_metadata_includes_window_lineage_and_turn_metadata`
- `just test -p codex-core` (fails locally: 46 failures and 1 timeout,
primarily missing `test_stdio_server` and shell snapshot timeouts)
- `just test -p codex-app-server-protocol`
- `just test -p codex-app-server
turn_moderation_metadata_emits_typed_notification_v2`
- `just test -p codex-app-server` (fails locally: 792 passed, 10 failed,
and 5 timed out; failures are in existing environment-sensitive tests,
primarily because nested macOS `sandbox-exec` is not permitted)
- `just write-app-server-schema --experimental --schema-root
/tmp/codex-app-server-schema-experimental`
## Why
Multi-agent v2 currently routes agent instructions through normal tool
arguments and inter-agent context. That means the parent model can emit
plaintext task text, Codex can persist it in history/rollouts, and the
recipient can receive it as ordinary assistant-message JSON.
This changes the v2 path so agent instructions stay encrypted between
model calls: Responses encrypts the `message` argument returned by the
model, Codex forwards only that ciphertext, and Responses decrypts it
internally for the recipient model.
## What changed
- Mark the v2 `message` parameter as encrypted for `spawn_agent`,
`send_message`, and `followup_task`.
- Treat multi-agent v2 tool `message` values as ciphertext
unconditionally.
- Store v2 inter-agent task text in
`InterAgentCommunication.encrypted_content` with empty plaintext
`content`.
- Convert encrypted inter-agent communications into the Responses
`agent_message` input item before sending the child request.
- Preserve `agent_message` items across history, rollout, compaction,
telemetry, and app-server schema paths.
- Leave multi-agent v1 unchanged.
## Message shape
The model still calls the v2 tools with a `message` argument, but that
value is now ciphertext:
```json
{
"name": "spawn_agent",
"arguments": {
"task_name": "worker",
"message": "<ciphertext>"
}
}
```
Codex stores the task as encrypted inter-agent communication:
```json
{
"author": "/root",
"recipient": "/root/worker",
"content": "",
"encrypted_content": "<ciphertext>",
"trigger_turn": true
}
```
When Codex builds the recipient request, it forwards the ciphertext
using the new Responses input item:
```json
{
"type": "agent_message",
"author": "/root",
"recipient": "/root/worker",
"content": [
{
"type": "encrypted_content",
"encrypted_content": "<ciphertext>"
}
]
}
```
Responses decrypts that item internally for the recipient model.
## Context impact
- Parent context no longer carries plaintext v2 agent task instructions
from these tool arguments.
- Codex rollout/history stores ciphertext for v2 agent instructions.
- Recipient requests receive an `agent_message` item instead of
assistant commentary JSON for encrypted task delivery.
- Plaintext completion/status notifications are still plaintext because
they are Codex-generated status messages, not encrypted model tool
arguments.
## Validation
- `just test -p codex-tools`
- `just test -p codex-protocol`
- `just test -p codex-rollout`
- `just test -p codex-rollout-trace`
- `just test -p codex-otel`
- `just write-app-server-schema`
## Summary
Adds a dedicated `codex.sandbox_outcome` telemetry event so we can query
sandbox edge outcomes without threading sandbox metadata through
tool-result output types.
This is meant to make sandbox failures and approved escalation retries
visible in OTEL while keeping the existing `codex.tool_result` event
shape focused on tool completion data.
## What changed
- Adds `SessionTelemetry::sandbox_outcome(...)`, which emits
`codex.sandbox_outcome` as both a log and trace event.
- Records the tool name, call id, sandbox outcome, initial attempt
duration, and escalated attempt duration when a retry runs.
- Emits `denied` when the sandbox blocks execution and no retry is run.
- Emits `timed_out` and `signal` when those sandbox errors surface from
tool execution.
- Emits `escalated` when the initial sandboxed attempt fails and the
approved unsandboxed retry succeeds.
- Adds OTEL coverage for the new event payload, including timing fields.
## Validation
- `RUST_MIN_STACK=8388608 just test -p codex-core
sandbox_outcome_event_records_outcome
handle_sandbox_error_user_approves_retry_records_tool_decision`
- `just test -p codex-otel
otel_export_routing_policy_routes_tool_result_log_and_trace_events
runtime_metrics_summary_collects_tool_api_and_streaming_metrics`
- `just fix -p codex-core`
- `just fix -p codex-otel`
## Summary
- accept non-empty model-defined reasoning effort values while
preserving built-in effort behavior
- propagate the non-Copy effort type through core, app-server, TUI,
telemetry, and persistence call sites
- preserve string wire encoding and expose an open-string schema for
clients
- update model selection and shortcut behavior for model-advertised
effort values
## Root cause
`ReasoningEffort` gained a string-backed custom variant, so it could no
longer implement `Copy` or rely on derived closed-enum serialization.
Existing consumers still moved effort values from shared references and
assumed a fixed built-in value set.
## Validation
- `just fmt`
- Local tests and compilation were not run per request; relying on CI.
## Summary
- drop the dead legacy profile usage metric and active-profile
conversation-start fields
- update role comments so they describe provider and service-tier
preservation without legacy config-profile wording
- pair the code cleanup with the file-backed profile docs update in
openai/developers-website#1476
## Testing
- `just fmt`
- `cargo test -p codex-otel`
- `cargo test -p codex-core` *(fails: existing stack overflow in
`mcp_tool_call::tests::guardian_mode_mcp_denial_returns_rationale_message`)*
- `cargo test -p codex-core --lib
mcp_tool_call::tests::guardian_mode_mcp_denial_returns_rationale_message`
*(fails with the same stack overflow)*
## Summary
- Add `list_available_plugins_to_install` as the inventory step for
plugin and connector install suggestions.
- Slim `request_plugin_install` so it only handles the actual
elicitation, instead of carrying the full discoverable list in its
prompt.
- Emit send-time telemetry when an install elicitation is dispatched,
including requested tool identity in the event payload.
- Emit install-result telemetry through `SessionTelemetry`, including
tool type, user response action, and completion status.
- Update registration and tests to cover the new two-step flow while
keeping the existing `tool_suggest` feature gate unchanged.
## Testing
- `just fmt`
- `cargo test -p codex-tools`
- `cargo test -p codex-core request_plugin_install`
- `cargo test -p codex-core list_available_plugins_to_install`
- `cargo test -p codex-core
install_suggestion_tools_can_be_registered_without_search_tool`
- `cargo test -p codex-otel
manager_records_plugin_install_suggestion_metric`
- `cargo test -p codex-otel
manager_records_plugin_install_elicitation_sent_metric`
- `just fix -p codex-core`
- `just fix -p codex-tools`
- `just fix -p codex-otel`
- `cargo check -p codex-core`
## Why
Remote compaction v2 was still using `context_compaction` as both the
request trigger and the compacted output shape. The Responses API now
has the landed contract for this flow: Codex sends a dedicated `{
"type": "compaction_trigger" }` input item, and the backend returns the
standard `compaction` output item with encrypted content.
This aligns the v2 path with that wire contract while preserving the
existing local compacted-history post-processing behavior.
## What changed
- Add `ResponseItem::CompactionTrigger` and regenerate the app-server
protocol schema fixtures.
- Send `compaction_trigger` from `remote_compaction_v2` instead of a
payload-less `context_compaction`.
- Collect exactly one backend `compaction` output item, then reuse the
existing compacted-history rebuilding path.
- Treat the trigger item as a transient request marker rather than model
output or persisted rollout/memory content.
## Verification
- `cargo test -p codex-protocol compaction_trigger`
- `cargo test -p codex-core remote_compact_v2`
- `cargo test -p codex-core compact_remote_v2`
- `cargo test -p codex-core
responses_websocket_sends_response_processed_after_remote_compaction_v2`
- `just write-app-server-schema`
- `cargo test -p codex-app-server-protocol schema_fixtures`
## Why
The MCP tool path had accumulated a few core-owned special cases: a
dedicated payload variant, resolver plumbing, a legacy `AfterToolUse`
translation path, and a side channel for parallel-call metadata. That
made `ToolRegistry` and the spec builder know more about MCP than they
needed to.
This change moves MCP-specific execution details back onto `ToolInfo`
and `McpHandler` so `codex-core` can treat MCP calls like normal
function calls while still preserving MCP-specific dispatch and
telemetry behavior where it belongs.
## What changed
- removed `resolve_mcp_tool_info`, `ToolPayload::Mcp`, `ToolKind`, and
the remaining registry-side MCP resolver path
- stored MCP routing metadata directly on `McpHandler` and `ToolInfo`,
including `supports_parallel_tool_calls`
- deleted the legacy `AfterToolUse` consumer in `core`, which removes
the need for handler-specific `after_tool_use_payload` implementations
- switched tool-result telemetry to handler-provided tags and kept
MCP-specific dispatch payload construction inside the handler
- simplified tool spec planning/building by passing `ToolInfo` directly
and dropping the direct/deferred MCP wrapper structs and the
parallel-server side table
## Testing
- `cargo check -p codex-core -p codex-mcp -p codex-otel`
- `cargo test -p codex-core
mcp_parallel_support_uses_exact_payload_server`
- `cargo test -p codex-core
direct_mcp_tools_register_namespaced_handlers`
- `cargo test -p codex-core
search_tool_description_lists_each_mcp_source_once`
- `cargo test -p codex-mcp
list_all_tools_uses_startup_snapshot_while_client_is_pending`
- `just fix -p codex-core -p codex-mcp -p codex-otel`
## Why
While investigating `codex exec hi` startup latency, the useful
questions were not "is startup slow?" but "which durable bucket is slow
in production?"
The path we observed has a few distinct stages:
1. `thread/start` creates the session
2. startup prewarm builds the turn context, tools, and prompt
3. startup prewarm warms the websocket
4. the first real turn resolves the prewarm
5. the model produces the first token
Before this PR, production telemetry had some of the raw measurements
already:
- aggregate startup-prewarm duration / age-at-first-turn metrics
- TTFT as a metric
- websocket request telemetry
But there was no coherent production event stream for the startup
breakdown itself, and TTFT was metric-only. That made it hard to answer
the same latency questions from OpenTelemetry-backed logs without adding
one-off local instrumentation.
## What changed
Add durable production telemetry on the existing `SessionTelemetry`
path:
- new `codex.startup_phase` OTel log/trace events plus
`codex.startup.phase.duration_ms`
- new `codex.turn_ttft` OTel log/trace events while preserving the
existing TTFT metric
The startup phase event is emitted for the coarse buckets we actually
observed while running `exec hi`:
- `thread_start_create_thread`
- `startup_prewarm_total`
- `startup_prewarm_create_turn_context`
- `startup_prewarm_build_tools`
- `startup_prewarm_build_prompt`
- `startup_prewarm_websocket_warmup`
- `startup_prewarm_resolve`
These phases are intentionally low-cardinality so they remain safe as
production telemetry tags.
## Why this shape
This keeps the instrumentation on the same production path as the rest
of the session telemetry instead of adding a local debug-only trace
mode. It also avoids changing startup behavior:
- prewarm still runs
- no control flow changes
- no extra remote calls
- no user-visible behavior changes
One boundary is intentional: very early process bootstrap that happens
before a session exists is not included here, because this PR uses
session-scoped production telemetry. The expensive buckets we were
trying to understand after `thread/start` are now covered durably.
## Verification
- `cargo test -p codex-otel`
- `cargo test -p codex-core turn_timing`
- `cargo test -p codex-core
regular_turn_emits_turn_started_without_waiting_for_startup_prewarm`
- `cargo test -p codex-core
interrupting_regular_turn_waiting_on_startup_prewarm_emits_turn_aborted`
- `cargo test -p codex-app-server thread_start`
- `just fix -p codex-otel -p codex-core -p codex-app-server`
I also ran `cargo test -p codex-core`; it built successfully and then
hit an existing unrelated stack overflow in
`tools::handlers::multi_agents::tests::tool_handlers_cascade_close_and_resume_and_keep_explicitly_closed_subtrees_closed`.
## Why
This adds the `remote_compaction_v2` client path so remote compaction
can run through the normal Responses stream and install a
`context_compaction` item that trigger a compaction.
The goal is to migrate some of the compaction logic on the client side
We keeps the v2 transport behind a feature flag while letting follow-up
requests reuse the compacted context instead of falling back to the
legacy compaction item shape.
## What changed
- add `ResponseItem::ContextCompaction` and refresh the generated
app-server / schema / TypeScript fixtures that expose response items on
the wire
- add `core/src/compact_remote_v2.rs` to send compaction through the
standard streamed Responses client, require exactly one
`context_compaction` output item, and install that item into compacted
history
- route manual compact and auto-compaction through the v2 path when
`remote_compaction_v2` is enabled, while keeping the existing remote
compaction path as the fallback
- preserve the new item type across history retention, follow-up request
construction, telemetry, rollout persistence, and rollout-trace
normalization
- add targeted coverage for the feature flag, `context_compaction`
serialization, rollout-trace normalization, and remote-compaction
follow-up behavior
## Verification
- added protocol tests for `context_compaction`
serialization/deserialization in `protocol/src/models.rs`
- added rollout-trace coverage for `context_compaction` normalization in
`rollout-trace/src/reducer/conversation_tests.rs`
- added remote compaction integration coverage for v2 follow-up reuse
and mixed compaction output streams in
`core/tests/suite/compact_remote.rs`
---------
Co-authored-by: Codex <noreply@openai.com>
## Why
Slow Codex turns are easier to debug when token usage is visible in the
trace itself, without joining against separate analytics. This adds
token usage to existing turn-handling spans for regular user turns only.
[Example
turn](https://openai.datadoghq.com/apm/trace/9d353efa2cb5de1f4c5b93dc33c3df04?colorBy=service&graphType=flamegraph&shouldShowLegend=true&sort=time&spanID=3555541504891512675&spanViewType=metadata&traceQuery=)
<img width="1447" height="967" alt="Screenshot 2026-04-24 at 3 03 07 PM"
src="https://github.com/user-attachments/assets/ab7bb187-e7fc-41f0-a366-6c44610b2b2c"
/>
## What Changed
Added response-level token fields on completed handle_responses spans:
gen_ai.usage.input_tokens
gen_ai.usage.cache_read.input_tokens
gen_ai.usage.output_tokens
codex.usage.reasoning_output_tokens
codex.usage.total_tokens
Added aggregate token fields on regular turn spans:
codex.turn.token_usage.*
Added an explicit regular-turn opt-in via
SessionTask::records_turn_token_usage_on_span() so this is not coupled
to span-name strings.
## Testing
- `cargo test -p codex-otel`
- `cargo test -p codex-core
turn_and_completed_response_spans_record_token_usage`
- `just fmt`
- `just fix -p codex-core`
- `just fix -p codex-otel`
- Manual local Electron/app-server smoke test: regular user turn emits
the new span fields
Known status: `cargo test -p codex-core` was attempted and failed in
unrelated existing areas: config approvals, request-permissions,
git-info ordering, and subagent metadata persistence.
## Summary
- Remove `ghost_snapshot` / `GhostCommit` from the Responses API surface
and generated SDK/schema artifacts.
- Keep legacy config loading compatible, but make undo a no-op that
reports the feature is unavailable.
- Clean up core history, compaction, telemetry, rollout, and tests to
stop carrying ghost snapshot items.
## Testing
- Unit tests passed for `codex-protocol`, `codex-core` targeted undo and
compaction flows, `codex-rollout`, and `codex-app-server-protocol`.
- Regenerated config and app-server schemas plus Python SDK artifacts
and verified they match the checked-in outputs.
Adds new events for streaming apply_patch changes from responses api.
This is to enable clients to show progress during file writes.
Caveat: This does not work with apply_patch in function call mode, since
that required adding streaming json parsing.
## Summary
- reduce public module visibility across Rust crates, preferring private
or crate-private modules with explicit crate-root public exports
- update external call sites and tests to use the intended public crate
APIs instead of reaching through module trees
- add the module visibility guideline to AGENTS.md
## Validation
- `cargo check --workspace --all-targets --message-format=short` passed
before the final fix/format pass
- `just fix` completed successfully
- `just fmt` completed successfully
- `git diff --check` passed
1. Use requirement-resolved config.features as the plugin gate.
2. Guard plugin/list, plugin/read, and related flows behind that gate.
3. Skip bad marketplace.json files instead of failing the whole list.
4. Simplify plugin state and caching.
CXC-410 Emit Env Var Status with `/feedback` report
Add more observability on top of #14611
[Unset](https://openai.sentry.io/issues/7340419168/?project=4510195390611458&query=019cfa8d-c1ba-7002-96fa-e35fc340551d&referrer=issue-stream)
[Set](https://openai.sentry.io/issues/7340426331/?project=4510195390611458&query=019cfa91-aba1-7823-ab7e-762edfbc0ed4&referrer=issue-stream)
<img width="1063" height="610" alt="image"
src="https://github.com/user-attachments/assets/937ab026-1c2d-4757-81d5-5f31b853113e"
/>
###### Summary
- Adds auth-env telemetry that records whether key auth-related env
overrides were present on session start and request paths.
- Threads those auth-env fields through `/responses`, websocket, and
`/models` telemetry and feedback metadata.
- Buckets custom provider `env_key` configuration to a safe
`"configured"` value instead of emitting raw config text.
- Keeps the slice observability-only: no raw token values or raw URLs
are emitted.
###### Rationale (from spec findings)
- 401 and auth-path debugging needs a way to distinguish env-driven auth
paths from sessions with no auth env override.
- Startup and model-refresh failures need the same auth-env diagnostics
as normal request failures.
- Feedback and Sentry tags need the same auth-env signal as OTel events
so reports can be triaged consistently.
- Custom provider config is user-controlled text, so the telemetry
contract must stay presence-only / bucketed.
###### Scope
- Adds a small `AuthEnvTelemetry` bundle for env presence collection and
threads it through the main request/session telemetry paths.
- Does not add endpoint/base-url/provider-header/geo routing attribution
or broader telemetry API redesign.
###### Trade-offs
- `provider_env_key_name` is bucketed to `"configured"` instead of
preserving the literal configured env var name.
- `/models` is included because startup/model-refresh auth failures need
the same diagnostics, but broader parity work remains out of scope.
- This slice keeps the existing telemetry APIs and layers auth-env
fields onto them rather than redesigning the metadata model.
###### Client follow-up
- Add the separate endpoint/base-url attribution slice if routing-source
diagnosis is still needed.
- Add provider-header or residency attribution only if auth-env presence
proves insufficient in real reports.
- Revisit whether any additional auth-related env inputs need safe
bucketing after more 401 triage data.
###### Testing
- `cargo test -p codex-core emit_feedback_request_tags -- --nocapture`
- `cargo test -p codex-core
collect_auth_env_telemetry_buckets_provider_env_key_name -- --nocapture`
- `cargo test -p codex-core
models_request_telemetry_emits_auth_env_feedback_tags_on_failure --
--nocapture`
- `cargo test -p codex-otel
otel_export_routing_policy_routes_api_request_auth_observability --
--nocapture`
- `cargo test -p codex-otel
otel_export_routing_policy_routes_websocket_connect_auth_observability
-- --nocapture`
- `cargo test -p codex-otel
otel_export_routing_policy_routes_websocket_request_transport_observability
-- --nocapture`
- `cargo test -p codex-core --no-run --message-format short`
- `cargo test -p codex-otel --no-run --message-format short`
---------
Co-authored-by: Codex <noreply@openai.com>
## Why
Once the repo-local lint exists, `codex-rs` needs to follow the
checked-in convention and CI needs to keep it from drifting. This commit
applies the fallback `/*param*/` style consistently across existing
positional literal call sites without changing those APIs.
The longer-term preference is still to avoid APIs that require comments
by choosing clearer parameter types and call shapes. This PR is
intentionally the mechanical follow-through for the places where the
existing signatures stay in place.
After rebasing onto newer `main`, the rollout also had to cover newly
introduced `tui_app_server` call sites. That made it clear the first cut
of the CI job was too expensive for the common path: it was spending
almost as much time installing `cargo-dylint` and re-testing the lint
crate as a representative test job spends running product tests. The CI
update keeps the full workspace enforcement but trims that extra
overhead from ordinary `codex-rs` PRs.
## What changed
- keep a dedicated `argument_comment_lint` job in `rust-ci`
- mechanically annotate remaining opaque positional literals across
`codex-rs` with exact `/*param*/` comments, including the rebased
`tui_app_server` call sites that now fall under the lint
- keep the checked-in style aligned with the lint policy by using
`/*param*/` and leaving string and char literals uncommented
- cache `cargo-dylint`, `dylint-link`, and the relevant Cargo
registry/git metadata in the lint job
- split changed-path detection so the lint crate's own `cargo test` step
runs only when `tools/argument-comment-lint/*` or `rust-ci.yml` changes
- continue to run the repo wrapper over the `codex-rs` workspace, so
product-code enforcement is unchanged
Most of the code changes in this commit are intentionally mechanical
comment rewrites or insertions driven by the lint itself.
## Verification
- `./tools/argument-comment-lint/run.sh --workspace`
- `cargo test -p codex-tui-app-server -p codex-tui`
- parsed `.github/workflows/rust-ci.yml` locally with PyYAML
---
* -> #14652
* #14651
CXC-392
[With
401](https://openai.sentry.io/issues/7333870443/?project=4510195390611458&query=019ce8f8-560c-7f10-a00a-c59553740674&referrer=issue-stream)
<img width="1909" height="555" alt="401 auth tags in Sentry"
src="https://github.com/user-attachments/assets/412ea950-61c4-4780-9697-15c270971ee3"
/>
- auth_401_*: preserved facts from the latest unauthorized response snapshot
- auth_*: latest auth-related facts from the latest request attempt
- auth_recovery_*: unauthorized recovery state and follow-up result
Without 401
<img width="1917" height="522" alt="happy-path auth tags in Sentry"
src="https://github.com/user-attachments/assets/3381ed28-8022-43b0-b6c0-623a630e679f"
/>
###### Summary
- Add client-visible 401 diagnostics for auth attachment, upstream auth classification, and 401 request id / cf-ray correlation.
- Record unauthorized recovery mode, phase, outcome, and retry/follow-up status without changing auth behavior.
- Surface the highest-signal auth and recovery fields on uploaded client bug reports so they are usable in Sentry.
- Preserve original unauthorized evidence under `auth_401_*` while keeping follow-up result tags separate.
###### Rationale (from spec findings)
- The dominant bucket needed proof of whether the client attached auth before send or upstream still classified the request as missing auth.
- Client uploads needed to show whether unauthorized recovery ran and what the client tried next.
- Request id and cf-ray needed to be preserved on the unauthorized response so server-side correlation is immediate.
- The bug-report path needed the same auth evidence as the request telemetry path, otherwise the observability would not be operationally useful.
###### Scope
- Add auth 401 and unauthorized-recovery observability in `codex-rs/core`, `codex-rs/codex-api`, and `codex-rs/otel`, including feedback-tag surfacing.
- Keep auth semantics, refresh behavior, retry behavior, endpoint classification, and geo-denial follow-up work out of this PR.
###### Trade-offs
- This exports only safe auth evidence: header presence/name, upstream auth classification, request ids, and recovery state. It does not export token values or raw upstream bodies.
- This keeps websocket connection reuse as a transport clue because it can help distinguish stale reused sessions from fresh reconnects.
- Misroute/base-url classification and geo-denial are intentionally deferred to a separate follow-up PR so this review stays focused on the dominant auth 401 bucket.
###### Client follow-up
- PR 2 will add misroute/provider and geo-denial observability plus the matching feedback-tag surfacing.
- A separate host/app-server PR should log auth-decision inputs so pre-send host auth state can be correlated with client request evidence.
- `device_id` remains intentionally separate until there is a safe existing source on the feedback upload path.
###### Testing
- `cargo test -p codex-core refresh_available_models_sorts_by_priority`
- `cargo test -p codex-core emit_feedback_request_tags_`
- `cargo test -p codex-core emit_feedback_auth_recovery_tags_`
- `cargo test -p codex-core auth_request_telemetry_context_tracks_attached_auth_and_retry_phase`
- `cargo test -p codex-core extract_response_debug_context_decodes_identity_headers`
- `cargo test -p codex-core identity_auth_details`
- `cargo test -p codex-core telemetry_error_messages_preserve_non_http_details`
- `cargo test -p codex-core --all-features --no-run`
- `cargo test -p codex-otel otel_export_routing_policy_routes_api_request_auth_observability`
- `cargo test -p codex-otel otel_export_routing_policy_routes_websocket_connect_auth_observability`
- `cargo test -p codex-otel otel_export_routing_policy_routes_websocket_request_transport_observability`
## Why
to support a new bring your own search tool in Responses
API(https://developers.openai.com/api/docs/guides/tools-tool-search#client-executed-tool-search)
we migrating our bm25 search tool to use official way to execute search
on client and communicate additional tools to the model.
## What
- replace the legacy `search_tool_bm25` flow with client-executed
`tool_search`
- add protocol, SSE, history, and normalization support for
`tool_search_call` and `tool_search_output`
- return namespaced Codex Apps search results and wire namespaced
follow-up tool calls back into MCP dispatch
This cleans up a bunch of metric plumbing that had started to drift.
The main change is making `codex-otel` the canonical home for shared
metric definitions and metric tag helpers. I moved the `turn/thread`
metric names that were still duplicated into the OTEL metric registry,
added a shared `metrics::tags` module for common tag keys and session
tag construction, and updated `SessionTelemetry` to build its metadata
tags through that shared path.
On the codex-core side, TTFT/TTFM now use the shared metric-name
constants instead of local string definitions. I also switched the
obvious remaining turn/thread metric callsites over to the shared
constants, and added a small helper so TTFT/TTFM can attach an optional
sanitized client.name tag from TurnContext.
This should make follow-on telemetry work less ad hoc:
- one canonical place for metric names
- one canonical place for common metric tag keys/builders
- less duplication between `codex-core` and `codex-otel`
## Summary
This is a purely mechanical refactor of `OtelManager` ->
`SessionTelemetry` to better convey what the struct is doing. No
behavior change.
## Why
`OtelManager` ended up sounding much broader than what this type
actually does. It doesn't manage OTEL globally; it's the session-scoped
telemetry surface for emitting log/trace events and recording metrics
with consistent session metadata (`app_version`, `model`, `slug`,
`originator`, etc.).
`SessionTelemetry` is a more accurate name, and updating the call sites
makes that boundary a lot easier to follow.
## Validation
- `just fmt`
- `cargo test -p codex-otel`
- `cargo test -p codex-core`
## Summary
This is a structural cleanup of `codex-otel` to make the ownership
boundaries a lot clearer.
For example, previously it was quite confusing that `OtelManager` which
emits log + trace event telemetry lived under
`codex-rs/otel/src/traces/`. Also, there were two places that defined
methods on OtelManager via `impl OtelManager` (`lib.rs` and
`otel_manager.rs`).
What changed:
- move the `OtelProvider` implementation into `src/provider.rs`
- move `OtelManager` and session-scoped event emission into
`src/events/otel_manager.rs`
- collapse the shared log/trace event helpers into
`src/events/shared.rs`
- pull target classification into `src/targets.rs`
- move `traceparent_context_from_env()` into `src/trace_context.rs`
- keep `src/otel_provider.rs` as a compatibility shim for existing
imports
- update the `codex-otel` README to reflect the new layout
## Why
`lib.rs` and `otel_provider.rs` were doing too many different jobs at
once: provider setup, export routing, trace-context helpers, and session
event emission all lived together.
This refactor separates those concerns without trying to change the
behavior of the crate. The goal is to make future OTEL work easier to
reason about and easier to review.
## Notes
- no intended behavior change
- `OtelManager` remains the session-scoped event emitter in this PR
- the `otel_provider` shim keeps downstream churn low while the
internals move around
## Validation
- `just fmt`
- `cargo test -p codex-otel`
- `just fix -p codex-otel`
jif