Commit Graph
27 Commits
Author SHA1 Message Date
cassirer-openai e977cb394d Preserve tool output versions and history order in rollout traces (#57311)
## Why

Tool calls can emit multiple outputs, and compaction can trim output content while retaining the same call and item IDs. The trace reducer rejected these versions as conflicting content. Late tool and code-cell events could also link outputs in map-key order instead of conversation order.

## What changed

- Preserve distinct tool output versions while retaining consistency checks for tool invocations.
- Link tool and code-cell outputs in thread history order, and resolve pending code cells after the entire event has been reduced.
- Prevent snapshot reconciliation from reusing the same conversation item twice when identical items move within history.

## Testing

Extend reducer regression tests for trimmed outputs, changed images and metadata, reordered identical messages, and late tool and code-cell events spanning the item-ID ordering boundary.

GitOrigin-RevId: eb4242d6a2d32d61c3caab9d3cc3cf6b374ce799
2026-10-26 23:55:28 +00:00
cassirer-openai f2df3e3b56 Preserve full MCP results in rollout traces (#57280)
## Why

MCP completion events can truncate large results and omit content removed for model consumption. Rollout traces need the original response, including content and metadata, as well as results for calls denied before reaching the backend.

## What changed

- Record MCP runtime results before callbacks, content filtering, and UI truncation, and record skipped calls as failures.
- Stop deriving MCP runtime completion records from protocol events to avoid duplicate records.
- Treat MCP runtime payloads as opaque results when reducing agent interactions.
- Buffer JSON trace writes and explicitly flush them.

## Testing

Extend app-server tests to verify that traces retain oversized MCP text, audio, structured content, and metadata while UI results remain truncated, and that denied calls produce exactly one failed runtime result with the denial reason.

GitOrigin-RevId: 57880770545eb5869420d7dc6fa4a15e481cae9d
2026-10-26 21:56:23 +00:00
cassirer-openai 36021e1ad5 Truncate rollout JSON summaries at UTF-8 character boundaries (#57199)
## Why

Truncating a JSON summary at the 240-byte limit can panic when the cutoff falls inside a multibyte UTF-8 character.

## What changed

Truncate at the nearest preceding character boundary before appending `...`.

## Testing

Extend the JSON tool-call reuse test with multibyte arguments and assert the truncated summary and original payload reference.

GitOrigin-RevId: e4b2af17eb6e3abaeb042772944305c6e1744bdf
2026-10-26 19:11:53 +00:00
cassirer-openai b6a4378862 Classify rollout tool calls by their runtime handler (#57194)
## Why

Tool interception and MCP aliases can route a requested callable to a different handler. Rollout traces need to reflect the handler that actually executes, including when an `exec_command` invocation is intercepted as `apply_patch`.

## What changed

Carry an explicit `ToolCallKind` in runtime-start events and use it to update the tool call's execution classification. This replaces MCP-specific payload inspection while preserving the original callable summary and invocation for intercepted patch and aliased MCP calls.

## Testing

Add a regression test verifying that an intercepted `exec_command` is recorded as `ApplyPatch`, retains its original summary and invocation, completes successfully, and creates no terminal operations or sessions.

GitOrigin-RevId: 0e72f3ef55cda2796e779efd16241e4a0dbf0c5b
2026-10-26 19:04:02 +00:00
cassirer-openai cfc11bfab3 Preserve failed write_stdin calls in rollout trace replay (#57193)
## Why

Invalid model arguments to `write_stdin` represent a tool failure, but parsing them could fail rollout trace replay.

## What changed

Skip terminal operation creation when dispatched `write_stdin` arguments cannot be parsed or lack a usable `session_id`. Preserve the generic tool call and its error, including the original arguments, so replay can continue.

## Testing

Add a regression test covering malformed JSON and missing or null `session_id` values. Verify that replay retains the failed call's arguments and error without creating terminal operations or sessions.

GitOrigin-RevId: 967e975a88072c169eb3e391d5c62e807d608fbf
2026-10-26 19:03:36 +00:00
cassirer-openai e5ea2ed50b Match rollout trace image content across request snapshots (#56342)
## Why

Repeated image outputs in different request payloads could fail replay with a call ID reuse error even when their content was unchanged.

## What changed

- Hash canonical JSON for non-text content parts and compare their `content_sha256` values independently of payload location.
- Preserve empty content arrays as empty bodies so they also match across snapshots.

## Testing

Add a regression test verifying that repeated image and empty tool outputs reuse conversation items, while changed image content with the same call and item IDs still fails replay.

GitOrigin-RevId: 07699312675764079f679fd2c12ba6679332d709
2026-10-23 03:39:48 +00:00
cassirer-openai 5b792948f9 Track accepted conversation history in rollout traces (#56312)
## Why

Model output alone does not establish that an item entered conversation history. Treating every response item as accepted can include discarded attempts, while relying on a later request misses accepted final output when a turn ends or is interrupted.

## What changed

- Record `HistoryItemAccepted` events and retain Responses item IDs, generating missing IDs consistently across item events.
- Build thread conversation history from request inputs, compaction checkpoints, and explicit acceptance. Keep unaccepted output accessible through its inference call.
- Preserve inference provenance across retries and late response records without overwriting newer request or compaction snapshots.
- Mark turns with errors as failed. Bump trace schemas to version 2 and reject incompatible bundles and event schemas during replay.

## Testing

Add reducer coverage for discarded attempts with identical text, late responses, compaction, inherited context, and legacy bundle rejection. Add CLI integration tests that run `debug trace-reduce` after success, stream failure, and interruption followed by shutdown, including responses without provider item IDs.

GitOrigin-RevId: 8e387a91d9b3f704df82847d630e6f0b08f998b3
2026-10-23 00:45:20 +00:00
cassirer-openai deede7201d Keep rollout trace terminal sessions distinct when process IDs are reused (#56011)
## Why

Runtime process handles are recyclable and thread-local. Using them as terminal session identities can merge separate commands or associate `write_stdin` operations with the wrong exec.

## What changed

- Give each exec a distinct terminal session and track the latest session for each thread-local process handle.
- Record the exec selected by `write_stdin` at runtime and use it to correct session membership during replay, including when dispatch is delayed until a handle is reused.
- Fall back to the latest session for the thread and handle when replaying older traces without runtime target events.

## Testing

Extend reducer coverage for process ID reuse across threads and successive execs, stdin session membership, and delayed runtime target resolution.

GitOrigin-RevId: 330c6fd26a4eebdef03ae600a030dc76920e0dd9
2026-10-21 23:13:31 +00:00
cassirer-openai d571eaf1f6 Preserve code cell response metadata in rollout traces (#55725)
## Why

Reducing code cell lifecycle events discarded their response payload references and lost the initial response status when the runtime status changed.

## What changed

Store the initial response status and separate raw payload IDs for the initial response and final result on `CodeCell`. Carry payload references through queued lifecycle events so they survive replay when the source item arrives later.

## Testing

Extend lifecycle tests to verify distinct initial and final payload IDs and the preserved initial status for yielded cells that complete and failed cells whose lifecycle events precede their source item.

GitOrigin-RevId: 8f49a3174ea487a5d91ebc2a6c97133aacaabfd2
2026-10-21 02:02:49 +00:00
cassirer-openai 3f31c0a040 Resolve MCP backend identity from runtime events in rollout traces (#55710)
## Why

MCP callable namespaces and tool names can differ from their backend identities or match built-in tool names. Classifying calls by name can misidentify MCP calls and incorrectly parse namespaced `write_stdin` calls as terminal operations.

## What changed

Resolve `ToolCallKind::Mcp` from the runtime invocation's server and tool as soon as the runtime starts, before MCP correlation assignment. Preserve the caller-facing summary and raw invocation, and skip built-in runtime handling for MCP calls.

Only create dispatch-based `write_stdin` terminal operations for calls without a namespace.

## Testing

Add replay regression coverage for prefixed and unprefixed MCP aliases, names that match built-in tools, and `write_stdin` arguments with and without a session ID. Verify backend classification, preserved callable identity and dispatch lifecycle, and the absence of terminal operations and sessions.

GitOrigin-RevId: 71f31335c1eb36b6b7e1a34383d10fc500bd2c4f
2026-10-21 00:53:48 +00:00
rb-oai 822e58cc3d Handle partial answers in realtime routing and thread search (#51260)
## Why

Messages with the `partial_answer` phase contain answer text without declaring a terminal answer. They should remain searchable and speakable without being treated as final messages.

## What changed

- Preserve explicit `partial_answer` phases in streamed and completed realtime handoffs, bypassing legacy channel-prefix parsing and routing them to the `speakable` channel.
- Treat partial answers as nonterminal in V1 handoffs and omit the final-message prefix.
- Include partial answers in thread occurrence searches and normalize them to the final conversation channel in rollout traces.

## Testing

Extend coverage for realtime phase routing, streamed and completed handoffs, partial-answer search results, and rollout normalization. Verify that turn summaries still exclude nonterminal messages.

GitOrigin-RevId: aa183da783f6e7e0b78ddd51073cba533297e121
2026-10-06 03:58:55 +00:00
Owen Lin c5d242fa79 Consolidate command execution output into aggregated_output (#50402)
## What changed

- Remove `stdout`, `stderr`, and `formatted_output` from `CommandExecutionItem` and `ExecCommandEndEvent`, retaining `aggregated_output` as the output field.
- Update command event producers, legacy conversion, rollout traces, and the precomputed schema. Use aggregated output for terminal trace results.
- Clear `interaction_input` in emitted and migrated command completion items.

## Testing

Update command execution and trace tests to use aggregated output, and assert that legacy rollout migration preserves `aggregatedOutput`.

GitOrigin-RevId: ed59827d4a5ec9a767084ee4feac72c2042440f2
2026-10-02 18:48:49 +00:00
jif b705b6b076 Report completed sub-agent activity on parent turns (#40437)
## Why

A Multi-Agent V2 child can finish after the parent turn that spawned it, so its
successful completion needs to remain associated with that parent turn in
notifications and history.

## What changed

- Add `completed` to `SubAgentActivityKind` and emit the corresponding started
  and completed item events on the spawning parent turn when a child succeeds.
- Persist completed activity as a canonical turn item in both legacy and
  paginated history, and expose it through app-server history and schemas.
- Render completed activity as terminal in the TUI without counting it as a
  separate tool call or tool-runtime event.

## Testing

- Cover successful completion in legacy and paginated history, late updates to
  completed parent turns, TUI rendering, analytics, and rollout tracing.
- Verify that terminal child errors do not emit completed activity.

GitOrigin-RevId: 6c71eb8b10e7327611c6af84f70e294e27d94d02
2026-08-24 17:07:21 +00:00
mandal-OAI 745603a5a1 Ignore passthrough metadata when reconciling rollout items (#36221)
## What changed

- Remove top-level `internal_chat_message_metadata_passthrough` from model items before rollout-trace normalization so replayed tool calls and outputs reuse their existing conversation items.
- Keep nested metadata model-visible and continue rejecting call ID reuse when that content changes.

## Testing

- Add reducer coverage for replayed tool search calls and tool outputs with top-level passthrough metadata, plus conflicting nested metadata.

GitOrigin-RevId: db81b9957770a2226f77db1d9df87e75572fc6de
2026-07-30 21:09:15 +00:00
Eric Traut 2edad72de3 Track prompt cache write token usage (#33454)
## What changed

- Parse `cache_write_tokens` from response input token details and carry it through token usage aggregation.
- Expose `cache_write_input_tokens` in protocol, app-server, exec, and TypeScript SDK usage events, defaulting it to zero for compatibility with older payloads.
- Report cache-write usage in analytics, metrics, traces, and rollout data alongside existing cache-read usage.

## Testing

- Add coverage for response parsing, usage event output, aggregation, analytics, metrics, and telemetry fields.

GitOrigin-RevId: 634e4fbe3086a8923c0ee13b622573e7d9d221bd
2026-07-16 00:22:53 +00:00
rka-oai 33cc928d33 [codex] Use input items for Responses Lite tools (#27946)
When using Responses Lite, we should all use `additional_tools` and a
developer item instead of the top level tools array & instructions
field. This keeps things 1-to-1.

Forced namespacing for _all_ tools will land in a following PR after
some coordination & fixes in Responses API (around collisions & return
items).

The goal is to eventually expand the scope of this to _all_ requests
from codex, but that will require larger coordination across providers &
slower rollout.
2026-06-22 23:56:16 -07:00
jif 5b22a8e5b1 feat: render typed envelopes for multi-agent v2 messages (#28368)
## Why

Multi-agent v2 messages need a consistent, model-visible envelope that
identifies what kind of interaction occurred, who sent it, and which
agent it targets. Previously, encrypted deliveries exposed only
`encrypted_content`, while child completion used the legacy
`<subagent_notification>` shape. That meant the client could not
consistently present `NEW_TASK`, `MESSAGE`, and `FINAL_ANSWER` using the
same format.

This change adds the routing envelope as plaintext while keeping task
and message payloads encrypted. No new Responses API field is required:
an encrypted delivery is represented as an `input_text` header
immediately followed by its existing `encrypted_content` item.

Every envelope now follows this shape:

```text
Message Type: <NEW_TASK | MESSAGE | FINAL_ANSWER>
Task name: <recipient agent path>
Sender: <author agent path>
Payload:
<message payload>
```

## Message types

### `NEW_TASK`

`NEW_TASK` is used when the recipient should begin a new turn, including
an initial `spawn_agent` task and a later `followup_task`.

For a root agent spawning `/root/worker`, the request contains a
plaintext envelope followed by the encrypted task:

```json
{
  "type": "agent_message",
  "author": "/root",
  "recipient": "/root/worker",
  "content": [
    {
      "type": "input_text",
      "text": "Message Type: NEW_TASK\nTask name: /root/worker\nSender: /root\nPayload:\n"
    },
    {
      "type": "encrypted_content",
      "encrypted_content": "<encrypted task payload>"
    }
  ]
}
```

Conceptually, the model receives:

```text
Message Type: NEW_TASK
Task name: /root/worker
Sender: /root
Payload:
Review the authentication changes and report any regressions.
```

### `MESSAGE`

`MESSAGE` is used for a queued `send_message` delivery. It communicates
with an existing agent without starting a new turn.

For `/root/worker` reporting progress to the root agent, the request
contains:

```json
{
  "type": "agent_message",
  "author": "/root/worker",
  "recipient": "/root",
  "content": [
    {
      "type": "input_text",
      "text": "Message Type: MESSAGE\nTask name: /root\nSender: /root/worker\nPayload:\n"
    },
    {
      "type": "encrypted_content",
      "encrypted_content": "<encrypted message payload>"
    }
  ]
}
```

Conceptually, the model receives:

```text
Message Type: MESSAGE
Task name: /root
Sender: /root/worker
Payload:
The protocol tests pass; I am checking the resume path now.
```

### `FINAL_ANSWER`

`FINAL_ANSWER` is emitted when a child agent reaches a terminal state
and reports its result to its parent. Completion payloads are already
available locally, so the complete envelope is represented as plaintext
rather than as a plaintext header plus encrypted content.

For `/root/worker` completing work for the root agent, the request
contains:

```json
{
  "type": "agent_message",
  "author": "/root/worker",
  "recipient": "/root",
  "content": [
    {
      "type": "input_text",
      "text": "Message Type: FINAL_ANSWER\nTask name: /root\nSender: /root/worker\nPayload:\nNo regressions found."
    }
  ]
}
```

The model-visible form is:

```text
Message Type: FINAL_ANSWER
Task name: /root
Sender: /root/worker
Payload:
No regressions found.
```

Errored, shut down, and missing agents also use `FINAL_ANSWER`, with a
terminal-status description in the payload.

## What changed

- Render `NEW_TASK` or `MESSAGE` in
`InterAgentCommunication::to_model_input_item`, based on whether the
encrypted delivery starts a turn.
- Replace the multi-agent v2 `<subagent_notification>` completion
payload with a model-visible `FINAL_ANSWER` envelope.
- Document `Task name`, `Sender`, and `Payload` consistently in the
multi-agent developer instructions.
- Prevent local-only history projections from treating an encrypted
message's plaintext header as the complete assistant message.
- Preserve rollout-trace interaction edges when an agent message
contains both plaintext and encrypted content.

Legacy multi-agent behavior remains unchanged.

## Verification

- `just test -p codex-protocol`
- `just test -p codex-rollout-trace`
- `just test -p codex-web-search-extension`
- `just test -p codex-core
encrypted_multi_agent_v2_spawn_sends_agent_message_to_child`
- `just test -p codex-core
plaintext_multi_agent_v2_completion_sends_agent_message`
- `just test -p codex-core
multi_agent_v2_followup_task_completion_notifies_parent_on_every_turn`
- `just test -p codex-core
multi_agent_v2_completion_queues_message_for_direct_parent`
2026-06-16 11:46:59 +02:00
Owen Lin 040dafa32d feat(core): add metadata field to ResponseItem (#28355)
## Description

This PR adds an optional `metadata` field to `ResponseItem` for
Responses API calls. Only mechanical plumbing, no actual values
populated and sent yet. Turns out just adding a new field to
`ResponseItem` has quite a large blast radius already.

This change is backwards compatible because `metadata` is optional and
omitted when absent, so existing response items and rollout history
without it still deserialize and requests that do not set it keep the
same wire shape. For provider compatibility, we strip out `metadata`
before non-OpenAI Responses requests so Azure and AWS Bedrock never see
this field.

My followup PR here will actually make use of it to start storing and
passing along `turn_id`: https://github.com/openai/codex/pull/28360

## What changed

- Added `ResponseItemMetadata` with optional `turn_id`, plus optional
`metadata` on Responses API item variants and inter-agent communication.
- Preserved item metadata through response-item rewrites such as
truncation, missing tool-output synthesis, compaction history
rebuilding, visible-history conversion, rollout/resume, and generated
app-server schemas/types.
- Strip item metadata from non-OpenAI Responses requests while
preserving it for OpenAI-shaped requests.
- Updated the mechanical fixture/test construction churn required by the
new optional field.
2026-06-15 15:05:28 -07:00
jif 8f2d6416ce Support plaintext agent messages (#27830)
## Why

Multi-agent v2 `send_message` deliveries already reach the receiving
model as typed `agent_message` items with encrypted content.
Child-completion notifications are generated by Codex itself, so their
content is plaintext and previously fell back to a serialized JSON
envelope inside an assistant message.

With plaintext `input_text` supported for `agent_message`, both delivery
paths can use the same model-visible type while preserving explicit
author and recipient metadata.

## What changed

- add plaintext `input_text` support to `AgentMessageInputContent` and
regenerate the affected app-server schemas
- preserve `InterAgentCommunication` as structured mailbox input instead
of converting it to assistant text
- record delivered communications as typed `agent_message` history items
- persist a dedicated rollout item so local delivery metadata such as
`trigger_turn` remains available without leaking into the Responses
request
- reconstruct typed agent messages on resume and preserve fork-turn
truncation behavior
- remove request-time assistant-content parsing
- preserve plaintext and encrypted inter-agent deliveries in stage-one
memory inputs
- normalize and link plaintext and encrypted agent messages in rollout
traces without treating inbound messages as child results
- cover the real MultiAgent V2 child-completion path end to end with
deterministic mailbox synchronization

## Verification

- `just test -p codex-core
plaintext_multi_agent_v2_completion_sends_agent_message`
- `just test -p codex-core input_queue_drains_mailbox_in_delivery_order
record_initial_history_reconstructs_typed_inter_agent_message
fork_turn_positions_use_inter_agent_delivery_metadata`
- `just test -p codex-memories-write
serializes_inter_agent_communications_for_memory`
- `just test -p codex-rollout-trace
agent_messages_preserve_routing_and_content
sub_agent_started_activity_creates_spawn_edge`
- `just test -p codex-rollout-trace
agent_result_edge_falls_back_to_child_thread_without_result_message`
- `just test -p codex-protocol -p codex-rollout -p
codex-app-server-protocol`
2026-06-12 13:50:04 -07:00
jif fae2709320 multi-agent: add path-based v2 activity tracking (#27007)
## Why

Multi-agent v2 identifies agents by canonical paths, but its tool
handlers still emitted the larger legacy collaboration begin/end events
built around nickname and role metadata. App-server, rollout-trace,
analytics, and TUI consumers therefore lacked one compact path-based
completion signal that behaved consistently across live events and
replay.

The TUI also needs a bounded `/agent` status surface for v2 agents. It
should use recent local activity for previews, refresh liveness without
loading full histories, and keep the legacy picker available when no
path-backed v2 agent is known.

## What changed

- Replace the v2 `spawn_agent`, `send_message`, `followup_task`, and
`interrupt_agent` legacy lifecycle emissions with a success-only
`SubAgentActivity` event. The event records the tool call ID, occurrence
time, affected thread, canonical agent path, and `started`,
`interacted`, or `interrupted` kind.
- Expose the activity as a completion-only app-server v2
`subAgentActivity` thread item in live notifications and reconstructed
history, regenerate the protocol schemas, and count it in sub-agent tool
analytics.
- Track canonical paths from live activity and loaded-thread metadata in
the TUI, and render the activity in live and replayed transcripts.
- Make `/agent` list running path-backed agents with summaries from
bounded local event buffers. Each summary is capped at 240 graphemes,
the scan is capped at six recent items, only the last three wrapped
lines are shown, and command output is omitted. Liveness falls back to
metadata-only `thread/read` when local turn state is unavailable.
- Persist the activity as a terminal rollout-trace runtime payload and
reduce it to the corresponding spawn, send, follow-up, or close
interaction edge. `interrupt_agent` is classified as a close-edge
operation.
- Preserve the legacy picker when no path-backed v2 agent is known.

## Compatibility

App-server v2 clients that consumed `collabAgentToolCall` begin/end
pairs for these tools must handle the new completion-only
`subAgentActivity` item. Legacy v1 collaboration behavior is unchanged.

## Screenshot

<img width="684" height="288" alt="Screenshot 2026-06-08 at 15 40 47"
src="https://github.com/user-attachments/assets/194b3cd0-619d-45fb-b587-cf3e2b1b8a1d"
/>

## Testing

- `just test -p codex-app-server-protocol`
- `just test -p codex-rollout-trace`
- Added focused coverage for activity analytics, terminal trace
serialization, spawn-edge reduction, `interrupt_agent` classification,
TUI status rendering without aggregated command output, and clearing
stale running state after a completed turn.
2026-06-09 12:14:48 +02:00
jif 5f4d06ef18 Encrypt multi-agent v2 message payloads (#26210)
## Why

Multi-agent v2 currently routes agent instructions through normal tool
arguments and inter-agent context. That means the parent model can emit
plaintext task text, Codex can persist it in history/rollouts, and the
recipient can receive it as ordinary assistant-message JSON.

This changes the v2 path so agent instructions stay encrypted between
model calls: Responses encrypts the `message` argument returned by the
model, Codex forwards only that ciphertext, and Responses decrypts it
internally for the recipient model.

## What changed

- Mark the v2 `message` parameter as encrypted for `spawn_agent`,
`send_message`, and `followup_task`.
- Treat multi-agent v2 tool `message` values as ciphertext
unconditionally.
- Store v2 inter-agent task text in
`InterAgentCommunication.encrypted_content` with empty plaintext
`content`.
- Convert encrypted inter-agent communications into the Responses
`agent_message` input item before sending the child request.
- Preserve `agent_message` items across history, rollout, compaction,
telemetry, and app-server schema paths.
- Leave multi-agent v1 unchanged.

## Message shape

The model still calls the v2 tools with a `message` argument, but that
value is now ciphertext:

```json
{
  "name": "spawn_agent",
  "arguments": {
    "task_name": "worker",
    "message": "<ciphertext>"
  }
}
```

Codex stores the task as encrypted inter-agent communication:

```json
{
  "author": "/root",
  "recipient": "/root/worker",
  "content": "",
  "encrypted_content": "<ciphertext>",
  "trigger_turn": true
}
```

When Codex builds the recipient request, it forwards the ciphertext
using the new Responses input item:

```json
{
  "type": "agent_message",
  "author": "/root",
  "recipient": "/root/worker",
  "content": [
    {
      "type": "encrypted_content",
      "encrypted_content": "<ciphertext>"
    }
  ]
}
```

Responses decrypts that item internally for the recipient model.

## Context impact

- Parent context no longer carries plaintext v2 agent task instructions
from these tool arguments.
- Codex rollout/history stores ciphertext for v2 agent instructions.
- Recipient requests receive an `agent_message` item instead of
assistant commentary JSON for encrypted task delivery.
- Plaintext completion/status notifications are still plaintext because
they are Codex-generated status messages, not encrypted model tool
arguments.

## Validation

- `just test -p codex-tools`
- `just test -p codex-protocol`
- `just test -p codex-rollout`
- `just test -p codex-rollout-trace`
- `just test -p codex-otel`
- `just write-app-server-schema`
2026-06-05 10:25:57 +02:00
cassirer-openai 702e6a3c64 [rollout-trace] Add a trace ID to MCP calls. (#22326)
This allows us to connect individual tool calls to the logs of the
invocations.
2026-05-13 16:03:33 +00:00
jif-oaiandCodex d927f61208 feat: add remote compaction v2 Responses client path (#20773)
## Why

This adds the `remote_compaction_v2` client path so remote compaction
can run through the normal Responses stream and install a
`context_compaction` item that trigger a compaction.

The goal is to migrate some of the compaction logic on the client side

We keeps the v2 transport behind a feature flag while letting follow-up
requests reuse the compacted context instead of falling back to the
legacy compaction item shape.

## What changed

- add `ResponseItem::ContextCompaction` and refresh the generated
app-server / schema / TypeScript fixtures that expose response items on
the wire
- add `core/src/compact_remote_v2.rs` to send compaction through the
standard streamed Responses client, require exactly one
`context_compaction` output item, and install that item into compacted
history
- route manual compact and auto-compaction through the v2 path when
`remote_compaction_v2` is enabled, while keeping the existing remote
compaction path as the fallback
- preserve the new item type across history retention, follow-up request
construction, telemetry, rollout persistence, and rollout-trace
normalization
- add targeted coverage for the feature flag, `context_compaction`
serialization, rollout-trace normalization, and remote-compaction
follow-up behavior

## Verification

- added protocol tests for `context_compaction`
serialization/deserialization in `protocol/src/models.rs`
- added rollout-trace coverage for `context_compaction` normalization in
`rollout-trace/src/reducer/conversation_tests.rs`
- added remote compaction integration coverage for v2 follow-up reuse
and mixed compaction output streams in
`core/tests/suite/compact_remote.rs`

---------

Co-authored-by: Codex <noreply@openai.com>
2026-05-04 14:15:01 +02:00
cassirer-openai df966996a7 [rollout-tracer] Match analysis messages on encrypted id. (#20123)
In some setups the summary or raw content can be dropped between
requests. This triggers a check in the reducer which expects that the
messages should remain identical between requests.

This PR relaxes the checks to only focus on the encrypted ID instead. It
also changes the reducer to keep the most rich version of the message
observed during the rollout (this ensures that we don't accidentally
lose the CoT nor summary when available).
2026-04-29 17:22:24 +00:00
cassirer-openai 89698ad1c3 [rollout-trace] Include x-request-id in rollout trace. (#20066)
## Why

Rollout traces need an identifier that can be used to correlate a Codex
inference with upstream Responses API, proxy, and engine logs. The
reduced trace model already exposed `upstream_request_id`, but it was
being populated from the Responses API `response.id`. That value is
useful for `previous_response_id` chaining, but it is not the transport
request id that upstream systems key on.

This PR separates those concepts so trace consumers can reliably answer
both questions:

- which Responses API response did this inference produce?
- which upstream request handled it?

## Structure

The change keeps the upstream request id at the same lifecycle level as
the provider stream:

- `codex-api` captures the `x-request-id` HTTP response header when the
SSE stream is created and exposes it on `ResponseStream`. Fixture and
websocket streams set the field to `None` because they do not have that
HTTP response header.
- `codex-core` carries that stream-level id into `InferenceTraceAttempt`
when recording terminal stream outcomes. Completed, failed, cancelled,
dropped-stream, and pre-response error paths all record the id when it
is available.
- `rollout-trace` now records both identifiers in raw terminal inference
events and response payloads: `response_id` for the Responses API
`response.id`, and `upstream_request_id` for `x-request-id`.
- The reducer stores both fields on `InferenceCall`. It also uses
`response_id` for `previous_response_id` conversation linking, which
removes the old accidental dependency on the misnamed
`upstream_request_id` field.
- Terminal inference reduction now consumes the full terminal payload
(`InferenceCompleted`, `InferenceFailed`, or `InferenceCancelled`) in
one place. That keeps status, partial payloads, response ids, and
upstream request ids consistent across success, failure, cancellation,
and late stream-mapper events.

## Why This Shape

`x-request-id` is a property of the HTTP/provider response envelope, not
an SSE event. Capturing it once in `codex-api` and plumbing it through
terminal trace recording avoids trying to infer the value from stream
contents, and it preserves the id even when the stream fails or is
cancelled after only partial output.

Keeping `response_id` separate from `upstream_request_id` also makes the
reduced trace model less surprising: `response_id` remains the
conversation-continuation id, while `upstream_request_id` is the
operational correlation id for upstream debugging.

## Validation

The PR updates trace and reducer coverage for:

- reading `x-request-id` from SSE response headers;
- storing the true upstream request id on completed inference calls;
- preserving upstream request ids for cancelled and late-cancelled
inference streams;
- keeping `previous_response_id` reconstruction tied to `response_id`
rather than transport request ids.
2026-04-28 21:11:17 +00:00
cassirer-openai 30c5c768de [codex] Trace cancelled inference streams (#19839)
Records cancelled inference streams when Codex stops consuming a
provider response before `response.completed`, preserving complete
output items observed before cancellation.

Also closes still-running inference calls when the owning turn ends, so
reduced rollout traces do not leave stale `Running` inference nodes.

Covered by focused reducer coverage and a core stream-drop test for
partial output preservation.
2026-04-27 21:58:29 +00:00
cassirer-openai 27d9673273 [rollout_trace] Add rollout trace crate (#18876)
## Summary

Adds the standalone `codex-rollout-trace` crate, which defines the raw
trace event format, replay/reduction model, writer, and reducer logic
for reconstructing model-visible conversation/runtime state from
recorded rollout data.

The crate-level design is documented in
[`codex-rs/rollout-trace/README.md`](https://github.com/openai/codex/blob/codex/rollout-trace-crate/codex-rs/rollout-trace/README.md).

## Stack

This is PR 1/5 in the rollout trace stack.

- [#18876](https://github.com/openai/codex/pull/18876): Add rollout
trace crate
- [#18877](https://github.com/openai/codex/pull/18877): Record core
session rollout traces
- [#18878](https://github.com/openai/codex/pull/18878): Trace tool and
code-mode boundaries
- [#18879](https://github.com/openai/codex/pull/18879): Trace sessions
and multi-agent edges
- [#18880](https://github.com/openai/codex/pull/18880): Add debug trace
reduction command

## Review Notes

This PR intentionally does not wire tracing into live Codex execution.
It establishes the data model and reducer contract first, with
crate-local tests covering conversation reconstruction, compaction
boundaries, tool/session edges, and code-cell lifecycle reduction. Later
PRs emit into this model.

The README is the best entry point for reviewing the intended trace
format and reduction semantics before diving into the reducer modules.
2026-04-21 21:54:05 +00:00