## What changed
- Add `accepted_at_unix_ms` and `accepted_seq` to conversation items, recording the first local `HistoryItemAccepted` event in trace order. Inherited context alone does not count as local acceptance.
- Add `started_code_cell_id` to model-requested `exec` tool calls when a matching code cell exists in the same thread and, when known, turn. Resolve links after replay so deferred cell starts can be matched.
- Default the new optional fields to absent when deserializing older traces and omit them from serialized output when unset.
## Testing
Extend reducer tests to cover duplicate acceptance events with out-of-order clock readings, inherited items without local acceptance, and `exec` calls with and without matching code cells.
GitOrigin-RevId: dfb594630f77d7c899398aa181c88a681587b1df
## Why
A completion marker missing its final newline can make an incomplete trace appear complete. Opaque JSON payload validation can also miss invalid UTF-8 inside strings.
## What changed
Require a newline-terminated `RecordingCompleted` record before setting `recording_complete`. Read raw payloads as UTF-8 text before validating their JSON structure.
## Testing
Extend the lifecycle replay test to cover a completion marker missing its final newline and an opaque payload containing invalid UTF-8.
GitOrigin-RevId: a0007ff449cdefeb44065ed9165ef60bf720dad9
## Why
Enabled rollout tracing could silently fail to initialize, and a terminal task status did not establish whether all recording had finished. Trace consumers need to distinguish a complete recording from a partial bundle, including when a task is interrupted.
## What changed
- Fail thread startup or resume if an enabled recorder cannot initialize, while cleaning up session resources.
- Stop recording after the first write or serialization failure and preserve that failure across shared producers.
- Append a closing record when the last writer handle is released without recording errors. Expose `recording_complete` in `state.json` independently of task success, and mark submission-channel closure as an aborted rollout.
- Reject replay logs with missing or reordered events, mismatched rollout IDs, events after completion, or missing or invalid JSON payloads. Valid prefixes remain replayable with `recording_complete` set to `false`.
## Testing
Add coverage for startup failure and recovery, shared-writer failures, finalization after child handles close, incomplete logs, and missing or malformed payloads. Extend session and CLI tests to check recording completion after shutdown, interruption, stream failure, and submission-channel closure.
GitOrigin-RevId: 30d62fa95912fb0442ad1199776d110884066c30
## Why
Reloading an evicted sub-agent disabled its rollout tracing, leaving later turns out of the trace. Replacement Code Mode sessions could also reuse cell IDs from earlier sessions.
## What changed
- Reattach reloaded children to the root rollout trace and emit `ThreadResumed` without duplicating `ThreadStarted`.
- Reopen existing threads during replay, preserving their identity and history while clearing stale runtime cell and terminal process mappings.
- Allocate Code Mode cell generations across sessions for both gRPC and stdio hosts to prevent ID reuse after reload.
- Record an aborted trace when session startup fails before the session loop begins.
## Testing
Add regression coverage for child eviction and reload, including a failed reload, continued inference tracing, distinct cell IDs, and preserved mail and follow-up context. Extend replay tests for history preservation and reused runtime handles, and host tests for cell IDs across session replacement and reconnection.
GitOrigin-RevId: 6c146c04b5cda9295afb7d973c0d5474b363a822
## Why
Tool calls can emit multiple outputs, and compaction can trim output content while retaining the same call and item IDs. The trace reducer rejected these versions as conflicting content. Late tool and code-cell events could also link outputs in map-key order instead of conversation order.
## What changed
- Preserve distinct tool output versions while retaining consistency checks for tool invocations.
- Link tool and code-cell outputs in thread history order, and resolve pending code cells after the entire event has been reduced.
- Prevent snapshot reconciliation from reusing the same conversation item twice when identical items move within history.
## Testing
Extend reducer regression tests for trimmed outputs, changed images and metadata, reordered identical messages, and late tool and code-cell events spanning the item-ID ordering boundary.
GitOrigin-RevId: eb4242d6a2d32d61c3caab9d3cc3cf6b374ce799
## Why
Tool interception and MCP aliases can route a requested callable to a different handler. Rollout traces need to reflect the handler that actually executes, including when an `exec_command` invocation is intercepted as `apply_patch`.
## What changed
Carry an explicit `ToolCallKind` in runtime-start events and use it to update the tool call's execution classification. This replaces MCP-specific payload inspection while preserving the original callable summary and invocation for intercepted patch and aliased MCP calls.
## Testing
Add a regression test verifying that an intercepted `exec_command` is recorded as `ApplyPatch`, retains its original summary and invocation, completes successfully, and creates no terminal operations or sessions.
GitOrigin-RevId: 0e72f3ef55cda2796e779efd16241e4a0dbf0c5b
## Why
Model output alone does not establish that an item entered conversation history. Treating every response item as accepted can include discarded attempts, while relying on a later request misses accepted final output when a turn ends or is interrupted.
## What changed
- Record `HistoryItemAccepted` events and retain Responses item IDs, generating missing IDs consistently across item events.
- Build thread conversation history from request inputs, compaction checkpoints, and explicit acceptance. Keep unaccepted output accessible through its inference call.
- Preserve inference provenance across retries and late response records without overwriting newer request or compaction snapshots.
- Mark turns with errors as failed. Bump trace schemas to version 2 and reject incompatible bundles and event schemas during replay.
## Testing
Add reducer coverage for discarded attempts with identical text, late responses, compaction, inherited context, and legacy bundle rejection. Add CLI integration tests that run `debug trace-reduce` after success, stream failure, and interruption followed by shutdown, including responses without provider item IDs.
GitOrigin-RevId: 8e387a91d9b3f704df82847d630e6f0b08f998b3
## Why
Runtime process handles are recyclable and thread-local. Using them as terminal session identities can merge separate commands or associate `write_stdin` operations with the wrong exec.
## What changed
- Give each exec a distinct terminal session and track the latest session for each thread-local process handle.
- Record the exec selected by `write_stdin` at runtime and use it to correct session membership during replay, including when dispatch is delayed until a handle is reused.
- Fall back to the latest session for the thread and handle when replaying older traces without runtime target events.
## Testing
Extend reducer coverage for process ID reuse across threads and successive execs, stdin session membership, and delayed runtime target resolution.
GitOrigin-RevId: 330c6fd26a4eebdef03ae600a030dc76920e0dd9
## Why
Reducing code cell lifecycle events discarded their response payload references and lost the initial response status when the runtime status changed.
## What changed
Store the initial response status and separate raw payload IDs for the initial response and final result on `CodeCell`. Carry payload references through queued lifecycle events so they survive replay when the source item arrives later.
## Testing
Extend lifecycle tests to verify distinct initial and final payload IDs and the preserved initial status for yielded cells that complete and failed cells whose lifecycle events precede their source item.
GitOrigin-RevId: 8f49a3174ea487a5d91ebc2a6c97133aacaabfd2
## Why
Rollout traces need an identifier that can be used to correlate a Codex
inference with upstream Responses API, proxy, and engine logs. The
reduced trace model already exposed `upstream_request_id`, but it was
being populated from the Responses API `response.id`. That value is
useful for `previous_response_id` chaining, but it is not the transport
request id that upstream systems key on.
This PR separates those concepts so trace consumers can reliably answer
both questions:
- which Responses API response did this inference produce?
- which upstream request handled it?
## Structure
The change keeps the upstream request id at the same lifecycle level as
the provider stream:
- `codex-api` captures the `x-request-id` HTTP response header when the
SSE stream is created and exposes it on `ResponseStream`. Fixture and
websocket streams set the field to `None` because they do not have that
HTTP response header.
- `codex-core` carries that stream-level id into `InferenceTraceAttempt`
when recording terminal stream outcomes. Completed, failed, cancelled,
dropped-stream, and pre-response error paths all record the id when it
is available.
- `rollout-trace` now records both identifiers in raw terminal inference
events and response payloads: `response_id` for the Responses API
`response.id`, and `upstream_request_id` for `x-request-id`.
- The reducer stores both fields on `InferenceCall`. It also uses
`response_id` for `previous_response_id` conversation linking, which
removes the old accidental dependency on the misnamed
`upstream_request_id` field.
- Terminal inference reduction now consumes the full terminal payload
(`InferenceCompleted`, `InferenceFailed`, or `InferenceCancelled`) in
one place. That keeps status, partial payloads, response ids, and
upstream request ids consistent across success, failure, cancellation,
and late stream-mapper events.
## Why This Shape
`x-request-id` is a property of the HTTP/provider response envelope, not
an SSE event. Capturing it once in `codex-api` and plumbing it through
terminal trace recording avoids trying to infer the value from stream
contents, and it preserves the id even when the stream fails or is
cancelled after only partial output.
Keeping `response_id` separate from `upstream_request_id` also makes the
reduced trace model less surprising: `response_id` remains the
conversation-continuation id, while `upstream_request_id` is the
operational correlation id for upstream debugging.
## Validation
The PR updates trace and reducer coverage for:
- reading `x-request-id` from SSE response headers;
- storing the true upstream request id on completed inference calls;
- preserving upstream request ids for cancelled and late-cancelled
inference streams;
- keeping `previous_response_id` reconstruction tied to `response_id`
rather than transport request ids.
Records cancelled inference streams when Codex stops consuming a
provider response before `response.completed`, preserving complete
output items observed before cancellation.
Also closes still-running inference calls when the owning turn ends, so
reduced rollout traces do not leave stale `Running` inference nodes.
Covered by focused reducer coverage and a core stream-drop test for
partial output preservation.
## Summary
Adds the standalone `codex-rollout-trace` crate, which defines the raw
trace event format, replay/reduction model, writer, and reducer logic
for reconstructing model-visible conversation/runtime state from
recorded rollout data.
The crate-level design is documented in
[`codex-rs/rollout-trace/README.md`](https://github.com/openai/codex/blob/codex/rollout-trace-crate/codex-rs/rollout-trace/README.md).
## Stack
This is PR 1/5 in the rollout trace stack.
- [#18876](https://github.com/openai/codex/pull/18876): Add rollout
trace crate
- [#18877](https://github.com/openai/codex/pull/18877): Record core
session rollout traces
- [#18878](https://github.com/openai/codex/pull/18878): Trace tool and
code-mode boundaries
- [#18879](https://github.com/openai/codex/pull/18879): Trace sessions
and multi-agent edges
- [#18880](https://github.com/openai/codex/pull/18880): Add debug trace
reduction command
## Review Notes
This PR intentionally does not wire tracing into live Codex execution.
It establishes the data model and reducer contract first, with
crate-local tests covering conversation reconstruction, compaction
boundaries, tool/session edges, and code-cell lifecycle reduction. Later
PRs emit into this model.
The README is the best entry point for reviewing the intended trace
format and reduction semantics before diving into the reducer modules.
tongzhou wang