Commit Graph
12 Commits
Author SHA1 Message Date
cassirer-openai ef6ce8100a Bound rollout trace capture with configurable storage limits (#61896)
## What changed

- Add `rollout_trace.max_payload_bytes` and `rollout_trace.max_bytes`, defaulting to 64 MiB per body or separated value and 256 MiB per bundle. Root and child threads share the bundle budget.
- Record explicit truncations and omissions when limits are reached, reserve space for later events, and stop at the last complete event if the event log fills.
- Reuse identical prepared-prompt items and store inline images separately from message structure. Preserve exact JSON bytes for complete composite payloads and surviving content and links in partial captures.
- Let reducers handle partial payloads while retaining lifecycle information and reporting `closed_with_partial_payloads` for closed bundles with incomplete captures.

## Testing

Add coverage for payload reuse and byte-exact reconstruction, truncation, compressed bundle budgets, later events after omissions, readable prefixes after event-space exhaustion, and CLI capture-limit configuration and startup failures.

GitOrigin-RevId: be99a68da427414a7a12bf8209f232c1a0bbe545
2026-11-16 22:26:31 +00:00
ioanbudea-OAI 400a1e1e82 Use native permission profiles for diagnostics and sandbox checks (#61369)
## What changed

- Derive sampling diagnostic and rollout trace labels directly from permission profiles instead of constructing legacy sandbox policies. Rename the trace metadata field from `sandbox_policy` to `permissions_profile`.
- Extract `with_workspace_write_cwd` to share default workspace metadata protections between legacy policy conversion and the TUI's sandbox-mode compatibility check. Build the check's workspace-write profile from filesystem and network permissions directly.

## Testing

Add a regression test confirming that the sandbox-mode compatibility check accepts standard workspace metadata protections and rejects a profile with an additional writable `.git` root.

GitOrigin-RevId: f20d23afd80dff6347865df3d6a94d6f5e90e1e0
2026-11-13 20:22:45 +00:00
cassirer-openai b8dde1eb40 Move rollout trace recording to a background worker (#61253)
## Why

Recording inference starts synchronously serializes prompts and writes trace payloads before model work can proceed. Defer that work so inference dispatch can continue while the trace is recorded.

## What changed

- Use one worker per trace bundle to write events in submission order, with up to eight pending jobs.
- Share prepared prompts and encoded requests with the worker, and build remote compaction snapshots there. Write payloads before publishing events that reference them.
- Add `TraceWriter::flush` and `ThreadTraceContext::flush` for live reads. Drain accepted jobs and seal the bundle on shutdown; suppress the closing marker after a recording failure.

## Testing

Add tests for deferred prompt serialization, ordered start and completion events, shutdown draining, and serialization failures that preserve earlier evidence without publishing a closing marker. Verify compaction snapshots are built on the worker and skipped when tracing is disabled.

GitOrigin-RevId: 32fd787bb1d281f1814df2f08c73b667628f8add
2026-11-13 04:36:03 +00:00
cassirer-openai d4b19eca27 Preserve agent message deliveries in rollout traces (#60673)
## Why

An accepted agent message may never appear in a later inference request or compaction snapshot. Rollout traces need to retain the delivery and its interaction edge even when the receiving agent makes no further request.

## What changed

Capture agent message payloads in `HistoryItemAccepted` events and create conversation items when those receipts are replayed. Reuse each item's ID when it later appears in a request or checkpoint, updating its body to the model-visible representation. Repeated receipts preserve that representation without duplicating the item.

## Testing

Extend the spawn-edge regression test to cover delivery before any child inference, stable item IDs across a later request, and preservation of trimmed content after a repeated receipt.

GitOrigin-RevId: c193d691e8240645800ce004c11f21e3f1c82915
2026-11-10 22:22:35 +00:00
cassirer-openai 297f5e33a5 Make rollout recording failures and completion explicit (#58166)
## Why

Enabled rollout tracing could silently fail to initialize, and a terminal task status did not establish whether all recording had finished. Trace consumers need to distinguish a complete recording from a partial bundle, including when a task is interrupted.

## What changed

- Fail thread startup or resume if an enabled recorder cannot initialize, while cleaning up session resources.
- Stop recording after the first write or serialization failure and preserve that failure across shared producers.
- Append a closing record when the last writer handle is released without recording errors. Expose `recording_complete` in `state.json` independently of task success, and mark submission-channel closure as an aborted rollout.
- Reject replay logs with missing or reordered events, mismatched rollout IDs, events after completion, or missing or invalid JSON payloads. Valid prefixes remain replayable with `recording_complete` set to `false`.

## Testing

Add coverage for startup failure and recovery, shared-writer failures, finalization after child handles close, incomplete logs, and missing or malformed payloads. Extend session and CLI tests to check recording completion after shutdown, interruption, stream failure, and submission-channel closure.

GitOrigin-RevId: 30d62fa95912fb0442ad1199776d110884066c30
2026-10-29 19:56:26 +00:00
cassirer-openai 68b67f0ff4 Preserve rollout tracing across sub-agent reloads (#57615)
## Why

Reloading an evicted sub-agent disabled its rollout tracing, leaving later turns out of the trace. Replacement Code Mode sessions could also reuse cell IDs from earlier sessions.

## What changed

- Reattach reloaded children to the root rollout trace and emit `ThreadResumed` without duplicating `ThreadStarted`.
- Reopen existing threads during replay, preserving their identity and history while clearing stale runtime cell and terminal process mappings.
- Allocate Code Mode cell generations across sessions for both gRPC and stdio hosts to prevent ID reuse after reload.
- Record an aborted trace when session startup fails before the session loop begins.

## Testing

Add regression coverage for child eviction and reload, including a failed reload, continued inference tracing, distinct cell IDs, and preserved mail and follow-up context. Extend replay tests for history preservation and reused runtime handles, and host tests for cell IDs across session replacement and reconnection.

GitOrigin-RevId: 6c146c04b5cda9295afb7d973c0d5474b363a822
2026-10-28 01:29:34 +00:00
cassirer-openai f2df3e3b56 Preserve full MCP results in rollout traces (#57280)
## Why

MCP completion events can truncate large results and omit content removed for model consumption. Rollout traces need the original response, including content and metadata, as well as results for calls denied before reaching the backend.

## What changed

- Record MCP runtime results before callbacks, content filtering, and UI truncation, and record skipped calls as failures.
- Stop deriving MCP runtime completion records from protocol events to avoid duplicate records.
- Treat MCP runtime payloads as opaque results when reducing agent interactions.
- Buffer JSON trace writes and explicitly flush them.

## Testing

Extend app-server tests to verify that traces retain oversized MCP text, audio, structured content, and metadata while UI results remain truncated, and that denied calls produce exactly one failed runtime result with the denial reason.

GitOrigin-RevId: 57880770545eb5869420d7dc6fa4a15e481cae9d
2026-10-26 21:56:23 +00:00
cassirer-openai b6a4378862 Classify rollout tool calls by their runtime handler (#57194)
## Why

Tool interception and MCP aliases can route a requested callable to a different handler. Rollout traces need to reflect the handler that actually executes, including when an `exec_command` invocation is intercepted as `apply_patch`.

## What changed

Carry an explicit `ToolCallKind` in runtime-start events and use it to update the tool call's execution classification. This replaces MCP-specific payload inspection while preserving the original callable summary and invocation for intercepted patch and aliased MCP calls.

## Testing

Add a regression test verifying that an intercepted `exec_command` is recorded as `ApplyPatch`, retains its original summary and invocation, completes successfully, and creates no terminal operations or sessions.

GitOrigin-RevId: 0e72f3ef55cda2796e779efd16241e4a0dbf0c5b
2026-10-26 19:04:02 +00:00
cassirer-openai 5b792948f9 Track accepted conversation history in rollout traces (#56312)
## Why

Model output alone does not establish that an item entered conversation history. Treating every response item as accepted can include discarded attempts, while relying on a later request misses accepted final output when a turn ends or is interrupted.

## What changed

- Record `HistoryItemAccepted` events and retain Responses item IDs, generating missing IDs consistently across item events.
- Build thread conversation history from request inputs, compaction checkpoints, and explicit acceptance. Keep unaccepted output accessible through its inference call.
- Preserve inference provenance across retries and late response records without overwriting newer request or compaction snapshots.
- Mark turns with errors as failed. Bump trace schemas to version 2 and reject incompatible bundles and event schemas during replay.

## Testing

Add reducer coverage for discarded attempts with identical text, late responses, compaction, inherited context, and legacy bundle rejection. Add CLI integration tests that run `debug trace-reduce` after success, stream failure, and interruption followed by shutdown, including responses without provider item IDs.

GitOrigin-RevId: 8e387a91d9b3f704df82847d630e6f0b08f998b3
2026-10-23 00:45:20 +00:00
cassirer-openai deede7201d Keep rollout trace terminal sessions distinct when process IDs are reused (#56011)
## Why

Runtime process handles are recyclable and thread-local. Using them as terminal session identities can merge separate commands or associate `write_stdin` operations with the wrong exec.

## What changed

- Give each exec a distinct terminal session and track the latest session for each thread-local process handle.
- Record the exec selected by `write_stdin` at runtime and use it to correct session membership during replay, including when dispatch is delayed until a handle is reused.
- Fall back to the latest session for the thread and handle when replaying older traces without runtime target events.

## Testing

Extend reducer coverage for process ID reuse across threads and successive execs, stdin session membership, and delayed runtime target resolution.

GitOrigin-RevId: 330c6fd26a4eebdef03ae600a030dc76920e0dd9
2026-10-21 23:13:31 +00:00
cassirer-openai 702e6a3c64 [rollout-trace] Add a trace ID to MCP calls. (#22326)
This allows us to connect individual tool calls to the logs of the
invocations.
2026-05-13 16:03:33 +00:00
cassirer-openai e3c8720a99 [rollout_trace] Add debug trace reduction command (#18880)
## Summary

Adds the debug CLI entry point for reducing recorded rollout traces.
This gives developers a direct way to inspect whether the emitted trace
stream reduces into the expected conversation/runtime model.

## Stack

This is PR 5/5 in the rollout trace stack.

- [#18876](https://github.com/openai/codex/pull/18876): Add rollout
trace crate
- [#18877](https://github.com/openai/codex/pull/18877): Record core
session rollout traces
- [#18878](https://github.com/openai/codex/pull/18878): Trace tool and
code-mode boundaries
- [#18879](https://github.com/openai/codex/pull/18879): Trace sessions
and multi-agent edges
- [#18880](https://github.com/openai/codex/pull/18880): Add debug trace
reduction command

## Review Notes

This PR is intentionally last: it depends on the trace crate, core
recorder, runtime/tool events, and session/agent edge data all existing.
The command should remain a debug/developer tool and avoid adding new
runtime behavior.

The useful review question is whether the CLI exposes the reducer in the
smallest practical way for local inspection without turning the debug
command into a supported user-facing workflow.
2026-04-24 01:56:48 +00:00