Commit Graph
20 Commits
Author SHA1 Message Date
felixxia-oai 80d7ca34bc Retain user instructions in guardian thread context (#42762)
## Why

Guardian review needs the original user instructions alongside verified answers, even after compaction, resume, or rollback. A summarized or legacy checkpoint cannot establish a complete instruction history.

## What changed

- Capture text user messages in retained context when `guardian_thread_context` is enabled, preserving their message identity through compaction and replay.
- Keep retained instructions and verified answers in arrival order, with independent size and completeness tracking for each family.
- Roll back retained evidence at the exact user-message boundary, including steers that share a turn ID, and apply the same semantics during rollout migration.
- Treat checkpoints without retained user messages as incomplete while preserving their verified answers.

## Testing

- Cover capture, compaction, resume, rollback, child forks, legacy checkpoints, storage bounds, and rollout migration.

GitOrigin-RevId: 72706731b54e5ac8645db61b446d2401afb42235
2026-09-04 13:24:42 +00:00
Owen Lin 69cebb5d15 Route rollout reads through the canonical JSON decoder (#42378)
## Why

Directly deserializing the flattened `RolloutLine` envelope can reject nested
decimal values, preventing affected paginated sessions from resuming.

## What changed

- Add canonical string, byte, and reverse-scanner helpers that decode rollout
  records through `serde_json::Value` before decoding the flattened item.
- Route rollout readers across session discovery, history, migration, search,
  thread storage, and transcript previews through those helpers.
- Remove `Deserialize` from `RolloutLine` so new readers cannot bypass the
  canonical persistence decoder.

## Testing

Add coverage that resumes a paginated rollout after a token-count record with
a decimal rate-limit value and verifies that ordinal sequencing continues.

GitOrigin-RevId: 49abac1e0751c073daa5a93a840d8a483fd2d013
2026-09-02 22:45:19 +00:00
felixxia-oai 8e3b180d49 Preserve retained answers across steer rollbacks (#42298)
## Why

A steer creates a new instruction boundary without creating a new turn ID. Retained `request_user_input` answers that share that turn ID must not all be discarded when only the later steer is rolled back.

## What changed

- Associate retained answers with their originating function call and instruction boundary.
- Keep answers whose source calls survive rollback, including through compaction, resume, and rollout migration.
- Remove answers whose source calls belong to the rolled-back instruction.

## Testing

- Cover live rollback and resume with answers before and after a steer.
- Cover rollout migration when delayed answers share a turn ID but originate from different calls.

GitOrigin-RevId: 225e7eb7b6a3ae5a5e7fa6fcb80cb7d259db90d0
2026-09-02 15:46:33 +00:00
felixxia-oai 5971d42847 Preserve verified answers across history compaction (#42293)
## Why

Compaction replaces the model history window, but host-verified user answers may
still be needed for the rest of the owning thread.

## What changed

- Add bounded, model-invisible retained context for verified
  `request_user_input` answers.
- Persist retained context in rollout events and compaction checkpoints, and
  restore it when a thread resumes.
- Remove answers when their source turns are rolled back, and prevent
  parent-local answers from being inherited by child forks.

## Testing

Add lifecycle coverage for deduplication and size limits, compaction and resume,
rollback, rollout migration, and child forks.

GitOrigin-RevId: c32900d3e8120a9899b71f0576683acd67a4b898
2026-09-02 15:07:11 +00:00
rka-oai 2c79ee6dac Add structured asynchronous user input requests (#42178)
## What changed

- Replace `send_user_message_async` with `request_user_input_async`, accepting one or more questions with optional suggested answers while allowing the turn to continue.
- Attach structured question metadata to asynchronous agent messages while retaining readable fallback text, and preserve it through app-server events, thread history, and generated schemas.
- Continue enabling the new tool for model catalogs that advertise either the old or new tool name.

## Testing

- Cover tool registration, question validation, emitted items, continued turn execution, history preservation, and backwards-compatible deserialization.

GitOrigin-RevId: ffc49aebde2e854c3c50a122aa52805fec3fc6b3
2026-09-02 02:09:35 +00:00
Owen Lin 5f79a92e39 Persist response token usage in rollout history (#41912)
## Why

Resumed threads need to continue per-turn and per-thread token totals without scanning arbitrarily far beyond the latest compaction checkpoint.

## What changed

- Add durable `TokenUsageRecord` rollout items with response, turn, thread, session, and root-turn attribution.
- Restore accumulated usage on resume and snapshot the latest record plus the compaction response ID in compaction checkpoints.
- Preserve root-turn lineage in persisted turn context, while ensuring forked child threads start with their own usage totals.

## Testing

- Cover usage accumulation across multiple responses and resumed turns.
- Cover local and remote compaction checkpoints, invalid remote compaction output, rollout reconstruction, and fork isolation.

GitOrigin-RevId: ef9e0c4a9102a08a2c382be4cdac68c84353c90a
2026-08-31 21:15:38 +00:00
Chris Dong 346c4db7c2 Track image generation request IDs in analytics (#40714)
## What changed

- Read `x-codex-imagegen-request-id` from image generation and edit responses.
- Propagate the ID to `codex_image_generation_event` analytics.
- Keep the ID in process only, excluding it from extension item serialization, generated types, app-server wire data, and rollout history.

## Testing

- Cover response-header extraction and missing-header behavior in the images client.
- Verify end-to-end analytics emission and the unchanged image-generation item wire shape.

GitOrigin-RevId: 9b8e09b7d655c0c2faa84f0eee99aabae583668d
2026-08-25 21:38:10 +00:00
guinness-oai 41ca88e95c Persist realtime events in the thread timeline (#40508)
## Why

Clients need a bounded view of realtime conversations that preserves the order of speech, agent work, and turn lifecycle events without loading the full thread history.

## What changed

- Persist realtime session boundaries, transcript segments, and promoted agent items in paginated thread rollouts.
- Add the experimental `thread/timeline/list` API to page ordinary items, realtime items, and turn boundaries in canonical rollout order. Include `activeRealtimeSessionAtPageStart` so each page can be rendered independently.
- Emit typed `thread/realtime/item/started`, `thread/realtime/item/transcript/delta`, and `thread/realtime/item/completed` notifications around durable realtime items.
- Keep the existing `thread/items/list` response and legacy thread history behavior unchanged.

## Testing

- Cover mixed timeline pagination, realtime-session state at page boundaries, rollout replay, subagent history prefixes, transcript streaming, steering, and promoted agent artifacts.

GitOrigin-RevId: faa2c420192d63e060e8fb32446e3c797422656e
2026-08-25 01:03:20 +00:00
Owen Lin 465eafacbc Harden startup rollout migration against concurrent updates (#40499)
## Why

Another Codex process can write, archive, or compress a rollout while startup
migration is inspecting it. This can leave a discovered path stale or make an
in-progress rollout look empty or busy.

## What changed

- Wait for rollout maintenance to finish before starting background migration.
- Re-read empty rollouts under their writer lock and retry busy rollouts on a
  later startup.
- Find archived or compressed rollouts again when their paths change after
  discovery, while keeping terminal failures from blocking the startup cursor.

## Testing

Added coverage for maintenance contention, writer-owned empty rollouts, busy
rollout retries after archive and compression moves, permanent failure skips,
pending recovery, and paths that move after discovery.

GitOrigin-RevId: 9708f212f2bbff88bd47a01ec4872bf6ce9af535
2026-08-25 00:03:23 +00:00
Owen Lin 90c67e6f33 Classify rollout migration failures (#39784)
## Why

Rollout migration outcomes expose free-form error messages, making failures hard to group without parsing text.

## What changed

- Add a serialized `RolloutMigrationFailureReason` to failed outcomes and classify failures across reading, conversion, SQLite materialization, publishing, and recovery.
- Add the failure reason as a low-cardinality tag on per-thread migration telemetry.

## Testing

- Cover missing SQLite metadata and invalid session metadata classifications.

GitOrigin-RevId: 2bff98483b484f4d02b72d52f39950ff99ebe429
2026-08-20 19:24:09 +00:00
joeflorencio-openai ce950dcf26 Add managed developer instructions to requirements (#39755)
## What changed

- Add `additional_developer_instructions` to managed requirements and expose it through `configRequirements/read` as `additionalDeveloperInstructions`, independently of ordinary developer instructions.
- Include the managed instructions in model context, emit explicit replacement or removal messages when requirements change, and preserve the current value across compaction, resume, and agent forks without duplication.
- Reject managed instructions whose rendered context exceeds 10,000 estimated tokens.

## Testing

- Cover requirements layering and API serialization, context updates and removal, size validation, repeated model requests, compaction and resume, rollout migration, and agent forks.

GitOrigin-RevId: bc0b70fb7988944c2f68176dff55f5ed61eb46c8
2026-08-20 17:36:22 +00:00
rka-oai fb356f3d2c Add async delivery metadata to agent messages (#39312)
## What changed

- Add an optional `delivery` field to agent message events and app-server
  `agentMessage` items, with `"async"` identifying a user-visible message sent
  without ending the current turn.
- Preserve the marker through legacy event conversion, thread history
  materialization, replay, and generated JSON and TypeScript schemas.
- Require delivery of async agent message completion notifications on the
  in-process app-server transport.

## Testing

- Cover preservation of async delivery metadata in thread history.
- Cover delivery-required classification for async agent messages.

GitOrigin-RevId: f92ba25de1293ee271404badfdf0d117d6530329
2026-08-18 23:05:52 +00:00
jif a397079287 Preserve MCP resource origins across compaction (#39192)
## Why

Compaction can remove the tool-call events that associate an MCP app widget with
the tool, account, and URI needed to authorize later resource reads.

## What changed

- Store a bounded MCP resource-origin checkpoint with each compacted rollout.
- Restore the checkpoint when resuming a session so existing widgets remain
  readable after compaction and restart.
- Discard saved origins when rollback rewrites history across the compaction
  boundary, and reject oversized or invalid checkpoints during restoration.

## Testing

- Extend the app-server MCP resource-origin test to compact a paginated thread,
  continue the conversation, and read the original widgets before and after a
  restart.
- Verify rollback clears provenance that no longer matches retained history.

GitOrigin-RevId: 7f91c6202fe8d9be21c78dda42fa205937c44263
2026-08-18 12:37:41 +00:00
jif 72fa74fbc9 Persist security risk scores in rollout history (#38363)
## What changed

- Add a `SecurityRiskScore` rollout item containing a category and numeric score.
- Persist the item in both thread history modes while excluding it from model context, user-visible thread history, search text, forks, and reconstructed conversation history.
- Re-export the score type from the extension API.

## Testing

- Cover serialization, persistence and loading, thread history projection, session reconstruction, append planning, and memory filtering.

GitOrigin-RevId: 1926fe366aeaa75052708a6da589f45a38eefb52
2026-08-13 12:04:29 +00:00
rka-oai 3a6f747d77 Preserve harness metadata across conversation history (#38058)
## What changed

- Wrap response items with optional harness-owned metadata while keeping the persisted response payload shape backward compatible.
- Store compacted-history metadata in an aligned sidecar and reject malformed sidecars.
- Carry metadata through history normalization, compaction, resume, fork, truncation, migration, and rollout recording without sending it to model providers.

## Testing

- Cover legacy rollout compatibility, metadata serialization and validation, compaction retention, and resume/fork reconstruction.

GitOrigin-RevId: 17a6127c9048ff984cb4df4d97c0751a38c29f5f
2026-08-11 19:30:12 +00:00
arun eswara edcec13372 Expose image generation usage-limit failures (#38024)
## What changed

- Add structured `usageLimitExceeded` metadata to failed image-generation
  items, including the image limit ID and optional reset timestamp.
- Preserve the failure metadata in completion events, app-server thread
  history, thread reads and resumes, and migrated legacy rollouts.
- Export the new failure type in the generated JSON and TypeScript schemas.

## Testing

- Cover usage-limit responses through item completion, persistence, and thread
  resume.
- Verify legacy rollout migration retains image-generation failure metadata.

GitOrigin-RevId: 91c31a2a47456b6b916ee03002ff314b05946dd4
2026-08-11 15:40:52 +00:00
Adam Perry @ OpenAI 63002bdb26 Extract persisted history types into a dedicated crate (#37871)
## What changed

- Add `codex-history` for model-history and persisted-rollout domain types, including `RolloutItem`, `RolloutLine`, `CompactedItem`, and initial/resumed history state.
- Re-export the persisted types from `codex-rollout` and update consumers to use the new crate boundary instead of `codex-protocol`.
- Preserve existing rollout serialization, including legacy numeric compacted-window IDs.

## Testing

- Add `codex-history` tests for rollout JSON round trips, compacted-history compatibility, persisted history modes, and multi-agent version selection.

GitOrigin-RevId: 944daa9297ddd231d3aebbdcb05fff4adf8b4e1b
2026-08-10 19:26:52 +00:00
Owen Lin 4bb7ee3472 Add rollout migration tooling and background migration (#37348)
## What changed

- Add `codex migrate-rollouts` with dry-run inspection by default, explicit `--apply`, thread filtering, optional I/O throttling, progress output, and JSON or verbose reports.
- Add the disabled-by-default `background_paginated_rollout_migration` feature to migrate legacy local sessions at startup before rollout compression begins.
- Persist a migration cursor and skipped-file fingerprints so later startups avoid full rescans while retrying changed files and recovering pending migrations.
- Coordinate migration with live writers and emit metrics for manual and startup runs.

## Testing

- Cover startup cursor advancement and lookback, pending migration recovery, live-writer coordination, changed empty rollouts, and progress reporting.

GitOrigin-RevId: 276ac506c50ebec5140fd319faca1bb998172061
2026-08-07 01:26:42 +00:00
Owen Lin aac9f84247 Preserve legacy semantics during rollout migration (#37191)
## Why

Legacy rollouts can contain historical rollbacks, compaction checkpoints, and
subagent copies of parent history. Migrating those records verbatim can change
the visible conversation or the model context used when a thread resumes, and
can retain large amounts of duplicated subagent history.

## What changed

- Plan and replay historical rollbacks by logical turn, preserving surviving
  lifecycle events and the correct compaction anchor for future resumes.
- Migrate subagent rollouts from a safe bounded replay when possible and record
  the history boundary so copied parent context is not projected as child turns.
- Include archived and compressed rollouts while keeping their storage form and
  location intact.
- Tolerate malformed, partial, oversized, and empty JSONL records during the
  migration scan.

## Testing

Add coverage for rollback and compaction combinations, subagent replay bounds,
archived and compressed rollouts, recovery, concurrent maintenance, and damaged
JSONL input.

GitOrigin-RevId: 3948e16d43f75df8571b0a92af12a181d1d08fdf
2026-08-06 00:54:14 +00:00
Owen Lin 6bb6e9045f Add legacy rollout migration to paginated history (#37175)
## What changed

- Add `LocalThreadStore::migrate_rollouts` with dry-run and apply modes,
  optional thread selection, throughput limiting, and per-rollout outcomes.
- Canonicalize legacy JSONL records into paginated history while preserving
  model-visible conversation context, including compressed rollouts and copied
  fork history.
- Publish replacements atomically and journal migrations so interrupted SQLite
  projections can be recovered. Coordinate migration with rollout compression
  and skip rollouts with active writers.
- Make paginated history a one-way SQLite promotion so stale metadata cannot
  downgrade a migrated thread.

## Testing

- Cover legacy record normalization, dry runs, idempotency, malformed input,
  compressed rollouts, active writers, interrupted migration recovery, and a
  cold app-server resume after migration.

GitOrigin-RevId: b9991b659f28ebb52da39ce62e8b4e0bae2ba2bb
2026-08-05 22:57:27 +00:00