Commit Graph
62 Commits
Author SHA1 Message Date
Brent Traut fe16bc0a97 Release superseded assistant text from live paginated history (#58751)
## Why

Live paginated history retains completed assistant text even when summaries only need the latest message. Release older text for threads with persisted history while preserving full resume responses.

## What changed

- Evict superseded assistant text from live turn state while preserving corrected items and the latest assistant message.
- Restore evicted text for full resume responses from prepared history pages, fetching remaining items by ID in bounded batches. Return an error if saved text is missing.
- Only load the active turn when it belongs in the requested initial page, and use the item ID index for targeted history reads.

## Testing

Add reducer coverage for text eviction and corrected messages. Extend running-thread resume coverage to compare full and summary items with live notifications, exercise restoration when the prepared page lacks the active turn, and check missing-text errors.

GitOrigin-RevId: 3229c0cea820f117dc6331312f1e6056a8d1d52c
2026-11-01 20:10:33 +00:00
Charlie Marsh 4969675f01 Resolve network approval prompts when their requests end (#58350)
## Why

A network request can disappear while its approval prompt is still pending. Leaving that prompt active allows a late cancellation to interrupt the turn even though the request no longer needs approval.

## What changed

- Track network prompt lifetimes in core and notify the app server when a prompt is resolved or withdrawn.
- Cancel the corresponding server request, emit `serverRequest/resolved`, and release its permission guard. Resolve outstanding network prompts during thread shutdown as well.
- Claim a live pending approval and validate its originating turn before applying a decision, so stale responses cannot interrupt a turn or persist policy changes.
- Distinguish network approvals that require the originating turn from proxy approvals that can be serviced by a later active turn.

## Testing

Extend regression coverage for disconnected network requests, late cancellation responses, turn completion during review, shutdown cleanup, mismatched turn IDs, and command approvals from background code running during a later turn.

GitOrigin-RevId: 78f656d92607ec21ddd0c5d781ed0947588cb2eb
2026-10-30 14:40:32 +00:00
Charlie Marsh 7f4d53e928 Release app-server thread runtimes after shutdown (#57788)
## Why

Waiting for the event channel to close after `ShutdownComplete` retains the thread runtime because the thread owns its event sender. Remaining subscribers still need late notifications after shutdown.

## What changed

- Release runtime references after `ShutdownComplete` while keeping the listener command queue available for ordered notifications and late request resolutions.
- Preserve other subscribers when a client unsubscribes or disconnects from an unloaded thread, and serialize terminal cleanup with cold resumes.
- Return the thread-closing error when `thread/resume` encounters a terminated runtime or a listener that cannot complete the request.
- Close the status watch when removing a thread so a later load gets a fresh watch.

## Testing

Add regression coverage for listeners attached before and after shutdown, verifying runtime release, preservation of remaining subscribers, late notification delivery, and rejection of resumes against a terminated runtime. Add a test that removing and reloading a thread closes the old status watch.

GitOrigin-RevId: 6dda8f83a0fbe06c9a713e9801356753a0de172e
2026-10-28 16:55:57 +00:00
Charlie Marsh 575cc5351e Fix app-server thread listener startup and shutdown races (#57787)
## Why

A thread runtime can shut down while listener setup is awaiting I/O, and a cold resume can start before the previous listener forwards `ShutdownComplete`. Listener replacement must preserve those buffered terminal events.

## What changed

- Coordinate listener installation and connection subscriptions with pending unloads, and reject closing or replaced runtimes.
- Allow a newly created runtime that has already shut down to drain its queued events, while enforcing one event reader per runtime.
- Track shutdown draining with a shared watch channel so cold resume and runtime replacement wait for terminal events to be processed.
- Serialize idle unloading with goal edits and cold resume, and recheck thread state after acquiring the lock.

GitOrigin-RevId: d05e348b0a81bddb2defa963d0cfd3fde08dbd03
2026-10-28 16:53:55 +00:00
Charlie Marsh a797dd7c6f Expose lifecycle timestamps on app-server thread items (#57172)
## What changed

- Add nullable `startedAtMs` and `completedAtMs` fields to `ThreadItem` in live notifications and turn history. Values are Unix milliseconds when recorded.
- Preserve recorded item times through history reconstruction, pagination, and resume, including command start times recorded during approval.
- Separate the Rust item payload into `ThreadItemPayload` while keeping the JSON payload flat, and update consumers, schemas, and generated TypeScript and Python types.

## Testing

Extend coverage for timestamp serialization, history reconstruction, paginated reads and resume, and persisted Python SDK items.

GitOrigin-RevId: ecbcee95af52db7c807b46a6e9deb30e5fba1f80
2026-10-26 18:13:00 +00:00
jif 07316174c6 Forward Guardian reviewer events to parent raw-event subscribers (#56124)
## What changed

Add the disabled-by-default `guardian_recorded_items` feature to forward recorded reviewer items and interruptions through the parent's raw-event subscription, preserving reviewer thread and turn IDs without modifying the parent's history or completing its turn.

Emit reviewer items as `rawResponseItem/completed` notifications and interruptions as interrupted `turn/completed` notifications. Mark opted-in reviewer requests with `guardian_recorded_items` metadata.

## Testing

Extend Guardian integration tests to cover reviewer identity, recorded-item metadata replay, concurrent reviewer interruptions, independent parent completion, and behavior without the feature enabled.

GitOrigin-RevId: beb4dbf6c6f2a800a2fbf3cfd11ae174a1382386
2026-10-22 09:20:00 +00:00
Charlie Marsh b1fd48c48e Avoid cloning events in the app-server thread listener (#55283)
Record whether an event is `ShutdownComplete` before moving it into
`apply_bespoke_event_handling`, preserving shutdown drain notification without
cloning the event.

GitOrigin-RevId: 9e4ca8c657d4d5b98056de8c7aba1391233994c1
2026-10-19 13:27:16 +00:00
jif d28be12577 Honor per-thread unload delays in app-server (#55270)
## Why

Thread unload timing used the server-wide delay, ignoring per-thread configuration overrides. Clients need to control how long individual idle, unsubscribed threads remain loaded.

## What changed

- Resolve unload timing from each loaded thread's `thread_unload_delay_secs` configuration. Setting it to `0` allows immediate unloading once the thread has no subscribers or activity.
- Allow this runtime retention override in `experimentalPredictionMode` forks while continuing to reject other configuration overrides. Reject negative values and delays too large to schedule.
- Replace the source-specific immediate-unload behavior with explicit configuration.

## Testing

Extend integration coverage for per-thread delays, unloading after active turns finish, fork overrides, and invalid prediction-fork delays. Verify that immediate unloading preserves persistent history and that cold resumes and ordinary nested forks use their default delay instead of retaining the previous runtime's override.

GitOrigin-RevId: fcb82bd73a101260c51e2a8476f94320c23ac929
2026-10-19 12:30:33 +00:00
jif 9032fc7683 Unload idle ephemeral composer prediction threads without delay (#55255)
## What changed

Set the app server's unload delay to zero for ephemeral threads whose `thread_source` is `composer_predictions`, excluding subagent sessions. These threads unload once they are idle and have no subscribers, while active turns continue to completion.

## Testing

Extend integration coverage to check immediate unloading after an active prediction turn finishes and after unsubscribing from a fork that inherits the prediction configuration. Verify that persistent threads and ephemeral threads without the prediction source retain the configured delay.

GitOrigin-RevId: 04a0e405b666c689ccf6f482055d24692b18ead0
2026-10-19 11:09:48 +00:00
Brent Traut 82883da25e Notify subscribers when thread read state changes (#52384)
## What changed

Add the experimental `thread/readState/changed` notification with `threadId` and the committed `readState` receipt. Send it to existing thread subscribers when terminal activity updates read state or a revert clears a removed unread position. Emit terminal read-state updates before `turn/completed` so completion remains the last notification for that result.

Expose the notification in generated JSON schemas, TypeScript types, and Python SDK models. Document experimental capability gating, subscription lifetime, notification opt-out, and refreshing receipts when concurrent responses disagree.

## Testing

Extend the read-state integration test to assert that the notification receipt matches the receipt returned by `thread/read` after terminal activity.

GitOrigin-RevId: 28935352a881b7a5e2e7f629173c3b2061bd7941
2026-10-09 02:59:21 +00:00
Brent Traut 0ada5d8806 Expose experimental durable thread read state in the app server (#52350)
## What changed

- Add experimental `readState` to `thread/read` and a thread-ID-keyed `readStates` map to `thread/list` for eligible durable local user threads. Each receipt contains `firstUnread` and an opaque `revision`.
- Publish new visible terminal results before `turn/completed`, after flushing the rollout. Reading snapshots leaves activity unread; replayed history does not publish new activity.
- Clear unread positions removed by a paginated thread revert while preserving explicit thread-start marks and concurrent receipt updates.

## Testing

Add an integration test covering completed-turn receipts, repeated reads without acknowledgement, matching list receipts, and revert cleanup with a changed revision.

GitOrigin-RevId: e7f6ddfbb1fc2fa95d1ef2876d7f0f7b9f99ff65
2026-10-09 01:22:30 +00:00
jif 2fdf047c96 Fix idle cleanup for disconnected multi-agent v2 children (#52081)
## Why

Persisted multi-agent v2 children created after the client disconnects need lifecycle listeners so their runtimes can be released once idle. Listener attachment must also wait for an earlier runtime's cleanup to finish.

## What changed

- Start lifecycle listeners for persisted v2 children even when no initialized clients are connected, and defer attachment until pending unload cleanup completes.
- Keep ephemeral children loaded so follow-up tasks retain their model context.
- Raise the default `thread_unload_delay_secs` from 60 seconds to 1800 seconds (30 minutes), updating the configuration documentation and sample.

## Testing

Add regression coverage for children spawned after disconnect with zero- and one-second unload delays, including waiting for pending cleanup, keeping running threads loaded, releasing idle runtimes, and preserving an ephemeral child's earlier reply across a follow-up. Update configuration tests for the new default and explicit overrides.

GitOrigin-RevId: 2fecc77b5c03237aa8fd7eacc4bd119b99d36e0a
2026-10-08 11:52:25 +00:00
jif d2a6dac20b Preserve agent state when unloading idle multi-agent v2 children (#52034)
## What changed

- Route app-server idle unloading of multi-agent v2 children through local eviction that preserves unread mail and environment selections for reload.
- Hold residency pins through input submission so newly reloaded children cannot be evicted before receiving their work.
- Guard listener cleanup against replacement runtimes and subscriptions, and reject local eviction of host-controlled threads.

## Testing

Add coverage for unsubscribe-driven eviction followed by `followup_task`, verifying restored conversation context and exactly-once delivery of queued mail. Extend tests to check that residency pins block eviction and that rejected host eviction preserves the thread and mailbox.

GitOrigin-RevId: e36723730af683d2081e495e637f0e6d23548b8d
2026-10-08 09:39:33 +00:00
jif 946bb6c583 Finish idle thread unloads after slow shutdown (#51420)
## Why

Idle thread unloading stopped waiting after ten seconds, leaving the thread loaded when resource cleanup took longer.

## What changed

Keep awaiting the same shutdown future after the ten-second deadline, emitting `codex.app_server.thread_shutdown_slow` as a warning. Once shutdown completes, remove the thread and send `thread/closed` without requiring another unload request.

## Testing

Add a regression test that blocks writer-lock cleanup past the warning deadline, verifies the thread remains loaded while cleanup is blocked, then releases the lock and checks for `thread/closed` and an empty loaded-thread list.

GitOrigin-RevId: 18c411f86fd69e710e472a4e3572d75f18682147
2026-10-06 17:10:19 +00:00
Yuzhu Shen 1ab8c6ef28 Fix thread unloading after disconnect during MCP startup (#50380)
## Why

If required MCP startup outlasts the requesting connection's cleanup, the newly created thread can remain loaded without subscribers because its listener is never started.

## What changed

Start the thread listener even when the requesting connection has already closed, allowing normal idle unloading to clean up the thread while preserving the `ConnectionClosed` result.

## Testing

Add a regression test that disconnects during blocked required MCP startup, then releases startup and verifies a `thread/closed` notification, an empty `thread/loaded/list`, and MCP process exit on Unix.

GitOrigin-RevId: 4ad912a82177266316042d806f5faa1713a48fa4
2026-10-02 17:40:09 +00:00
Charlie Marsh 05a93a8a1d Avoid cloning excluded turn items during thread resume (#46511)
## Why

Resuming a running thread with a summary initial page cloned every active-turn item before discarding those outside the summary, including potentially large reasoning payloads.

## What changed

- Snapshot the active turn using the requested `TurnItemsView`, copying only the first user message and final agent message for summaries and no items for `NotLoaded`.
- Preserve full snapshots when the resume response includes full turns, even if its initial page requests a summary.
- Share item selection between snapshots and page responses, and skip selection when a turn already has the requested view.

## Testing

Add unit coverage for summary selection, completed-turn metadata, and metadata-only snapshots. Extend running-thread resume coverage for default summaries, explicit full pages, and full turns returned alongside a summary page.

GitOrigin-RevId: 6ddcd93c44dbc8083827ac82f442b31730a4edde
2026-09-18 23:37:11 +00:00
Charlie Marsh dfb265764d Avoid cloning active turn items for metadata-only thread resumes (#46510)
## Why

Resuming a running thread cloned the active turn's items even when the response did not need them.

## What changed

Add a metadata-only turn snapshot and use it for resume status checks and initial pages with `TurnItemsView::NotLoaded`. Only snapshot items when including turns or requesting an initial page with items, preserving item data for `Summary` and `Full` views.

## Testing

Add lifecycle coverage for metadata snapshots and extend running-thread resume tests to check metadata-only responses and populated `Summary` and `Full` pages.

GitOrigin-RevId: 281a2b7e1faf3a3fc07e007a065b16235eda08b6
2026-09-18 23:33:08 +00:00
Charlie Marsh 17baabd01b Separate thread startup metadata from replay history (#46294)
## Why

`CodexThread` retained the full `SessionConfiguredEvent`, including initial replay messages, and cloned it even for callers that only needed thread or session IDs.

## What changed

Store `ThreadStartupMetadata` without replay history and expose it by reference through `startup_metadata()`. Update callers to read IDs from this metadata. When rejoining a running thread, rebuild the startup response using the supplied resume history for `initial_messages`.

## Testing

Extend the resume test to verify that rejoining reuses the loaded thread, preserves startup metadata, and still returns the original replay messages as a prefix.

GitOrigin-RevId: 9d656063088c28f091d29e5862e4b9d894b7642c
2026-09-17 19:55:20 +00:00
Eric Traut 91d54f1667 Restore collaboration mode when resuming threads (#45519)
## Why

Resuming a thread initialized its collaboration mode to Default, losing the saved Plan mode and its developer instructions. Reconnecting clients also lacked a server-reported mode to reconcile changes made by another client.

## What changed

- Restore the saved collaboration mode from the latest matching `ThreadSettingsApplied` event, falling back to the last legacy `TurnContext`. Apply the effective model and reasoning effort while retaining the saved mode and developer instructions.
- Include `collaborationMode` in `thread/resume` responses and update the generated schemas and bindings.
- Use the restored mode in the TUI, including the first prompt after resume. Prefer the server's mode when restoring disconnected input, while preserving the local selection for older servers that omit it.

## Testing

Add regression coverage for persisted and legacy collaboration modes, model and reasoning-effort overrides, the resumed Plan mode display and first prompt, and reconnect behavior with and without a server-reported mode.

GitOrigin-RevId: ed064516e7fae1c1668152ab448f510cbcacfe06
2026-09-14 21:33:45 +00:00
Owen Lin 3052bbcf8c Remove the deprecated thread/rollback API (#44915)
## What changed

Remove `thread/rollback`, its request and response types, generated bindings, and the core `Op::ThreadRollback` operation. Requests now follow the generic unknown-method rejection path. Document `thread/revert` as the alternative for paginated threads.

Keep historical `ThreadRolledBack` markers and legacy error deserialization so existing rollouts remain compatible with replay and migration.

## Testing

Adapt retained-context and Guardian history tests to append legacy rollback markers and resume threads, preserving coverage of surviving instructions, answers, and review history.

GitOrigin-RevId: b3da1becdf86b1869275aacb0ffc2817cee5af2e
2026-09-11 21:32:04 +00:00
Matthew Zeng c62d191c4c Expose disabled plugin settings in the app-server API (#44905)
## What changed

- Accept `disabledPluginIds` in `thread/settings/update` and `turn/start`. A supplied list replaces the saved selection; omission or `null` preserves it, and `[]` clears it.
- Return the selection in thread start, resume, and fork responses and `thread/settings/updated` notifications. Update generated schemas and client types.
- Restore disabled plugin IDs from the history retained at the requested fork boundary, preserving explicit overrides.

The selection persists across resume but does not yet filter plugin capabilities.

## Testing

Add coverage for replacing, preserving, and clearing selections without inference; turn-start notifications; resume; and fork boundaries across legacy and paginated history, with loaded and restarted parents. Add a core regression test for history restoration and explicit clearing.

GitOrigin-RevId: 654a8c2a0527228d422c0dd4919228447e2663db
2026-09-11 20:27:37 +00:00
Eric Traut 102e1763b9 Keep app-server thread RPCs active until delegated work completes (#43950)
## Why

Thread startup, running-thread resume, and rollback could outlive their request handlers, allowing connection draining and request serialization to finish too early.

## What changed

- Await thread startup and listener completion for running-thread resume and rollback.
- Release the thread-list permit before waiting for resume completion so the listener can finish rollback responses.
- Release pending rollback waiters when clearing a listener, and discard queued requests when their connection closes.

## Testing

Add coverage for releasing queued requests and rollback waiters, and pipeline resume during rollback to check that both responses complete. Update diagnostics expectations to allow startup to remain in flight after its response is queued.

GitOrigin-RevId: 138156179afbc40de29965820db7fa9cd3669cc7
2026-09-09 00:25:02 +00:00
Eric Traut 5e26f7621c Make the app-server thread unload delay configurable (#42320)
## What changed

- Add the top-level `thread_unload_delay_secs` configuration key for the
  app-server, with a 60-second default instead of the previous 30-minute
  fixed delay.
- Allow zero-delay unloading and reject values that cannot fit in a
  monotonic-clock deadline.
- Reset the unload countdown when a thread gains a subscriber or becomes
  active, and preserve active turns even when the configured delay is zero.
- Document that unloading ephemeral threads discards their in-memory state.

## Testing

- Cover configuration parsing, defaults, overrides, and overflow rejection.
- Cover unsubscribe, resubscribe, delayed and immediate unloading,
  notifications, persisted-thread resume, and WebSocket disconnect behavior.

GitOrigin-RevId: edd46f6b49bbdafbf606bff74378e8f569e95977
2026-09-02 18:18:54 +00:00
guinness-oai 64c9cde458 Record realtime conversation history in Core (#41924)
## Why

Realtime history should be recorded consistently for every Core host, including
when no app-server event listener is attached.

## What changed

- Move transcript segmentation, session boundaries, and backing-agent artifact
  promotion into Core for paginated threads.
- Persist canonical realtime items through the thread store in event order and
  emit dedicated history lifecycle events for hosts to present.
- Translate those Core events into the existing app-server realtime item
  notifications without app-server persisting the items a second time.

## Testing

- Cover Core-only persistence across repeated sessions, ephemeral sessions,
  accepted and rejected steering, typed input ordering, and artifact promotion.
- Verify app-server notifications correspond to the persisted timeline.

GitOrigin-RevId: 7cbef14129d77f6d6d7099b733be91f5279c55f5
2026-08-31 22:38:23 +00:00
guinness-oai 41ca88e95c Persist realtime events in the thread timeline (#40508)
## Why

Clients need a bounded view of realtime conversations that preserves the order of speech, agent work, and turn lifecycle events without loading the full thread history.

## What changed

- Persist realtime session boundaries, transcript segments, and promoted agent items in paginated thread rollouts.
- Add the experimental `thread/timeline/list` API to page ordinary items, realtime items, and turn boundaries in canonical rollout order. Include `activeRealtimeSessionAtPageStart` so each page can be rendered independently.
- Emit typed `thread/realtime/item/started`, `thread/realtime/item/transcript/delta`, and `thread/realtime/item/completed` notifications around durable realtime items.
- Keep the existing `thread/items/list` response and legacy thread history behavior unchanged.

## Testing

- Cover mixed timeline pagination, realtime-session state at page boundaries, rollout replay, subagent history prefixes, transcript streaming, steering, and promoted agent artifacts.

GitOrigin-RevId: faa2c420192d63e060e8fb32446e3c797422656e
2026-08-25 01:03:20 +00:00
rhan-oai d21794d6ba Reload Multi-Agent V2 children through their parent (#40477)
## Why

Multi-Agent V2 children are owned by their parent at runtime. Resuming an
unloaded child directly could rebuild it from caller-provided settings instead
of the parent's current authority.

## What changed

- Route child reloads through their loaded immediate parent and reject direct
  cold resume when that parent is unavailable.
- Preserve the child's recorded model, provider, reasoning effort, and role
  while inheriting the parent's execution policy and MCP extensions.
- Validate cached environments against the parent, intersect local permission
  profiles, and reject stale remote or workspace-root configurations.
- Keep paginated cold-resume token usage attribution when attaching to the
  reloaded child.

## Testing

Add coverage for owner validation, nested-parent reloads, permission and
environment changes, role instruction restoration, paginated usage replay,
and app-server child resume behavior.

GitOrigin-RevId: 84014762bcdb57f94fc23974c9c1af55dc1babf1
2026-08-24 20:47:10 +00:00
xli-oai 0cc80b8db5 Support turn cost telemetry for custom model providers (#39785)
## What changed

- Route turn-cost queries for non-OpenAI providers through the configured
  provider endpoint and authentication, while retaining the existing OpenAI
  API-key path and excluding Amazon Bedrock.
- Observe turns only when their model provider matches the worker's provider.
- Retry custom-provider authentication failures during periodic availability
  probes and ensure client authentication takes precedence over provider
  headers.

## Testing

- Add coverage for provider matching, custom-provider authentication retries,
  ChatGPT-auth rejection, and header precedence.

GitOrigin-RevId: 04a7b28e8e3e18a510ae6fface5193af40d114c0
2026-08-20 19:28:32 +00:00
xli-oai 04caa22c82 Emit API-key turn cost telemetry from app-server (#39006)
## What changed

- Track app-server turn lifecycle events for OpenAI API-key sessions when OTLP telemetry export is enabled.
- Query the turn-cost endpoint after completed or aborted turns and emit `codex.turn_cost` with the estimated USD cost, interruption state, model, speed, and reasoning effort.
- Preserve provider organization and project headers, wait for all observed responses to be priced, and retry pending or transiently failed queries with bounded tracking.
- Re-probe availability after authentication changes and shut the worker down with other app-server background tasks.

## Testing

- Cover late API-key login, transient probe failures, telemetry after thread removal, and turns with multiple responses.
- Verify turn-cost requests use API-key authentication and provider scope headers and deserialize priced response data.

GitOrigin-RevId: a303162a54798b554fcb2e9e2b372e1cf9f49f39
2026-08-17 13:20:58 +00:00
efrazer-oai 9341b38310 Add experimental thread queue APIs to app server (#38456)
## What changed

- Add experimental `thread/queue/add`, `list`, `update`, `delete`, `reorder`, and `start` requests for persistent queued user submissions.
- Automatically dispatch queued submissions in FIFO order after completed or failed turns, while leaving the queue paused after interruption.
- Emit `thread/queue/changed` notifications and expose paginated queue contents with stable submission and client message IDs.
- Export the new protocol types through the generated JSON Schema and TypeScript bindings.

## Testing

- Add app-server integration coverage for capability gating, CRUD, pagination, capacity limits, notifications, automatic dispatch, cold-thread resume, interruption, and explicit queue starts.

GitOrigin-RevId: 89caf5577192199bda77cf6c372a59d26cdd910e
2026-08-13 23:38:02 +00:00
Owen Lin 4343b2bdc4 Add app-server support for reverting paginated threads (#38440)
## What changed

- Add the experimental `thread/revert` request, which replaces a loaded paginated thread's durable history with the prefix before `beforeTurnId` while preserving the thread ID.
- Interrupt any active turn, reload the replacement history without unloading the thread, preserve mutable thread settings, and emit `thread/reverted`.
- Return backward pagination cursors for the retained turns and items, reject stale rollout paths after a revert, and document that local file changes are unaffected.

## Testing

- Cover history replacement, pagination, stale paths, missing turns, subsequent turns, active-turn interruption, and preserved thread settings.

GitOrigin-RevId: 435a8163ad4f310cc47219b436f6bdde21f2b83a
2026-08-13 21:45:02 +00:00
efrazer-oai bc8b25ea02 Add durable user-message queue dispatch (#37204)
## What changed

- Add a storage-neutral queue extension for listing, adding, editing, reordering, deleting, and explicitly starting queued user messages.
- Dispatch queued messages in FIFO order when a thread becomes idle after a completed turn, while leaving them queued after interruptions or failures.
- Wait for user input to be persisted before removing its queue item, and report hook rejection, task termination, and persistence failures distinctly.
- Snapshot local image and audio attachments into validated data URLs before storing them, enforce input limits, and emit `ThreadQueueChanged` events after mutations.

## Testing

- Cover queue mutations, automatic and explicit dispatch, interruption and failure behavior, hook rejection, invalid records, input limits, and local attachment snapshotting.

GitOrigin-RevId: 14f39d7c8cbaccb767a6da84b82cced0edfcb4b9
2026-08-06 03:11:42 +00:00
Felipe Coury 8bfa49e350 Paginate transcript history in the TUI (#36948)
## Why

Paginated threads should not require the TUI to load and render their entire
history when a session is resumed.

## What changed

- Hydrate a bounded initial history page for paginated resumes and forks,
  respecting the configured terminal reflow row budget.
- Load older transcript items in the background as users navigate upward, while
  preserving the viewport, live output, and backtrack selection.
- Reconcile review-mode prompts across page boundaries and keep restored token
  usage hidden until it can be attributed to a persisted turn.

## Testing

- Added coverage for paginated resume and fork hydration, older-page merging,
  transcript prepending, backtrack behavior, review prompt filtering, and token
  usage restoration.

GitOrigin-RevId: 42160b943a9f6a1ba0cfe4b195c24ddf410b747d
2026-08-04 18:59:32 +00:00
Charlie Marsh 1ae2b9880e Avoid cloning rollout history when truncating forks (#35982)
## What changed

- Consume owned rollout vectors when truncating fork and subagent history, reusing their allocations instead of copying slices.
- Project ephemeral fork responses and token-usage ownership before moving the source history into the new thread.
- Preserve canonical `beforeTurnId` validation for rolled-back and synthetic legacy turns.

## Testing

- Add truncation tests covering rolled-back and synthetic legacy `beforeTurnId` values.

GitOrigin-RevId: 5785d004b74d7e057b82379a172676b023bcfb4f
2026-07-29 15:41:39 +00:00
felixxia-oai 5dd992acd3 Route extension warnings to app-server threads (#35168)
## Why

Extension warnings need a stable thread target even when they are emitted outside an active turn.

## What changed

- Add `ExtensionWarning` and a dedicated `ExtensionEventSink::emit_warning` path with an explicit thread ID and optional turn ID.
- Deliver extension warnings as thread-scoped app-server `warning` notifications, preserving listener ordering and waiting briefly for a subscriber when no listener is active.
- Move skills warnings to the new path, limiting provider catalog warnings to four messages of 256 bytes each.

## Testing

- Cover warning routing, ordering, UTF-8-safe truncation, subscriber fallback, and invalid thread IDs.
- Verify skills warning bounds and app-server delivery of executor catalog budget warnings.

GitOrigin-RevId: 4dcb1ba7f17c0edb1a9d548f9568e0c734bffc15
2026-07-24 12:28:29 +00:00
Charlie Marsh 1e20272fa5 Avoid cloning thread history for token usage replay (#34361)
## What changed

- Resolve the turn ID for restored token usage before sending resume and fork responses, avoiding a clone of the full thread history.
- Add `ThreadHistoryBuilder::active_turn_id` so attribution can inspect the active turn without materializing its items.
- Stop history reconstruction at the latest persisted `TokenCount` and verify that a later tail turn does not receive the replayed usage.

GitOrigin-RevId: 69c4a9fe04770d1a938470afc933431e61670094
2026-07-20 14:52:42 +00:00
Owen Lin 0fb559f0f6 Support legacy views for paginated thread history (#34085)
## Why

Clients that use full-history resume or request full items in turn pages should
work consistently across legacy and paginated threads.

## What changed

- Materialize complete turns and items from the paginated projection for
  `thread/resume` and `thread/turns/list` when requested.
- Support `initialTurnsPage` during paginated resume, including merging a live
  active turn while preserving the requested page size and cursors.
- Keep metadata-only resume behavior and backwards cursors available through
  `excludeTurns: true`.

## Testing

- Extend app-server coverage for full paginated history, full item views, and
  initial resume pages on running threads.

GitOrigin-RevId: e340f44d631ce135895c95d7e1baa7bfecd86e71
2026-07-19 00:01:51 +00:00
Owen Lin da61f7d8e1 Enable paginated thread history in app-server (#33364)
## What changed

- Allow `thread/start` to select experimental `historyMode: "paginated"` when the thread store supports turn and item pagination.
- Resume paginated threads with bounded model context and require `excludeTurns: true` instead of loading full history.
- Return `turnsBackwardsCursor` and `itemsBackwardsCursor` from `thread/resume` so clients can hydrate durable history through `thread/turns/list` and `thread/items/list` while newer records arrive through live notifications.
- Reject operations that still require full history, including `initialTurnsPage`, `thread/read` with `includeTurns: true`, `thread/rollback`, and detached review.
- Make item cursors independent of the optional `turnId` filter.

## Testing

- Cover paginated thread start, cold and live resume, backwards cursor hydration, metadata reads, unsupported operations, and stores without pagination support.

GitOrigin-RevId: c42f9b268e29d049843330f474fa3ecde12f5b8a
2026-07-15 15:29:13 +00:00
Dylan Hurd 1c9f890c05 Expose exact per-response usage in raw app-server events (#32985)
## Why

Clients that account for individual upstream Responses API calls need the exact
usage for each completion rather than accumulated or estimated thread usage.

## What changed

- Add the opt-in `rawResponse/completed` app-server notification with the
  `threadId`, `turnId`, `responseId`, and upstream `usage` payload.
- Emit the transient event for normal turns and compaction requests when
  `experimentalRawEvents` is enabled. The event is not persisted or replayed,
  and `usage` is `null` when the upstream response omits it.
- Export the notification in the app-server schemas and document its behavior.

## Testing

- Cover exact token breakdowns for turn and compaction completions.

GitOrigin-RevId: 49040a060bf1784c1ff1dd62112efb4a778428c4
2026-07-14 06:39:42 +00:00
pakrym-oai 393f64565a Scope runtime workspace roots to execution environments (#32952)
## Why

Multi-environment turns can run against different filesystems, so a single
thread-wide workspace-root list cannot accurately describe every selected
environment.

## What changed

- Add optional `runtimeWorkspaceRoots` to each app-server environment selection.
  Omitted roots default to that environment's `cwd`, while an empty list selects
  no roots.
- Carry environment-native roots through turn selection, model-visible context,
  permission materialization, and tool sandboxing.
- Keep the top-level `runtimeWorkspaceRoots` field as a compatibility input when
  constructing default environment selections; explicit selections use their
  own roots.

## Testing

- Cover thread and turn API defaults, overrides, empty roots, and schema
  round-tripping.
- Verify foreign roots remain model-visible and remote execution applies the
  target environment's roots before sandbox selection.

GitOrigin-RevId: c6e7a62883c45c676c153055f46a55ee15a323a4
2026-07-14 04:41:08 +00:00
Owen Lin 2342b2c2a6 feat(rollout): persist TurnItems for paginated thread rollouts (#30188)
## Description

This PR makes new threads with `history_mode = "paginated"` persist
`ItemCompleted(item: <turn_item>)` in their rollout JSONL file.

Legacy threads keep persisting the existing legacy events. Because the
format is selected per thread, a rollout is either legacy or paginated;
we do not need to support mixed rollouts containing both
representations.

This PR depends on [#31473](https://github.com/openai/codex/pull/31473).

## Why

Paginated thread history needs stable turn/item IDs and completed item
snapshots so the later SQLite projector can materialize appended rollout
JSONL without rebuilding the whole thread.

Keeping the legacy persistence policy unchanged avoids changing
historical rollouts or the readers that still consume them.

## What changed

- Made rollout filtering history-mode aware. Paginated threads keep
completed canonical `ItemCompleted` events and drop their redundant
legacy projections; legacy threads keep the existing event set.
- Made forks inherit the source thread history mode, so copied legacy
history is never filtered as paginated.
- Made paginated threads assign IDs to locally-created response items
even when `Feature::ItemIds` is off, and reject streamed output items
that arrive without server IDs.
- Updated legacy turn replay, rollout list/search, and SQLite metadata
extraction to understand completed canonical user-message items.
2026-07-08 19:55:03 -07:00
Owen Lin ba5dd1fd3a feat(core): emit canonical hook prompt items (#31630)
## Description

This PR moves hook prompts onto the canonical `TurnItem` lifecycle in
core.

Stop hooks now record their `ResponseItem` through the existing
lifecycle path, which emits `ItemStarted` and `ItemCompleted`.
App-server consumes those events directly instead of deriving a hook
prompt from `RawResponseItem`.

## Why

Hook prompts were the only `ThreadItem` app-server synthesized from
`RawResponseItem`. This brings them in line with other core-owned turn
items while preserving legacy rollout replay.

## What changed

- Route stop-hook prompts through
`record_response_item_and_emit_turn_item`.
- Materialize canonical hook prompts in `ThreadHistoryBuilder`.
- Remove `RawResponseItem` to `ThreadItem` synthesis while preserving
legacy rollout replay.
- Add focused coverage for lifecycle emission and canonical and legacy
history materialization.
2026-07-08 13:42:06 -07:00
alexsong-oai 841f30598c [codex] Attribute app-server analytics by thread originator (#29935)
## Why

Desktop Work threads and regular Codex threads can share the same
app-server connection. App-server analytics currently copy
`product_client_id` from connection metadata for every thread-scoped
event, so Work thread activity is attributed to the Desktop connection
instead of the thread's resolved originator. This prevents analytics
from distinguishing the two products on a shared connection.

## What changed

- Publish the resolved originator after a thread is materialized,
covering new, resumed, forked, and subagent threads.
- Store that originator in the analytics reducer's existing per-thread
state.
- Override only `app_server_client.product_client_id` for thread, turn,
tool, review, goal, guardian, and compaction events while preserving the
connection's client name, version, and transport metadata.
- Fall back to the connection-wide product client ID when a thread has
no originator override.
- Preserve persisted originators in thread initialization analytics for
resume and fork flows.

## Validation

- `just test -p codex-analytics
thread_originator_overrides_shared_connection_across_thread_events
subagent_events_keep_thread_originator_with_explicit_turn_connection`
- `just test -p codex-app-server
turn_start_tracks_thread_originator_in_analytics
thread_start_tracks_thread_initialized_analytics
thread_fork_tracks_thread_initialized_analytics
thread_resume_tracks_thread_initialized_analytics`
- `just test -p codex-core thread_manager`
2026-06-25 18:15:48 -07:00
Shijie Rao df1199fddb [codex] Add Ultra reasoning effort (#29899)
## Why

Ultra should be one user-facing reasoning selection for work that
benefits from both maximum reasoning and proactive multi-agent
delegation. Without it, clients must coordinate maximum reasoning with
the experimental `multiAgentMode` setting, even though the inference
backend still expects its existing `max` effort value.

This change makes reasoning effort the source of truth: clients select
`ultra`, core derives proactive multi-agent behavior when the turn is
eligible for multi-agent V2, and inference requests continue to use the
backend-compatible `max` value.

## What changed

- Add `ultra` as a first-class reasoning effort and preserve
model-catalog ordering when exposing it to clients.
- Convert `ultra` to `max` at the inference request boundary, including
Responses HTTP/WebSocket requests, startup prewarm, compaction, and
memory summarization.
- Derive effective multi-agent mode per turn from effective reasoning
effort:
  - eligible multi-agent V2 + `ultra` → `proactive`
  - eligible multi-agent V2 + any other effort → `explicitRequestOnly`
- V1 or otherwise ineligible sessions → no multi-agent mode instruction
- Keep the derived effective mode in turn context history so successive
turns can emit a developer-message update only when the effective mode
changes.
- Remove selected multi-agent mode from core session configuration, turn
construction, thread settings, resume/fork restoration, and subagent
spawn plumbing. Subagents inherit reasoning effort and derive their own
effective mode.
- Retain the experimental app-server `multiAgentMode` fields for wire
compatibility while marking them deprecated. Request values are accepted
but ignored; compatibility response fields report `explicitRequestOnly`.
- Display Ultra in the TUI using the order supplied by `model/list`.

## Validation

- `just test -p codex-core ultra_reasoning_uses_max_for_requests`
- `just test -p codex-tui model_reasoning_selection_popup`
2026-06-24 20:13:52 -07:00
Shijie Rao 7abfcf220b Expose thread-level multi-agent mode (#28792)
## Why

Once multi-agent mode can be selected per turn, clients also need to
choose the initial selection when creating a thread and observe that
selection through lifecycle and settings APIs.

The selected value is intentionally distinct from the effective
model-visible value: no client selection is represented as `null`, even
though an eligible multi-agent v2 turn derives `explicitRequestOnly` as
its effective default.

## What changed

- Add the optional experimental `thread/start.multiAgentMode` parameter
and pass it through thread creation.
- Preserve an omitted initial value as an unset selection rather than
eagerly storing `explicitRequestOnly`.
- Apply an explicit `thread/start` selection to the first turn through
the session configuration established at thread creation.
- Restore the latest persisted effective mode as the selected baseline
on cold resume when rollout history contains one.
- Inherit the optional selected mode from a loaded parent when creating
related runtime threads.
- Return the current selected `multiAgentMode` from `thread/start`,
`thread/resume`, `thread/fork`, and thread settings, using `null` when
no mode is selected.
- Keep lifecycle reporting independent from model capability and feature
eligibility; core turn construction remains responsible for calculating
and persisting the effective mode.

## Not covered

- Clearing an existing loaded-session selection back to unset through
`turn/start`; omitted or `null` currently retains the session's
selection.
- A TUI control, slash command, or `config.toml` preference.

## Verification

- `CARGO_INCREMENTAL=0 just test -p codex-app-server-protocol`
- `CARGO_INCREMENTAL=0 just test -p codex-app-server multi_agent_mode`

The focused app-server coverage verifies explicit `thread/start`
initialization, first-turn prompting, nullable reporting for an omitted
selection, and retention of selections that are not currently
runtime-eligible.

## Stack

Stacked on #28685. This PR contains only the thread initialization and
lifecycle/settings API layer.
2026-06-19 10:50:44 +02:00
pakrym-oai f3c1283411 Pair thread environment settings (#26687)
## Why

Thread cwd and environment selections are a single logical setting in
core: updating one without the other can silently desynchronize the
next-turn execution context. This change makes that relationship
explicit in the internal thread settings flow while preserving the
existing app-server public API shape.

## What changed

- Moved the cwd/environment pair through internal
`ThreadSettingsOverrides.environment_settings` instead of a top-level
internal `cwd` field.
- Kept `thread/settings/update` public params unchanged, with app-server
translating top-level `cwd` into the paired internal settings shape.
- Moved `Op::UserInput` environment overrides into thread settings so
user turns and settings updates use the same core path.
- Updated core, app-server, MCP, memories, sample, and test callsites to
construct the paired settings shape.

## Verification

- `git diff --check`
- Local test run starting after PR creation.
2026-06-08 13:55:15 -07:00
Eric Traut 479a14cf59 [2 of 2] Finish moving goal runtime to extension (#26548)
## Stack

1. [#26547](https://github.com/openai/codex/pull/26547) - [1 of 2] Align
goal extension with core behavior
2. [#26548](https://github.com/openai/codex/pull/26548) - [2 of 2] Move
goal runtime to extension

## Why

This PR completes the switch of the goal behavior to the
extension-backed runtime and removes the old core goal implementation.

## What Changed

- Installs the goal extension for app-server `ThreadManager` sessions.
- Routes app-server thread goal `get`, `set`, and `clear` through
`GoalService`.
- Uses thread-idle lifecycle emission after goal resume and snapshot
ordering so the extension can decide whether to continue the goal.
- Forwards extension goal updates through a FIFO async app-server
notification path so backpressure does not drop them or reorder updates.
- Keeps review turns from enabling goal runtime behavior.
- Plans extension tools before dynamic tools so built-in goal tool names
keep their old precedence when goals are enabled.
- Removes the old core goal runtime, core goal tool handlers, and core
goal tool specs.
- Updates tests that were coupled to the core-owned goal runtime while
leaving the legacy `<goal_context>` compatibility path in core for old
threads.
- Removes the stale cargo-shear ignore now that `codex-goal-extension`
is used by the workspace.
- Keeps realtime event matching exhaustive after removing the old
goal-specific realtime text path.


## Validation

- Ran manual `/goal` runs in TUI. Validated time accounting matched
wall-clock time and goal lifecycle state transitions.
2026-06-05 14:17:30 -07:00
Brent Traut 2a1158b8e2 feat(app-server): include turns page on thread resume (#23534)
## Summary

The client currently calls `thread/resume` to establish live updates and
immediately follows it with `thread/turns/list` to hydrate recent turns.
This lets `thread/resume` return that page directly, eliminating a round
trip and the ordering/deduplication gap between the two calls.

Experimental clients opt in with `initialTurnsPage: { limit,
sortDirection, itemsView }`. The response returns `initialTurnsPage` as
a `TurnsPage`, including cursors for paging further back in history.
Keeping the controls in a nested opt-in object provides the useful
`thread/turns/list` knobs without spreading page-specific parameters
across `thread/resume`.

## Verification

- `just fmt`
- `just write-app-server-schema --experimental`
- `just write-app-server-schema`
- `cargo test -p codex-app-server-protocol`
- `cargo test -p codex-app-server
thread_resume_initial_turns_page_matches_requested_turns_list_page
--tests`
- `cargo test -p codex-app-server
thread_resume_rejoins_running_thread_even_with_override_mismatch
--tests`
- `just fix -p codex-app-server-protocol -p codex-app-server`
2026-05-28 09:18:13 -07:00
Eric Traut 771a4e74ac Add thread/settings/update app-server API (#23502)
## Why

App-server clients need a way to update a thread's next-turn settings
without starting a turn, adding transcript content, or waiting for turn
lifecycle events. This gives settings UI a direct path for durable
thread settings while clients observe the eventual effective state
through a notification.

This is a simplified rework of PR
https://github.com/openai/codex/pull/22509. In particular, it changes
the `thread/settings/update` api to return immediately rather than
waiting and returning the effective (updated) thread settings. This
makes the new api consistent with `turn/start` and greatly reduces the
complexity of the implementation relative to the earlier attempt.

## What Changed

- Adds experimental `thread/settings/update` with partial-update request
fields and an empty acknowledgment response.
- Adds experimental `thread/settings/updated`, carrying full effective
`ThreadSettings` and scoped by `threadId` to subscribed clients for the
affected thread.
- Shares durable settings validation with `turn/start`, including
`sandboxPolicy` plus `permissions` rejection and `serviceTier: null`
clearing.
- Emits the same settings notification when `turn/start` overrides
change the stored effective thread settings.
- Regenerates app-server protocol schema fixtures and updates
`app-server/README.md`.
2026-05-20 11:03:20 -07:00
jif-oai 7ee7fe239f chore: isolate thread goal storage behind GoalStore (#23295)
## Why

Thread goal persistence is being prepared for a dedicated storage
boundary. Before that split, goal-specific reads, writes, accounting,
and cleanup were exposed directly on `StateRuntime`, so core and
app-server callsites stayed coupled to the full runtime instead of a
goal-specific store.

This PR introduces that boundary without changing the goal wire API or
current persistence behavior. Callers now go through
`StateRuntime::thread_goals()` and the new `GoalStore`, while
`GoalStore` still uses the existing state DB pool underneath.

## What changed

- Added `GoalStore` in `state/src/runtime/goals.rs` and exposed it from
`StateRuntime` via `thread_goals()`.
- Moved thread-goal reads, writes, status updates, pause, delete, and
usage accounting onto `GoalStore`.
- Updated core session goal handling, app-server goal RPCs, resume
snapshots, and goal tests to use the store boundary.
- Kept thread deletion responsible for cascading goal cleanup by
deleting the goal through the store only after a thread row is removed.

## Testing

- Existing goal persistence, resume, and accounting tests were updated
to exercise the new `GoalStore` access path.
2026-05-18 14:47:05 +02:00
Michael Bolin 83bbb4f326 app-server: stop returning thread permission profiles (#22792)
## Why

The app-server thread lifecycle API should no longer expose the full
`PermissionProfile` value. After the permissions-profile migration,
clients should round-trip only the active profile identity through
`activePermissionProfile` and `permissions` when that identity is known.

The full profile is server-side config. Treating a response-derived
legacy sandbox projection as a new local profile can lose named-profile
restrictions and accidentally widen permissions on the next turn. The
legacy `sandbox` response field remains only as the
compatibility/display fallback.

## What Changed

- Removed `permissionProfile` from `ThreadStartResponse`,
`ThreadResumeResponse`, and `ThreadForkResponse`.
- Stopped populating that field in app-server thread start/resume/fork
responses.
- Updated embedded exec/TUI response mapping to derive display
permission state from local config or the legacy sandbox fallback
instead of a response profile value.
- Added a TUI turn override shape that distinguishes preserving server
permissions, selecting an active profile id, and sending a legacy
sandbox for an explicit local override.
- Preserved remote app-server permissions across turns by sending
`permissions` only when an `activePermissionProfile` id is known, and
otherwise sending no sandbox override unless the user selected a local
override.
- Kept embedded `thread/resume` hydration server-authored when
`activePermissionProfile` is absent, which matches the live-thread
attach path where the server ignores requested overrides.
- Updated the app-server README to remove the obsolete lifecycle
response `permissionProfile` reference. The remaining
`permissionProfile` README references are request-side permission
overrides.
- Regenerated app-server JSON schema and TypeScript fixtures.
- Kept the generated typed response enum exempt from
`large_enum_variant`, matching the existing payload enum exemption after
the lifecycle response variants shrank.

## How To Review

Start with `codex-rs/app-server-protocol/src/protocol/v2/thread.rs` to
confirm the response shape, then check the response construction in
`codex-rs/app-server/src/request_processors`. The generated schema and
TypeScript fixture changes are mechanical follow-through from the
protocol removal.

The TUI behavior is the delicate part: review
`codex-rs/tui/src/app_server_session.rs` for response hydration and
turn-start override projection, then
`codex-rs/tui/src/app/thread_routing.rs` for the decision about whether
the next turn should preserve the server snapshot, send an active
profile id, or send a legacy sandbox for an explicit local override.

## Verification

- `just write-app-server-schema`
- `cargo test -p codex-app-server-protocol
thread_lifecycle_responses_default_missing_optional_fields`
- `cargo test -p codex-exec
session_configured_from_thread_response_uses_permission_profile_from_config`
- `cargo test -p codex-tui --lib thread_response`
- `cargo test -p codex-tui turn_permissions_`
- `cargo test -p codex-tui
resume_response_restores_turns_from_thread_items`
- `cargo test -p codex-analytics
track_response_only_enqueues_analytics_relevant_responses`
- `just fix -p codex-analytics`
- `just fix -p codex-app-server-protocol`
- `just fix -p codex-tui`
- `just argument-comment-lint`

---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/22792).
* #22795
* __->__ #22792
2026-05-15 12:45:48 -07:00