## Why
Guardian can reuse an opaque parent compaction only when the checkpoint's
recorded producer hash matches the scoring model. Missing or incompatible
provenance must not allow an asynchronous score or cached approval to omit
compacted context.
## What changed
- Record the producing model's compatibility hash on compaction history items
and expose the latest checkpoint hash through conversation snapshots.
- In thread-owned context mode, fail closed and require synchronous review when
checkpoint reuse is enabled but the latest checkpoint is incompatible or has
unknown provenance.
- Reject incompatible checkpoints in the sampler while preserving the legacy
context behavior.
## Testing
Add coverage for local, remote, resumed, compacted, and rolled-back histories,
including matching, mismatched, missing, and empty compatibility hashes.
GitOrigin-RevId: 1ce959cb276edc602fd7dfa442133c7fe5e53857
## What changed
- Include the complete upstream `response.usage` object in
`ResponseUsageMetadata.metadata` while retaining the existing `amount` field.
- Expose the metadata through `rawResponse/completed` notifications and update
the generated JSON and TypeScript schemas.
## Testing
- Cover SSE, WebSocket, turn, and compaction completion paths.
GitOrigin-RevId: 6d79b1b4c92ac11abf43ec98f380b4868288a701
## Why
Resumed threads need to continue per-turn and per-thread token totals without scanning arbitrarily far beyond the latest compaction checkpoint.
## What changed
- Add durable `TokenUsageRecord` rollout items with response, turn, thread, session, and root-turn attribution.
- Restore accumulated usage on resume and snapshot the latest record plus the compaction response ID in compaction checkpoints.
- Preserve root-turn lineage in persisted turn context, while ensuring forked child threads start with their own usage totals.
## Testing
- Cover usage accumulation across multiple responses and resumed turns.
- Cover local and remote compaction checkpoints, invalid remote compaction output, rollout reconstruction, and fork isolation.
GitOrigin-RevId: ef9e0c4a9102a08a2c382be4cdac68c84353c90a
## What changed
- Store the most recently captured execution `StepContext` in the active turn state.
- Keep speculative model-fallback captures from replacing that context, and update it only when the fallback is selected for remote compaction.
- Preserve the retained context when a turn completes or is interrupted.
## Testing
- Add coverage for completed and interrupted turns, successful primary compaction, fallback compaction, and interruption while a fallback is running.
GitOrigin-RevId: c7dee36ec8e88ba2f4fb4a9c0322e5c86a0caf0e
## What changed
- Add a positive `output_token_limit` setting to each entry under an MCP server's `tools` configuration.
- Apply the most restrictive limit when plugin and user policies overlap, while keeping approval policy independent.
- Carry the effective MCP output budget in conversation history so tool output, post-tool hook responses, and resumed sessions use the same truncation limit.
## Testing
- Cover configuration parsing, serialization, schema validation, and plugin policy merging.
- Cover MCP output below and above the configured limit, post-tool hook responses, and session resume.
GitOrigin-RevId: d0beb4fca9ba6055d9e1d31c137373b465d50d61
## What changed
- Share the root thread's current service tier across its agent tree.
- Apply that tier to each subagent request when the selected child model supports it, including remote compaction requests.
- Propagate root service-tier updates to existing and newly spawned subagents without rewriting child-owned settings.
- Remove per-spawn and role-level service-tier overrides so routing remains controlled by the root thread.
## Testing
- Cover supported and unsupported child models, full-history forks, live tier changes, remote compaction, new subagents, and reloaded subagents.
GitOrigin-RevId: 6c299e224e90b41a558788fb7aa4d5f336fa383a
## What changed
- Parse optional `usage_metadata.amount` values from Responses API completion events without converting their string representation.
- Propagate the metadata through SSE and WebSocket streams, regular turns, and remote compaction.
- Add `usageMetadata` to the app-server `rawResponse/completed` notification and its generated schemas.
## Testing
- Cover missing, null, zero, and high-precision amounts, including completion events emitted during compaction.
GitOrigin-RevId: b16d81e8350eab0dc7ff613d0dc9dbed73041713
## Why
Thread settings can change while a turn is running, and delegated review or
compaction steps can select a different model. Each request needs a consistent
set of settings resolved against the model that will execute it.
## What changed
- Capture immutable resolved settings for each model step, including model
metadata, reasoning options, service tier, approvals, and personality.
- Keep in-flight steps on their captured settings while applying thread updates
to subsequent work.
- Resolve inherited reasoning summaries and filter service tiers against the
selected model and feature configuration for review and compaction steps.
## Testing
Added coverage for settings updates during paused turns, model changes,
previous-model compaction, review-model defaults, and service-tier filtering.
GitOrigin-RevId: fca4b3b305379f1bf26a5cac75ee0dd1747ac1f6
## Why
Remote compaction's retained-message budget counted text but not images, so
image-heavy history could retain more context than the budget represented.
## What changed
- Add the opt-in `compaction_image_budget` feature to charge retained images
using the existing image size estimate.
- Keep images and their adjacent labels atomic when truncating a boundary
message, while preserving existing text, audio, metadata, and
client-authored developer-message behavior.
- Stop backfilling older messages when an image at the truncation boundary
does not fit.
## Testing
- Add unit coverage for image boundaries, annotations, audio, text-only
messages, and client-authored developer messages.
- Add an integration test covering repeated remote compaction with the feature
enabled, disabled, and left at its default.
GitOrigin-RevId: 88868bbd09d2858939dbae3f8402e1f497b4e3c6
## Why
Rebuilding a truncated message from its content fields can drop passthrough
metadata and leave positional content classifications out of sync with the
items that remain.
## What changed
- Convert message content and its classifications into `AnnotatedContent`
before applying the token budget, then write both back together.
- Preserve other message and envelope metadata during truncation.
- Assign the `unknown` classification to content from legacy messages that do
not provide classifications.
## Testing
Expanded truncation coverage to verify that retained text and image items keep
their matching classifications and that unrelated passthrough metadata is
preserved.
GitOrigin-RevId: f84e2ebc3e97392e45f321dec3c171713864bdcd
## What changed
- Exclude descendant-authored `MESSAGE` progress updates from the history retained after remote compaction v2.
- Continue retaining descendant-authored tasks while omitting progress updates and final answers from subsequent requests.
## Testing
- Extend the remote compaction follow-up test to cover descendant progress updates and descendant-authored tasks.
GitOrigin-RevId: 3e3e60bb108d27e99e2a4c222eed4b9d02fd3829
## Why
Client-authored developer instructions should remain available after a context
window is compacted when `retain_client_developer_messages` is enabled.
## What changed
- Preserve annotated client-authored developer messages during remote compaction
v2 and local token-budget context resets.
- Apply the existing retained-message token budget when carrying those messages
into the new context window.
## Testing
- Cover enabled and disabled retention for remote compaction v2, manual
token-budget compaction, and mid-turn automatic compaction.
GitOrigin-RevId: 4cb3782cb86ef46458413a17a3255d0016fd675f
## What changed
- Wrap response items with optional harness-owned metadata while keeping the persisted response payload shape backward compatible.
- Store compacted-history metadata in an aligned sidecar and reject malformed sidecars.
- Carry metadata through history normalization, compaction, resume, fork, truncation, migration, and rollout recording without sending it to model providers.
## Testing
- Cover legacy rollout compatibility, metadata serialization and validation, compaction retention, and resume/fork reconstruction.
GitOrigin-RevId: 17a6127c9048ff984cb4df4d97c0751a38c29f5f
## What changed
- Classify HTTP connection failures separately from other network errors without exposing request URLs.
- For sampling requests, retry connection failures with exponential delays from 5 to 60 seconds and show a `Reconnecting... waiting for network` stream error.
- Preserve the normal stream retry budget while waiting for the provider to become reachable. Keep the existing bounded retry behavior for other retryable errors.
## Testing
- Verify connection errors are classified without leaking URL contents.
- Verify a turn recovers after its provider becomes reachable and still applies the configured retry limit to a subsequent incomplete stream.
GitOrigin-RevId: 646553290c865a1332abd30c4a64ed9266bbfc6f
## Why
Image resize notices describe the history item immediately before them, so
remote compaction must not preserve or discard them independently of that item.
## What changed
- Group recognized `image_resize_notice` developer messages with their source
history items when filtering compacted history.
- Account for each group together when trimming or truncating history to a token
budget, preserving the notice only when its source is retained.
## Testing
- Extend the remote compaction v2 follow-up test to verify that a retained user
image keeps its resize notice, while notices for discarded items and
unrecognized developer notices are omitted.
GitOrigin-RevId: 82bb70a46eb68b07b4601e7fb345921d0789f442
## What changed
- Parse `codex_rollout_budget_units` from completed Responses API usage into
`TokenUsage`.
- Keep the provider-only value out of serialized protocol, JSON schema, and
TypeScript representations.
## Testing
- Cover the value in unit and end-to-end SSE response parsing tests.
GitOrigin-RevId: a53d4202beb9c8985d25894cb10e7c01f20b2a44
## What changed
- Retain bounded, non-completion agent messages in remote v2 compaction history so delegated tasks remain available on follow-up turns.
- Account for encrypted agent-message content when estimating token usage and place restored initial context before the latest real user or agent message.
- Exclude child completion messages from retained follow-up history and strip inherited parent agent messages when forking a child.
## Testing
- Extend compaction, history-estimation, and agent-fork tests to cover encrypted delegated tasks, child completions, context ordering, and fork sanitization.
GitOrigin-RevId: ba4392e9cfdf4ecbcdc9c7d8581d58a8d2d78260
## What changed
- Preserve `encrypted_function_args` on function calls so an empty list can mark plaintext collaboration arguments across request replay.
- Deliver `spawn_agent`, `send_message`, and `followup_task` payloads as structured plaintext agent messages when that marker is present; retain encrypted delivery otherwise.
- Redact plaintext collaboration arguments from tool and communication logs, and omit the metadata when sending requests to non-OpenAI providers.
## Testing
- Cover serialization of empty encrypted-argument metadata and plaintext versus encrypted subagent message delivery.
- Verify plaintext tool arguments are redacted and provider-specific metadata is removed from non-OpenAI requests.
GitOrigin-RevId: 64db98ff0b61a3af2f04ed609292363f2e2362a8
## What changed
- Wrap `CodexErrorDetails` and an optional retry delay in `CodexErr`, allowing any mapped error to preserve server-provided retry timing.
- Generate the payload-free `CodexErrKind` classification alongside the error details and reuse it for analytics.
- Update error handling sites to inspect `CodexErr::details()` while preserving existing display, debug, protocol mapping, and retryability behavior.
## Testing
- Add coverage for legacy debug formatting, error-specific retryability, and retry-delay propagation through API error mapping.
GitOrigin-RevId: d3ab8a305f2a2ee21d0c0a8c8c388b06dda9c59a
## Why
Interrupting a turn while MCP tools were still starting could abort before the
submitted user input was recorded in conversation history.
## What changed
- Build and retain the MCP tool list and tool router as part of each step
snapshot, using the turn cancellation token during tool discovery.
- Record turn hooks and submitted input when cancellation occurs during startup
prewarming, pre-sampling compaction, or first-step capture.
- Reuse the captured router for sampling, compaction, and prompt debugging so
advertised and executable tools share the same request-scoped view.
## Testing
Add coverage that interrupts each affected MCP startup phase and verifies that
the user prompt remains in history before the turn-aborted marker.
GitOrigin-RevId: 6e96c6814357a55545d69b580e28f35933cfadf7
## What changed
- Assign IDs to client-created response items in every session, including streamed items, forked history, compaction results, and non-OpenAI provider requests.
- Preserve server-provided IDs and stable generated IDs across persisted and resumed conversations while continuing to omit unprefixed server IDs from outbound requests.
- Retire `features.item_ids` as a configurable feature. Existing configuration and CLI toggles remain accepted as compatibility no-ops.
## Testing
- Cover default ID assignment, persistence and resume behavior, remote compaction, provider requests, WebSocket requests, and removed-feature compatibility.
GitOrigin-RevId: 7a09e2d07d2606c2ef1e8e2283d36cb555557eae
## What changed
- Pass only compaction message and window metadata into
`Session::replace_compacted_history`.
- Build the persisted `CompactedItem` there after assigning any missing response
item IDs, so its replacement history matches the live history without callers
cloning or assembling it themselves.
GitOrigin-RevId: a6106c33b6d3eef39ac0b563a50e17f6782970dc
## What changed
- Add an `ExtensionData` store to each `StepContext`.
- Pass the step store to context, world-state, turn-input, and tool contributors so extensions can use capabilities bound to the current sampling step.
- Preserve the captured step context when rebuilding initial context during compaction.
GitOrigin-RevId: 51f3105d3a793f60534f122f9be8247c131c2ca5
## Why
Remote compaction can process large histories. Repeatedly estimating and replacing the full history, and cloning it when tracing is disabled, adds avoidable CPU and memory overhead.
## What changed
- Estimate each history item's token count once, update the total as trailing tool outputs are rewritten, and replace history only after all rewrites are selected.
- Preserve unclamped token totals while calculating removed tokens so saturated estimates do not hide overflow.
- Snapshot compaction input history only when rollout tracing is enabled.
- Reuse the v2 request input instead of cloning it before adding the compaction trigger.
## Testing
- Cover enabled and disabled compaction trace contexts through `is_enabled()` assertions.
GitOrigin-RevId: 8de9c9704ba29532fba05430eb204bbae2e8bf83
## What changed
- Add remote and local audio variants to core and app-server user inputs, including generated JSON and TypeScript schemas.
- Preserve audio fields when converting app-server inputs and response content into thread items.
- Keep unsupported audio out of model requests, TUI rendering, text extraction, and truncated tool output, using explicit text placeholders where audio is submitted as a model input.
## Testing
- Cover app-server media conversion, unsupported-input placeholders, TUI omission, and tool-output truncation.
GitOrigin-RevId: 16152bf3af674575ce569764a8b472e032e7569d
## What changed
- Parse `cache_write_tokens` from response input token details and carry it through token usage aggregation.
- Expose `cache_write_input_tokens` in protocol, app-server, exec, and TypeScript SDK usage events, defaulting it to zero for compatibility with older payloads.
- Report cache-write usage in analytics, metrics, traces, and rollout data alongside existing cache-read usage.
## Testing
- Add coverage for response parsing, usage event output, aggregation, analytics, metrics, and telemetry fields.
GitOrigin-RevId: 634e4fbe3086a8923c0ee13b622573e7d9d221bd
## Why
Clients that account for individual upstream Responses API calls need the exact
usage for each completion rather than accumulated or estimated thread usage.
## What changed
- Add the opt-in `rawResponse/completed` app-server notification with the
`threadId`, `turnId`, `responseId`, and upstream `usage` payload.
- Emit the transient event for normal turns and compaction requests when
`experimentalRawEvents` is enabled. The event is not persisted or replayed,
and `usage` is `null` when the upstream response omits it.
- Export the notification in the app-server schemas and document its behavior.
## Testing
- Cover exact token breakdowns for turn and compaction completions.
GitOrigin-RevId: 49040a060bf1784c1ff1dd62112efb4a778428c4
## Why
`TurnContext` stored model and reasoning settings both as resolved turn fields and inside `CollaborationMode`, requiring the copies to stay synchronized when the model changed.
## What changed
- Store the mode and collaboration-mode developer instructions directly on `TurnContext`.
- Reconstruct `CollaborationMode` from those values plus the turn's current model and reasoning effort when callers need it.
- Read the mode directly for plan-mode checks, lifecycle events, and telemetry.
GitOrigin-RevId: b8fccafdd5d5cbccc84bc70d542588087ddf0093
## Why
Compacting a resumed conversation can fail when its previous model is no longer
available. Some model-specific failures, such as a model-not-found response, are
not represented as invalid requests and therefore did not trigger the existing
fallback to the currently selected model.
## What changed
- Share a retry predicate between both remote compaction implementations.
- Fall back to the current model for invalid requests, unexpected statuses,
context-window and usage-limit errors, server failures, and exhausted retries.
## Testing
Add an integration test that resumes a conversation whose previous model returns
model-not-found and verifies that compaction and the next turn use the renamed
model.
GitOrigin-RevId: c3857beae2ce79b50589deb13c2e59710ce14725
## Why
Pre-sampling compaction intentionally uses the previous turn's model
when the compaction compatibility hash changes or when switching to a
model with a smaller context window. This keeps compaction aligned with
the settings that produced the history, but it can block the next turn
when a resumed ChatGPT thread still references a model slug that has
since been retired. The Codex backend rejects that compaction request
before the user's currently selected model gets a chance to sample.
This PR lets those threads recover without changing previous-model
compaction behavior for API-key authentication or custom providers. It
is stacked on #31316, which is a behavior-preserving extraction of the
individual remote compaction attempts; this PR contains the fallback
behavior.
## What changed
- For automatic previous-model compaction, capture the selected model's
request context when using ChatGPT authentication with the OpenAI
provider and the selected model differs from the previous model.
- If the previous-model attempt returns an `InvalidRequest`, retry
compaction once with the selected model for both `/responses/compact`
and Responses Compaction V2.
- Complete history processing, lifecycle events, and token accounting
with the context of the model that successfully compacted the thread.
- If the fallback also fails, return the original previous-model error
so the retry does not change the user-visible failure.
- Record fallback attempts with reason, implementation, and outcome
telemetry.
- Leave API-key authentication, custom providers, same-model turns, and
non-`InvalidRequest` failures on their existing paths.
## Testing
- `just test -p codex-core -E 'test(pre_sampling_compact) |
test(model_unavailable_error)'` (10 tests)
- Added integration coverage for a resumed thread whose model was
renamed, a model downshift using Responses Compaction V2, and API-key
authentication with a custom provider.
## Why
This PR is a behavior-preserving refactor only. It does not add a
fallback, change which model is used for compaction, or otherwise change
compaction behavior. The behavioral change is implemented in the stacked
follow-up, #30319.
Pre-sampling compaction deliberately uses the previous turn's context
when the compaction compatibility hash changes or when switching to a
model with a smaller context window. That preserves the model settings
that produced the history being compacted, but the previous context is
not always usable. For example, a resumed thread can still reference a
model slug that has since been retired, causing compaction to fail
before the currently selected model can sample.
#30319 addresses that failure mode by retrying compaction with the
current turn's selected model when the backend rejects the
previous-model attempt. This PR performs only that preparatory refactor.
## What changed
- Extracted one legacy `/responses/compact` request attempt into
`compact_remote_request.rs`.
- Extracted one Responses-based remote compaction request attempt into
`compact_remote_v2_attempt.rs`.
- Kept hooks, lifecycle events, analytics, window advancement, history
processing and installation, and error behavior unchanged in the
existing orchestration paths.
- Preserved standalone Responses-based compaction's owned client-session
lifetime through lifecycle completion.
## Testing
- `just test -p codex-core -E 'test(remote_compact)'` (22 tests)
## Why
With deferred executors, an environment can become ready between two
sampling requests in the same turn. The model-visible environment
update, advertised tools, and eventual tool execution must all describe
the same request-time view.
Otherwise, a request built while only environment B is ready can
advertise a tool without an `environment_id`; if higher-priority
environment A becomes ready before execution, that call could silently
run in A instead.
This PR is stacked on #29527.
## Design
`run_turn` captures one `Arc<StepContext>` at each sampling-request
boundary. That step owns the request's `TurnContext` and environment
snapshot.
- World-state environment updates and tool planning borrow that same
step.
- `ToolCallRuntime` retains the `Arc` while asynchronous tool calls
execute.
- `ToolInvocation` carries the step to handlers; its temporary `turn`
compatibility field is derived from the same object.
- `ToolRouter` does not retain `StepContext`; it only uses it while
constructing the request's tool set.
- With `DeferredExecutor` disabled, step capture keeps using the
environments frozen at turn start.
Simply: every sampling request gets one consistent picture of its
environments, from what the model sees through where its tool calls run.
## What changed
- Build environment-dependent tool specs from the current request's
`StepContext`.
- Use that same step for unified exec, legacy shell, `apply_patch`,
`view_image`, and `request_permissions` execution.
- Hide environment-backed tools, including `request_permissions`, while
no environment is attached.
- Resolve legacy shell paths and metadata from the selected step
environment instead of the stale turn-start environment.
- Capture explicit steps at non-turn-loop boundaries such as compaction,
prompt debug, and startup prewarm.
- Reconcile prompt-debug history from the same step used to build its
tools.
## Follow-up
- Bind yielded code-mode cells to the tool runtime that created them, so
nested calls made after yielding continue to use the originating
request's `StepContext`.
## Test plan
- `just test -p codex-core
deferred_executor_updates_context_and_tools_after_startup`
- `just test -p codex-core
environment_count_controls_environment_backed_tools`
- `just test -p codex-core
build_prompt_input_includes_context_and_user_message`
## Why
Follow-up to #29249 and its [compaction review
thread](https://github.com/openai/codex/pull/29249#discussion_r3455055101).
During a turn, environment readiness can change between sampling
requests. Inline compaction must render the same model-visible
`WorldState` used by the request it follows. Rebuilding that state
during compaction can observe a newer environment, make replacement
history disagree with what the model saw, and suppress the next
environment update.
## What changed
- Make `run_turn` own the current `Arc<WorldState>` and replace it only
between sampling requests.
- Build each state from an explicitly chosen environment snapshot, diff
deferred-executor steps against the turn-owned state, and retain the
latest state in `ContextManager` only for cross-turn and resume
tracking.
- Pass the exact turn-owned state into inline compaction and explicit
new-context-window replacement.
- Carry that state with
`InitialContextInjection::BeforeLastUserMessage`, so replacement context
and its stored baseline cannot come from different snapshots.
- Remove obsolete state-recapture helpers and ambiguous TurnContext-only
WorldState builders.
- Add an integration test that moves an environment from starting to
ready during a paused turn, triggers compaction, and verifies the next
request receives the readiness update exactly once.
## Test plan
- `just test -p codex-core
deferred_executor_compaction_preserves_then_updates_environment_once`
- `just test -p codex-core process_compacted_history`
- `just test -p codex-core mid_turn_continuation_compaction`
- `just test -p codex-core build_initial_context`
- `just test -p codex-core
ignores_session_prefix_messages_when_truncating`
## Summary
- express remote compaction result handling as an exhaustive match
- preserve the special `TurnAborted` path without emitting a generic
compaction error
- rely on the standard `test_codex` provider setup in the compaction
budget test
Follow-up to review feedback on #28707.
## Testing
- `just test -p codex-core
compaction_budget_exhaustion_aborts_without_error_or_retry`
- `just fmt`
## Description
This PR is a followup to https://github.com/openai/codex/pull/28355 and
starts assigning `internal_chat_message_metadata_passthrough.turn_id` to
durable Responses API items created during a turn.
The goal is that those items keep the `turn_id` that introduced them
when Codex resends stateless HTTP context, reconstructs history for
resume/fork paths, or reuses websocket response state.
## What changed
- Set `internal_chat_message_metadata_passthrough.turn_id` when missing
as response items enter durable history, initial/replacement history,
inter-agent communication history, and local compaction summaries.
- Preserve existing item turn IDs instead of overwriting them during
persistence, resume reconstruction, compaction, forked history, and
websocket incremental reuse.
- Keep `compaction_trigger` fieldless because it is a request control,
not a durable response item.
- Update focused history/request assertions and fixtures for stateless
requests, websocket incrementals, compaction, thread injection, prompt
debug, and related CI coverage.
## Description
This PR cuts Codex over from generic `ResponseItem.metadata` (introduced
here: https://github.com/openai/codex/pull/28355) to
`ResponseItem.internal_chat_message_metadata_passthrough`, which is the
blessed path and has strongly-typed keys.
For now we have to drop this MAv2 usage of `metadata`:
https://github.com/openai/codex/pull/28561 until we figure out where
that should live.
## Why
The rendered `<token_budget>` fragment identifies the thread and current
context window, but it does not expose enough lineage to identify the
first window in the thread or the immediately preceding window. Those
IDs also need to remain stable across compaction, resume, and rollback.
## What changed
- Track first, previous, and current UUIDv7 context-window IDs in
auto-compaction state.
- Render `thread_id`, `first_window_id`, `previous_window_id`, and the
current window ID in the full `<token_budget>` fragment.
- Persist the first and previous window IDs in compacted rollout
checkpoints and restore them during rollout reconstruction.
- Preserve compatibility with older compacted records that do not
contain the new optional fields.
- Update focused state, rendering, reconstruction, rollback, and
serialization coverage.
## Validation
- `just test -p codex-core token_budget`
- `just test -p codex-protocol compacted_item::tests`
- `just test -p codex-core tracks_prefill_and_window_boundaries`
- `just test -p codex-core
reconstruct_history_uses_replacement_history_verbatim`
- `just test -p codex-core
thread_rollback_restores_cleared_reference_context_item_after_compaction`
## Stack
Depends on #28494.
## Description
This PR propagates shared rollout-budget exhaustion through the existing
`CodexErr::TurnAborted` task result.
Each thread records its model usage against the same ledger. Once the
ledger is exhausted, that usage update and all later usage updates
return `TurnAborted`. The task wrapper emits the normal aborted-turn
event and lifecycle instead of completing the turn.
This is intentionally a soft boundary: there is no cross-thread
`Op::Interrupt` fanout. An in-flight thread can finish its current
response before it observes the exhausted ledger, but every thread
aborts at its next usage-accounting boundary.
## Tests
The integration coverage verifies that:
- the response that exhausts the budget aborts its turn;
- a later response also aborts because the shared ledger remains
exhausted; and
- sub-agent usage draws from the same shared ledger; and
- local and remote-v2 compaction abort without retrying or emitting a
generic error.
Local checks:
- `just test -p codex-core
exhausted_budget_aborts_current_and_later_turns`
- `just test -p codex-core subagent_usage_draws_from_the_shared_budget`
- `just test -p codex-core
abort_regular_task_emits_marker_before_turn_aborted`
- `just test -p codex-core
compaction_budget_exhaustion_aborts_without_error_or_retry`
- `just fix -p codex-core`
- `just fmt`
- `git diff --check`
The full workspace test suite was not run locally.
## Why
Remote v2 compaction can return replacement-history items without IDs.
Because replacement history is installed directly, those items bypass
normal history preparation and remain ID-less in later Responses
requests even when the `item_ids` feature is enabled.
## What changed
- Pass the active `TurnContext` into `replace_compacted_history`.
- When `item_ids` is enabled, assign missing IDs before installing and
persisting replacement history.
- Rebuild `CompactedItem` from the prepared history so live and
persisted replacement histories match.
- Add integration coverage requiring IDs on every ID-capable input item
in the initial, remote v2 compaction, and post-compaction requests.
## Test plan
- `just test -p codex-core response_item_ids`
- `just test -p codex-core websocket_v2_test_codex_shell_chain`
- `just test -p codex-core remote_compaction_parity_pre_turn_auto`
- `just test -p codex-app-server
thread_inject_items_adds_raw_response_items_to_thread_history`
## Why
Client-created response items enter history without IDs, so their
identity is lost across rollout persistence and resume. IDs should be
assigned once at the history-recording boundary, while IDs returned by
the server must remain unchanged.
The Responses API validates item IDs using type-specific prefixes.
Locally generated IDs therefore use the matching prefix plus a
hyphenated UUIDv7, keeping them valid while distinguishable from
server-generated IDs. Because this changes persisted history and
provider request shapes, the behavior is opt-in behind the
under-development `item_ids` feature. Compaction triggers remain request
controls whose API shape does not accept an ID.
## What changed
- Register the disabled-by-default `item_ids` feature and expose it in
`config.schema.json`.
- Make supported optional `ResponseItem` IDs serializable and expose
them in the generated app-server schemas.
- When `item_ids` is enabled, assign an ID during conversation-history
preparation if an item has no ID.
- Generate type-prefixed, hyphenated UUIDv7 IDs using the Responses API
item conventions.
- Preserve existing server IDs without rewriting them.
- Persist assigned IDs in rollouts and include them in subsequent
Responses requests.
- Remove the unsupported ID field from `CompactionTrigger` and document
why it has no ID.
- Add integration coverage for enabled ID persistence, preservation of
server IDs, and omission of generated IDs while the feature is disabled.
`prepare_conversation_items_for_history` is the single response-item ID
allocation boundary.
## Test plan
- `just test -p codex-features`
- `just test -p codex-core
response_item_ids_persist_across_resume_and_preserve_server_ids`
- `just test -p codex-core
non_openai_responses_requests_omit_item_turn_metadata`
- `just test -p codex-core
resize_all_images_prepares_failures_before_history_insertion`
- `just test -p codex-protocol`
- `just test -p codex-app-server-protocol`
- `just test -p codex-api azure_default_store_attaches_ids_and_headers`
## Why
The token-budget context currently identifies a context window by its
thread-local sequence number. A UUIDv7 gives the model a stable opaque
identity that remains fixed for a window and rotates when compaction or
`new_context` starts the next one.
## What changed
- Preserve the existing monotonic value as `window_number` and add a
UUIDv7 `window_id` to `CompactedItem`.
- Generate and rotate the UUID with auto-compaction window state,
persist it alongside the number, and reconstruct it on resume and
rollback.
- Accept legacy compacted rollout records where the numeric `window_id`
represented the window number.
- Use the UUID only in token-budget context; existing request headers
and metadata continue using `thread_id:window_number`.
## Testing
- `just test -p codex-protocol compacted_item::tests`
- `just test -p codex-core token_budget`
## Stack
Depends on #28746. This PR implements shared rollout-budget accounting
and model-visible reminders using the configuration defined in #28746.
# Description / Main changes to Core:
`AgentControl` will now be the area where "rollout level" features &
accounting will have to live. It is incorrectly named for this
responsibility, but I think it can hold all the necessary shared state &
features (rollout token budget, mutliple thread interruption
responsibilitym etc)
In this PR, we have one "token ledger" that each thread will subtract
from when sampling. The "charge" will occur when response.completed() is
done and the calculation will be done on the responses api usage
carrier. The calculation will weigh sampling and pre-fill tokens as
specified.
Every time the budget crosses the configured reminder threshold, a
developer message is appended before the thread's next request
This remaining budget will _always_ be restated/reminded after a
compaction event.
Expiration and fan-out interruption will be in the stacked follow-up
(and also live in Agent Control).
## Reminders
"You have weighted {session_tokens_left} tokens left in the shared
session token budget."
The first request in each thread context receives the current remainder.
Later reminders are emitted after aggregate weighted usage crosses a
configured interval. If several intervals are crossed before a thread
sends another request, Core inserts one reminder with the latest
remainder.
Compaction response usage is charged before the next context starts. The
next reminder is appended after the compaction summary, leaving the
initial context content stable.
## Tests
Integration coverage verifies:
- weighted output and non-cached input accounting
- initial and periodic reminders
- shared accounting between a root and sub-agent
- post-compaction remainder and message placement
Local checks:
- `just fmt`
- `just test -p codex-core rollout_budget`
- `git diff --check`
The full workspace test suite was not run locally.
## Why
`ResponseItem` variants do not have a consistent internal ID shape: some
variants carry required IDs, some carry optional IDs, and some cannot
represent an ID at all. The existing fields also use inconsistent serde,
TypeScript, and JSON-schema annotations. A single enum-level access path
is needed before history recording can assign and retain IDs.
This PR establishes that internal model only. It intentionally does not
generate or serialize IDs; allocation and wire persistence are isolated
in the stacked follow-up.
## What changed
- Give every concrete `ResponseItem` variant an `Option<String>` ID
field.
- Apply the same internal-only annotations to every ID field:
`#[serde(default, skip_serializing)]`, `#[ts(skip)]`, and
`#[schemars(skip)]`.
- Add `ResponseItem::id()` and `ResponseItem::set_id()` as the shared
accessors.
- Preserve IDs when history items are rewritten for truncation.
- Adapt consumers that previously assumed reasoning and image-generation
IDs were required.
- Regenerate app-server schemas so the hidden fields are represented
consistently.
The serde catch-all `ResponseItem::Other` remains ID-less because it
must remain a unit variant.
## Test plan
- `cargo check --tests -p codex-core -p codex-api -p codex-rollout-trace
-p codex-image-generation-extension`
- `just test -p codex-protocol`
- `just test -p codex-app-server-protocol`
- `just test -p codex-api -p codex-rollout-trace -p
codex-image-generation-extension`
- `just test -p codex-core event_mapping`
## Why
`TurnContext` had accumulated dead fields and cached projections of
values already owned by its per-turn `Config` or `ModelInfo`. Keeping
both copies made ownership unclear and allowed artificial split-brain
states, such as a compatibility hash differing from the model metadata
it came from.
`Prompt` similarly carried a write-only personality after personality
selection had already been materialized into its base instructions.
This makes the canonical owner explicit: configuration-backed values
come from `config`, model-derived values come from `model_info`, and
prompts contain only data consumed by request construction.
## What changed
- Remove the unused `ghost_snapshot`, `codex_self_exe`, and
`thread_source` fields.
- Remove duplicate `comp_hash`, `truncation_policy`, `features`,
`shell_environment_policy`, `codex_linux_sandbox_exe`, `compact_prompt`,
and `tool_mode` fields.
- Read those values directly from `TurnContext::config` or
`TurnContext::model_info` at their consumers.
- Remove the write-only `Prompt::personality` field and its constructor
assignments.
- Preserve review-turn inheritance of the parent turn's shell policy,
Linux sandbox executable, and compact prompt through the review config.
## Testing
- `cargo check -p codex-core --tests`
## Description
This PR adds an optional `metadata` field to `ResponseItem` for
Responses API calls. Only mechanical plumbing, no actual values
populated and sent yet. Turns out just adding a new field to
`ResponseItem` has quite a large blast radius already.
This change is backwards compatible because `metadata` is optional and
omitted when absent, so existing response items and rollout history
without it still deserialize and requests that do not set it keep the
same wire shape. For provider compatibility, we strip out `metadata`
before non-OpenAI Responses requests so Azure and AWS Bedrock never see
this field.
My followup PR here will actually make use of it to start storing and
passing along `turn_id`: https://github.com/openai/codex/pull/28360
## What changed
- Added `ResponseItemMetadata` with optional `turn_id`, plus optional
`metadata` on Responses API item variants and inter-agent communication.
- Preserved item metadata through response-item rewrites such as
truncation, missing tool-output synthesis, compaction history
rebuilding, visible-history conversion, rollout/resume, and generated
app-server schemas/types.
- Strip item metadata from non-OpenAI Responses requests while
preserving it for OpenAI-shaped requests.
- Updated the mechanical fixture/test construction churn required by the
new optional field.
## What
Introduce a `CodexResponsesMetadata` struct that defines all the core
metadata we send to Responses API. Example fields are `thread_id`,
`turn_id`, `window_id`, etc.
Going forward, `client_metadata["x-codex-turn-metadata"]` will be the
canonical way Codex sends metadata to Responses API across both HTTP and
websocket transports.
For now, we continue to emit the existing top-level HTTP headers and
top-level `client_metadata` fields from the same
`CodexResponsesMetadata` struct for compatibility reasons.
Also, app-server clients who specify additional
`responsesapi_client_metadata` via `turn/start` and `turn/steer` will
have those fields merged into
`client_metadata["x-codex-turn-metadata"]`, but cannot override the
reserved fields that core uses (i.e. the fields in
`CodexResponsesMetadata`).
## Why
Responses API request instrumentation is the source of truth for
downstream Codex analytics that join requests by Codex IDs such as
session, thread, turn, and context window. Before this change, those
values were assembled through several request-specific paths: HTTP
request bodies, websocket handshake headers, websocket `response.create`
payloads, compaction requests, and the rich `x-codex-turn-metadata`
envelope all had their own wiring.
That made metadata propagation easy to drift across API-key/direct
Responses API requests, ChatGPT-auth/proxied requests, websocket
requests, and compaction requests. It also made additions like
`window_id` error-prone because a field could be added to one transport
projection but missed in another.
## What changed
- Added `CodexResponsesMetadata` as the core-owned snapshot for Codex
metadata sent to ResponsesAPI.
- Render `client_metadata["x-codex-turn-metadata"]`, flat
`client_metadata` projections, and direct compatibility headers from
that same snapshot.
- Include the known Codex-owned fields in the turn metadata blob,
including installation/session/thread/turn/window IDs, request kind,
lineage, sandbox/workspace metadata, timing, and compaction details.
- Treat app-server `responsesapi_client_metadata` as enrichment for the
Codex turn metadata blob while preventing those extras from overriding
Codex-owned fields.
- Use the same metadata path for normal turns, websocket prewarm, local
compaction, remote v1 compaction, and remote v2 compaction.
- Keep websocket connection-only preconnect metadata separate so
handshakes carry compatibility identity headers without inventing a fake
turn metadata blob.
## Verification
- `cargo check -p codex-core`
- `just fix -p codex-core`
## Summary
- replace raw compaction `error` analytics with `codex_error_kind` and
`codex_error_http_status_code`
- derive compaction error telemetry from `CodexErr` using the same
`CodexErrKind` mapping and HTTP status helper used by turn events
- remove the pre-compact hook stop reason from the internal compaction
outcome now that it is no longer emitted as raw analytics text
## Why
Compaction `error` was a raw `CodexErr::to_string()` value, which can
carry free-form provider or user-derived text. Structured Codex error
fields preserve useful low-cardinality telemetry without sending the raw
string.
## Validation
- `just fmt`
- `just test -p codex-analytics`
- `just test -p codex-core
compact::tests::build_token_limited_compacted_history_appends_summary_message`
Attempted `just test -p codex-core`; the changed crate compiled, but the
full target failed in unrelated environment-dependent tests such as
missing helper binaries and shell snapshot timeouts.
## Summary
- add nullable `cached_input_tokens` to the compaction analytics event
- populate it from response usage for compaction v2
- leave it `null` for other compaction implementations
This adds visibility into prompt-cache usage for v2 compaction without
changing compaction behavior.
## Testing
- `just test -p codex-analytics`
- `just test -p codex-core
collect_compaction_output_accepts_additional_output_items`
## Why
The model should be able to see bounded context-window budget metadata
when the `token_budget` feature is enabled. The full-window message is
only injected with full context, while normal turns get a smaller
follow-up only when reported usage first crosses a budget threshold.
## What changed
- Added the `TokenBudget` feature flag.
- Added `<token_budget>` developer fragments for full context-window
metadata and current-window remaining tokens.
- Inserted the threshold message during normal turn handling by
comparing token usage before and after sampling, avoiding persistent
threshold bookkeeping.
- Added core integration coverage for full-context-only metadata and
25/50/75 percent threshold messages.
## Verification
- `just test -p codex-core token_budget`
- `git diff --check`
## Why
Compaction window identity is part of session history, not model-client
transport state. Persisting it with the compacted rollout item lets
resumed threads continue from the reconstructed window without keeping
mutable window state on `ModelClient`.
## What changed
- Added `window_id` to `CompactedItem` and stamp it when
`replace_compacted_history` installs compacted history.
- Moved auto-compact window id ownership into `AutoCompactWindow` /
`SessionState`; `ModelClient` now receives the request window id from
callers instead of storing it.
- Returned `window_id` from rollout reconstruction for resume.
Reconstruction uses the newest surviving compacted item's stored
`window_id` when present, and falls back to the legacy compacted-item
count when it is absent.
- Kept fork startup at the fresh default window id and updated direct
model-client tests to pass explicit test window ids.
## Validation
- `cargo check -p codex-core --tests`
felixxia-oai