## What changed
- Accept `fileId` alongside the existing `url` form for app-server image inputs, and forward file references to the Responses API as `file_id`. Update generated schemas and client types.
- Preserve file references, image detail hints, and mixed inline/file image ordering through user-message events, thread history, and rollout migration. Retain file images when truncating tool output.
- Pass file references through image preparation without resolving them, while keeping resize-notice numbering correct. Omit them from unsupported TUI display and Guardian image context.
- Reject image-edit requests whose recent-image window includes a file reference, preventing selection of an older inline image instead.
## Testing
Add coverage for serialization, request and rollout preservation, mixed-image history ordering, incomplete ordering metadata, tool-output truncation, and rejection of unsupported image-edit selections.
GitOrigin-RevId: 6ca20a8577155cc934b720803c3b7b3bffdf972a
## What changed
- Use one renderer for captured requests, raw request bodies, and input items. Show only appended items when inputs extend the previous request and settings match; start a new snapshot window otherwise and explain the boundary.
- Add optional request settings and tool inventory deltas. Normalize volatile values with stable labels, preserve multiline content, and fingerprint omitted content so changes remain visible.
- Migrate existing context snapshots and expand compaction snapshots to include request history.
## Testing
Add focused tests for window boundaries, cache key changes, tool deltas, shared item rendering, stable normalization, and fingerprints. Assert that the follow-up request after pre-turn compaction includes the working directory override.
GitOrigin-RevId: 6445b014081208ea432b1f3e2ac835c5e4955cb0
## Why
Running root threads retained their startup global instructions, so edits to global `AGENTS.md` files did not take effect during an active session.
## What changed
- Reload global instructions when capturing model-request context, including after tools within the same turn. Apply changes without repeating unchanged instructions or rediscovering repository instructions when the environment and trust level are unchanged.
- Preserve the last successful global instructions on read failures, suppress recurring warnings until recovery, and clear instructions when their source is removed or blank.
- Serialize refreshes and allow cancellation without blocking subsequent requests.
- Give new subagents the parent's applied instruction snapshot and update Guardian reviewer reuse to account for refreshed instructions.
## Testing
Add regression coverage for live edits and removal, read failures and recovery, warning suppression, cancellation, subagent inheritance, and Guardian reviewer reuse. Update resume, fork, and compaction tests to verify refreshed instructions.
GitOrigin-RevId: f7e9b399740fa4f45e482c070f6e901ee7cf85cd
## Why
Remote compaction already uses the normal Responses API, leaving the legacy `/responses/compact` runner unused.
## What changed
- Remove the legacy runner, endpoint client, request types, analytics variant, and obsolete test helpers and snapshots.
- Move tool-output trimming and its metadata tests into `compact_remote_history` for reuse by remote compaction v2.
- Consolidate retained-history filtering in the v2 implementation and remove redundant `RemoteCompactionV2` settings from tests.
## Testing
Extend the retained-history unit test to explicitly check that hook prompts survive compaction.
GitOrigin-RevId: 999bb1391aa22c3c281d6a96f98817d6c9e46ed9
## What changed
- Route manual and automatic remote compaction through the v2 implementation.
- Retire the `remote_compaction_v2` feature toggle while continuing to advertise it in the model client beta features header.
- Migrate compaction tests from `/responses/compact` to streamed `/responses` requests with `compaction_trigger` items and remove legacy parity coverage.
## Testing
Add streamed compaction coverage for tool search output trimming and session base instruction token estimates. Update app-server tests to verify started/completed notifications for both local and streamed remote compaction.
GitOrigin-RevId: cea392a6166a9b116ff83708fd9126d5546476e4
## Why
Saved model selections can outlive their bundled catalog entries and still need migration guidance.
## What changed
- Remove `gpt-5.2` and `gpt-5.4-mini` from the bundled model catalog.
- Preserve migration from saved `gpt-5.4-mini` selections to `gpt-5.6-luna` when the old model is absent, while respecting catalog-provided migration metadata.
- Update tests and snapshots to use current models or explicit test fixtures, and remove the obsolete prompt fixture.
## Testing
Extend migration tests to cover catalog metadata precedence, saved selections, replacement model and reasoning-effort persistence, acknowledged prompts, and missing or hidden migration targets.
GitOrigin-RevId: 82beea504cc4a15699677c28a61ec0939af3069c
## Why
With reasoning effort overrides enabled, compaction used the selected effort even when sampling requests still used an earlier pinned effort. Successful compaction also left that old pin active in the new context window.
## What changed
- Use the pinned request effort for local and remote compaction when it matches the model.
- Resolve fallback models' effort without mutating the live pin, preserving it if compaction fails.
- Reset the pin after successful compaction so the next sampling request establishes the selected effort as its baseline without a redundant `configuration_update`.
## Testing
Add regression coverage for remote compaction success and failure, fallback model effort selection, pin preservation during fallback lookup, and compaction after resuming with overrides disabled.
GitOrigin-RevId: 35e666b0c8026553500715ddbe7bef02cd759f4a
## Why
Directly deserializing the flattened `RolloutLine` envelope can reject nested
decimal values, preventing affected paginated sessions from resuming.
## What changed
- Add canonical string, byte, and reverse-scanner helpers that decode rollout
records through `serde_json::Value` before decoding the flattened item.
- Route rollout readers across session discovery, history, migration, search,
thread storage, and transcript previews through those helpers.
- Remove `Deserialize` from `RolloutLine` so new readers cannot bypass the
canonical persistence decoder.
## Testing
Add coverage that resumes a paginated rollout after a token-count record with
a decimal rate-limit value and verifies that ordinal sequencing continues.
GitOrigin-RevId: 49abac1e0751c073daa5a93a840d8a483fd2d013
## Why
Resumed threads need to continue per-turn and per-thread token totals without scanning arbitrarily far beyond the latest compaction checkpoint.
## What changed
- Add durable `TokenUsageRecord` rollout items with response, turn, thread, session, and root-turn attribution.
- Restore accumulated usage on resume and snapshot the latest record plus the compaction response ID in compaction checkpoints.
- Preserve root-turn lineage in persisted turn context, while ensuring forked child threads start with their own usage totals.
## Testing
- Cover usage accumulation across multiple responses and resumed turns.
- Cover local and remote compaction checkpoints, invalid remote compaction output, rollout reconstruction, and fork isolation.
GitOrigin-RevId: ef9e0c4a9102a08a2c382be4cdac68c84353c90a
## What changed
- Default `tools.update_plan.enabled` to `false`; users can explicitly enable it to expose `update_plan`.
- Remove bundled `update_plan` guidance from model, collaboration-mode, multi-agent, compaction, prewarm, and goal-continuation prompts when the tool is disabled.
- Preserve custom base instructions, model catalog instructions, collaboration policies, and user goal text even when they mention planning or `update_plan`.
## Testing
- Cover default and explicitly enabled tool registration, prompt consistency across request paths, and preservation of custom instructions.
GitOrigin-RevId: a53964e6e72f98e2557dd3090fdb7caccb956527
## What changed
- Cover the default, explicitly enabled, and explicitly disabled states of
`tools.update_plan.enabled`.
- Verify prompt tool lists remain consistent across requests, including when
custom base and developer instructions are configured.
- Explicitly enable `update_plan` in existing fixtures that depend on the tool.
GitOrigin-RevId: 3be279cb5195dfc5892c2c8ff6a1461fac879309
## Why
Resuming a thread without an explicit `cwd` should restore that thread's latest
retained setting. Forked history can contain settings copied from another
thread, while compaction can move the latest setting outside the replay window.
## What changed
- Record the owning thread ID on new settings snapshots and only use snapshots
owned by the resumed thread when restoring `cwd`. Older snapshots without an
owner remain readable but do not override the startup `cwd`.
- Checkpoint the current settings after compaction and serialize checkpoints
with settings updates so the retained history contains an accepted, current
snapshot.
- Cover resume behavior across compaction, forks, reverts, legacy histories,
and concurrent settings updates.
GitOrigin-RevId: 1efc9cf55472d6b88c465c4efe44672a232dbf64
## Why
Planning tools for a candidate model must not change the selected model or its
Responses Lite tool inventory. Otherwise, preparing a fallback can overwrite
metadata before the current model's request is sent.
## What changed
- Build tool registries and visible specs from an explicit `ModelInfo`.
- Store the finalized tool mode, code-mode names, namespace inventory, and
terminal and child-management capabilities on `ToolRouter`.
- Publish namespace inventory only after selecting the step's router, and use
that router's mode when dispatching and recording tool calls.
## Testing
Added coverage for candidate-model isolation, fallback compaction metadata,
code-mode worker selection, normalized tool names, and router capabilities.
GitOrigin-RevId: 56df2de8f42c7e93a4c4dcd117720e40a0cca3db
## Why
Thread settings can change while a turn is running, and delegated review or
compaction steps can select a different model. Each request needs a consistent
set of settings resolved against the model that will execute it.
## What changed
- Capture immutable resolved settings for each model step, including model
metadata, reasoning options, service tier, approvals, and personality.
- Keep in-flight steps on their captured settings while applying thread updates
to subsequent work.
- Resolve inherited reasoning summaries and filter service tiers against the
selected model and feature configuration for review and compaction steps.
## Testing
Added coverage for settings updates during paused turns, model changes,
previous-model compaction, review-model defaults, and service-tier filtering.
GitOrigin-RevId: fca4b3b305379f1bf26a5cac75ee0dd1747ac1f6
## What changed
- Represent compaction summaries and Guardian-approved actions as typed contextual fragments.
- Inject subagent notifications through the same fragment path.
- Annotate the resulting request items with `compaction.summary`, `guardian.approved_action`, and `multi_agent.subagent_notification` content kinds while preserving their existing roles and text.
## Testing
- Verify the content-kind metadata on compacted summaries, approved actions, and subagent notifications.
GitOrigin-RevId: 01c93e7f08b2dc1c1e85ba69e82bfb5f56e6cca5
## Why
Local compaction rebuilds preserved user input as a single text item. Carrying
over the original per-item content kinds can leave its annotation metadata out
of sync with the rebuilt content.
## What changed
- Set the rebuilt item's content kind to `user.text` when the original message
had content-kind metadata.
- Preserve the remaining message and harness metadata.
## Testing
- Cover metadata normalization in compacted history.
- Verify persisted replacement history contains the merged text and a matching
`user.text` content kind.
GitOrigin-RevId: cd10b8ec68ef8ec2a272a90182e69875b60dc35c
## What changed
- Remove the legacy `shell_command` handler and runtime, leaving `exec_command`
and `write_stdin` as the shell execution tools.
- Treat legacy `shell_command` model metadata as `unified_exec`, and normalize
legacy user opt-outs so they do not disable command execution. Managed feature
requirements and `shell_tool` can still disable it.
- Preserve shell approvals, sandboxing, zsh-fork support, and output truncation
through the unified execution path.
## Testing
- Cover legacy configuration and model-metadata compatibility.
- Exercise unified shell execution, approvals, truncation, and `apply_patch`
serialization across the app-server and core test suites.
GitOrigin-RevId: 5c2fd6164fc3519cdae4944cb9db276b8467311c
## What changed
- Run local and remote compaction test turns with approvals disabled and an unrestricted permission profile.
- Keep the full test harness available where compaction turns need the configured model and working directory.
- Enable the Windows sandbox for managed-network unified exec process-event coverage.
- Update request snapshots to reflect the resulting `danger-full-access` sandbox metadata.
GitOrigin-RevId: 38eb46ca6ab28246d738e31eab900b2825072c7d
## What changed
- Add `TurnInputRequest` and typed submission results for atomically starting a turn, steering the active turn, or declining input with a specific reason.
- Expose `start_or_steer_turn`, `start_turn_if_idle`, and `steer_turn` on `CodexThread`, and migrate Core consumers to these APIs.
- Make app-server `turn/start` steer an active regular turn and return that turn's ID. Reject incompatible output schemas and non-steerable turns without applying settings or enqueueing input.
## Testing
- Cover concurrent start-or-steer submissions, accepted and rejected settings updates, output-schema compatibility, idle-start rejection, and app-server steering.
GitOrigin-RevId: dd9b5528d76ec650c019e97af420bc13190ea86a
## What changed
- Wrap response items with optional harness-owned metadata while keeping the persisted response payload shape backward compatible.
- Store compacted-history metadata in an aligned sidecar and reject malformed sidecars.
- Carry metadata through history normalization, compaction, resume, fork, truncation, migration, and rollout recording without sending it to model providers.
## Testing
- Cover legacy rollout compatibility, metadata serialization and validation, compaction retention, and resume/fork reconstruction.
GitOrigin-RevId: 17a6127c9048ff984cb4df4d97c0751a38c29f5f
## What changed
- Add `codex-history` for model-history and persisted-rollout domain types, including `RolloutItem`, `RolloutLine`, `CompactedItem`, and initial/resumed history state.
- Re-export the persisted types from `codex-rollout` and update consumers to use the new crate boundary instead of `codex-protocol`.
- Preserve existing rollout serialization, including legacy numeric compacted-window IDs.
## Testing
- Add `codex-history` tests for rollout JSON round trips, compacted-history compatibility, persisted history modes, and multi-agent version selection.
GitOrigin-RevId: 944daa9297ddd231d3aebbdcb05fff4adf8b4e1b
## Why
Legacy thread histories did not use the cross-process writer ownership guard already applied to paginated histories.
## What changed
- Acquire and retain a writer lock whenever a legacy or paginated thread is created or resumed.
- Apply the same ownership checks when archiving or deleting either history mode.
- Release ownership when initialization fails or the active thread shuts down, allowing another process to resume it.
## Testing
- Cover competing create and resume operations for both history modes.
- Cover archive and deletion conflicts for legacy and paginated descendants.
GitOrigin-RevId: 1be79b01966ad2634ed79eb96c0aa0db8eafe2fa
## What changed
- Assign IDs to client-created response items in every session, including streamed items, forked history, compaction results, and non-OpenAI provider requests.
- Preserve server-provided IDs and stable generated IDs across persisted and resumed conversations while continuing to omit unprefixed server IDs from outbound requests.
- Retire `features.item_ids` as a configurable feature. Existing configuration and CLI toggles remain accepted as compatibility no-ops.
## Testing
- Cover default ID assignment, persistence and resume behavior, remote compaction, provider requests, WebSocket requests, and removed-feature compatibility.
GitOrigin-RevId: 7a09e2d07d2606c2ef1e8e2283d36cb555557eae
## What changed
- Model permission instructions as a world-state section keyed by a stable hash of their rendered developer message.
- Re-emit permission context when its model-visible contents change or the retained fragment is missing, while avoiding duplicates when matching instructions already exist in history, including bundled developer messages.
- Normalize CRLF line endings when hashing world-state fragments so equivalent content produces the same snapshot.
## Testing
- Add snapshot coverage for unchanged, changed, absent, and unknown permission state.
- Cover bundled retained permissions and restored world-state baselines during rollout reconstruction.
GitOrigin-RevId: 7d8d212460c1690ca617859f66dcd825987f1e23
## Why
Compacting a resumed conversation can fail when its previous model is no longer
available. Some model-specific failures, such as a model-not-found response, are
not represented as invalid requests and therefore did not trigger the existing
fallback to the currently selected model.
## What changed
- Share a retry predicate between both remote compaction implementations.
- Fall back to the current model for invalid requests, unexpected statuses,
context-window and usage-limit errors, server failures, and exhausted retries.
## Testing
Add an integration test that resumes a conversation whose previous model returns
model-not-found and verifies that compaction and the next turn use the renamed
model.
GitOrigin-RevId: c3857beae2ce79b50589deb13c2e59710ce14725
## Why
Pre-sampling compaction intentionally uses the previous turn's model
when the compaction compatibility hash changes or when switching to a
model with a smaller context window. This keeps compaction aligned with
the settings that produced the history, but it can block the next turn
when a resumed ChatGPT thread still references a model slug that has
since been retired. The Codex backend rejects that compaction request
before the user's currently selected model gets a chance to sample.
This PR lets those threads recover without changing previous-model
compaction behavior for API-key authentication or custom providers. It
is stacked on #31316, which is a behavior-preserving extraction of the
individual remote compaction attempts; this PR contains the fallback
behavior.
## What changed
- For automatic previous-model compaction, capture the selected model's
request context when using ChatGPT authentication with the OpenAI
provider and the selected model differs from the previous model.
- If the previous-model attempt returns an `InvalidRequest`, retry
compaction once with the selected model for both `/responses/compact`
and Responses Compaction V2.
- Complete history processing, lifecycle events, and token accounting
with the context of the model that successfully compacted the thread.
- If the fallback also fails, return the original previous-model error
so the retry does not change the user-visible failure.
- Record fallback attempts with reason, implementation, and outcome
telemetry.
- Leave API-key authentication, custom providers, same-model turns, and
non-`InvalidRequest` failures on their existing paths.
## Testing
- `just test -p codex-core -E 'test(pre_sampling_compact) |
test(model_unavailable_error)'` (10 tests)
- Added integration coverage for a resumed thread whose model was
renamed, a model downshift using Responses Compaction V2, and API-key
authentication with a custom provider.
## Why
With deferred executors, a turn can begin before a remote environment
attaches. AGENTS.md discovery previously ran only during session setup,
so instructions from a later environment never reached the model or the
session instruction sources.
WorldState persistence has now landed, so this uses the durable
model-visible baseline directly instead of carrying a temporary
resume/fork compatibility path.
## What
- Add an `AgentsMdManager` in `SessionServices` to own host
instructions, loaded state, and refresh caching.
- When `DeferredExecutor` is enabled, refresh AGENTS.md when attached
environment selections change and freeze the result in the corresponding
`StepContext`.
- Represent AGENTS.md as a persisted WorldState section for every
session, with bounded initial, replacement, and removal updates.
- Remove duplicate AGENTS.md state and rendering from
`SessionConfiguration` and `TurnContext`.
- Build initial context, per-request updates, and compaction context
from the same step-scoped value.
- On resume and fork, compare current instructions with the restored
WorldState baseline and inject a replacement exactly once when they
differ.
Builds on #29833, #29835, and #29837.
## Tests
- Covers a remote environment becoming ready mid-turn, with AGENTS.md
appearing on the next request exactly once and updating canonical
instruction sources.
- Covers full, unchanged, replaced, and removed AGENTS.md WorldState
rendering.
- Covers changed instructions across cold resume and fork without
duplicate reinjection.
- Covers remote-v2 compaction retaining creation-time instructions in
the live session and cold resume appending one replacement when the
source changed.
- Ran focused `codex-core` AGENTS.md, WorldState, and context-update
test suites.
## Description
This PR cuts Codex over from generic `ResponseItem.metadata` (introduced
here: https://github.com/openai/codex/pull/28355) to
`ResponseItem.internal_chat_message_metadata_passthrough`, which is the
blessed path and has strongly-typed keys.
For now we have to drop this MAv2 usage of `metadata`:
https://github.com/openai/codex/pull/28561 until we figure out where
that should live.
## What
- make Fjord's centralized response-item image preparation unconditional
for new and resumed history
- have local user images and `view_image` outputs always defer decoding
and resizing to that path
- retain `resize_all_images` as an ignored, removed compatibility key
for released clients
- delete the flag-off producer paths and obsolete policy-specific tests
## Why
Centralized preparation is now the intended image path. Keeping the
runtime feature checks also kept two image-processing implementations
alive and allowed client config to select the legacy behavior.
This is a clean replacement for #28975, rebuilt from the latest `main`.
## How
`prepare_response_items` now runs whenever items enter history and
whenever persisted history is reconstructed. Producers emit deferred
image data, so malformed images become the existing model-visible
placeholder instead of failing the session at the producer.
## Test plan
- `just fmt`
- `just fix -p codex-core -p codex-features`
- `just test -p codex-features` — 52 passed
- focused affected `codex-core` set — 20 passed
- `just test -p codex-core handle_accepts_explicit_high_detail` — 1
passed
- full `just test -p codex-core` attempt — 2,723 passed; 88 unrelated
environment failures from read-only `~/.codex` SQLite state and
unavailable integration helper binaries
## Summary
- add a default-on `auto_compaction` feature flag as an internal escape
hatch
- skip pre-turn, model-switch/hash, and mid-turn automatic compaction
when the flag is disabled
- preserve manual `/compact` behavior and surface the existing
context-window error when the provider runs out of room
- add integration coverage for disabled pre-turn and mid-turn compaction
## Motivation
Long-running SPO optimization rollouts need the option to preserve their
full context and fail on context exhaustion instead of entering another
compaction window. This deliberately uses the existing feature-flag
mechanism rather than adding a dedicated public config or app-server
API.
Disable it with:
```sh
codex --disable auto_compaction
```
## Testing
- `just test -p codex-features` — 51 passed
- `just test -p codex-core auto_compaction_feature_disabled` — 2 passed
- `just fix -p codex-core -p codex-features`
- `just write-config-schema`
- `just test -p codex-core` — the new compaction tests passed; the
overall local run had 54 unrelated environment failures, primarily
missing first-party test binaries and shell-snapshot timeouts
## Why
Make it possible to load AGENTS.md from remote exec-servers whose OS is
different than app-server.
## What
- keep `AGENTS.md` discovery and provenance as `PathUri`, with
root-aware parent and ancestor traversal
- expose lifecycle instruction sources as legacy app-server path strings
in events while retaining `PathUri` internally
- preserve and test mixed POSIX and Windows paths in model context and
TUI status output
- cover remote Windows loading end to end by seeding the Wine prefix
through host filesystem APIs
- fix bug in `PathUri`'s parent() implementation that would erase
Windows drive letters
## Why
`ResponseItem` variants do not have a consistent internal ID shape: some
variants carry required IDs, some carry optional IDs, and some cannot
represent an ID at all. The existing fields also use inconsistent serde,
TypeScript, and JSON-schema annotations. A single enum-level access path
is needed before history recording can assign and retain IDs.
This PR establishes that internal model only. It intentionally does not
generate or serialize IDs; allocation and wire persistence are isolated
in the stacked follow-up.
## What changed
- Give every concrete `ResponseItem` variant an `Option<String>` ID
field.
- Apply the same internal-only annotations to every ID field:
`#[serde(default, skip_serializing)]`, `#[ts(skip)]`, and
`#[schemars(skip)]`.
- Add `ResponseItem::id()` and `ResponseItem::set_id()` as the shared
accessors.
- Preserve IDs when history items are rewritten for truncation.
- Adapt consumers that previously assumed reasoning and image-generation
IDs were required.
- Regenerate app-server schemas so the hidden fields are represented
consistently.
The serde catch-all `ResponseItem::Other` remains ID-less because it
must remain a unit variant.
## Test plan
- `cargo check --tests -p codex-core -p codex-api -p codex-rollout-trace
-p codex-image-generation-extension`
- `just test -p codex-protocol`
- `just test -p codex-app-server-protocol`
- `just test -p codex-api -p codex-rollout-trace -p
codex-image-generation-extension`
- `just test -p codex-core event_mapping`
The workspace denies `clippy::expect_used` in production. Although
`clippy.toml` allows `expect` in tests, Bazel Clippy compiles
integration-test helper code in a way that does not receive that
exemption, which encouraged verbose `unwrap_or_else(... panic!(...))`
and equivalent `match`/`let else` forms.
This allows `clippy::expect_used` once at each integration-test crate
root (including aggregated suites and test-support libraries), then
replaces manual panic-based Result and Option unwraps with
`expect`/`expect_err`. Standalone `tests/*.rs` files remain their own
crate roots. Intentional assertion and unexpected-variant panics remain
unchanged, and the production `expect_used = "deny"` lint remains in
place.
The cleanup is mechanical and net-negative in line count.
## Description
This PR adds an optional `metadata` field to `ResponseItem` for
Responses API calls. Only mechanical plumbing, no actual values
populated and sent yet. Turns out just adding a new field to
`ResponseItem` has quite a large blast radius already.
This change is backwards compatible because `metadata` is optional and
omitted when absent, so existing response items and rollout history
without it still deserialize and requests that do not set it keep the
same wire shape. For provider compatibility, we strip out `metadata`
before non-OpenAI Responses requests so Azure and AWS Bedrock never see
this field.
My followup PR here will actually make use of it to start storing and
passing along `turn_id`: https://github.com/openai/codex/pull/28360
## What changed
- Added `ResponseItemMetadata` with optional `turn_id`, plus optional
`metadata` on Responses API item variants and inter-agent communication.
- Preserved item metadata through response-item rewrites such as
truncation, missing tool-output synthesis, compaction history
rebuilding, visible-history conversion, rollout/resume, and generated
app-server schemas/types.
- Strip item metadata from non-OpenAI Responses requests while
preserving it for OpenAI-shaped requests.
- Updated the mechanical fixture/test construction churn required by the
new optional field.
## Why
We already have the machinery to support multiple environments on a
single thread, but we only show the model the contents of `AGENTS.md`
files in the primary environment.
We should show the model all of the relevant project instructions when
we know there's more than one environment.
## Known Gaps
As discussed in the RFC, this implementation:
1. doesn't handle environments being added/removed to/from the thread
after its creation
2. it doesn't enforce an aggregate context budget across environments,
and instead applies the configured project maximum independently to each
environment
## Implementation
- Discover project instructions in environment order with an independent
byte budget per environment and preserve source provenance/order.
- Keep the legacy fragment byte-for-byte when exactly one environment
contributes project instructions; use environment-labeled sections when
two or more environments contribute.
- Freeze the complete rendered fragment in `LoadedAgentsMd`, insert it
directly into requests, and recognize both layouts in contextual and
memory filtering.
- Add exact rendering, independent-budget, source-order,
creation-snapshot, and consumer coverage without changing app-server
schemas.
## Why
Remote compaction v2 is ready to become the default for providers that
already support remote compaction. Leaving it behind an
under-development opt-in keeps eligible sessions on the legacy
remote-compaction path.
This does not broaden provider eligibility: OpenAI and Azure move to v2,
while Bedrock and OSS providers retain their existing local-compaction
behavior.
## What changed
- Mark `remote_compaction_v2` stable and enable it by default.
- Make tests that intentionally cover legacy remote compaction
explicitly disable v2.
- Update parity coverage so v2 exercises the production default and only
legacy mode opts out.
## Verification
- `just test -p codex-core
auto_compact_runs_after_resume_when_token_usage_is_over_limit
auto_compact_counts_encrypted_reasoning_before_last_user
auto_compact_runs_when_reasoning_header_clears_between_turns
responses_lite_compact_request_uses_lite_transport_contract`
## Summary
- snapshot `comp_hash` into `TurnContext` when the turn is created and
use that snapshot as the downstream source of truth
- persist the turn hash in rollout context and recover it into
previous-turn settings during resume and fork replay
- compact existing history with the previous model only when both
adjacent turns provide hashes and the values differ
- record `comp_hash_changed` as the compaction reason
- cover ordinary transitions, resume, and missing-hash compatibility
with end-to-end tests
## Why
History produced under one compaction-compatible model configuration may
not be safe to carry directly into another. Compacting at the turn
boundary converts that history before context updates and the new user
message are added. Persisting the turn snapshot in `TurnContextItem`
makes the same protection work after resuming a rollout.
A missing hash is not treated as evidence of incompatibility. `None →
Some`, `Some → None`, and `None → None` do not trigger compaction; only
`Some(previous) → Some(current)` with unequal values does.
## Stack
- depends on #27532
- #27532 is based directly on `main`
## Testing
- `just test -p codex-core pre_sampling_compact_` — 6 passed
- `just test -p codex-core
turn_context_item_uses_turn_context_comp_hash_snapshot` — passed
- `just fix -p codex-core -p codex-protocol -p codex-analytics -p
codex-models-manager`
## Why
Global instruction behavior spans thread creation, resume, forks,
subagents, and compaction. Characterization coverage is needed before
changing those semantics so preserved history can be distinguished from
newly loaded configuration.
## What changed
- Extends the existing `agents_md` suite with fresh-thread, warning,
resume, fork, and subagent lifecycle coverage.
- Extends the existing `compact` suite with manual, mid-turn, and
remote-v2 compaction coverage.
- Asserts rendered instruction fragments, reported source paths, and
structured request history before and after instruction-file mutations.
## Why
Thread cwd and environment selections are a single logical setting in
core: updating one without the other can silently desynchronize the
next-turn execution context. This change makes that relationship
explicit in the internal thread settings flow while preserving the
existing app-server public API shape.
## What changed
- Moved the cwd/environment pair through internal
`ThreadSettingsOverrides.environment_settings` instead of a top-level
internal `cwd` field.
- Kept `thread/settings/update` public params unchanged, with app-server
translating top-level `cwd` into the paired internal settings shape.
- Moved `Op::UserInput` environment overrides into thread settings so
user turns and settings updates use the same core path.
- Updated core, app-server, MCP, memories, sample, and test callsites to
construct the paired settings shape.
## Verification
- `git diff --check`
- Local test run starting after PR creation.
## Why
Thread settings cwd overrides are expected to be resolved before they
enter core. Keeping this boundary as a plain `PathBuf` made it easy for
core/session code to keep fallback normalization and relative-path
resolution logic in places that should only receive an already-resolved
cwd.
This is intentionally the absolute-cwd-only slice: it does not change
environment selection stickiness or cwd-to-default-environment fallback
behavior.
## What changed
- Changes `ThreadSettingsOverrides.cwd`,
`CodexThreadSettingsOverrides.cwd`, and `SessionSettingsUpdate.cwd` to
use `AbsolutePathBuf`.
- Removes core-side cwd normalization/resolution from session settings
updates.
- Updates affected core/app-server test helpers and callsites to pass
existing absolute cwd values or use `abs()` helpers.
## Validation
Opening as draft so CI can start while local validation continues.
## Summary
- Add `request_kind` values for foreground turn, startup prewarm,
compaction, and detached memory model requests.
- Attach compaction dispatch metadata to local Responses, legacy
`/v1/responses/compact`, and remote v2 compact requests.
- Add the existing logical context-window identifier as `window_id` on
turn-owned model request metadata.
- Keep identity fields optional for detached memory requests, while
still emitting `request_kind="memory"` in non-git/no-sandbox workspaces.
## Root Cause
`x-codex-turn-metadata` has more than one producer. Foreground turns and
compaction requests own a real turn and should carry that turn identity.
Detached memory stage-one requests do not own a foreground turn, so
absent identity fields are valid rather than missing data. Startup
websocket prewarm is also a model request, but it has `generate=false`
and must not be counted as a foreground turn.
`thread_source` or session source identifies where a thread came from
(for example review, guardian, or another subagent). `request_kind`
identifies what the current outbound model request is doing (`turn`,
`prewarm`, `compaction`, or `memory`). A review or guardian thread can
issue either a normal turn request or a compaction request, so source
cannot replace request kind.
## Behavior / Impact
- Ordinary foreground requests send `request_kind="turn"`, their real
identity fields, and `window_id="<thread_id>:<window_generation>"`.
- Startup websocket warmup requests send `request_kind="prewarm"` so
they are not counted as foreground turns.
- Compaction requests send `request_kind="compaction"`, their real
owning turn identity, the existing `window_id`, and
`compaction.{trigger,reason,implementation,phase,strategy}`.
- Detached memory stage-one requests send `request_kind="memory"`
without `session_id`, `thread_id`, `turn_id`, or `window_id`; when no
workspace metadata exists, the kind-only header is still emitted.
- `session_id`, `thread_id`, `turn_id`, and `window_id` remain optional
in the header schema because detached memory requests do not own a
foreground turn or context window.
- `window_id` is not a new ID system: it is copied from the already-sent
`x-codex-window-id` / WS client metadata value at model-request dispatch
time.
- Existing `x-codex-window-id` HTTP/WS emission, value format,
generation advancement, resume behavior, and fork reset behavior are
unchanged.
- `request_kind`, `window_id`, and upstream turn-owned identity fields
remain schema-owned; input `responsesapi_client_metadata` cannot replace
their canonical values.
- No table, DAG, export, app-server API, or MCP `_meta` schema changes
are included.
A compaction attempt stopped by a pre-compact hook issues no model
request and therefore has no request header; its outcome remains in
analytics events. Status, error, duration, and token deltas also remain
analytics fields rather than request-header fields.
Future detached-memory attribution using a real initiating turn ID as
`trigger_turn_id` is intentionally not part of this PR.
## Sync With Main
- Final pushed head `716342e79` is rebased onto `origin/main@0d37db4b2`.
- The metadata conflict came from upstream `#24160`, which added
`forked_from_thread_id` on the same `turn_metadata` surface. Resolution
preserves that field and its protection from client metadata override
alongside this PR's request-kind, compaction, and window-id fields.
- While resolving the overlapping commits, I removed an accidental
recursive model-request overlay and a duplicate detached-memory header
builder before completing the rebase.
## Latency / User Experience Boundary
- Foreground turns perform no new filesystem, git, or network work. New
fields are inserted into metadata already serialized for outgoing
requests.
- Compaction issues the same model/HTTP requests with the same prompt,
model, service tier, and sampling settings; only metadata bytes change.
- Startup prewarm already sent metadata; it is now correctly classified
as `prewarm`.
- Non-git detached memory now sends a small kind-only metadata header
rather than no header.
- This client diff adds no user-visible latency mechanism beyond
negligible serialization and header bytes on already-existing requests.
## Validation
On conflict-resolved head `1d35c2cfb` based on `origin/main@487521733`:
- `just fmt` (passed)
- `just fix -p codex-core` (passed)
- `git diff --check origin/main...HEAD` (passed)
- `just test -p codex-core -E 'test(turn_metadata) |
test(websocket_first_turn_uses_startup_prewarm_and_create) |
test(responses_stream_includes_turn_metadata_header_for_git_workspace_e2e)
|
test(responses_websocket_forwards_turn_metadata_on_initial_and_incremental_create)
| test(remote_compact_v2_retries_failures_with_stream_retry_budget) |
test(window_id_advances_after_compact_persists_on_resume_and_resets_on_fork)'`
(`23 passed`; `bench-smoke` passed)
- `just test -p codex-app-server -E
'test(turn_start_forwards_client_metadata_to_responses_request_v2) |
test(turn_start_forwards_client_metadata_to_responses_websocket_request_body_v2)
| test(auto_compaction_remote_emits_started_and_completed_items)'` (`3
passed`; `bench-smoke` passed)
- `just test -p codex-memories-write` (`29 passed`; `bench-smoke`
passed)
## Summary
- Bump the workspace Rust toolchain from `1.93.0` to `1.95.0` across
Cargo, Bazel, CI, release workflows, devcontainers, and the Codex
environment config.
- Refresh `MODULE.bazel.lock` so the Bazel Rust toolchain artifacts
match the new version.
- Leave purpose-specific toolchains unchanged, including the
`argument-comment-lint` nightly and the upstream `rusty_v8` `1.91.0`
build pin.
- Includes fixes for new lints from `just fix` and a few codex-authored
fixes for lints without a suggestion.
## Summary
Adds experimental `additionalContext` support to `turn/start` and
`turn/steer` so clients can provide ephemeral external context, such as
browser or automation state, without turning that plumbing into a
visible user prompt or triggering user-prompt lifecycle behavior.
## API Shape
The parameter shape is:
```ts
additionalContext?: Record<string, {
value: string
kind: "untrusted" | "application"
}> | null
```
Example:
```json
{
"additionalContext": {
"browser_info": {
"value": "Active tab is CI failures.",
"kind": "untrusted"
},
"automation_info": {
"value": "CI rerun is in progress.",
"kind": "application"
}
}
}
```
The keys are opaque and caller-defined.
## Context Injection
When provided, accepted entries are inserted into model context as
hidden contextual message items, not as visible thread user-message
items.
`kind: "untrusted"` entries are inserted with role `user`:
```text
<external_${key}>${value}</external_${key}>
```
`kind: "application"` entries are inserted with role `developer`:
```text
<${key}>${value}</${key}>
```
Values are not escaped. Each value is truncated to 1k approximate tokens
before wrapping.
For `turn/start`, accepted additional context is inserted before normal
user input. For `turn/steer`, additional context is merged only when the
steer includes non-empty user input; context-only steers still reject as
empty input.
## Dedupe Strategy
`AdditionalContextStore` lives on session state and stores the latest
complete additional-context map.
Each `turn/start` or non-empty `turn/steer` treats its
`additionalContext` as the current complete set of values. Entries are
injected only when the key is new or the exact entry for that key
changed, including `value` or `kind`. After merging, the store is
replaced with the provided map, so omitted keys are removed from the
retained set and can be injected again later if reintroduced.
Omitting `additionalContext`, passing `null`, or passing an empty object
resets the store to empty and injects nothing.
## What Changed
- Threads experimental v2 `additionalContext` through app-server into
core turn start and steer handling.
- Adds separate contextual fragment types for untrusted user-role
context and application developer-role context.
- Uses pending response input items so additional context can be
combined with normal user input without treating it as prompt text.
- Adds integration coverage for start/steer flow, role routing,
dedupe/reset behavior, deletion/re-add behavior, hook-blocked input
behavior, empty context-only steer rejection, external-fragment marker
matching, and truncation.
## Why
`model_auto_compact_token_limit` has only been able to budget the full
active context. That makes it hard to set a small "growth since
compaction" budget for sessions that preserve a large carried window
prefix: the preserved prefix can consume the whole budget and force
immediate repeated compaction.
This PR adds an opt-in `body_after_prefix` scope so callers can apply
`model_auto_compact_token_limit` to sampled output and later growth
after the current carried prefix, while still forcing compaction before
the full model context window is exhausted.
## What changed
- Adds `AutoCompactTokenLimitScope` with the existing `total` behavior
as the default and a new `body_after_prefix` mode:
[`config_types.rs`](https://github.com/openai/codex/blob/973806b1cb35792555bead994cb3ed94656eb171/codex-rs/protocol/src/config_types.rs#L24-L37).
- Threads `model_auto_compact_token_limit_scope` through config loading,
`Config`, `core-api`, and app-server v2 schema/TypeScript generation.
- Records the first observed input-token count for a `body_after_prefix`
compaction window and uses it as the baseline when deciding whether the
scoped auto-compaction budget is exhausted:
[`turn.rs`](https://github.com/openai/codex/blob/973806b1cb35792555bead994cb3ed94656eb171/codex-rs/core/src/session/turn.rs#L743-L781).
- Keeps a hard context-window cap in `body_after_prefix`, so scoped
budgeting cannot let the active context overrun the usable window.
## Verification
Added compact-suite coverage for the two key behaviors:
`body_after_prefix` does not re-compact just because the carried prefix
is larger than the scoped budget, and it still compacts when the total
active context reaches the configured context window:
[`compact.rs`](https://github.com/openai/codex/blob/973806b1cb35792555bead994cb3ed94656eb171/codex-rs/core/tests/suite/compact.rs#L3003-L3128).
**Stack position:** [5 of 7]
## Summary
This PR adds `Op::ThreadSettings`, a queued settings-only update
mechanism for changing stored thread settings without starting a new
turn. It also removes the legacy `Op::OverrideTurnContext` in the same
layer, so reviewers can see the replacement and deletion together.
## Changes
- Add `Op::ThreadSettings` for settings-only queued updates.
- Emit `ThreadSettingsApplied` with the effective thread settings
snapshot after core applies an update.
- Route settings-only updates through the same submission queue as user
input.
- Migrate remaining `OverrideTurnContext` tests and callers to the
queued `Op::ThreadSettings` path.
- Delete `Op::OverrideTurnContext` from the core protocol and submission
loop.
This stack addresses #20656 and #22090.
## Stack
1. [1 of 7] [Add thread settings to
UserInput](https://github.com/openai/codex/pull/23080)
2. [2 of 7] [Remove
UserInputWithTurnContext](https://github.com/openai/codex/pull/23081)
3. [3 of 7] [Remove
UserTurn](https://github.com/openai/codex/pull/23075)
4. [4 of 7] [Placeholder for OverrideTurnContext
cleanup](https://github.com/openai/codex/pull/23087)
5. [5 of 7] [Replace OverrideTurnContext with
ThreadSettings](https://github.com/openai/codex/pull/22508) (this PR)
6. [6 of 7] [Add app-server thread settings
API](https://github.com/openai/codex/pull/22509)
7. [7 of 7] [Sync TUI thread
settings](https://github.com/openai/codex/pull/22510)
**Stack position:** [1 of 7]
## Summary
The first three PRs in this stack are a cleanup pass before the actual
thread settings API work.
Today, core has several overlapping "user input" ops: `UserInput`,
`UserInputWithTurnContext`, and `UserTurn`. They differ mostly in how
much next-turn state they carry, which makes the later queued thread
settings update harder to reason about and review.
This PR starts that cleanup by adding the shared
`ThreadSettingsOverrides` payload and allowing `Op::UserInput` to carry
it. Existing variants remain in place here, so this layer is mostly a
behavior-preserving API shape change plus mechanical constructor
updates.
## End State After PR3
By the end of PR3, `Op::UserInput` is the only "user input" core op. It
can carry optional thread settings overrides for callers that need to
update stored defaults with a turn, while callers without updates use
empty settings. `Op::UserInputWithTurnContext` and `Op::UserTurn` are
deleted.
## End State After PR5
By the end of PR5, core will have only two ops for this area:
- `Op::UserInput` for user-input-bearing submissions.
- `Op::ThreadSettings` for settings-only updates.
## Stack
1. [1 of 7] [Add thread settings to
UserInput](https://github.com/openai/codex/pull/23080) (this PR)
2. [2 of 7] [Remove
UserInputWithTurnContext](https://github.com/openai/codex/pull/23081)
3. [3 of 7] [Remove
UserTurn](https://github.com/openai/codex/pull/23075)
4. [4 of 7] [Placeholder for OverrideTurnContext
cleanup](https://github.com/openai/codex/pull/23087)
5. [5 of 7] [Replace OverrideTurnContext with
ThreadSettings](https://github.com/openai/codex/pull/22508)
6. [6 of 7] [Add app-server thread settings
API](https://github.com/openai/codex/pull/22509)
7. [7 of 7] [Sync TUI thread
settings](https://github.com/openai/codex/pull/22510)
## Summary
- Add optional image detail to user image inputs across core, app-server
v2, thread history/event mapping, and the generated app-server
schemas/types.
- Preserve requested detail when serializing Responses image inputs:
omitted detail stays on the existing `high` default, while explicit
`original` keeps local images on the original-resolution path.
- Support `high`/`original` consistently for tool image outputs,
including MCP `codex/imageDetail`, code-mode image helpers, and
`view_image`.
Krish Chainani