Commit Graph
40 Commits
Author SHA1 Message Date
ioanbudea-OAI ab541fb990 Encapsulate thread metadata construction (#60524)
## What changed

Move `ThreadMetadataBuilder` into its own module, make its fields private, and replace direct field assignments with fluent `with_*` methods. Initialize `updated_at` and `recency_at` from the creation time and default the approval mode to `OnRequest`.

Update `apply_rollout_items` to accept a boxed `ThreadMetadata` seed, keeping callers' async frames small while preserving existing database fields except for the seed rollout path. Add `build_thread_metadata` to use the state runtime's configured provider, and use `thread_id_from_rollout_path` directly for the doctor's filename fallback.

GitOrigin-RevId: a27e4c6b22832e902922dbdd60d7f5c11b4a1a83
2026-11-10 04:56:49 +00:00
cooper-oai 1c414e0d65 Use thread runtime auth for Guardian Decisions and memory quota checks (#60064)
## Why

Guardian Decisions requests used bearer keys directly, bypassing runtime authentication. Memory startup quota checks used the app-level auth manager instead of the thread's provider auth context.

## What changed

- Bind Decisions credentials to runtime authentication and use route-aware HTTP clients with sensitive header protection. Preserve workload identity credentials, reuse the saved-login runtime binding, and isolate explicit API key bindings.
- Select the staging Decisions endpoint when the ChatGPT environment is staging.
- Use the thread's `ProviderAuthContext` for memory startup and quota checks, preserving account-bound network policy and applying runtime authentication.
- Keep custom destinations independent of runtime assertions.

## Testing

Add regression tests for dual-header and assertion-only authentication, fresh Decisions assertions, saved versus explicit credential isolation, workload identity, staging routing, redirect header protection, and memory quota authentication with custom destinations.

GitOrigin-RevId: 27bc7821a1cad1f5d1d2a71680527ea91ace95db
2026-11-07 05:50:00 +00:00
mandal-OAI 02bc25c89f Configure memory consolidation service tiers independently (#57568)
## Why

Memory consolidation inherits the session's service tier. Give background consolidation its own setting, defaulting to Standard.

## What changed

Add `memories.consolidation_service_tier` with a default of `"default"` and apply it to the consolidation agent. Normalize recognized tier aliases, including `"fast"` to `"priority"`, and expose the setting in the configuration schema.

## Testing

Add coverage for configuration parsing and alias normalization, Standard consolidation despite an Ultrafast session tier, and an explicit `"flex"` consolidation override.

GitOrigin-RevId: 58031ad56196152d212531d56e7fe9813f83317e
2026-10-27 21:51:26 +00:00
dhruvgupta-oai 333d7bde0f Bound memory baseline files and recover incomplete snapshots (#57240)
## Why

Memory baseline snapshots need bounded file reads, and recovery must keep existing memory files visible to consolidation rather than treating them as already processed.

## What changed

- Limit baseline files to 10 MiB, skip oversized working files, and check stored blob sizes before reading them.
- Version baselines and replace missing, outdated, or unusable metadata with an empty baseline, preserving working files as additions. Also invalidate the baseline when a tracked file grows beyond the limit.
- Serialize baseline initialization, resets, and diffs with a cross-process lock. Isolate repository access and pin index writes to the baseline worktree.

## Testing

Add coverage for oversized files, legacy baseline migration, interrupted resets, cross-process locking, and consolidation with unchanged inputs after baseline recovery.

GitOrigin-RevId: 34ecc9e5700a88ed37eff83a90f08d8a9b0d9f29
2026-10-26 20:40:53 +00:00
josh-oai c0cfbfbb98 Extend memory extension resource retention to 14 days (#54033)
## What changed

Increase memory extension resource retention from 7 to 14 days before pruning.

## Testing

Update pruning and startup test fixtures for the 14-day retention window, including resources at the exact cutoff.

GitOrigin-RevId: 5044a1262c8806245d90d567cd84b21aa0d3da43
2026-10-14 16:53:24 +00:00
jif 0d58012ec2 Test memory extraction budgets with excluded items and ciphertext (#53963)
Extend phase-one request tests for both memory versions with reasoning,
developer messages, and assistant ciphertext whose combined size exceeds
the reader budget while individual records remain below the record limit.
Verify that earlier extraction evidence and assistant text alongside large
ciphertext remain in the request, with `encrypted_content` excluded.

GitOrigin-RevId: f0e85e368e5a94d491d0aa4ae1688f47d873ccd9
2026-10-14 12:07:20 +00:00
jif 5670b1c18e Bound rollout input for memory extraction (#53940)
## What changed

Read rollout evidence from newest to oldest for both memory versions, retaining up to 8 MiB of sanitized response items and inter-agent communications, then restore chronological order. Skip malformed records and records larger than 1 MiB, and require valid session metadata in the first record within that limit.

## Testing

Add coverage for the retained byte budget, newest-record ordering, malformed and oversized records, and unsupported session metadata. Extend extraction request tests for both memory versions to verify that evidence after an oversized record reaches the model.

GitOrigin-RevId: 55ccbc2477ecb059830d3a55be12bcbc21235303
2026-10-14 10:19:14 +00:00
alishobeiri-oai de8fab6d7a Upgrade rmcp to 3.5.1 and harden lifecycle tests (#52972)
## What changed

- Update the pinned `rmcp` dependency and Cargo/Bazel lockfiles, and migrate MCP handlers and fixtures to `ClientConfig` and `ServerConfig`.
- Stop requiring responses for cancelled elicitation requests in cancellation tests while retaining checks for cleanup and continued connection usability.
- Handle client disconnects during WebSocket writes in the TUI test server, use an asynchronous HTTP mock for loopback proxy testing, and improve remote executor test diagnostics and the Windows subprocess fixture.
- Use `invalid_prompt` for the memory worker failure fixture.

## Testing

Add a regression test covering transport EOF during both form elicitation and user verification. Keep the client alive while checking that pending response channels close, the timeout pause clears, and the service exits with `QuitReason::Closed`.

GitOrigin-RevId: c0166b83fab3b59d84a5207c6630cb6adcf3b230
2026-10-10 19:43:24 +00:00
pakrym-oai 5ef96ab278 Add opt-in output token replay for OpenAI requests (#52742)
## What changed

- Add the disabled-by-default `output_token_replay` feature to request `output.encrypted_content` from OpenAI providers.
- Preserve encrypted message and tool-call output, message and function-call status, and output-text `annotations` and `logprobs` through history and replay. Update the generated protocol schemas and SDK types.
- Strip output ciphertext from requests to other providers while retaining it in stored history, including after replay is disabled. Remove encrypted content from memory extraction inputs.

## Testing

Add integration coverage for the feature/provider combinations, metadata replay across follow-up requests, and history preservation when resuming with replay disabled.

GitOrigin-RevId: 77729a021b6771933402aebe74ece8eb39c057e2
2026-10-10 01:25:00 +00:00
jif 69f7140559 Isolate the memory startup metadata test from Git enrichment (#49000)
## Why

Best-effort workspace enrichment can time out independently of the detached
request metadata checked by the memory startup test.

## What changed

Keep the phase-one service-tier and detached-metadata test fixture outside Git
and assert that `workspaces` is absent. Update the header assertion message to
refer to turn metadata.

GitOrigin-RevId: 9541668fc62a28d1ff3131d7f71b351149b6e152
2026-09-28 14:46:32 +00:00
joeytrasatti-openai 742472c525 Set turn triggers for guardian and memory requests (#44298)
## What changed

Populate `turn_trigger` in request metadata with `guardian_review` for guardian reviews, `guardian_classifier` for classifier requests, and `memory_consolidation` for both detached memory requests and consolidation agent turns.

## Testing

Extend request metadata assertions to cover each trigger across guardian reviews, classifier requests, and both memory startup phases.

GitOrigin-RevId: 2dc3c173ede6563b942b8099ad9901c32967b969
2026-09-09 20:32:57 +00:00
jif 2cbbf0c9b5 Add memory dual writing and v2 readiness reporting (#43827)
## Why

Allow v2 memories to build in the background while the selected memory version continues supplying context, and let clients check whether v2 has enough consolidated data and a valid summary to use.

## What changed

- Add `memories.dual_write`, disabled by default, to run v1 and v2 extraction and consolidation concurrently with separate stores and directories.
- Add the experimental `memory/status` endpoint, returning `v2ConsolidatedThreads` and `v2Ready`. Readiness requires a valid v2 summary and a consolidation thread count meeting `minConsolidatedThreads` (default 20, supported range 1–4096).
- Persist the largest thread count from a successful consolidation across pruning and clear it on memory reset.
- Share v2 summary validation between consolidation and status reporting, and tag memory metrics with `memory_version`.

## Testing

Add tests for independent dual-write outputs, preserving existing notes without importing them into v2, and readiness behavior across consolidation ownership checks, missing summaries, configurable thresholds, thread deletion, and memory reset.

GitOrigin-RevId: 54417cc60bf419062401355c0da8c79119995f61
2026-09-08 14:23:23 +00:00
jif 553df1c691 Add dedicated memory v2 consolidation and read prompts (#43813)
## What changed

- Consolidate v2 rollout summaries into `memory_summary.md` without generating `raw_memories.md` or requiring `MEMORY.md`. Validate the summary's required sections and size below 10,000 UTF-8 bytes.
- Add v2 read instructions for selective history retrieval, evidence-grounded preferences, citations, and explicit memory edits. Split injected instructions into fragments to preserve the complete summary within fragment byte limits.
- Record memory citation usage in the store selected by `memories.version`.

## Testing

Add coverage for v2 consolidation without a handbook, summary validation, version-isolated memory reads, and resetting both memory versions while preserving threads.

GitOrigin-RevId: 1d895fb4a23a973f1a45ba03be07a1f480c10227
2026-09-08 12:58:11 +00:00
jif 74d3a5bf10 Add summary-only extraction for memory v2 (#43800)
## Why

Memory summaries should preserve task history and the scope of user corrections and preferences without turning task-specific instructions into general claims about the user.

## What changed

- Give memory v2 dedicated extraction prompts that emphasize chronological task history, evidence provenance, and uncertainty, with working directory and branch hints.
- Require only `rollout_summary` and `rollout_slug` for v2, store an empty raw memory, and redact secrets before truncating summaries with a 9,000-byte budget.
- Split v2 input into memory context messages while preserving the v1 prompts and output contract.

## Testing

Extend startup tests to check v2 output storage, schema, branch hints, and redaction before truncation. Verify that budgeted input preserves human and agent evidence in order, omits media payloads and lower-priority noise, and keeps message chunks bounded.

GitOrigin-RevId: 89e1c797450bd8f8e192fbe7161b5159bff8f931
2026-09-08 11:25:15 +00:00
jif 6924ce636b Prioritize human evidence in memory v2 extraction (#43799)
## Why

Memory extraction has a limited rollout budget. Select evidence by source so human input and assistant final responses take priority over commentary, context, and tool output.

## What changed

- Use tiered selection for memory v2, preferring newer evidence within each tier and rendering selected items in chronological order with omission markers.
- Keep answered `request_user_input` questions alongside human replies, label evidence sources, replace message media with placeholders, and bound tool output and individual rows.
- Add typed memory context fragments with bounded messages and Unicode-safe extraction chunking.

## Testing

Add tests for memory v2 startup using tiered input and for extraction chunks preserving Unicode evidence within message bounds.

GitOrigin-RevId: 1b2bba4e3264b4fda271e429ad3b7d5f69caed59
2026-09-08 11:12:01 +00:00
pakrym-oai d05e6d5f46 Establish root turn identity for independent tasks and memory requests (#42900)
## Why

Background and empty-input turns could lack a `root_turn_id`, and detached memory requests omitted turn identity entirely. Tasks without a root could also adopt one from coalesced mailbox input.

## What changed

- Set `root_turn_id` to the task's turn ID at task start unless an inherited or recovered root is already present.
- Stop assigning roots from mailbox input or steering an active turn.
- Give detached memory requests a fresh UUID for both `turn_id` and `root_turn_id`, included in turn headers and client metadata while still omitting session and thread identity.

## Testing

Extend assertions for background turns, empty input, goal continuations, manual compaction, and detached memory requests. Exercise mailbox coalescing with both independent and inherited roots to verify the task keeps its established root.

GitOrigin-RevId: ee0f2f1b43542a9f5884f694c1d94341b476779c
2026-09-05 01:04:12 +00:00
pakrym-oai 868c9edb0d Assign stable IDs to generated Responses input items (#41349)
## What changed

- Assign type-prefixed IDs to Guardian classifier input items before sending the request, preserving existing IDs and reusing generated IDs across WebSocket retries.
- Add a prefixed ID to the memory phase-one user message.

## Testing

- Verify retried Guardian requests contain the same unique, prefixed item IDs.
- Verify the memory phase-one request includes a prefixed user-message ID.

GitOrigin-RevId: e01da49d9a661b6bfd2088f091a3ddf69d601e40
2026-08-28 15:10:58 +00:00
jif 7c6eb0eef1 Scope stop hooks for memory consolidation (#40587)
## Why

Background memory consolidation must honor managed completion policies without
running project completion checks or sending user-facing notifications.

## What changed

- Add a memory-consolidation stop target that selects managed policy hooks and
  executor-scoped cleanup while excluding user, project, session, and plugin
  stop hooks.
- End memory consolidation with an error when a managed stop hook rejects it,
  and propagate that turn error into agent status.
- Disable legacy completion notifications for the background memory worker.

## Testing

- Cover hook selection, executor cleanup, managed rejection, agent error
  status, and notification suppression.

GitOrigin-RevId: be350679ddcde2dde8a408b0972f08facff01a37
2026-08-25 10:29:26 +00:00
pakrym-oai 7b5b3bd5a2 Identify detached memory requests as memory consolidation (#40186)
## What changed

- Set `thread_source` to `memory_consolidation` in turn metadata for detached memory requests.
- Verify that startup requests carry matching turn metadata in the request header and nested `client_metadata`.

GitOrigin-RevId: 318d6f097e07463ac7a1c52b35a926532e67b83d
2026-08-23 04:59:28 +00:00
sayan-oai d75c85f651 Separate thread settings from environment configuration (#39597)
## Why

Environment-owned permission profiles and workspace roots must remain effective
without being persisted or restored as thread-owned settings.

## What changed

- Keep thread settings snapshots and restore data separate from the effective
  configuration supplied by the primary environment.
- Use resolved environment permissions, active profiles, workspace roots, and
  network policy when building turn context and session events.
- Preserve environment selections when agents are evicted, resumed, or spawned,
  and validate owner configuration when it becomes ready.
- Start memory initialization only after the primary environment is configured.

## Testing

- Extend remote-environment and multi-agent tests to cover settings persistence,
  owner permission profiles, workspace roots, spawning, eviction, and resume.

GitOrigin-RevId: d82f126981cd0190233020bbdd61d96c70558868
2026-08-20 05:29:12 +00:00
jif a04940cb12 Reject symbolic links in memory workspaces (#39205)
## What changed

- Reject a symbolic link used as the memory root.
- Recursively remove symbolic links before preparing or seeding the memory workspace.
- Remove and reject symbolic links created during consolidation, including when the worker fails, so they cannot affect files outside the workspace.

## Testing

Add Unix startup tests covering pre-existing extension links and links created by both successful and failed consolidation workers.

GitOrigin-RevId: f96e03812773f7beaa16455dd77370ee47d8a71f
2026-08-18 14:14:57 +00:00
rka-oai 3a6f747d77 Preserve harness metadata across conversation history (#38058)
## What changed

- Wrap response items with optional harness-owned metadata while keeping the persisted response payload shape backward compatible.
- Store compacted-history metadata in an aligned sidecar and reject malformed sidecars.
- Carry metadata through history normalization, compaction, resume, fork, truncation, migration, and rollout recording without sending it to model providers.

## Testing

- Cover legacy rollout compatibility, metadata serialization and validation, compaction retention, and resume/fork reconstruction.

GitOrigin-RevId: 17a6127c9048ff984cb4df4d97c0751a38c29f5f
2026-08-11 19:30:12 +00:00
Adam Perry @ OpenAI 63002bdb26 Extract persisted history types into a dedicated crate (#37871)
## What changed

- Add `codex-history` for model-history and persisted-rollout domain types, including `RolloutItem`, `RolloutLine`, `CompactedItem`, and initial/resumed history state.
- Re-export the persisted types from `codex-rollout` and update consumers to use the new crate boundary instead of `codex-protocol`.
- Preserve existing rollout serialization, including legacy numeric compacted-window IDs.

## Testing

- Add `codex-history` tests for rollout JSON round trips, compacted-history compatibility, persisted history modes, and multi-agent version selection.

GitOrigin-RevId: 944daa9297ddd231d3aebbdcb05fff4adf8b4e1b
2026-08-10 19:26:52 +00:00
sayan-oai 4ca25a2c4e Include sandbox mode in response metadata (#37507)
## What changed

- Add the effective permission profile as `sandbox_mode` in turn metadata for regular, prewarm, compaction, and detached memory requests.
- Reserve `sandbox_mode` so client-provided metadata cannot override the computed value.

## Testing

- Cover metadata headers and request bodies for read-only and danger-full-access modes.

GitOrigin-RevId: 13c690cc99bc2691023f281ca527af049bb1286f
2026-08-07 22:41:20 +00:00
jif 5ad367fb95 Wait for memory consolidation shutdown before finishing jobs (#36140)
## Why

A phase-two memory job could release its lease while its consolidation agent was
still shutting down, allowing another worker to race the existing agent.

## What changed

- Await consolidation-agent shutdown before validating artifacts and completing
  the job.
- Keep the lease until it expires if shutdown fails, and remove the agent from
  the thread manager only after shutdown succeeds.

## Testing

Update startup tests to wait for the phase-two job lifecycle to finish and verify
that the consolidation thread has been removed.

GitOrigin-RevId: 7ac3a7369529822ac95fd2ee9dcb27d39c191d20
2026-07-30 11:52:38 +00:00
Adam Perry @ OpenAI c769a05340 Honor the configured SQLite home across state consumers (#34994)
## Why

Codex and SQLite data can use separate home directories, but state consumers
could reconstruct database paths from the Codex home instead of consistently
using the resolved SQLite configuration.

## What changed

- Pass `SqliteConfig` through the core, rollout, state runtime, and thread store
  instead of passing a directory and rebuilding the configuration downstream.
- Use that shared configuration for state, logs, memories, goals, and paginated
  thread-history database access, including integrity checks and cleanup.
- Reject state database handles whose SQLite configuration does not match the
  requesting store.

## Testing

Add coverage with separate Codex and SQLite homes that verifies startup
backfill, thread listing, and paginated history all use the configured SQLite
directory.

GitOrigin-RevId: 1de1cdd1d6ff1d70bbb6c360c8352e6543fb8ebf
2026-07-23 19:19:35 +00:00
jif 54b8f112a3 Preserve parent sandbox enforcement for memory consolidation (#32441)
## What changed

- Pass the parent turn's effective permission profile to the memory consolidation agent, including thread-level permission and legacy sandbox overrides.
- Preserve disabled and externally enforced permission profiles instead of replacing them with a managed sandbox.
- Continue restricting consolidation to the memory root without network access when the parent uses Codex-managed permissions.

## Testing

- Add coverage for disabled, external, and managed parent permission profiles.

GitOrigin-RevId: 9ca3be0e41dc14d858053f6f70e33f4ae7c578e1
2026-07-11 15:20:49 +00:00
Owen Lin 5c19155cbd Add ordinals to paginated rollout records (#32332)
## Why

Paginated thread history needs durable ordering so consumers can process a rollout suffix without rebuilding all earlier history.

## What changed

- Add optional, zero-based ordinals to `RolloutLine` records in paginated rollouts while leaving legacy rollout serialization unchanged.
- Continue ordinals from the last valid record when appending or resuming, including after gaps or an incomplete tail, and reject overflow without appending.
- Add a stateless `project_rollout_line` helper that maps canonical turn lifecycle and completed-item records into thread-history change sets.

## Testing

- Cover ordinal assignment, legacy compatibility, resume and tail recovery, overflow handling, and thread-history projection for completed, failed, and interrupted turns.

GitOrigin-RevId: 3a9bb6cd2a1f674a9f154342e96e9aa3d8330781
2026-07-11 04:15:42 +00:00
jif 52c9605dc7 Validate memory consolidation artifacts before succeeding (#32193)
## Why

A completed Phase 2 agent run does not guarantee that its required outputs were
created correctly. Treating a clean workspace as success can also preserve a
state where those outputs are missing.

## What changed

- Require `MEMORY.md` to be a file and `memory_summary.md` to start with `v1`
  before marking consolidation successful.
- Fail completed runs with invalid artifacts without resetting the workspace
  baseline, allowing the job to be retried.
- Run consolidation for a clean workspace when its required artifacts are
  invalid instead of taking the no-change success path.

## Testing

Added coverage for rejecting an invalid summary and retrying a clean workspace
whose consolidation artifacts are missing.

GitOrigin-RevId: ac57b2ba9d062c5203ca51afd716795a91814bb3
2026-07-10 15:40:35 +00:00
Owen Lin 5b95745eae core: rename metadata -> internal_chat_message_metadata_passthrough (#28968)
## Description
This PR cuts Codex over from generic `ResponseItem.metadata` (introduced
here: https://github.com/openai/codex/pull/28355) to
`ResponseItem.internal_chat_message_metadata_passthrough`, which is the
blessed path and has strongly-typed keys.

For now we have to drop this MAv2 usage of `metadata`:
https://github.com/openai/codex/pull/28561 until we figure out where
that should live.
2026-06-22 11:11:25 -07:00
Owen Lin 040dafa32d feat(core): add metadata field to ResponseItem (#28355)
## Description

This PR adds an optional `metadata` field to `ResponseItem` for
Responses API calls. Only mechanical plumbing, no actual values
populated and sent yet. Turns out just adding a new field to
`ResponseItem` has quite a large blast radius already.

This change is backwards compatible because `metadata` is optional and
omitted when absent, so existing response items and rollout history
without it still deserialize and requests that do not set it keep the
same wire shape. For provider compatibility, we strip out `metadata`
before non-OpenAI Responses requests so Azure and AWS Bedrock never see
this field.

My followup PR here will actually make use of it to start storing and
passing along `turn_id`: https://github.com/openai/codex/pull/28360

## What changed

- Added `ResponseItemMetadata` with optional `turn_id`, plus optional
`metadata` on Responses API item variants and inter-agent communication.
- Preserved item metadata through response-item rewrites such as
truncation, missing tool-output synthesis, compaction history
rebuilding, visible-history conversion, rollout/resume, and generated
app-server schemas/types.
- Strip item metadata from non-OpenAI Responses requests while
preserving it for OpenAI-shaped requests.
- Updated the mechanical fixture/test construction churn required by the
new optional field.
2026-06-15 15:05:28 -07:00
Adam Perry @ OpenAI 5a56caf18c [codex] Remove async_trait from first-party code (#27475)
## Why

First-party async traits should expose their `Send` contracts explicitly
without requiring `async_trait`. This completes the migration pattern
established in #27303 and #27304.

## What changed

- Replaced the remaining first-party `async_trait` traits with native
return-position `impl Future + Send` where statically dispatched and
explicit boxed `Send` futures where object safety is required.
- Kept implementations behavior-preserving, outlining existing async
bodies into inherent methods where that keeps the diff reviewable.
- Removed all direct first-party `async-trait` dependencies and the
workspace dependency declaration.
- Added a cargo-deny policy that permits `async-trait` only through the
remaining transitive wrapper crates.
- Updated `rand` from 0.8.5 to 0.8.6 to resolve RUSTSEC-2026-0097 and
keep the full cargo-deny check passing.

## Validation

- `just test -p codex-exec-server`: 216 passed, 2 skipped.
- `just test -p codex-model-provider`: 39 passed.
- `just test -p codex-core` and `just test`: changed tests passed;
remaining failures are environment-sensitive suites unrelated to this
migration.
- `cargo deny check`
- `just fix`
- `just fmt`
- `cargo shear`
- `just bazel-lock-check`
2026-06-11 18:16:39 -07:00
Celia Chen 51fc4b0559 feat: use provider defaults for memory models (#27129)
## Why

Memory startup used hardcoded OpenAI model slugs for extraction and
consolidation. That works for the default OpenAI-compatible path, but
provider-specific backends can require different model identifiers. In
particular, Amazon Bedrock should use its Bedrock model ID for these
background memory requests instead of the OpenAI `gpt-5.4-mini` /
`gpt-5.4` slugs.

## What Changed

- Added provider-owned preferred memory model methods alongside
`approval_review_preferred_model`.
- Updated memory extraction and consolidation to resolve their default
model through the active `ModelProvider`.
- Added Amazon Bedrock overrides so both memory stages use
`openai.gpt-5.4` through Bedrock’s provider-specific model ID.
- Kept explicit `memories.extract_model` and
`memories.consolidation_model` config overrides taking precedence.
- Added startup coverage for default OpenAI and Bedrock memory model
selection.

#closes #26288
2026-06-09 23:49:09 +00:00
jif-oai d5ec93f379 Move memories root setup out of core config (#24758)
## Why

Config loading should not create or write-authorize the memories root
just because memory support exists. Memory startup is the code path that
actually materializes that tree.

## What

- Stop creating the memories root during Config load and remove it from
legacy workspace-write projections.
- Grant the memories root read access only when the memories feature and
use_memories are enabled.
- Create the memories root inside memories startup before seeding
extension instructions.
- Update config and startup tests around the ownership boundary.

## Tests

- just fmt
- just fix -p codex-core
- just fix -p codex-memories-write
- just test -p codex-core
memory_tool_makes_memories_root_readable_without_creating_or_widening_writes
workspace_write_includes_configured_writable_root_once_without_memories_root
permission_profile_override_keeps_memories_root_out_of_legacy_projection
permissions_profiles_allow_direct_write_roots_outside_workspace_root
default_permissions_profile_populates_runtime_sandbox_policy
- just test -p codex-memories-write memories_startup_creates_memory_root

Note: a broader just test -p codex-core run is not clean in this
sandbox; it hit missing test_stdio_server plus seatbelt, realtime, and
environment-sensitive failures. The changed config tests above pass.
2026-05-28 11:51:24 +02:00
ningyi-oai bee78806a9 [codex] add compaction metadata to turn headers (#24368)
## Summary
- Add `request_kind` values for foreground turn, startup prewarm,
compaction, and detached memory model requests.
- Attach compaction dispatch metadata to local Responses, legacy
`/v1/responses/compact`, and remote v2 compact requests.
- Add the existing logical context-window identifier as `window_id` on
turn-owned model request metadata.
- Keep identity fields optional for detached memory requests, while
still emitting `request_kind="memory"` in non-git/no-sandbox workspaces.

## Root Cause
`x-codex-turn-metadata` has more than one producer. Foreground turns and
compaction requests own a real turn and should carry that turn identity.
Detached memory stage-one requests do not own a foreground turn, so
absent identity fields are valid rather than missing data. Startup
websocket prewarm is also a model request, but it has `generate=false`
and must not be counted as a foreground turn.

`thread_source` or session source identifies where a thread came from
(for example review, guardian, or another subagent). `request_kind`
identifies what the current outbound model request is doing (`turn`,
`prewarm`, `compaction`, or `memory`). A review or guardian thread can
issue either a normal turn request or a compaction request, so source
cannot replace request kind.

## Behavior / Impact
- Ordinary foreground requests send `request_kind="turn"`, their real
identity fields, and `window_id="<thread_id>:<window_generation>"`.
- Startup websocket warmup requests send `request_kind="prewarm"` so
they are not counted as foreground turns.
- Compaction requests send `request_kind="compaction"`, their real
owning turn identity, the existing `window_id`, and
`compaction.{trigger,reason,implementation,phase,strategy}`.
- Detached memory stage-one requests send `request_kind="memory"`
without `session_id`, `thread_id`, `turn_id`, or `window_id`; when no
workspace metadata exists, the kind-only header is still emitted.
- `session_id`, `thread_id`, `turn_id`, and `window_id` remain optional
in the header schema because detached memory requests do not own a
foreground turn or context window.
- `window_id` is not a new ID system: it is copied from the already-sent
`x-codex-window-id` / WS client metadata value at model-request dispatch
time.
- Existing `x-codex-window-id` HTTP/WS emission, value format,
generation advancement, resume behavior, and fork reset behavior are
unchanged.
- `request_kind`, `window_id`, and upstream turn-owned identity fields
remain schema-owned; input `responsesapi_client_metadata` cannot replace
their canonical values.
- No table, DAG, export, app-server API, or MCP `_meta` schema changes
are included.

A compaction attempt stopped by a pre-compact hook issues no model
request and therefore has no request header; its outcome remains in
analytics events. Status, error, duration, and token deltas also remain
analytics fields rather than request-header fields.

Future detached-memory attribution using a real initiating turn ID as
`trigger_turn_id` is intentionally not part of this PR.

## Sync With Main
- Final pushed head `716342e79` is rebased onto `origin/main@0d37db4b2`.
- The metadata conflict came from upstream `#24160`, which added
`forked_from_thread_id` on the same `turn_metadata` surface. Resolution
preserves that field and its protection from client metadata override
alongside this PR's request-kind, compaction, and window-id fields.
- While resolving the overlapping commits, I removed an accidental
recursive model-request overlay and a duplicate detached-memory header
builder before completing the rebase.

## Latency / User Experience Boundary
- Foreground turns perform no new filesystem, git, or network work. New
fields are inserted into metadata already serialized for outgoing
requests.
- Compaction issues the same model/HTTP requests with the same prompt,
model, service tier, and sampling settings; only metadata bytes change.
- Startup prewarm already sent metadata; it is now correctly classified
as `prewarm`.
- Non-git detached memory now sends a small kind-only metadata header
rather than no header.
- This client diff adds no user-visible latency mechanism beyond
negligible serialization and header bytes on already-existing requests.

## Validation
On conflict-resolved head `1d35c2cfb` based on `origin/main@487521733`:
- `just fmt` (passed)
- `just fix -p codex-core` (passed)
- `git diff --check origin/main...HEAD` (passed)
- `just test -p codex-core -E 'test(turn_metadata) |
test(websocket_first_turn_uses_startup_prewarm_and_create) |
test(responses_stream_includes_turn_metadata_header_for_git_workspace_e2e)
|
test(responses_websocket_forwards_turn_metadata_on_initial_and_incremental_create)
| test(remote_compact_v2_retries_failures_with_stream_retry_budget) |
test(window_id_advances_after_compact_persists_on_resume_and_resets_on_fork)'`
(`23 passed`; `bench-smoke` passed)
- `just test -p codex-app-server -E
'test(turn_start_forwards_client_metadata_to_responses_request_v2) |
test(turn_start_forwards_client_metadata_to_responses_websocket_request_body_v2)
| test(auto_compaction_remote_emits_started_and_completed_items)'` (`3
passed`; `bench-smoke` passed)
- `just test -p codex-memories-write` (`29 passed`; `bench-smoke`
passed)
2026-05-27 11:09:33 -07:00
jif-oai aad59a0916 Move memory state to a dedicated SQLite DB (#24591)
## Summary

Generated memory rows and their stage-one/stage-two job state currently
live in `state_5.sqlite` alongside thread metadata. That makes memory
cleanup and regeneration share the main state schema even though those
rows are memory-pipeline data and can be rebuilt independently from the
durable thread records.

This PR moves the memory-owned tables into a dedicated
`memories_1.sqlite` runtime database while keeping thread metadata in
`state_5.sqlite`.

## Changes

- Adds a separate memories DB runtime, migrator, path helpers, telemetry
kind, and Bazel compile data for `state/memory_migrations`.
- Introduces `MemoryStore` behind `StateRuntime::memories()` and moves
memory table/job operations onto that store.
- Drops the old memory tables from the state DB and recreates their
schema in `state/memory_migrations/0001_memories.sql`.
- Updates memory startup, citation usage tracking, rollout pollution
handling, `debug clear-memories`, and app-server `memory/reset` to
operate through the memories DB.
- Preserves cross-DB behavior by hydrating thread metadata from the
state DB when selecting visible memory outputs and checking stage-one
staleness.

## Verification

- Added/updated `codex-state` tests for deleted-thread memory visibility
and already-polluted phase-two enqueue behavior.
- Updated `debug clear-memories`, app-server `memory/reset`, and
memories startup tests to seed and assert memory rows through
`memories_1.sqlite`.
2026-05-26 20:07:25 +02:00
Eric Traut a668379abf [5 of 7] Replace OverrideTurnContext with ThreadSettings (#22508)
**Stack position:** [5 of 7]

## Summary

This PR adds `Op::ThreadSettings`, a queued settings-only update
mechanism for changing stored thread settings without starting a new
turn. It also removes the legacy `Op::OverrideTurnContext` in the same
layer, so reviewers can see the replacement and deletion together.

## Changes

- Add `Op::ThreadSettings` for settings-only queued updates.
- Emit `ThreadSettingsApplied` with the effective thread settings
snapshot after core applies an update.
- Route settings-only updates through the same submission queue as user
input.
- Migrate remaining `OverrideTurnContext` tests and callers to the
queued `Op::ThreadSettings` path.
- Delete `Op::OverrideTurnContext` from the core protocol and submission
loop.

This stack addresses #20656 and #22090.

## Stack

1. [1 of 7] [Add thread settings to
UserInput](https://github.com/openai/codex/pull/23080)
2. [2 of 7] [Remove
UserInputWithTurnContext](https://github.com/openai/codex/pull/23081)
3. [3 of 7] [Remove
UserTurn](https://github.com/openai/codex/pull/23075)
4. [4 of 7] [Placeholder for OverrideTurnContext
cleanup](https://github.com/openai/codex/pull/23087)
5. [5 of 7] [Replace OverrideTurnContext with
ThreadSettings](https://github.com/openai/codex/pull/22508) (this PR)
6. [6 of 7] [Add app-server thread settings
API](https://github.com/openai/codex/pull/22509)
7. [7 of 7] [Sync TUI thread
settings](https://github.com/openai/codex/pull/22510)
2026-05-18 21:03:51 -07:00
Ahmed IbrahimandCodex be1d3cff93 2- Use string service tiers in session protocol (#20971)
## Summary
- break service tier session/op/app-server protocol fields from the
closed enum to string tier ids
- send the service tier string directly through model requests, prewarm,
compaction, memories, and TUI/app-server turn starts
- regenerate app-server protocol JSON/TypeScript schemas, removing the
standalone ServiceTier TS enum

## Verification
- just fmt
- cargo check -p codex-core -p codex-app-server -p codex-tui
- just write-app-server-schema

---------

Co-authored-by: Codex <noreply@openai.com>
2026-05-06 18:00:21 +03:00
jif-oaiandCodex a98623511b feat: add session_id (#20437)
## Summary

Related to
https://openai.slack.com/archives/C095U48JNL9/p1777537279707449
TLDR:
We update the meaning of session ids and thread ids:
* thread_id stays as now
* session_id become a shared id between every thread under a /root
thread (i.e. every sub-agent share the same session id)

This PR introduces an explicit `SessionId` and threads it through the
protocol/client boundary so `session_id` and `thread_id` can diverge
when they need to, while preserving compatibility for older serialized
`session_configured` events.

---------

Co-authored-by: Codex <noreply@openai.com>
2026-05-06 10:48:37 +02:00
jif-oaiandCodex 431ebeaef7 feat: split memories part 2 (#19860)
Keep extracting memories out of core and moving the write trigger in the
app-server
This is temporary and it should move at the client level as a follow-up
This makes core fully independant from `codex-memories-write`

---------

Co-authored-by: Codex <noreply@openai.com>
2026-04-28 13:03:28 +02:00