Commit Graph
18 Commits
Author SHA1 Message Date
Dmitriy Kovalenko b91cedc080 Centralize TOML config deserialization (#58198)
## What changed

- Add a non-inlined `TryFrom<toml::Value>` implementation for `ConfigToml` and route callers through it to share the generated deserializer in `codex-config`.
- Reuse the ignored-field deserialization helper for strict validation, returning path-aware errors while preserving existing fallbacks for CLI validation and ignored-config warnings.

GitOrigin-RevId: 0208b314416ad4d3d7fbf3a30560a0c8c4035abd
2026-10-30 00:36:00 +00:00
Eric Traut b725da3b6d Add Pro Max plan support and update Pro display names (#47971)
## What changed

- Recognize `promax` in authentication, account responses, and backend rate limits; update generated schemas and client types.
- Display `prolite` as “Pro”, `pro` as “Pro (More)”, and `promax` as “Pro (Max)” in account labels, status, and analytics.
- Include Pro Max in experimental context eligibility, paid-plan tooltips, and usage-limit guidance for purchasing credits.

## Testing

Extend coverage for Pro Max plan parsing, account and rate-limit responses, experimental context eligibility, and plan display names. Update the login token test to expect “Pro (More)”.

GitOrigin-RevId: 197e00fa7a99cbce125c8f1d43a451ec1ed5c1fb
2026-09-25 00:20:02 +00:00
Abhinav b64de2f3ad Use the originating model when recording conversation history (#44243)
## Why

History recording used the model captured at turn start, so switching models within a turn could apply stale image preparation and tool-output truncation rules.

## What changed

Pass explicit model metadata through conversation recording. Record model responses and tool results using their originating step's settings, and capture the current model for pending input recorded during startup interruption or task completion.

## Testing

Add regression coverage for tool results across a model switch, explicit model selection for plain and annotated history, and image input retained during interruption and completion. Extend model-switching tests to verify that prepared media and item identity survive switching back to a multimodal model in live, resumed, and forked threads.

GitOrigin-RevId: 5bc750c30c334acb9b802cb80fdea01daffa84a1
2026-09-09 17:46:00 +00:00
pmccrary-oai 6af345407d Gate experimental context by model capability at session startup (#43147)
## Why

Experimental context activation previously checked the provider and account eligibility without checking model support. Child sessions also inherited token-budget activation from their parent, even when starting fresh with a different model.

## What changed

- Add `ModelInfo.supports_experimental_context`, defaulting to `false`, and enable it for the bundled `gpt-6-astra` model. Require this capability when activating experimental context.
- Snapshot configured token-budget preferences before startup activation. Restore them for fresh child sessions before applying their starting model's defaults, while history forks retain their parent's activation.
- Pass unresolved token-budget preferences to child sessions so they can use their own model's prompts.

## Testing

Extend coverage for unsupported models, model-switch guidance under explicit and experimental activation, and child configuration from both active and inactive parents. Verify that omitted capability metadata defaults to `false`.

GitOrigin-RevId: 02df9e171682267232fa923d5ea3f7af36527808
2026-09-06 02:28:38 +00:00
pmccrary-oai cff76fa96f Add experimental context management activation (#42385)
## What changed

- Add the under-development `features.context_management.experimental_mode` configuration and schema.
- For eligible ChatGPT Plus, Pro, and Pro Lite sessions using the Codex backend, enable token-budget context, history notes, and the `new_context` tool when experimental context management is configured.
- Keep the feature disabled for custom providers, provider credentials, non-Codex endpoints, and temporary structured threads.

## Testing

- Cover subscription eligibility, backend restrictions, configuration resolution, token-budget guidance, and temporary-thread overrides.

GitOrigin-RevId: 19402af19c4d69e09a7cdb3687ee7d233f6a562a
2026-09-02 23:37:39 +00:00
pmccrary-oai d58d0e5841 Allow models to enable token budgeting by default (#41803)
## What changed

- Add model metadata flags for enabling token budgeting and its history-notes extension.
- Apply those activation defaults once at thread startup when the user has not explicitly configured token budgeting, while respecting managed feature requirements.
- Continue resolving model-owned prompts for the active model so model switches can use their own token-budget guidance.

## Testing

- Update the token-budget integration test to verify that model metadata activates the feature without manual configuration.

GitOrigin-RevId: 73a32623eaeabb9243e240c43da7a5e7e93e9a19
2026-08-31 08:31:23 +00:00
Abhinav 8228e9b867 Resolve token budgets from each step's active model (#41162)
## Why

Model settings can change between steps in the same turn. Token-budget defaults,
context-window limits, and model-visible guidance need to stay aligned with the
model captured for each step.

## What changed

- Preserve the turn's original token-budget preferences and resolve model-owned
  defaults when each `StepContext` is created.
- Use the step's model to determine context-window availability and token-budget
  guidance, while keeping explicit user settings unchanged.
- Emit one-time replacement or removal notices when context-window guidance
  changes, including compatibility with previously persisted world-state
  snapshots.
- Add `ModelInfo::usable_context_window()` to distinguish reserved-headroom
  capacity from the resolved context window and auto-compaction limit.

## Testing

- Cover mid-turn model switches with model defaults, explicit settings, config
  reloads, missing context windows, and missing destination guidance.
- Cover unchanged, replaced, removed, blank, and legacy guidance transitions.

GitOrigin-RevId: 91100a9f3625bef8a79ecd158eb028051c309eee
2026-08-27 15:01:22 +00:00
pmccrary-oai d8ec270183 Rename the history notes extension config option (#39830)
## What changed

Rename the token-budget configuration key and its Rust field from
`use_history_notes_history` to `use_history_notes_extension`. Update the config
schema, extension gating, explicit-setting detection, and tests to use the new
name consistently.

GitOrigin-RevId: b6a514e23f02095fcd1d35dc42eeb7394b800c77
2026-08-21 01:51:48 +00:00
pmccrary-oai daa48072f4 Add history and notes tools for token-budget sessions (#39827)
## Why

Token-budget sessions need a way to recover prior conversation context and preserve working state across context-window transitions.

## What changed

- Add direct-model `history` tools for listing windows and items, reading items, and searching conversation contents.
- Add direct-model `notes` tools for listing, reading, searching, appending, and writing persistent notes.
- Route tool calls through the configured Codex backend with trusted session and agent context, bounded request arguments, and truncation-aware output handling.
- Expose the extension when `features.token_budget.use_history_notes_history` is enabled with an OpenAI provider and Codex backend authentication.

## Testing

- Cover tool registration, configuration changes, provider and authentication requirements, backend request context, encrypted output preservation, and request and response limits.

GitOrigin-RevId: 43b259f01014ba3f30803dd7cd5634942407a9bb
2026-08-21 01:24:52 +00:00
Boyang Niu 69b6152c17 Track context windows per agent (#37347)
## Why

Forked subagents can inherit a parent's compacted history, but their context-window metadata needs to identify the child and start a distinct window lineage.

## What changed

- Always identify context windows by agent name and remove the `features.token_budget.mode` setting.
- Persist the agent path in world state and emit context-window metadata as a standalone developer message when the identity changes.
- Reset inherited compaction metadata to the forked subagent's initial context window and restore that window in session state.

## Testing

Extend the full-history subagent test to verify parent and child world-state snapshots, agent identities, distinct window IDs, and the reset child checkpoint.

GitOrigin-RevId: 6584201e57b3d2fba0ce626b89a8b3b0b3c752db
2026-08-07 01:05:33 +00:00
Boyang Niu c607da9f37 Make token budget context identity configurable (#36970)
## What changed

- Add a `features.token_budget.mode` setting with `thread` and `name` values.
- Default context-window metadata to the thread ID while allowing configurations to retain the agent name.
- Preserve the selected identity mode when applying model token-budget defaults and replaying locked configuration.

## Testing

- Cover configuration parsing and both identity formats in token-budget context messages.

GitOrigin-RevId: f6e6d2c49e09e70c0352e803bb65784ef77e5bee
2026-08-04 20:48:40 +00:00
rka-oai fbe65995bb Support model-owned token budget defaults (#35608)
## What changed

- Add token-budget settings to model catalog messages and apply them when the
  feature is enabled without explicit token-budget configuration.
- Keep explicit user settings authoritative, reject invalid catalog defaults,
  and preserve resolved defaults in exported config locks.
- Manage context-window guidance through world state so it updates once when
  the active model changes while retaining prior conversation history.

## Testing

- Cover catalog defaults, explicit overrides, disabled features, invalid
  values, config-lock replay, and model switching.

GitOrigin-RevId: 54544fefaa14b09f5ef5bad9967a13c52b87b0fe
2026-07-27 12:52:37 +00:00
rka-oai 768330dd6c Add a fallback phase before automatic context rollover (#33255)
## Why

Token-budget sessions need a final opportunity to preserve important state before the current context is compacted.

## What changed

- When `auto_compact_fallback_prompt` is configured, reserve `auto_compact_fallback_buffer_tokens` beyond the base auto-compaction limit.
- At the base limit, inject the fallback prompt once as a developer message while keeping the normal tool surface available. Continue reporting zero base-window tokens during this reserve.
- Roll over when the reserve or full model context is exhausted, and skip the fallback when a new context was already requested.

## Testing

Added coverage for using the fallback reserve, rolling over after it is exhausted, and bypassing the fallback for an explicit `new_context` request.

GitOrigin-RevId: 32983ecadc5cb4927e15ecd7b1e896cb52a9ad3b
2026-07-15 05:45:43 +00:00
Michael Bolin 77e7ce1374 fix: scope context remaining to body window (#29665)
## Why

With `model_auto_compact_token_limit_scope = "body_after_prefix"`, the
persistent prefix should not count against the active body window.
`get_context_remaining` and the token-budget reminder should report the
same usable body-after-prefix window that auto-compaction uses, rather
than the total token count since the session began.

This is stacked on #29664 so the mechanical move from `turn.rs` is
isolated from the behavior fix.

## What

- Extends `ContextWindowTokenStatus` with `context_remaining_tokens`.
- Updates `get_context_remaining` to use the shared context-window
accounting.
- Adds integration coverage for body-after-prefix reminder timing and
`get_context_remaining` output.

## Testing

- `just test -p codex-core body_after_prefix_window`
- `just test -p codex-core auto_compact_body_after_prefix`
- `just fix -p codex-core`
2026-06-23 23:08:54 +00:00
pakrym-oai aaf737fa59 [codex] simplify token budget context (#29295)
## Why

The token-budget feature currently adds remaining-token messages
whenever usage crosses the 25%, 50%, and 75% thresholds. Those periodic
inserts create prompt churn without requiring action, while the
near-compaction reminder and explicit `get_context_remaining` tool
already cover actionable and on-demand budget information.

The context-window lineage block is also easier to scan as plain labeled
text than as a `<token_budget>`-wrapped fragment.

## What changed

- Stop recording automatic remaining-token messages at percentage
thresholds.
- Render context-window lineage in `First`, `Current`, `Previous` order
with colon-separated labels.
- Omit the `Previous` line for the first context window.
- Remove `<token_budget>` wrappers from newly rendered lineage,
near-compaction reminders, and `get_context_remaining` output.
- Keep recognizing legacy wrapped fragments so existing rollouts remain
compatible.
- Remove the post-sampling token snapshot that was only needed by the
periodic threshold path.

## Testing

- `just test -p codex-core token_budget` (11 tests passed)
2026-06-20 21:50:09 -07:00
pakrym-oai 6df037d47f [codex] add configurable token budget compaction reminder (#29255)
## Why

The token-budget feature reports coarse remaining-context milestones,
but it does not give the model a configurable wrap-up prompt before
automatic compaction. A strict threshold-crossing check can also miss
resumed or reconfigured windows that are already inside the threshold.

## What changed

- Add structured `[features.token_budget]` configuration for an absolute
`reminder_threshold_tokens` and bounded `reminder_message_template`;
`{n_remaining}` is expanded when the reminder is delivered.
- Compute remaining tokens against the next effective auto-compaction
boundary, including scoped `body_after_prefix` accounting and the full
context-window limit.
- Make reminder delivery level-triggered before and after sampling, with
one-shot state owned by `AutoCompactWindow` and re-armed on compaction,
`new_context`, restore, or history replacement.
- Leave the existing initial full-window token-budget context, 25/50/75%
notices, and token-budget tools unchanged.
- Persist the resolved feature configuration in the session config lock
and regenerate the config schema.

## Validation

- `just test -p codex-core token_budget`
- `just test -p codex-core
token_budget_reminder_emits_after_crossing_compaction_threshold`
- `just test -p codex-core auto_compact_window`
- `just test -p codex-core
lock_contains_prompts_and_materializes_features`
- `just test -p codex-features`
- `just test -p codex-config`
2026-06-20 19:13:42 -07:00
pakrym-oai 172b2218a5 core: remove redundant TurnContext and Prompt fields (#28638)
## Why

`TurnContext` had accumulated dead fields and cached projections of
values already owned by its per-turn `Config` or `ModelInfo`. Keeping
both copies made ownership unclear and allowed artificial split-brain
states, such as a compatibility hash differing from the model metadata
it came from.

`Prompt` similarly carried a write-only personality after personality
selection had already been materialized into its base instructions.

This makes the canonical owner explicit: configuration-backed values
come from `config`, model-derived values come from `model_info`, and
prompts contain only data consumed by request construction.

## What changed

- Remove the unused `ghost_snapshot`, `codex_self_exe`, and
`thread_source` fields.
- Remove duplicate `comp_hash`, `truncation_policy`, `features`,
`shell_environment_policy`, `codex_linux_sandbox_exe`, `compact_prompt`,
and `tool_mode` fields.
- Read those values directly from `TurnContext::config` or
`TurnContext::model_info` at their consumers.
- Remove the write-only `Prompt::personality` field and its constructor
assignments.
- Preserve review-turn inheritance of the parent turn's shell policy,
Linux sandbox executable, and compact prompt through the review config.

## Testing

- `cargo check -p codex-core --tests`
2026-06-16 22:17:24 -07:00
pakrym-oai 658af936fd [codex] Add token budget context feature (#27438)
## Why

The model should be able to see bounded context-window budget metadata
when the `token_budget` feature is enabled. The full-window message is
only injected with full context, while normal turns get a smaller
follow-up only when reported usage first crosses a budget threshold.

## What changed

- Added the `TokenBudget` feature flag.
- Added `<token_budget>` developer fragments for full context-window
metadata and current-window remaining tokens.
- Inserted the threshold message during normal turn handling by
comparing token usage before and after sampling, avoiding persistent
threshold bookkeeping.
- Added core integration coverage for full-context-only metadata and
25/50/75 percent threshold messages.

## Verification

- `just test -p codex-core token_budget`
- `git diff --check`
2026-06-10 20:07:06 -07:00