mirror of
https://github.com/openai/codex.git
synced 2026-11-05 20:10:33 +00:00
Standardize model-visible fragments
Fold Apps, Skills, and Plugins into the model-visible fragment registry as ordered developer-envelope fragments, keep the turn-state/contextual-user framework intact, and refresh prompt layout snapshots/docs around the new shape. Co-authored-by: Codex <noreply@openai.com>
This commit is contained in:
Charles Cunningham
co-authored by
Codex
Codex
parent
cf0223887f
commit
c54d3211bc
@@ -0,0 +1,382 @@
|
||||
# Model-visible context
|
||||
|
||||
This document describes how model-visible prompt context is represented in the
|
||||
current `codex-rs` codebase, and how to add new context without breaking
|
||||
resume, compaction, backtracking, or history parsing.
|
||||
|
||||
The key modules are:
|
||||
|
||||
- [`codex-rs/core/src/model_visible_context.rs`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/model_visible_context.rs):
|
||||
shared fragment abstractions, roles, marker helpers, and diff params
|
||||
- [`codex-rs/core/src/model_visible_fragments.rs`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/model_visible_fragments.rs):
|
||||
concrete fragment definitions, the central registry, shared contextual-user
|
||||
detection, and turn-state fragment assembly
|
||||
- [`codex-rs/core/src/context_manager/updates.rs`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/context_manager/updates.rs):
|
||||
the shared developer/contextual-user envelope builders and steady-state
|
||||
per-turn diff assembly
|
||||
- [`codex-rs/core/src/codex.rs`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/codex.rs):
|
||||
initial-context assembly and the normal runtime path that decides whether to
|
||||
inject full context or only diffs
|
||||
|
||||
## Core model
|
||||
|
||||
Model-visible prompt context falls into three buckets:
|
||||
|
||||
1. Turn-state fragments.
|
||||
These are derived from current durable turn/session state and are the ones
|
||||
that must survive history-mutating flows such as resume, compaction,
|
||||
backtracking, and fork by being rebuilt from current state plus an optional
|
||||
persisted baseline.
|
||||
2. Registered runtime fragments.
|
||||
These are not derived from `TurnContext` diffs, but they are still modeled
|
||||
as typed fragments because they are emitted into model-visible history and,
|
||||
for contextual-user fragments, must still be recognized during later history
|
||||
parsing.
|
||||
3. Developer-only one-off text injections.
|
||||
These are not currently represented as dedicated registered fragments. They
|
||||
are developer-role text added in a few call sites where no contextual-user
|
||||
detection or turn-state diffing is needed.
|
||||
|
||||
The single most important distinction is whether the model-visible state is:
|
||||
|
||||
- durable turn/session state that should be rebuilt from `TurnContext`
|
||||
- or a one-off event/message that is only relevant because it just happened
|
||||
|
||||
That determines whether the fragment needs `build(...)`.
|
||||
|
||||
## Two-envelope rule
|
||||
|
||||
Turn-state context assembly always produces exactly two envelopes:
|
||||
|
||||
- one `developer` message
|
||||
- one contextual `user` message
|
||||
|
||||
Those envelopes are assembled by the shared builders in
|
||||
[`updates.rs`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/context_manager/updates.rs).
|
||||
Adjacent text content items are explicitly separated with `\n\n` because, from
|
||||
the model's perspective, adjacent text `ContentItem`s inside one message are
|
||||
effectively concatenated.
|
||||
|
||||
This two-envelope rule applies to turn-state context assembly:
|
||||
|
||||
- [`Codex::build_initial_context(...)`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/codex.rs#L3342)
|
||||
- [`build_settings_update_items(...)`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/context_manager/updates.rs#L105)
|
||||
|
||||
It does not mean every model-visible message in the system is forced into those
|
||||
two envelopes. Runtime/session-prefix messages may still be emitted as separate
|
||||
messages when the event itself is what needs to be recorded, and some
|
||||
initial-only contextual/developer additions are appended to the envelopes
|
||||
outside registry-driven `build(...)` assembly.
|
||||
|
||||
## Current architecture
|
||||
|
||||
### 1. Shared fragment trait
|
||||
|
||||
Every named fragment type implements
|
||||
[`ModelVisibleContextFragment`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/model_visible_context.rs#L180).
|
||||
|
||||
That trait owns:
|
||||
|
||||
- `type Role`
|
||||
- `render_text()`
|
||||
- optional `build(...)` for turn-state fragments
|
||||
- optional contextual-user detection via `contextual_user_markers()` or
|
||||
`matches_contextual_user_text()`
|
||||
- standard conversions such as `into_message()` and `into_response_input_item()`
|
||||
|
||||
Roles are:
|
||||
|
||||
- `DeveloperContextRole`
|
||||
- `ContextualUserContextRole`
|
||||
|
||||
### 2. Central registry
|
||||
|
||||
Current fragment types are registered exactly once in
|
||||
[`REGISTERED_MODEL_VISIBLE_FRAGMENTS`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/model_visible_fragments.rs#L123).
|
||||
|
||||
That registry is used for:
|
||||
|
||||
- contextual-user fragment detection during history parsing
|
||||
- turn-state fragment assembly for both envelopes by calling `build(...)`
|
||||
|
||||
Registration is therefore an integration step, not just bookkeeping. Defining
|
||||
the type is not enough; the fragment is not part of the system until it is
|
||||
registered.
|
||||
|
||||
### 3. Build semantics
|
||||
|
||||
[`ModelVisibleContextFragment::build(...)`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/model_visible_context.rs#L187)
|
||||
is the canonical hook for turn-state fragments.
|
||||
|
||||
It receives:
|
||||
|
||||
- the current `TurnContext`
|
||||
- `reference_context_item: Option<&TurnContextItem>`
|
||||
- `TurnContextDiffParams`
|
||||
|
||||
`reference_context_item` is the persisted durable baseline already represented
|
||||
in model-visible history. A turn-state fragment should compare against that
|
||||
baseline to avoid emitting duplicate prompt state. When there is no baseline
|
||||
(`None`), the fragment should decide whether to emit full current context or
|
||||
fall back to `TurnContextDiffParams` for other relevant previous-turn/session
|
||||
state such as `previous_turn_settings`.
|
||||
|
||||
Turn-state fragments return:
|
||||
|
||||
- `Some(fragment)` when model-visible state should be injected
|
||||
- `None` when the current state is already represented and no update is needed
|
||||
|
||||
Fragments that are not turn-state-derived leave `build(...)` as the default
|
||||
`None`.
|
||||
|
||||
## Initial context vs steady-state diffs
|
||||
|
||||
The code intentionally treats these differently.
|
||||
|
||||
### Full initial context
|
||||
|
||||
[`Codex::build_initial_context(...)`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/codex.rs#L3342)
|
||||
always builds turn-state fragments with `reference_context_item = None`. In
|
||||
other words, it asks every turn-state fragment for the full current prompt
|
||||
state, not a diff.
|
||||
|
||||
That is the right behavior for:
|
||||
|
||||
- first real user turn in a thread
|
||||
- any path that must re-establish canonical prompt context from scratch
|
||||
|
||||
There is also
|
||||
[`build_initial_context_without_reference_context_item(...)`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/codex.rs#L3445),
|
||||
which currently delegates to the same full-context behavior and exists to make
|
||||
the “ignore any existing baseline” intent explicit in compaction rebuild paths.
|
||||
|
||||
### Steady-state turn updates
|
||||
|
||||
[`build_settings_update_items(...)`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/context_manager/updates.rs#L105)
|
||||
passes the current `reference_context_item` into the registry-driven turn-state
|
||||
builder loop, so registered turn-state fragments can emit only the minimal
|
||||
developer/contextual-user diffs for the next turn.
|
||||
|
||||
### Normal runtime path
|
||||
|
||||
[`record_context_updates_and_set_reference_context_item(...)`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/codex.rs#L3479)
|
||||
uses this rule:
|
||||
|
||||
- if there is no baseline, inject full initial context
|
||||
- otherwise inject only turn-state diffs
|
||||
|
||||
After each real user turn it persists the latest `TurnContextItem` and advances
|
||||
the in-memory baseline even if no model-visible diff message was emitted. That
|
||||
is what keeps later diffing aligned with current state.
|
||||
|
||||
## Contextual-user detection
|
||||
|
||||
Contextual-user fragments share the `user` role with real user messages, so
|
||||
history parsing must be able to distinguish:
|
||||
|
||||
- contextual state
|
||||
- true user intent
|
||||
|
||||
That is why contextual-user fragments need stable detection.
|
||||
|
||||
Preferred path:
|
||||
|
||||
- implement `contextual_user_markers()` when the fragment has stable fixed
|
||||
wrappers
|
||||
|
||||
Fallback path:
|
||||
|
||||
- override `matches_contextual_user_text()` when detection is genuinely custom
|
||||
(for example the current AGENTS.md wrapper)
|
||||
|
||||
The shared detection entrypoint is
|
||||
[`is_contextual_user_fragment(...)`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/model_visible_fragments.rs#L843).
|
||||
It first checks the current registry, then applies a very small legacy shim for
|
||||
historical user-role wrappers that were shipped previously but are no longer
|
||||
current fragment types.
|
||||
|
||||
Current legacy shim:
|
||||
|
||||
- old user-role `<subagent_notification>...</subagent_notification>` messages
|
||||
|
||||
Those legacy checks should stay bounded and local to detection. Do not create
|
||||
fake current fragment types just to recognize old persisted history.
|
||||
|
||||
## Current fragment inventory
|
||||
|
||||
### Registered turn-state developer fragments
|
||||
|
||||
These implement `build(...)` and participate in both full initial context and
|
||||
steady-state diffs:
|
||||
|
||||
- `ModelInstructionsUpdateFragment`
|
||||
- `PermissionsUpdateFragment`
|
||||
- `CustomDeveloperInstructionsUpdateFragment`
|
||||
- `CollaborationModeUpdateFragment`
|
||||
- `RealtimeUpdateFragment`
|
||||
- `PersonalityUpdateFragment`
|
||||
- `AppsInstructionsFragment`
|
||||
- `SkillsSectionFragment`
|
||||
- `PluginInstructions`
|
||||
|
||||
The capability-guidance fragments are intentionally developer-role, rendered in
|
||||
`Apps -> Skills -> Plugins` order, and use stable XML-ish wrappers
|
||||
(`<apps_instructions>`, `<skills_instructions>`, `<plugins_instructions>`) so
|
||||
prompt-layout tests can normalize them reliably.
|
||||
|
||||
### Registered runtime developer fragments
|
||||
|
||||
These are typed and registered, but not built from `TurnContext` diffs:
|
||||
|
||||
- `SubagentRosterContext`
|
||||
- `SubagentNotification`
|
||||
|
||||
### Registered turn-state contextual-user fragments
|
||||
|
||||
These implement `build(...)` and participate in both full initial context and
|
||||
steady-state diffs:
|
||||
|
||||
- `UserInstructionsFragment`
|
||||
- `AgentsMdInstructions`
|
||||
- `JsReplInstructionsFragment`
|
||||
- `ChildAgentsInstructionsFragment`
|
||||
- `EnvironmentContext`
|
||||
|
||||
Some of these are true steady-state diff fragments (`UserInstructionsFragment`,
|
||||
`AgentsMdInstructions`, `EnvironmentContext`). Others intentionally rebuild only
|
||||
when there is no baseline and therefore behave as initial-context fragments
|
||||
expressed through the same `build(...)` hook (`JsReplInstructionsFragment`,
|
||||
`ChildAgentsInstructionsFragment`).
|
||||
|
||||
### Registered runtime contextual-user fragments
|
||||
|
||||
These are typed and registered for rendering/detection, but not built from
|
||||
`TurnContext` diffs:
|
||||
|
||||
- `SkillInstructions`
|
||||
- `UserShellCommandFragment`
|
||||
- `TurnAbortedMarker`
|
||||
|
||||
`SkillInstructions`, `UserShellCommandFragment`, and `TurnAbortedMarker` are
|
||||
currently emitted as standalone contextual-user messages.
|
||||
|
||||
## Current non-registry developer text injections
|
||||
|
||||
The current codebase still has a few developer-role prompt injections that are
|
||||
not dedicated registered fragments. These are accurate descriptions of current
|
||||
behavior, not the preferred path for new work.
|
||||
|
||||
Current examples include:
|
||||
|
||||
- memory-tool initial developer instructions
|
||||
- commit-attribution guidance in the initial developer envelope
|
||||
- explicit plugin-mention developer hints
|
||||
- a few one-off runtime developer messages such as stop-hook and hook-provided
|
||||
additional context
|
||||
|
||||
These are acceptable because they are developer-role only and do not need
|
||||
contextual-user detection or turn-state diff reconstruction. But for new
|
||||
model-visible context, prefer a dedicated typed fragment unless the message is
|
||||
truly an isolated one-off developer event.
|
||||
|
||||
## Choosing the right representation
|
||||
|
||||
### Use a registered turn-state fragment when:
|
||||
|
||||
- the context is derived from `TurnContext` / durable session state
|
||||
- resume, fork, compaction, or backtracking should be able to rebuild it
|
||||
- it belongs in one of the two turn-state envelopes
|
||||
|
||||
Examples:
|
||||
|
||||
- permissions policy
|
||||
- collaboration mode
|
||||
- realtime start/end state
|
||||
- apps guidance
|
||||
- skills catalog guidance
|
||||
- plugin capability guidance
|
||||
- custom user instructions
|
||||
- AGENTS.md instructions
|
||||
- JS REPL guidance
|
||||
- child-AGENTS guidance
|
||||
- environment context
|
||||
|
||||
### Use a registered runtime fragment when:
|
||||
|
||||
- the context is model-visible
|
||||
- it is not derived from `TurnContext` diffs
|
||||
- it still benefits from standardized rendering and, for contextual-user
|
||||
fragments, shared detection
|
||||
|
||||
These fragments may either:
|
||||
|
||||
- be appended to an envelope in a non-diff initial-context path
|
||||
- or be emitted as standalone runtime/session-prefix messages
|
||||
|
||||
Examples:
|
||||
|
||||
- turn-aborted marker
|
||||
- user shell command record
|
||||
- skill injection
|
||||
- subagent roster / subagent notification
|
||||
|
||||
### Use a plain developer text wrapper only when:
|
||||
|
||||
- the message is developer-role only
|
||||
- it is a narrow one-off or initial-only addition
|
||||
- it does not need contextual-user detection
|
||||
- it does not need to participate in turn-state diff reconstruction
|
||||
|
||||
If you are tempted to use a plain `String` or `DeveloperTextFragment`, stop and
|
||||
verify that this is really just an isolated developer-only event and not new
|
||||
shared prompt state.
|
||||
|
||||
## Environment context rule
|
||||
|
||||
Use `<environment_context>` only for environment facts derived from
|
||||
`TurnContext` that may need turn-to-turn refresh semantics.
|
||||
|
||||
Current fields are:
|
||||
|
||||
- `cwd`
|
||||
- `shell`
|
||||
- optional `current_date`
|
||||
- optional `timezone`
|
||||
- optional network allow/deny summaries
|
||||
|
||||
Do not put other guidance in `<environment_context>`, especially:
|
||||
|
||||
- permissions/policy instructions
|
||||
- skills or plugins
|
||||
- AGENTS instructions
|
||||
- subagent notices
|
||||
|
||||
Those should stay in their own fragment types.
|
||||
|
||||
## Blessed path for new work
|
||||
|
||||
When adding new model-visible context:
|
||||
|
||||
1. Decide whether the model should see it at all.
|
||||
2. Decide whether it is durable turn/session state or a one-off runtime event.
|
||||
3. If it should be represented as a named fragment, define a typed fragment in
|
||||
[`model_visible_fragments.rs`](/Users/ccunningham/code/codex-worktree-tria/codex-rs/core/src/model_visible_fragments.rs).
|
||||
4. Implement `ModelVisibleContextFragment`.
|
||||
5. Set `type Role` correctly.
|
||||
6. If it is turn-state context, implement `build(...)`.
|
||||
7. If it is contextual-user, provide stable detection with
|
||||
`contextual_user_markers()` or custom `matches_contextual_user_text()`.
|
||||
8. Register it exactly once in `REGISTERED_MODEL_VISIBLE_FRAGMENTS`, in prompt
|
||||
order.
|
||||
9. Use the shared envelope builders or fragment conversions rather than
|
||||
hand-constructing model-visible `ResponseItem::Message` payloads in new
|
||||
turn-state paths.
|
||||
|
||||
Rule of thumb:
|
||||
|
||||
- “This is durable prompt state” => registered typed fragment, usually with
|
||||
`build(...)`
|
||||
- “This is a one-off contextual/runtime marker” => registered typed fragment,
|
||||
usually without `build(...)`
|
||||
- “This is an isolated developer-only text event” => plain developer text is
|
||||
sometimes acceptable, but it is the exception, not the default
|
||||
Reference in New Issue
Block a user