## Why
Message roles and content classifications do not identify who supplied each context part. Source attribution needs to survive prompt assembly and history truncation, including messages that combine text from different producers.
## What changed
- Add per-part `ContentItemMetadata` for user input, developer instructions, `AGENTS.md`, skills, harness context, command hooks, and tool context with optional callable namespaces.
- Carry attribution alongside content and classifications through contextual fragments, extension prompts, hook output, and agent messages. Keep tool-provided context-window hints separate from harness text.
- Preserve the source of retained text slices during compaction and attribute generated truncation notices to the harness. Default missing historical attribution to unknown.
- Attribute host MCP calls using cached tool namespaces only when catalog revisions remain stable, without starting servers or refreshing catalogs for attribution.
- Send the new metadata to Responses Lite and strip it from requests to endpoints that do not support it.
## Testing
Extend compaction tests to cover mixed sources, missing or extra metadata entries, and discarded media. Update Responses Lite request assertions to verify endpoint-specific attribution handling.
GitOrigin-RevId: b0ebc929b7c60f3ec49d1affbf897312d63edcc8
## Why
Paginated history previously persisted completed commands' full `aggregated_output`, allowing large command outputs to inflate stored history.
## What changed
- Limit persisted command `aggregated_output` to 64 KiB, including a truncation marker. Preserve the beginning and end of the output without splitting UTF-8 characters.
- Apply the shared persistence policy to rollout recording, thread-store appends, and legacy-to-paginated history migration.
- Measure persisted payload bytes after truncation so persistence metrics reflect the stored representation.
## Testing
Add coverage for the byte limit, UTF-8 boundaries, unchanged inputs that fit, and markers that exceed the budget. Verify capped output in paginated thread history and migrated legacy history, and verify persistence metrics use the truncated payload size.
GitOrigin-RevId: 01d420d396ad34b42dc5dc208c0bf88d890639ee
## Why
Token-budget truncation uses byte counts to estimate tokens, but previously scanned the entire string to count characters, including the discarded middle.
## What changed
Use UTF-8 boundary lookups to select the retained prefix and suffix for token-budget truncation. Skip character counting for a zero-token budget and defer it to paths that need character counts.
## Testing
Add regression coverage for multibyte characters crossing both retained boundaries, checking the truncated output and original token estimate.
GitOrigin-RevId: 37fee0c8d4ede8e74b491d3d641b0ea0e9003dd4
## Why
The 900-byte recap prompt limit leaves little room for completed progress, unresolved caveats, and recent corrections. Recaps need this context to distinguish completed work from pending requests.
## What changed
- Introduce a shared `RecapPrompt` with a 32 KiB ceiling for instructions and history, using an 8,192-token estimate. Instruct summaries to retain the broader goal, completed outcomes, and unresolved availability or validation caveats.
- Select up to eight answered exchanges plus a pending request, preserving adjacent steering and progress messages. Drop older whole exchanges before excerpting both ends of oversized messages, while retaining the newest answer and pending correction.
- Require a `summary` and nullable `next_action`, bounded to 700 and 200 characters respectively. Reject malformed or oversized responses and display an optional, separately styled `Next:` line.
## Testing
Add coverage for UTF-8 prompt bounds, exchange selection, pending corrections, excerpt boundaries, strict response parsing, and next-action rendering. Extend the recap generation integration test to verify bounded history and the structured response schema.
GitOrigin-RevId: 6c6a84bc602a168418c1952feb1d286ec0cb9e28
- move the shared byte-based middle truncation logic from `core` into
`codex-utils-string`
- keep token-specific truncation in `codex-core` so rollout can reuse
the shared helper in the next stacked PR
---------
Co-authored-by: Codex <noreply@openai.com>
tongzhou wang