Commit Graph
14 Commits
Author SHA1 Message Date
Owen Lin be48ae396e Account for JSON overhead when truncating MCP tool results (#50470)
## Why

Truncating the preview alone can leave the serialized MCP tool result above its byte budget because JSON escaping and the result wrapper add bytes.

## What changed

Measure the complete serialized replacement and progressively reduce the preview budget until the result fits or the preview is empty.

## Testing

Strengthen the regression test with quotes, backslashes, and newlines. Assert that the serialized result fits within the 1,024-byte budget and that truncating it again returns the existing result unchanged.

GitOrigin-RevId: ebd97b1a6ca71343941a96e472cb313028ac680c
2026-10-03 00:13:58 +00:00
Owen Lin 820f85cf59 Truncate oversized MCP results in paginated thread history (#50458)
## Why

Paginated thread history can persist large MCP results in completed tool-call items. Bound these stored results to avoid retaining multi-megabyte payloads.

## What changed

Apply a 64 KiB preview budget when persisting oversized MCP results in paginated history. Replace the result with a truncated text preview of its serialized JSON, preserving `is_error` and clearing `structuredContent` and `_meta`. Leave small results unchanged.

Share the truncation helper with the existing MCP tool-call event path.

## Testing

Add helper tests for unchanged small results and oversized results with structured content and metadata. Add a thread-history test verifying that persisted MCP results contain a bounded preview with the beginning, end, and truncation marker while preserving tool arguments.

GitOrigin-RevId: 09dce73c70bfd357c9b9630f456e9e5c1d8492cf
2026-10-02 23:15:02 +00:00
Krish Chainani 7b8b17b97a Support image references by file ID in inputs and tool outputs (#45794)
## What changed

- Accept `fileId` alongside the existing `url` form for app-server image inputs, and forward file references to the Responses API as `file_id`. Update generated schemas and client types.
- Preserve file references, image detail hints, and mixed inline/file image ordering through user-message events, thread history, and rollout migration. Retain file images when truncating tool output.
- Pass file references through image preparation without resolving them, while keeping resize-notice numbering correct. Omit them from unsupported TUI display and Guardian image context.
- Reject image-edit requests whose recent-image window includes a file reference, preventing selection of an older inline image instead.

## Testing

Add coverage for serialization, request and rollout preservation, mixed-image history ordering, incomplete ordering metadata, tool-output truncation, and rejection of unsupported image-edit selections.

GitOrigin-RevId: 6ca20a8577155cc934b720803c3b7b3bffdf972a
2026-09-15 21:19:08 +00:00
Krish Chainani 5a66d460d3 Refactor image content to use a shared ImageReference type (#45543)
## What changed

Represent images in `ContentItem` and `FunctionCallOutputContentItem` with `ImageReference::Inline`, flattened to preserve the existing `image_url` wire format. Update image producers and consumers and regenerate app-server schemas and SDK artifacts.

Preserve the Python SDK's `InputImageContentItem` and `InputImageFunctionCallOutputContentItem` class names during generation.

## Testing

Add a regression test for stable Python image class names and adapt existing image tests to the shared representation.

GitOrigin-RevId: c38a780ac3314c2ac2deb3afc1b93b94b6f93fec
2026-09-14 23:25:09 +00:00
Abhinav aa88a0333c Preserve tool output truncation budgets across resume and fork (#44248)
## Why

Replaying tool outputs under a different model can expand or shrink the history shown to the model if truncation uses the new model's budget.

## What changed

Save the originating history truncation budget on function and custom tool outputs and reuse it during replay, preserving existing tool-specific overrides. Include the existing 20% serialization allowance once, before converting byte budgets to tokens.

Rename the metadata field to `history_truncation_token_limit` while retaining `fallback_token_limit_override` as its serialized name for compatibility.

## Testing

Add regression coverage for resume and fork with different model budgets, including custom tool outputs. Extend unit coverage for existing overrides and byte-budget conversion.

GitOrigin-RevId: a62c8fd3e43198e2252b150fba72f022939c1070
2026-09-09 17:55:00 +00:00
pakrym-oai f742dabc6f Support per-tool MCP output limits (#41421)
## What changed

- Add a positive `output_token_limit` setting to each entry under an MCP server's `tools` configuration.
- Apply the most restrictive limit when plugin and user policies overlap, while keeping approval policy independent.
- Carry the effective MCP output budget in conversation history so tool output, post-tool hook responses, and resumed sessions use the same truncation limit.

## Testing

- Cover configuration parsing, serialization, schema validation, and plugin policy merging.
- Cover MCP output below and above the configured limit, post-tool hook responses, and session resume.

GitOrigin-RevId: d0beb4fca9ba6055d9e1d31c137373b465d50d61
2026-08-28 21:40:23 +00:00
pakrym-oai 75cb7c903d Preserve MCP tool output as content items (#40737)
## What changed

- Convert unstructured MCP results into typed function-call output items instead of serializing the entire content array as a text string.
- Keep structured MCP results as serialized text and preserve media, encrypted content, and unknown content through their existing item conversions.
- Drop empty text items during output truncation so they do not consume API array slots.

## Testing

- Cover text-only, mixed unstructured, structured, and image-sanitized MCP results.
- Verify empty text items are discarded under byte- and token-based truncation policies.

GitOrigin-RevId: fa1c5b7dee6f003a094265379dcabdd060386a48
2026-08-25 23:42:50 +00:00
nhamidi-oai 6f785632b0 Preserve audio across history and tool outputs (#34385)
## Why

Audio attachments could be dropped from tool outputs and reconstructed thread history, while inline base64 payload size could overstate their model context cost.

## What changed

- Preserve remote and local audio attachments in legacy user-message events and thread history.
- Keep audio from function and custom tool outputs when the model supports it and the output fits the truncation budget; otherwise replace it with an omission marker.
- Estimate inline audio cost from its duration, with a serialized-size fallback for malformed or unsupported data URLs, so compaction and truncation use model-visible cost instead of raw base64 size.
- Replace audio in messages and tool outputs with an explanatory placeholder for models without audio input support.

## Testing

Add coverage for history round trips, dynamic tool responses, modality filtering, duration-based context estimates, output-budget truncation, and remote compaction.

GitOrigin-RevId: 5ed4628850a668f8383b7cd73db71b144cc02cd1
2026-07-20 17:36:44 +00:00
nhamidi-oai 97dbd01a3d Add audio variants to user input protocols (#33923)
## What changed

- Add remote and local audio variants to core and app-server user inputs, including generated JSON and TypeScript schemas.
- Preserve audio fields when converting app-server inputs and response content into thread items.
- Keep unsupported audio out of model requests, TUI rendering, text extraction, and truncated tool output, using explicit text placeholders where audio is submitted as a model input.

## Testing

- Cover app-server media conversion, unsupported-input placeholders, TUI omission, and tool-output truncation.

GitOrigin-RevId: 16152bf3af674575ce569764a8b472e032e7569d
2026-07-17 23:14:37 +00:00
Ahmed Ibrahim 952656356a [codex] Warn clearly when code mode output is truncated (#28467)
## Summary

- make `formatted_truncate_text` prepend `Warning: truncated output
(original token count: N)` above the existing `Total output lines`
header
- update direct formatter, unified-exec, user-shell, and code-mode
expectations
- add core unit coverage that runs in Bazel without requiring the
skipped V8-backed code-mode integration suite

## Validation

- `cargo test -p codex-utils-output-truncation -- --nocapture` (17
passed)
- `cargo test -p codex-core --lib
truncated_text_output_starts_with_warning -- --nocapture`
- `cargo test -p codex-core --test all
clamps_model_requested_max_output_tokens_to_policy -- --nocapture` (2
passed)
- `cargo test -p codex-core --test all
unified_exec_formats_large_output_summary -- --nocapture`
- `cargo test -p codex-core --test all
user_shell_command_output_is_truncated_in_history -- --nocapture`
- Bazel CI exercises the shared formatter and downstream integration
expectations
2026-06-16 10:37:06 -07:00
sayan-oai 34aad43684 add encryptedcontent to functioncalloutput (#23500)
add new `EncryptedContent` variant to `FunctionCallOutputContentItem`
ahead of standalone websearch.

we need to be able to receive and pass encrypted function call output
from the new web search endpoint back to responsesapi, as we cannot
expose direct search results.
2026-05-19 23:47:48 -07:00
Charlie MarshandCodex 54ef99a365 Disable empty Cargo test targets (#21584)
## Summary

`cargo test` has entails both running standard Rust tests and doctests.
It turns out that the doctest discovery is fairly slow, and it's a cost
you pay even for crates that don't include any doctests.

This PR disables doctests with `doctest = false` for crates that lack
any doctests.

For the collection of crates below, this speeds up test execution by
>4x.

E.g., before this PR:

```
Benchmark 1: cargo test     -p codex-utils-absolute-path     -p codex-utils-cache     -p codex-utils-cli     -p codex-utils-home-dir     -p codex-utils-output-truncation     -p codex-utils-path     -p codex-utils-string     -p codex-utils-template     -p codex-utils-elapsed     -p codex-utils-json-to-toml
  Time (mean ± σ):      1.849 s ±  4.455 s    [User: 0.752 s, System: 1.367 s]
  Range (min … max):    0.418 s … 14.529 s    10 runs
```

And after:

```
Benchmark 1: cargo test     -p codex-utils-absolute-path     -p codex-utils-cache     -p codex-utils-cli     -p codex-utils-home-dir     -p codex-utils-output-truncation     -p codex-utils-path     -p codex-utils-string     -p codex-utils-template     -p codex-utils-elapsed     -p codex-utils-json-to-toml
  Time (mean ± σ):     428.6 ms ±   6.9 ms    [User: 187.7 ms, System: 219.7 ms]
  Range (min … max):   418.0 ms … 436.8 ms    10 runs
```

For a single crate, with >2x speedup, before:

```
Benchmark 1: cargo test -p codex-utils-string
  Time (mean ± σ):     491.1 ms ±   9.0 ms    [User: 229.8 ms, System: 234.9 ms]
  Range (min … max):   480.9 ms … 512.0 ms    10 runs
```

And after:

```
Benchmark 1: cargo test -p codex-utils-string
  Time (mean ± σ):     213.9 ms ±   4.3 ms    [User: 112.8 ms, System: 84.0 ms]
  Range (min … max):   206.8 ms … 221.0 ms    13 runs
```

Co-authored-by: Codex <noreply@openai.com>
2026-05-07 15:44:17 -07:00
pakrym-oai 53b1570367 Update image outputs to default to high detail (#18386)
Do not assume the default `detail`.
2026-04-18 11:01:12 -07:00
Ahmed IbrahimandCodex 062fa7a2bb Move string truncation helpers into codex-utils-string (#15572)
- move the shared byte-based middle truncation logic from `core` into
`codex-utils-string`
- keep token-specific truncation in `codex-core` so rollout can reuse
the shared helper in the next stacked PR

---------

Co-authored-by: Codex <noreply@openai.com>
2026-03-24 15:45:40 -07:00