nhamidi-oai 6f785632b0 Preserve audio across history and tool outputs (#34385) ...

## Why

Audio attachments could be dropped from tool outputs and reconstructed thread history, while inline base64 payload size could overstate their model context cost.

## What changed

- Preserve remote and local audio attachments in legacy user-message events and thread history.
- Keep audio from function and custom tool outputs when the model supports it and the output fits the truncation budget; otherwise replace it with an omission marker.
- Estimate inline audio cost from its duration, with a serialized-size fallback for malformed or unsupported data URLs, so compaction and truncation use model-visible cost instead of raw base64 size.
- Replace audio in messages and tool outputs with an explanatory placeholder for models without audio input support.

## Testing

Add coverage for history round trips, dynamic tool responses, modality filtering, duration-based context estimates, output-budget truncation, and remote compaction.

GitOrigin-RevId: 5ed4628850a668f8383b7cd73db71b144cc02cd1

2026-07-20 17:36:44 +00:00

..

agent

Add audio variants to user input protocols (#33923)

2026-07-17 23:14:37 +00:00

apps

Make Apps guidance react to MCP availability (#30226)

2026-07-06 14:29:19 -07:00

bin

…

config

Remove unused Rust helpers (#34345)

2026-07-20 13:45:29 +00:00

context

Avoid rendering generated images twice (#34378)

2026-07-20 16:47:03 +00:00

context_manager

Preserve audio across history and tool outputs (#34385)

2026-07-20 17:36:44 +00:00

guardian

Add audio variants to user input protocols (#33923)

2026-07-17 23:14:37 +00:00

mcp_tool_call

Report MCP error codes with server attribution (#29969)

2026-06-24 21:08:39 -07:00

plugins

Make plugin guidance react to environment readiness (#30223)

2026-07-06 07:05:40 -07:00

realtime_conversation

Route realtime V3 handoffs by response channel (#33903)

2026-07-17 21:12:40 +00:00

sandboxing

Launch managed network proxies on remote executors (#33906)

2026-07-17 21:20:14 +00:00

session

Stop retrying turns with invalid tool images (#34380)

2026-07-20 16:57:03 +00:00

state

Centralize thread MCP connections in McpRuntime (#33889)

2026-07-17 18:43:14 +00:00

tasks

Launch managed network proxies on remote executors (#33906)

2026-07-17 21:20:14 +00:00

tools

Preserve audio across history and tool outputs (#34385)

2026-07-20 17:36:44 +00:00

unified_exec

Launch managed network proxies on remote executors (#33906)

2026-07-17 21:20:14 +00:00

utils

…

agent_communication.rs

Log multi-agent communication lifecycle (#30872)

2026-07-01 18:11:09 -07:00

agents_md_manager.rs

Instrument environment and plugin resolution paths (#33223)

2026-07-15 03:27:16 +00:00

agents_md_tests.rs

Keep active-turn environments stable across settings updates (#33658)

2026-07-16 18:23:43 +00:00

agents_md.rs

Keep active-turn environments stable across settings updates (#33658)

2026-07-16 18:23:43 +00:00

apply_patch_tests.rs

path-uri: clarify host-native path conversion (#29501)

2026-06-23 00:02:33 +00:00

apply_patch.rs

apply-patch: carry paths as PathUri (#28854)

2026-06-18 19:31:19 +00:00

attestation.rs

…

audio_preparation_tests.rs

Forward audio inputs to the Responses API (#33932)

2026-07-18 00:26:24 +00:00

audio_preparation.rs

Preserve audio across history and tool outputs (#34385)

2026-07-20 17:36:44 +00:00

client_common_tests.rs

[codex] Support sequential cutoff reasoning summaries (#31306)

2026-07-06 23:23:56 -04:00

client_common.rs

[codex] Use input items for Responses Lite tools (#27946)

2026-06-22 23:56:16 -07:00

client_tests.rs

Require prefixes for outbound response item IDs (#32312)

2026-07-11 02:29:14 +00:00

client.rs

Avoid cloning Responses WebSocket payloads (#34381)

2026-07-20 17:18:25 +00:00

codex_delegate_tests.rs

Keep active-turn environments stable across settings updates (#33658)

2026-07-16 18:23:43 +00:00

codex_delegate.rs

Separate session state from session I/O (#33209)

2026-07-15 02:16:53 +00:00

codex_thread.rs

Separate session state from session I/O (#33209)

2026-07-15 02:16:53 +00:00

command_canonicalization_tests.rs

…

command_canonicalization.rs

…

compact_model_fallback.rs

Broaden remote compaction model fallback (#32881)

2026-07-13 21:54:04 +00:00

compact_remote_request.rs

chore: extract remote compaction request attempts (#31316)

2026-07-07 12:57:19 -07:00

compact_remote_v2_attempt.rs

Expose exact per-response usage in raw app-server events (#32985)

2026-07-14 06:39:42 +00:00

compact_remote_v2.rs

Add audio variants to user input protocols (#33923)

2026-07-17 23:14:37 +00:00

compact_remote.rs

Derive collaboration settings from turn context (#32900)

2026-07-13 23:48:42 +00:00

compact_tests.rs

Require prefixes for outbound response item IDs (#32312)

2026-07-11 02:29:14 +00:00

compact_token_budget.rs

Derive collaboration settings from turn context (#32900)

2026-07-13 23:48:42 +00:00

compact.rs

Add audio variants to user input protocols (#33923)

2026-07-17 23:14:37 +00:00

config_lock.rs

Use plugin-service MCP as the hosted plugin runtime (#27198)

2026-06-10 12:54:21 +02:00

connectors_tests.rs

[codex-apps] Filter optional file fields by tool schema (#31686)

2026-07-09 14:32:23 -07:00

connectors.rs

Reuse MCP tool catalogs across sessions (#33184)

2026-07-14 22:25:39 +00:00

consequential_tool_message_templates.json

…

current_time.rs

[codex] route sleep through time providers (#29973)

2026-06-24 22:17:43 -07:00

elicitation_tests.rs

elicitations: Move to shared ElicitationService (#30627)

2026-07-06 11:20:32 -07:00

elicitation.rs

elicitations: Move to shared ElicitationService (#30627)

2026-07-06 11:20:32 -07:00

environment_selection.rs

Keep active-turn environments stable across settings updates (#33658)

2026-07-16 18:23:43 +00:00

event_mapping_tests.rs

Forward audio inputs to the Responses API (#33932)

2026-07-18 00:26:24 +00:00

event_mapping.rs

Forward audio inputs to the Responses API (#33932)

2026-07-18 00:26:24 +00:00

exec_env_tests.rs

core: expose permission profile to shell tools (#29941)

2026-06-25 19:00:23 +00:00

exec_env.rs

core: expose permission profile to shell tools (#29941)

2026-06-25 19:00:23 +00:00

exec_policy_tests.rs

Migrate legacy exec policy allow rules (#34271)

2026-07-20 06:13:29 +00:00

exec_policy_windows_tests.rs

[codex] Respect Windows sandbox backend in exec policy (#26307)

2026-06-05 11:20:52 -07:00

exec_policy.rs

Migrate legacy exec policy allow rules (#34271)

2026-07-20 06:13:29 +00:00

exec_tests.rs

Remove generated-default filesystem path variants (#33632)

2026-07-16 16:13:53 +00:00

exec.rs

Launch managed network proxies on remote executors (#33906)

2026-07-17 21:20:14 +00:00

function_tool.rs

…

git_info_tests.rs

Pipeline bounded AGENTS.md and Git root probes (#29870)

2026-06-24 22:58:34 +01:00

hook_runtime.rs

Add audio variants to user input protocols (#33923)

2026-07-17 23:14:37 +00:00

image_preparation_tests.rs

[codex] replace remote images with model-visible error text (#29417)

2026-06-22 16:41:00 -07:00

image_preparation.rs

[codex] Use input items for Responses Lite tools (#27946)

2026-06-22 23:56:16 -07:00

installation_id.rs

…

lib.rs

Avoid rendering generated images twice (#34378)

2026-07-20 16:47:03 +00:00

mcp_openai_file.rs

Keep active-turn environments stable across settings updates (#33658)

2026-07-16 18:23:43 +00:00

mcp_skill_dependencies.rs

Pin MCP runtimes to model steps (#30101)

2026-06-26 00:53:07 +01:00

mcp_tool_approval_templates.rs

Uprev Rust toolchain pins to 1.95.0 (#24684)

2026-05-26 20:59:47 -07:00

mcp_tool_call_tests.rs

Add audio output support to dynamic tools and code mode (#34080)

2026-07-18 23:22:13 +00:00

mcp_tool_call.rs

Add audio output support to dynamic tools and code mode (#34080)

2026-07-18 23:22:13 +00:00

mcp_tool_exposure_test.rs

Build MCP tool runtimes before router planning (#33149)

2026-07-14 20:36:29 +00:00

mcp_tool_exposure.rs

Build MCP tool runtimes before router planning (#33149)

2026-07-14 20:36:29 +00:00

mcp.rs

Add batched executor capability discovery (#33852)

2026-07-17 15:45:48 +00:00

memory_usage.rs

Tag shell tool telemetry by command category (#32887)

2026-07-13 22:34:16 +00:00

mention_syntax.rs

…

network_policy_decision_tests.rs

fix: attribut network requests to the exact exec on linux (#29697)

2026-07-06 14:51:34 -07:00

network_policy_decision.rs

…

original_image_detail.rs

…

otel_init.rs

Remove unused Rust helpers (#34345)

2026-07-20 13:45:29 +00:00

prompt_debug.rs

Separate session state from session I/O (#33209)

2026-07-15 02:16:53 +00:00

realtime_context_tests.rs

feat(app-server): add history_mode to thread (#29927)

2026-06-26 09:12:42 -07:00

realtime_context.rs

feat(app-server): list descendant threads by ancestor (#29591)

2026-06-24 13:08:14 -07:00

realtime_conversation_tests.rs

Route realtime V3 handoffs by response channel (#33903)

2026-07-17 21:12:40 +00:00

realtime_conversation.rs

Seed realtime V3 sessions with initial text items (#34067)

2026-07-18 20:59:57 +00:00

realtime_prompt.rs

[codex] Consolidate shared prompts in codex-prompts (#25151)

2026-06-01 18:45:07 +00:00

responses_metadata.rs

fix: retry rejected previous-model compaction with selected model (#30319)

2026-07-07 14:26:03 -07:00

responses_retry_tests.rs

Add context to sampling retry logs (#33963)

2026-07-18 06:34:53 +00:00

responses_retry.rs

Add context to sampling retry logs (#33963)

2026-07-18 06:34:53 +00:00

rollout_budget.rs

[codex] configure rollout budget reminder thresholds (#29423)

2026-06-22 13:25:48 -07:00

rollout.rs

app-server: remove experimental persist_extended_history bool flag (#25712)

2026-06-01 23:33:42 +00:00

safety_tests.rs

apply-patch: carry paths as PathUri (#28854)

2026-06-18 19:31:19 +00:00

safety.rs

chore(core) rm AskForApproval::OnFailure (#28418)

2026-06-23 12:13:54 -07:00

sandbox_tags_tests.rs

…

sandbox_tags.rs

…

session_prefix_tests.rs

core: surface terminal subagent errors to parent agents (#28375)

2026-06-16 14:34:54 +02:00

session_prefix.rs

core: surface terminal subagent errors to parent agents (#28375)

2026-06-16 14:34:54 +02:00

session_rollout_init_error.rs

feat(app-server): add history_mode to thread (#29927)

2026-06-26 09:12:42 -07:00

session_startup_prewarm.rs

Trace startup prewarm tasks (#33155)

2026-07-14 20:52:12 +00:00

shell_snapshot_tests.rs

Preserve zsh tied PATH exports in shell snapshots (#34293)

2026-07-20 08:29:12 +00:00

shell_snapshot.rs

Preserve zsh tied PATH exports in shell snapshots (#34293)

2026-07-20 08:29:12 +00:00

shell_tests.rs

[codex] simplify shell snapshot ownership (#27756)

2026-06-15 08:18:13 -07:00

shell.rs

[codex] simplify shell snapshot ownership (#27756)

2026-06-15 08:18:13 -07:00

skills.rs

Add a skill invocation extension contributor (#32200)

2026-07-10 15:52:04 +00:00

spawn.rs

core: stop threading SandboxPolicy through exec (#25700)

2026-06-03 10:41:41 -07:00

state_db_bridge.rs

…

stream_events_utils_tests.rs

Build MCP tool runtimes before router planning (#33149)

2026-07-14 20:36:29 +00:00

stream_events_utils.rs

Avoid rendering generated images twice (#34378)

2026-07-20 16:47:03 +00:00

test_support.rs

Remove unused Rust helpers (#34345)

2026-07-20 13:45:29 +00:00

thread_manager_tests.rs

Add batched executor capability discovery (#33852)

2026-07-17 15:45:48 +00:00

thread_manager.rs

Forward thread originators from standalone extensions (#33677)

2026-07-16 19:31:26 +00:00

thread_rollout_truncation_tests.rs

Preserve thread context when retrying or editing turns (#33211)

2026-07-15 02:35:49 +00:00

thread_rollout_truncation.rs

Preserve thread context when retrying or editing turns (#33211)

2026-07-15 02:35:49 +00:00

turn_diff_tracker_tests.rs

path-uri: clarify host-native path conversion (#29501)

2026-06-23 00:02:33 +00:00

turn_diff_tracker.rs

core: cache turn diff rendering (#27489)

2026-06-10 17:17:44 -07:00

turn_metadata_tests.rs

fix(core): restore thread_source in x-codex-turn-metadata (#29455)

2026-06-22 09:05:27 -07:00

turn_metadata.rs

fix(core): restore thread_source in x-codex-turn-metadata (#29455)

2026-06-22 09:05:27 -07:00

turn_timing_tests.rs

Preserve namespaces on custom tool calls (#30302)

2026-06-27 09:54:56 -07:00

turn_timing.rs

[codex] Support sequential cutoff reasoning summaries (#31306)

2026-07-06 23:23:56 -04:00

user_shell_command_tests.rs

…

user_shell_command.rs

core: remove redundant TurnContext and Prompt fields (#28638)

2026-06-16 22:17:24 -07:00

util_tests.rs

…

util.rs

Remove unused Rust helpers (#34345)

2026-07-20 13:45:29 +00:00

web_search.rs

Remove unused Rust helpers (#34345)

2026-07-20 13:45:29 +00:00

windows_sandbox_read_grants_tests.rs

windows-sandbox: pass workspace roots to runner (#24108)

2026-05-28 15:26:55 -07:00

windows_sandbox_read_grants.rs

windows-sandbox: pass workspace roots to runner (#24108)

2026-05-28 15:26:55 -07:00

windows_sandbox_tests.rs

config: remove legacy profile v1 resolution (#24051)

2026-05-22 12:13:52 +02:00

windows_sandbox.rs

Remove unused Rust helpers (#34345)

2026-07-20 13:45:29 +00:00