nhamidi-oai 643de86a19 Add audio output support to dynamic tools and code mode (#34080) ...

## What changed

- Add `inputAudio` content items to dynamic tool responses, app-server events, thread history, and generated protocol schemas.
- Add an `audio()` code-mode helper that accepts inline data URLs, audio URL objects, and MCP audio blocks.
- Convert MCP audio blocks into model input when audio is supported, and replace unsupported audio with an explanatory text item.
- Reject non-data audio URLs and track audio item counts in dynamic tool analytics.

## Testing

- Cover audio serialization, protocol round trips, thread-history conversion, MCP modality filtering, code-mode helper inputs, and invalid URL handling.

GitOrigin-RevId: 1ed52a8f9c62d4840fb71c5ec736b4a3566243d6

2026-07-18 23:22:13 +00:00

..

snapshots

Track permission instructions in world state (#33944)

2026-07-18 02:45:55 +00:00

abort_tasks.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

additional_context.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

agent_execution.rs

Namespace multi-agent v2 tools under collaboration (#29067)

2026-06-23 14:15:20 +02:00

agent_jobs.rs

Honor configured model defaults for spawned agents (#33631)

2026-07-16 16:08:43 +00:00

agent_websocket.rs

Require prefixes for outbound response item IDs (#32312)

2026-07-11 02:29:14 +00:00

agents_md.rs

Scope runtime workspace roots to execution environments (#32952)

2026-07-14 04:41:08 +00:00

apply_patch_cli.rs

Scope runtime workspace roots to execution environments (#32952)

2026-07-14 04:41:08 +00:00

approvals.rs

Isolate core tests from shell and rollout persistence (#33858)

2026-07-17 15:58:05 +00:00

auto_review.rs

Respect model support for reasoning summaries (#32290)

2026-07-10 23:39:21 +00:00

catalog_permission_messages.rs

Support model catalog permission messages (#33147)

2026-07-14 20:14:29 +00:00

cli_stream.rs

Update models.json (#31684)

2026-07-09 03:40:08 +00:00

client_websockets.rs

Honor workspace spend controls in rate-limit handling (#33187)

2026-07-14 22:43:26 +00:00

client.rs

Gate audio history by model input modalities (#33982)

2026-07-18 09:13:38 +00:00

code_mode_elicitation.rs

code-mode: make all approvals trigger elicitation pause (#31650)

2026-07-08 15:27:04 -07:00

code_mode.rs

Add audio output support to dynamic tools and code mode (#34080)

2026-07-18 23:22:13 +00:00

codex_delegate.rs

Revert "[core] Support interleaved response items" (#31261)

2026-07-06 10:09:54 -07:00

collaboration_instructions.rs

Track collaboration mode instructions in world state (#33876)

2026-07-17 17:22:39 +00:00

compact_remote_parity.rs

Stabilize remote compaction parity against dynamic skill catalogs (#31585)

2026-07-08 15:59:36 +01:00

compact_remote.rs

Seed realtime V3 sessions with initial text items (#34067)

2026-07-18 20:59:57 +00:00

compact_resume_fork.rs

Support openai/form extended form elicitations (#27500)

2026-06-18 11:54:49 -07:00

compact.rs

Track permission instructions in world state (#33944)

2026-07-18 02:45:55 +00:00

current_time_reminder.rs

[codex] impl delivery_mode: current time reminders on response boundaries (#30033)

2026-06-25 19:28:50 +00:00

deprecation_notice.rs

Add indexed web search mode (#28489)

2026-06-19 05:35:57 -07:00

exec_policy.rs

Strengthen forced rm command detection (#33464)

2026-07-16 01:12:14 +00:00

exec.rs

Add network environment ID plumbing (#28766)

2026-06-18 14:09:38 +02:00

extension_sandbox.rs

Use the image generation extension by default (#31596)

2026-07-09 12:25:19 +09:00

external_auth.rs

[codex] Add externally provided Codex auth (#31274)

2026-07-07 17:43:19 -07:00

fork_thread.rs

Share resumed rollout history (#28426)

2026-06-23 10:23:25 -04:00

guardian_review.rs

Support model catalog templates for Guardian policy prompts (#33177)

2026-07-14 22:01:11 +00:00

hooks_mcp.rs

Represent MCP authentication with an enum (#29924)

2026-06-24 19:51:51 -07:00

hooks.rs

Add SessionEnd hooks for thread teardown (#33895)

2026-07-17 19:58:56 +00:00

image_rollout.rs

feat(core): store turn_id on ResponseItem metadata (#28360)

2026-06-22 16:45:14 -07:00

items.rs

Use the image generation extension by default (#31596)

2026-07-09 12:25:19 +09:00

json_result.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

live_cli.rs

[codex] Use expect in integration tests (#28441)

2026-06-15 21:53:47 -07:00

mcp_auth_elicitation.rs

[codex] Enable auth elicitation by default (#28772)

2026-07-07 11:48:27 -07:00

mcp_auth_refresh.rs

Reuse MCP tool catalogs across sessions (#33184)

2026-07-14 22:25:39 +00:00

mcp_refresh_cleanup.rs

Pin MCP runtimes to model steps (#30101)

2026-06-26 00:53:07 +01:00

mcp_tool_cache.rs

Expand MCP tool catalog cache regression coverage (#33308)

2026-07-15 10:18:51 +00:00

mcp_tool_exposure.rs

Centralize thread MCP connections in McpRuntime (#33889)

2026-07-17 18:43:14 +00:00

mcp_turn_metadata.rs

[codex-rs] Add writes app approval mode (#30482)

2026-07-07 13:13:59 -04:00

mod.rs

Seed realtime V3 sessions with initial text items (#34067)

2026-07-18 20:59:57 +00:00

model_overrides.rs

[5 of 7] Replace OverrideTurnContext with ThreadSettings (#22508)

2026-05-18 21:03:51 -07:00

model_runtime_selectors.rs

Unify multi-agent settings under agents (#33550)

2026-07-16 09:00:51 +00:00

model_switching.rs

Gate audio history by model input modalities (#33982)

2026-07-18 09:13:38 +00:00

model_visible_layout.rs

Refresh step world state for all sessions (#33665)

2026-07-16 18:45:42 +00:00

models_cache_ttl.rs

Respect model support for reasoning summaries (#32290)

2026-07-10 23:39:21 +00:00

models_etag_responses.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

multi_agent_mode.rs

Suppress empty multi-agent mode messages (#33862)

2026-07-17 16:10:50 +00:00

multi_agent_resume.rs

Restore agent roles when reloading v2 sub-agents (#33657)

2026-07-16 18:20:14 +00:00

multi_exec_server_sandbox.rs

Test workspace write isolation across exec servers (#33861)

2026-07-17 16:07:24 +00:00

network_approval.rs

Scope runtime workspace roots to execution environments (#32952)

2026-07-14 04:41:08 +00:00

openai_file_mcp.rs

[codex-apps] Filter optional file fields by tool schema (#31686)

2026-07-09 14:32:23 -07:00

otel.rs

Track prompt cache write token usage (#33454)

2026-07-16 00:22:53 +00:00

override_updates.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

pending_input.rs

Respect final-answer boundaries for queued agent mail (#33367)

2026-07-15 15:48:12 +00:00

permissions_messages.rs

Support model catalog permission messages (#33147)

2026-07-14 20:14:29 +00:00

personality.rs

Support model catalog permission messages (#33147)

2026-07-14 20:14:29 +00:00

plugins.rs

Make Apps guidance react to MCP availability (#30226)

2026-07-06 14:29:19 -07:00

prompt_cache_key.rs

Use session IDs for prompt cache keys (#33035)

2026-07-14 10:37:23 +00:00

prompt_caching.rs

Track permission instructions in world state (#33944)

2026-07-18 02:45:55 +00:00

prompt_debug_tests.rs

feat(core): store turn_id on ResponseItem metadata (#28360)

2026-06-22 16:45:14 -07:00

quota_exceeded.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

realtime_conversation.rs

Seed realtime V3 sessions with initial text items (#34067)

2026-07-18 20:59:57 +00:00

realtime_initial_items.rs

Seed realtime V3 sessions with initial text items (#34067)

2026-07-18 20:59:57 +00:00

remote_env.rs

Refresh step world state for all sessions (#33665)

2026-07-16 18:45:42 +00:00

remote_models.rs

Respect model support for reasoning summaries (#32290)

2026-07-10 23:39:21 +00:00

request_compression.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

request_permissions_tool.rs

[codex] Use expect in integration tests (#28441)

2026-06-15 21:53:47 -07:00

request_permissions.rs

Separate exec permission paths from core models (#33200)

2026-07-15 01:31:40 +00:00

request_plugin_install.rs

[plugins] Track plugin install requests by ID (#29684)

2026-06-24 21:29:11 +00:00

request_user_input.rs

[codex] Use expect in integration tests (#28441)

2026-06-15 21:53:47 -07:00

responses_api_proxy_headers.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

responses_lite.rs

Use the image generation extension by default (#31596)

2026-07-09 12:25:19 +09:00

resume_warning.rs

Include terminal errors in turn completion events (#32280)

2026-07-10 22:04:03 +00:00

resume.rs

Update models.json (#21818)

2026-07-08 16:24:12 -07:00

review.rs

feat(core): emit canonical review mode items (#31473)

2026-07-08 09:59:50 -07:00

rmcp_client.rs

Preserve encrypted content in MCP tool outputs (#33509)

2026-07-16 05:38:36 +00:00

rollout_budget.rs

Stabilize shared rollout budget test (#31587)

2026-07-08 16:10:28 +01:00

rollout_list_find.rs

Support thread-level originator overrides (#29477)

2026-06-23 17:23:38 -07:00

safety_buffering.rs

[codex] Read retry model from buffering events (#31262)

2026-07-06 17:04:26 +00:00

safety_check_downgrade.rs

Update models.json (#21818)

2026-07-08 16:24:12 -07:00

search_tool.rs

[codex] Restore v1 delegation guidance (#30511)

2026-06-28 20:34:47 -07:00

shell_command.rs

test: branch on target OS instead of runner flavor (#29712)

2026-06-23 14:27:13 -07:00

shell_serialization.rs

test: branch on target OS instead of runner flavor (#29712)

2026-06-23 14:27:13 -07:00

shell_snapshot.rs

[codex] Use expect in integration tests (#28441)

2026-06-15 21:53:47 -07:00

skill_approval.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

skills.rs

core tests: rename automatic environment builder (#29728)

2026-06-23 21:45:06 +00:00

spawn_agent_description.rs

Expose spawn agent types only when roles are configured (#33572)

2026-07-16 10:28:36 +00:00

sqlite_state.rs

Track inherited paginated rollout prefixes (#33930)

2026-07-18 00:06:49 +00:00

stream_error_allows_next_turn.rs

Include terminal errors in turn completion events (#32280)

2026-07-10 22:04:03 +00:00

stream_no_completed.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

subagent_notifications.rs

Validate reasoning effort after applying spawn roles (#33656)

2026-07-16 18:16:44 +00:00

token_budget.rs

Add a fallback phase before automatic context rollover (#33255)

2026-07-15 05:45:43 +00:00

tool_harness.rs

[codex] Use expect in integration tests (#28441)

2026-06-15 21:53:47 -07:00

tool_parallelism.rs

[codex] Use expect in integration tests (#28441)

2026-06-15 21:53:47 -07:00

tools.rs

Preserve namespaces on custom tool calls (#30302)

2026-06-27 09:54:56 -07:00

truncation.rs

Represent MCP authentication with an enum (#29924)

2026-06-24 19:51:51 -07:00

turn_state.rs

[codex] Use expect in integration tests (#28441)

2026-06-15 21:53:47 -07:00

unified_exec_process_events.rs

perf(skills): reuse walk inventory for host loading (#31566)

2026-07-09 09:54:24 +01:00

unified_exec_zsh_fork_approvals.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

unified_exec.rs

Run write_stdin concurrently across terminal sessions (#33645)

2026-07-16 16:58:41 +00:00

unstable_features_warning.rs

[codex] Remove child AGENTS.md prompt experiment (#28993)

2026-06-18 16:13:07 -07:00

user_notification.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

user_shell_cmd.rs

[codex] Warn clearly when code mode output is truncated (#28467)

2026-06-16 10:37:06 -07:00

view_image.rs

Scope runtime workspace roots to execution environments (#32952)

2026-07-14 04:41:08 +00:00

web_search.rs

Update models.json (#21818)

2026-07-08 16:24:12 -07:00

websocket_fallback.rs

Pair thread environment settings (#26687)

2026-06-08 13:55:15 -07:00

window_headers.rs

[codex] Use expect in integration tests (#28441)

2026-06-15 21:53:47 -07:00

windows_sandbox.rs

Remove generated-default filesystem path variants (#33632)

2026-07-16 16:13:53 +00:00

workspace_roots.rs

Propagate workspace roots to exec-server sandboxes (#32214)

2026-07-10 16:55:01 +00:00