Add audio output support to dynamic tools and code mode (#34080)

## What changed

- Add `inputAudio` content items to dynamic tool responses, app-server events, thread history, and generated protocol schemas.
- Add an `audio()` code-mode helper that accepts inline data URLs, audio URL objects, and MCP audio blocks.
- Convert MCP audio blocks into model input when audio is supported, and replace unsupported audio with an explanatory text item.
- Reject non-data audio URLs and track audio item counts in dynamic tool analytics.

## Testing

- Cover audio serialization, protocol round trips, thread-history conversion, MCP modality filtering, code-mode helper inputs, and invalid URL handling.

GitOrigin-RevId: 1ed52a8f9c62d4840fb71c5ec736b4a3566243d6
This commit is contained in:
nhamidi-oai
2026-07-18 23:22:13 +00:00
committed by copyberry
parent 312caf176a
commit 643de86a19
46 changed files with 987 additions and 72 deletions
@@ -42,6 +42,9 @@ impl IntoProtocol<FunctionCallOutputContentItem>
.or(Some(DEFAULT_IMAGE_DETAIL)),
}
}
codex_code_mode::FunctionCallOutputContentItem::InputAudio { audio_url } => {
FunctionCallOutputContentItem::InputAudio { audio_url }
}
}
}
}