Add opt-in file-backed tool audio (#61394)

## What changed

- Add the disabled-by-default `file_backed_tool_audio` feature to prepare fresh function and custom tool audio through the attachment store before persistence. Store returned file IDs in history and preserve them on replay without uploading again.
- Validate and budget mixed text and audio output before uploading retained clips, preserving audio token charges and producer limits. Resolve tool audio durations before continuation compaction checks.
- Keep inline audio on backend upload failures and use text placeholders for invalid or rejected clips. Replace unavailable tool audio files with placeholders in prompts while retaining the original history, so restored files can be used again.
- Treat file-backed tool audio as a competing source when attributing audio or unknown-file errors to user uploads.

## Testing

Add coverage for direct tool and Code Mode audio across follow-up turns, file expiration, restoration, and resume in both legacy and paginated history. Extend tests for truncation before upload, upload failure handling, bounded error placeholders, and error attribution.

GitOrigin-RevId: 66c1e7699c7d92f98c3a2d0f5cecbbd63147eb6c
This commit is contained in:
Hung
2026-11-13 23:02:16 +00:00
committed by copyberry
parent f941a5fb92
commit 5158e9780e
32 changed files with 1283 additions and 110 deletions
+8
View File
@@ -2532,6 +2532,14 @@ impl Default for FunctionCallOutputBody {
}
impl FunctionCallOutputPayload {
pub fn contains_audio(&self) -> bool {
self.content_items().is_some_and(|parts| {
parts
.iter()
.any(|part| matches!(part, FunctionCallOutputContentItem::InputAudio { .. }))
})
}
pub fn from_text(content: String) -> Self {
Self {
body: FunctionCallOutputBody::Text(content),