Add audio file references with explicit runtime rejection (#58430)

## What changed

- Allow audio inputs to carry uploaded file references via `fileId` in the app-server API and `file_id` in model content, preserving existing inline audio shapes and Python class names.
- Preserve mixed inline and file-backed audio ordering through history reconstruction and migration, and budget file-backed audio conservatively when duration is unknown.
- Keep uploaded audio disabled at runtime: reject it before inference or raw history injection, preserve accompanying text, and prevent rejected attachments from reaching follow-up or resumed requests.

## Testing

Add coverage for wire-format round trips, SDK compatibility, mixed audio history ordering, and rejection during turn start, steering, task completion, and raw injection.

GitOrigin-RevId: 70895c47a2494c0c9936348ad1250eb1b7b892a6
This commit is contained in:
Hung
2026-10-30 20:53:25 +00:00
committed by copyberry
parent b9858217d8
commit 2546265e3e
83 changed files with 2122 additions and 300 deletions
@@ -2731,6 +2731,32 @@
"type": "object"
},
{
"anyOf": [
{
"properties": {
"url": {
"type": "string"
}
},
"required": [
"url"
],
"title": "UrlUserInput",
"type": "object"
},
{
"properties": {
"fileId": {
"type": "string"
}
},
"required": [
"fileId"
],
"title": "FileIdUserInput",
"type": "object"
}
],
"properties": {
"type": {
"enum": [
@@ -2738,14 +2764,10 @@
],
"title": "AudioUserInputType",
"type": "string"
},
"url": {
"type": "string"
}
},
"required": [
"type",
"url"
"type"
],
"title": "AudioUserInput",
"type": "object"