Gate audio history by model input modalities (#33982)

## What changed

- Add `audio` to the model input modalities exposed by the protocol and generated app-server schemas.
- Preserve audio in prompts for models that advertise audio input, and replace historical audio with an omission marker for models that do not.

## Testing

- Cover prompt normalization with and without audio support.
- Extend the model-switching integration test to verify that switching from a multimodal model to a text-only model strips prior image and audio content.

GitOrigin-RevId: c18e47ad602720216303ff9abb5bca4d508391ee
This commit is contained in:
nhamidi-oai
2026-07-18 09:13:38 +00:00
committed by copyberry
parent 5c0e582c59
commit 56395bddaf
11 changed files with 146 additions and 20 deletions
+7 -2
View File
@@ -157,6 +157,8 @@ pub enum InputModality {
Text,
/// Image attachments included in user turns.
Image,
/// Audio attachments included in user turns.
Audio,
}
/// Backward-compatible default when `input_modalities` is omitted on the wire.
@@ -1101,12 +1103,15 @@ mod tests {
"context_window": null,
"auto_compact_token_limit": null,
"effective_context_window_percent": 95,
"experimental_supported_tools": [],
"input_modalities": ["text", "image"]
"experimental_supported_tools": []
}))
.expect("deserialize model info");
assert_eq!(model.availability_nux, None);
assert_eq!(
model.input_modalities,
vec![InputModality::Text, InputModality::Image]
);
assert!(!model.include_skills_usage_instructions);
assert!(model.supports_reasoning_summary_parameter);
assert!(!model.supports_image_detail_original);