Add Frameless Bidi support for realtime conversations (#33261)

## What changed

- Add realtime conversation version `v3`, which preserves the V1 Codex Voice behavior while using Frameless Bidi `delegation.*` events.
- Translate audio, transcripts, handoffs, session context, and lifecycle events between the app server and the Frameless Bidi wire protocol.
- Support `v3` over WebSocket and WebRTC, including the Frameless `/live` endpoint, session configuration, headers, and default model selection.
- Update the app-server protocol schemas and documentation for the new version.

## Testing

- Add unit coverage for Frameless event parsing, outbound messages, context chunking, URL construction, and call creation.
- Add app-server end-to-end coverage for WebSocket delegation and WebRTC session startup.

GitOrigin-RevId: 79a3307bc209e1a54582ebd2febc07c909fca016
This commit is contained in:
jiayuhuang-openai
2026-07-15 05:59:37 +00:00
committed by copyberry
parent 768330dd6c
commit 2e1607ee2f
24 changed files with 1117 additions and 147 deletions
+4 -3
View File
@@ -211,9 +211,9 @@ pub struct ConversationStartParams {
pub codex_responses_as_items: bool,
/// Optional prefix added to automatic Codex response items when `codex_responses_as_items` is set.
pub codex_response_item_prefix: Option<String>,
/// Optional prefix added to automatic V1 Codex commentary sent with
/// `conversation.handoff.append` when `codex_responses_as_items` is not set. Final answers are
/// sent without the prefix.
/// Optional prefix added to automatic V1 or V3 Codex commentary sent through the selected
/// Bidi handoff wire event when `codex_responses_as_items` is not set. Final answers are sent
/// without the prefix.
pub codex_response_handoff_prefix: Option<String>,
/// Overrides the configured realtime model for this session only.
pub model: Option<String>,
@@ -1621,6 +1621,7 @@ pub enum RealtimeConversationVersion {
V1,
#[default]
V2,
V3,
}
#[derive(Debug, Clone, Deserialize, Serialize, PartialEq, JsonSchema, TS)]