Commit Graph
5585 Commits
Author SHA1 Message Date
rhan-oai fb1e0bcd15 merge commit for archive created by Sapling 2026-04-10 12:58:40 -07:00
rhan-oai c36ffd6555 [codex-analytics] denormalize thread metadata onto turn events 2026-04-10 12:26:37 -07:00
rhan-oai 24c54673fe Merge c815dc6870 into sapling-pr-archive-rhan-oai 2026-04-10 11:49:16 -07:00
rhan-oai c815dc6870 [codex-analytics] denormalize thread metadata onto turn events 2026-04-10 11:45:41 -07:00
rhan-oai b940eee9ab [codex-analytics] add steering metadata 2026-04-10 11:37:58 -07:00
rhan-oai 761d713d78 merge commit for archive created by Sapling 2026-04-10 11:20:31 -07:00
rhan-oai 0294922b78 [codex-analytics] denormalize thread metadata onto turn events 2026-04-10 11:16:21 -07:00
rhan-oai 2b2db7bb97 [codex-analytics] add steering metadata 2026-04-10 11:16:21 -07:00
rhan-oai 62c5934b0e merge commit for archive created by Sapling 2026-04-09 17:26:29 -07:00
rhan-oai af2db0f369 [codex-analytics] add token usage metadata 2026-04-09 17:24:11 -07:00
rhan-oai 3cbd03b1af [codex-analytics] denormalize thread metadata onto turn events 2026-04-09 17:24:11 -07:00
rhan-oai dd2f563418 [codex-analytics] feature plumbing and emittance 2026-04-09 17:24:11 -07:00
rhan-oai 135cd488f7 [codex-analytics] add steering metadata 2026-04-09 17:24:11 -07:00
rhan-oai 6b0d203517 Merge 49a1322ec3 into sapling-pr-archive-rhan-oai 2026-04-09 17:02:59 -07:00
rhan-oai 49a1322ec3 [codex-analytics] denormalize thread metadata onto turn events 2026-04-09 16:53:36 -07:00
rhan-oai 459295c066 [codex-analytics] add steering metadata 2026-04-09 16:53:36 -07:00
rhan-oai 3f7d0bfb0f [codex-analytics] add token usage metadata 2026-04-09 16:53:36 -07:00
rhan-oai 9098f5f1af [codex-analytics] feature plumbing and emittance 2026-04-09 16:53:32 -07:00
rhan-oai 0020942090 merge commit for archive created by Sapling 2026-04-09 16:41:29 -07:00
rhan-oai 69dd52ae19 [codex-analytics] denormalize thread metadata onto turn events 2026-04-09 16:39:39 -07:00
rhan-oai ba4de0d6d6 [codex-analytics] add steering metadata 2026-04-09 16:39:39 -07:00
rhan-oai e8af2e57f1 [codex-analytics] add token usage metadata 2026-04-09 16:39:39 -07:00
rhan-oai b7e0069d45 [codex-analytics] feature plumbing and emittance 2026-04-09 16:39:39 -07:00
rhan-oai 6e1dd7e34d merge commit for archive created by Sapling 2026-04-09 15:34:04 -07:00
rhan-oai 92f515bdf8 [codex-analytics] denormalize thread metadata onto turn events 2026-04-09 15:31:46 -07:00
rhan-oai 05cd2af4d5 [codex-analytics] add steering metadata 2026-04-09 15:31:46 -07:00
Ahmed Ibrahim ecca34209d Omit empty app-server instruction overrides (#17258)
## Summary
- omit serialized Responses instructions when an app-server base
instruction override is empty
- skip empty developer instruction messages and add v2 coverage for the
empty-override request shape

## Validation
- just fmt
- git diff --check
2026-04-09 15:29:35 -07:00
rhan-oai 0681ff6622 [codex-analytics] add token usage metadata 2026-04-09 15:26:54 -07:00
rhan-oai f58ed8990c [codex-analytics] feature plumbing and emittance 2026-04-09 15:26:51 -07:00
rhan-oai e87d8f3d37 Merge f81b862ef3 into sapling-pr-archive-rhan-oai 2026-04-09 14:50:25 -07:00
rhan-oai 350c4b4174 merge commit for archive created by Sapling 2026-04-09 14:48:54 -07:00
rhan-oai f81b862ef3 [codex-analytics] denormalize thread metadata onto turn events 2026-04-09 14:39:39 -07:00
rhan-oai d86cec92be [codex-analytics] add steering metadata 2026-04-09 14:39:31 -07:00
rhan-oai d7940aa8fd [codex-analytics] add token usage metadata 2026-04-09 14:38:31 -07:00
Ruslan Nigmatullin ff1ab61e4f app-server: Fix clippy by removing extra mut (#17262) 2026-04-09 14:30:18 -07:00
rhan-oai cb42dd28f8 [codex-analytics] feature plumbing and emittance 2026-04-09 14:23:25 -07:00
rhan-oai b3da3d2d8a merge commit for archive created by Sapling 2026-04-09 14:11:17 -07:00
ef330eff6d feat(tui): Ctrl+O copy hotkey and harden copy-as-markdown behavior (#16966)
## TL;DR

- New `Ctrl+O` shortcut on top of the existing `/copy` command, allowing
users to copy the latest agent response without having to cancel a plan
or type `/copy`
- Copy server clipboard to the client over SSH (OSC 52)
- Fixes linux copy behavior: a clipboard handle has to be kept alive
while the paste happens for the contents to be preserved
- Uses arboard as primary mechanism on Windows, falling back to
PowerShell copy clipboard function
- Works with resumes, rolling back during a session, etc.

Tested on macOS, Linux/X11, Windows WSL2, Windows cmd.exe, Windows
PowerShell, Windows VSCode PowerShell, Windows VSCode WSL2, SSH (macOS
-> macOS).

## Problem

The TUI's `/copy` command was fragile. It relied on a single
`last_copyable_output` field that was bluntly cleared on every rollback
and thread reconfiguration, making copied content unavailable after
common operations like backtracking. It also had no keyboard shortcut,
requiring users to type `/copy` each time. The previous clipboard
backend mixed platform selection policy with low-level I/O in a way that
was hard to test, and it did not keep the Linux clipboard owner alive —
meaning pasted content could vanish once the process that wrote it
dropped its `arboard::Clipboard`.

This addresses the text-copy failure modes reported in #12836, #15452,
and #15663: native Linux clipboard access failing in remote or
unreachable-display environments, copy state going blank even after
visible assistant output, and local Linux X11 reporting success while
leaving the clipboard empty.

## Shortcut rationale

The copy hotkey is `Ctrl+O` rather than `Alt+C` because Alt/Option
combinations are not delivered consistently by macOS terminal emulators.
Terminal.app and iTerm2 can treat Option as text input or as a
configurable Meta/Esc prefix, and Option+C may be consumed or
transformed before the TUI sees an `Alt+C` key event. `Ctrl+O` is a
stable control-key chord in Terminal.app, iTerm2, SSH, and the existing
cross-platform terminal stack.

## Mental model

Agent responses are now tracked as a bounded, ordinal-indexed history
(`agent_turn_markdowns: Vec<AgentTurnMarkdown>`) rather than a single
nullable string. Each completed agent turn appends an entry keyed by its
ordinal (the number of user turns seen so far). Rollbacks pop entries
whose ordinal exceeds the remaining turn count, then use the visible
transcript cells as a best-effort fallback if the ordinal history no
longer has a surviving entry. This means `/copy` and `Ctrl+O` reflect
the most recent surviving agent response after a backtrack, instead of
going blank.

The clipboard backend was rewritten as `clipboard_copy.rs` with a
strategy-injection design: `copy_to_clipboard_with` accepts closures for
the OSC 52, arboard, and WSL PowerShell paths, making the selection
logic fully unit-testable without touching real clipboards. On Linux,
the `Clipboard` handle is returned as a `ClipboardLease` stored on
`ChatWidget`, keeping X11/Wayland clipboard ownership alive for the
lifetime of the TUI. When native copy fails under WSL, the backend now
tries the Windows clipboard through PowerShell before falling back to
OSC 52.

## Non-goals

- This change does not introduce rich-text (HTML) clipboard support; the
copied content is raw markdown.
- It does not add a paste-from-history picker or multi-entry clipboard
ring.
- WSL support remains a best-effort fallback, not a new configuration
surface or guarantee for every terminal/host combination.

## Tradeoffs

- **Bounded history (256 entries)**: `MAX_AGENT_COPY_HISTORY` caps
memory. For sessions with thousands of turns this silently drops the
oldest entries. The cap is generous enough for realistic sessions.
- **`saw_copy_source_this_turn` flag**: Prevents double-recording when
both `AgentMessage` and `TurnComplete.last_agent_message` fire for the
same turn. The flag is reset on turn start and on turn complete,
creating a narrow window where a race between the two events could
theoretically skip recording. In practice the protocol delivers them
sequentially.
- **Transcript fallback on rollback**:
`last_agent_markdown_from_transcript` walks the visible transcript cells
to reconstruct plain text when the ordinal history has been fully
truncated. This path uses `AgentMessageCell::plain_text()` which joins
rendered spans, so it reconstructs display text rather than the original
raw markdown. It keeps visible text copyable after rollback, but
responses with markdown-specific syntax can diverge from the original
source.
- **Clipboard fallback ordering**: SSH still uses OSC 52 exclusively
because native/PowerShell clipboard access would target the wrong
machine. Local sessions try native clipboard first, then WSL PowerShell
when running under WSL, then OSC 52. This adds one process-spawn
fallback for WSL users but keeps the normal desktop and SSH paths
simple.

## Architecture

```
chatwidget.rs
├── agent_turn_markdowns: Vec<AgentTurnMarkdown>  // ordinal-indexed history
├── last_agent_markdown: Option<String>            // always == last entry's markdown
├── completed_turn_count: usize                    // incremented when user turns enter history
├── saw_copy_source_this_turn: bool                // dedup guard
├── clipboard_lease: Option<ClipboardLease>        // keeps Linux clipboard owner alive
│
├── record_agent_markdown(&str)                    // append/update history entry
├── truncate_agent_turn_markdowns_to_turn_count()  // rollback support
├── copy_last_agent_markdown()                     // public entry point (slash + hotkey)
└── copy_last_agent_markdown_with(fn)              // testable core

clipboard_copy.rs
├── copy_to_clipboard(text) -> Result<Option<ClipboardLease>>
├── copy_to_clipboard_with(text, ssh, wsl, osc52_fn, arboard_fn, wsl_fn)
├── ClipboardLease { _clipboard on linux }
├── arboard_copy(text)          // platform-conditional native clipboard path
├── wsl_clipboard_copy(text)    // WSL PowerShell fallback
├── osc52_copy(text)            // /dev/tty -> stdout fallback
├── SuppressStderr              // macOS stderr redirect guard
├── is_ssh_session()
└── is_wsl_session()

app_backtrack.rs
├── last_agent_markdown_from_transcript()  // reconstruct from visible cells
└── truncate call sites in trim/apply_confirmed_rollback
```

## Observability

- `tracing::warn!` on native clipboard failure before OSC 52 fallback.
- `tracing::debug!` on `/dev/tty` open/write failure before stdout
fallback.
- History cell messages: "Copied last message to clipboard", "Copy
failed: {error}", "No agent response to copy" appear in the TUI
transcript.

## Tests

- `clipboard_copy.rs`: Unit tests cover OSC 52 encoding roundtrip,
payload size rejection, writer output, SSH-only OSC52 routing, non-WSL
native-to-OSC52 fallback, WSL native-to-PowerShell fallback, WSL
PowerShell-to-OSC52 fallback, and all-error reporting via strategy
injection.
- `chatwidget/tests/slash_commands.rs`: Updated existing `/copy` tests
to use `last_agent_markdown_text()` accessor. Added coverage for the
Linux clipboard lease lifecycle, missing
`TurnComplete.last_agent_message` fallback through completed assistant
items, replayed legacy agent messages, stale-output prevention after
rollback, and the `Ctrl+O` no-output hotkey path.
- `app_backtrack.rs`: Added
`agent_group_count_ignores_context_compacted_marker` verifying that
info-event cells don't inflate the agent group count.

---------

Co-authored-by: Felipe Coury <felipe.coury@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 18:10:38 -03:00
rhan-oai 3dab8506dc [codex-analytics] denormalize thread metadata onto turn events 2026-04-09 14:07:24 -07:00
rhan-oai f695c726df [codex-analytics] add steering metadata 2026-04-09 14:07:19 -07:00
Matthew Zeng d7f99b0fa6 [mcp] Expand tool search to custom MCPs. (#16944)
- [x] Expand tool search to custom MCPs.
- [x] Rename several variables/fields to be more generic.

Updated tool & server name lifecycles:

**Raw Identity**

ToolInfo.server_name is raw MCP server name.
ToolInfo.tool.name is raw MCP tool name.
MCP calls route back to raw via parse_tool_name() returning
(tool.server_name, tool.tool.name).
mcpServerStatus/list now groups by raw server and keys tools by
Tool.name: mod.rs:599
App-server just forwards that grouped raw snapshot:
codex_message_processor.rs:5245

**Callable Names**

On list-tools, we create provisional callable_namespace / callable_name:
mcp_connection_manager.rs:1556
For non-app MCP, provisional callable name starts as raw tool name.
For codex-apps, provisional callable name is sanitized and strips
connector name/id prefix; namespace includes connector name.
Then qualify_tools() sanitizes callable namespace + name to ASCII alnum
/ _ only: mcp_tool_names.rs:128
Note: this is stricter than Responses API. Hyphen is currently replaced
with _ for code-mode compatibility.

**Collision Handling**

We do initially collapse example-server and example_server to the same
base.
Then qualify_tools() detects distinct raw namespace identities behind
the same sanitized namespace and appends a hash to the callable
namespace: mcp_tool_names.rs:137
Same idea for tool-name collisions: hash suffix goes on callable tool
name.
Final list_all_tools() map key is callable_namespace + callable_name:
mcp_connection_manager.rs:769

**Direct Model Tools**

Direct MCP tool declarations use the full qualified sanitized key as the
Responses function name.
The raw rmcp Tool is converted but renamed for model exposure.

**Tool Search / Deferred**

Tool search result namespace = final ToolInfo.callable_namespace:
tool_search.rs:85
Tool search result nested name = final ToolInfo.callable_name:
tool_search.rs:86
Deferred tool handler is registered as "{namespace}:{name}":
tool_registry_plan.rs:248
When a function call comes back, core recombines namespace + name, looks
up the full qualified key, and gets the raw server/tool for MCP
execution: codex.rs:4353

**Separate Legacy Snapshot**

collect_mcp_snapshot_from_manager_with_detail() still returns a map
keyed by qualified callable name.
mcpServerStatus/list no longer uses that; it uses
McpServerStatusSnapshot, which is raw-inventory shaped.
2026-04-09 13:34:52 -07:00
rhan-oai 954b866946 Merge 5a701c08c1 into sapling-pr-archive-rhan-oai 2026-04-09 13:33:10 -07:00
rhan-oai 2f9a1ff0f1 [codex-analytics] add token usage metadata 2026-04-09 13:28:52 -07:00
rhan-oai 5a701c08c1 [codex-analytics] denormalize thread metadata onto turn events 2026-04-09 13:28:52 -07:00
rhan-oai dbb736c5e1 [codex-analytics] feature plumbing and emittance 2026-04-09 13:28:52 -07:00
rhan-oai 714613c189 [codex-analytics] add steering metadata 2026-04-09 13:28:52 -07:00
rhan-oai 517959d3cd merge commit for archive created by Sapling 2026-04-09 13:24:02 -07:00
rhan-oai 3eb168c1d8 [codex-analytics] denormalize thread metadata onto turn events 2026-04-09 13:04:45 -07:00
rhan-oai f751c655ac [codex-analytics] add steering metadata 2026-04-09 13:01:00 -07:00
rhan-oai fc992fe268 [codex-analytics] add token usage metadata 2026-04-09 13:01:00 -07:00