Commit Graph

5070 Commits

Author SHA1 Message Date
Michael Bolin
2538a5fd23 merge commit for archive created by Sapling 2025-11-20 15:23:13 -08:00
Michael Bolin
781e03664c feat: codex-shell-tool-mcp 2025-11-20 15:23:05 -08:00
Michael Bolin
782ed9af2e feat: waiting for an elicitation should not count against a shell tool timeout 2025-11-20 15:23:05 -08:00
Michael Bolin
208863fdd6 feat: update process_exec_tool_call() to take a cancellation token 2025-11-20 15:23:04 -08:00
Celia Chen
9bce050385 [app-server & core] introduce new codex error code and v2 app-server error events (#6938)
This PR does two things:
1. populate a new `codex_error_code` protocol in error events sent from
core to client;
2. old v1 core events `codex/event/stream_error` and `codex/event/error`
will now both become `error`. We also show codex error code for
turncompleted -> error status.

new events in app server test:
```
< {
<   "method": "codex/event/stream_error",
<   "params": {
<     "conversationId": "019aa34c-0c14-70e0-9706-98520a760d67",
<     "id": "0",
<     "msg": {
<       "codex_error_code": {
<         "response_stream_disconnected": {
<           "http_status_code": 401
<         }
<       },
<       "message": "Reconnecting... 2/5",
<       "type": "stream_error"
<     }
<   }
< }

 {
<   "method": "error",
<   "params": {
<     "error": {
<       "codexErrorCode": {
<         "responseStreamDisconnected": {
<           "httpStatusCode": 401
<         }
<       },
<       "message": "Reconnecting... 2/5"
<     }
<   }
< }

< {
<   "method": "turn/completed",
<   "params": {
<     "turn": {
<       "error": {
<         "codexErrorCode": {
<           "responseTooManyFailedAttempts": {
<             "httpStatusCode": 401
<           }
<         },
<         "message": "exceeded retry limit, last status: 401 Unauthorized, request id: 9a1b495a1a97ed3e-SJC"
<       },
<       "id": "0",
<       "items": [],
<       "status": "failed"
<     }
<   }
< }
```
2025-11-20 23:06:55 +00:00
Michael Bolin
4d03259976 merge commit for archive created by Sapling 2025-11-20 15:05:45 -08:00
Michael Bolin
9fb1355710 feat: codex-shell-tool-mcp 2025-11-20 15:05:30 -08:00
Michael Bolin
a0e90b219c feat: waiting for an elicitation should not count against a shell tool timeout 2025-11-20 15:05:30 -08:00
Michael Bolin
aec9655023 feat: update process_exec_tool_call() to take a cancellation token 2025-11-20 14:59:48 -08:00
iceweasel-oai
3f92ad4190 add deny ACEs for world writable dirs (#7022)
Our Restricted Token contains 3 SIDs (Logon, Everyone, {WorkspaceWrite
Capability || ReadOnly Capability})

because it must include Everyone, that left us vulnerable to directories
that allow writes to Everyone. Even though those directories do not have
ACEs that enable our capability SIDs to write to them, they could still
be written to even in ReadOnly mode, or even in WorkspaceWrite mode if
they are outside of a writable root.

A solution to this is to explicitly add *Deny* ACEs to these
directories, always for the ReadOnly Capability SID, and for the
WorkspaceWrite SID if the directory is outside of a workspace root.

Under a restricted token, Windows always checks Deny ACEs before Allow
ACEs so even though our restricted token would allow a write to these
directories due to the Everyone SID, it fails first because of the Deny
ACE on the capability SID
2025-11-20 14:50:33 -08:00
Ahmed Ibrahim
54ee302a06 Attempt to fix unified_exec_formats_large_output_summary flakiness (#7029)
second attempt to fix this test after
https://github.com/openai/codex/pull/6884. I think this flakiness is
happening because yield_time is too small for a 10,000 step loop in
python.
2025-11-20 14:38:04 -08:00
Ahmed Ibrahim
44fa06ae36 fix flaky test: approval_matrix_covers_all_modes (#7028)
looks like it sometimes flake around 30. let's give it more time.
2025-11-20 14:37:42 -08:00
pakrym-oai
856f97f449 Delete shell_command feature (#7024) 2025-11-20 14:14:56 -08:00
Michael Bolin
c21cd389bc merge commit for archive created by Sapling 2025-11-20 13:50:11 -08:00
Michael Bolin
5f81440698 feat: codex-shell-tool-mcp 2025-11-20 13:50:02 -08:00
zhao-oai
fe7a3f0c2b execpolicycheck command in codex cli (#7012)
adding execpolicycheck tool onto codex cli

this is useful for validating policies (can be multiple) against
commands.

it will also surface errors in policy syntax:
<img width="1150" height="281" alt="Screenshot 2025-11-19 at 12 46
21 PM"
src="https://github.com/user-attachments/assets/8f99b403-564c-4172-acc9-6574a8d13dc3"
/>

this PR also changes output format when there's no match in the CLI.
instead of returning the raw string `noMatch`, we return
`{"noMatch":{}}`

this PR is a rewrite of: https://github.com/openai/codex/pull/6932 (due
to the numerous merge conflicts present in the original PR)

---------

Co-authored-by: Michael Bolin <mbolin@openai.com>
2025-11-20 16:44:31 -05:00
Michael Bolin
7cfd859afb feat: waiting for an elicitation should not count against a shell tool timeout 2025-11-20 13:43:05 -08:00
Michael Bolin
d41c652890 feat: update process_exec_tool_call() to take a cancellation token 2025-11-20 13:43:05 -08:00
zhao-oai
c30ca0d5b6 increasing user shell timeout to 1 hour (#7025)
setting user shell timeout to an unreasonably high value since there
isn't an easy way to have a command run without timeouts

currently, user shell commands timeout is 10 seconds
2025-11-20 13:39:16 -08:00
Weiller Carvalho
a8a6cbdd1c fix: route feedback issue links by category (#6840)
## Summary
- TUI feedback note now only links to the bug-report template when the
category is bug/bad result.
- Good result/other feedback shows a thank-you+thread ID instead of
funneling people to file a bug.
- Added a helper + unit test so future changes keep the behavior
consistent.

## Testing
  - just fmt
  - just fix -p codex-tui
  - cargo test -p codex-tui

  Fixes #6839
2025-11-20 13:20:03 -08:00
Dmitri Khokhlov
e4257f432e codex-exec: allow resume --last to read prompt #6717 (#6719)
### Description

- codex exec --json resume --last "<prompt>" bailed out because clap
treated the prompt as SESSION_ID. I removed the conflicts_with flag and
reinterpret that positional as a prompt when
--last is set, so the flow now keeps working in JSON mode.
(codex-rs/exec/src/cli.rs:84-104, codex-rs/exec/src/lib.rs:75-130)
- Added a regression test that exercises resume --last in JSON mode to
ensure the prompt is accepted and the rollout file is updated.
(codex-rs/exec/tests/suite/resume.rs:126-178)

### Testing

  - just fmt
  - cargo test -p codex-exec
  - just fix -p codex-exec
  - cargo test -p codex-exec

#6717

Signed-off-by: Dmitri Khokhlov <dkhokhlov@cribl.io>
2025-11-20 13:10:49 -08:00
Jeremy Rose
2c793083f4 tui: centralize markdown styling and make inline code cyan (#7023)
<img width="762" height="271" alt="Screenshot 2025-11-20 at 12 54 06 PM"
src="https://github.com/user-attachments/assets/10021d63-27eb-407b-8fcc-43740e3bfb0f"
/>
2025-11-20 21:06:22 +00:00
Lionel Cheng
e150798baf Bumped number of fuzzy search results from 8 to 20 (#7013)
I just noticed that in the VSCode / Codex extension when you type @ the
number of results is around 70:

- small video of searching for `mod.rs` inside `codex` repository:
https://github.com/user-attachments/assets/46e53d31-adff-465e-b32b-051c4c1c298c

- while in the CLI the number of results is currently of 8 which is
quite small:
<img width="615" height="439" alt="Screenshot 2025-11-20 at 09 42 04"
src="https://github.com/user-attachments/assets/1c6d12cb-3b1f-4d5b-9ad3-6b12975eaaec"
/>

I bumped it to 20. I had several cases where I wanted a file and did not
find it because the number of results was too small

Signed-off-by: lionel-oai <lionel@openai.com>
Co-authored-by: lionel-oai <lionel@openai.com>
2025-11-20 12:33:12 -08:00
Kyuheon Kim
33a6cc66ab fix(cli): correct mcp add usage order (#6827)
## Summary
- add an explicit `override_usage` string to `AddArgs` so clap prints
`<NAME>` before the command/url choice, matching the actual parser and
docs

### Before

Usage: codex mcp add [OPTIONS] <COMMAND|--url <URL>> <NAME>


### After

Usage: codex mcp add [OPTIONS] <NAME> [--url <URL> | -- <COMMAND>...]

---------

Signed-off-by: kyuheon-kr <kyuheon.kr@gmail.com>
2025-11-20 12:32:12 -08:00
pakrym-oai
52d0ec4cd8 Delete tiktoken-rs (#7018) 2025-11-20 11:15:04 -08:00
LIHUA
397279d46e Fix: Improve text encoding for shell output in VSCode preview (#6178) (#6182)
## 🐛 Problem

Users running commands with non-ASCII characters (like Russian text
"пример") in Windows/WSL environments experience garbled text in
VSCode's shell preview window, with Unicode replacement characters (�)
appearing instead of the actual text.

**Issue**: https://github.com/openai/codex/issues/6178

## 🔧 Root Cause

The issue was in `StreamOutput<Vec<u8>>::from_utf8_lossy()` method in
`codex-rs/core/src/exec.rs`, which used `String::from_utf8_lossy()` to
convert shell output bytes to strings. This function immediately
replaces any invalid UTF-8 byte sequences with replacement characters,
without attempting to decode using other common encodings.

In Windows/WSL environments, shell output often uses encodings like:

- Windows-1252 (common Windows encoding)
- Latin-1/ISO-8859-1 (extended ASCII)

## 🛠️ Solution

Replaced the simple `String::from_utf8_lossy()` call with intelligent
encoding detection via a new `bytes_to_string_smart()` function that
tries multiple encoding strategies:

1. **UTF-8** (fast path for valid UTF-8)
2. **Windows-1252** (handles Windows-specific characters in 0x80-0x9F
range)
3. **Latin-1** (fallback for extended ASCII)
4. **Lossy UTF-8** (final fallback, same as before)

## 📁 Changes

### New Files

- `codex-rs/core/src/text_encoding.rs` - Smart encoding detection module
- `codex-rs/core/tests/suite/text_encoding_fix.rs` - Integration tests

### Modified Files

- `codex-rs/core/src/lib.rs` - Added text_encoding module
- `codex-rs/core/src/exec.rs` - Updated StreamOutput::from_utf8_lossy()
- `codex-rs/core/tests/suite/mod.rs` - Registered new test module

##  Testing

- **5 unit tests** covering UTF-8, Windows-1252, Latin-1, and fallback
scenarios
- **2 integration tests** simulating the exact Issue #6178 scenario
- **Demonstrates improvement** over the previous
`String::from_utf8_lossy()` approach

All tests pass:

```bash
cargo test -p codex-core text_encoding
cargo test -p codex-core test_shell_output_encoding_issue_6178
```

## 🎯 Impact

-  **Eliminates garbled text** in VSCode shell preview for non-ASCII
content
-  **Supports Windows/WSL environments** with proper encoding detection
-  **Zero performance impact** for UTF-8 text (fast path)
-  **Backward compatible** - UTF-8 content works exactly as before
-  **Handles edge cases** with robust fallback mechanism

## 🧪 Test Scenarios

The fix has been tested with:

- Russian text ("пример")
- Windows-1252 quotation marks (""test")
- Latin-1 accented characters ("café")
- Mixed encoding content
- Invalid byte sequences (graceful fallback)

## 📋 Checklist

- [X] Addresses the reported issue
- [X] Includes comprehensive tests
- [X] Maintains backward compatibility
- [X] Follows project coding conventions
- [X] No breaking changes

---------

Co-authored-by: Josh McKinney <joshka@openai.com>
2025-11-20 11:04:11 -08:00
pakrym-oai
30ca89424c Always fallback to real shell (#6953)
Either cmd.exe or `/bin/sh`.
2025-11-20 10:58:46 -08:00
Eric Traut
d909048a85 Added feature switch to disable animations in TUI (#6870)
This PR adds support for a new feature flag `tui.animations`. By
default, the TUI uses animations in its welcome screen, "working"
spinners, and "shimmer" effects. This animations can interfere with
screen readers, so it's good to provide a way to disable them.

This change is inspired by [a
PR](https://github.com/openai/codex/pull/4014) contributed by @Orinks.
That PR has faltered a bit, but I think the core idea is sound. This
version incorporates feedback from @aibrahim-oai. In particular:
1. It uses a feature flag (`tui.animations`) rather than the unqualified
CLI key `no-animations`. Feature flags are the preferred way to expose
boolean switches. They are also exposed via CLI command switches.
2. It includes more complete documentation.
3. It disables a few animations that the other PR omitted.
2025-11-20 10:40:08 -08:00
Michael Bolin
00f32326ff Merge a2b797a6a9 into sapling-pr-archive-bolinfest 2025-11-20 08:33:46 -08:00
Michael Bolin
a2b797a6a9 feat: codex-shell-tool-mcp 2025-11-20 08:33:36 -08:00
jif-oai
888c6dd9e7 fix: command formatting for user commands (#7002) 2025-11-20 17:29:15 +01:00
hanson-openai
b5dd189067 Allow unified_exec to early exit (if the process terminates before yield_time_ms) (#6867)
Thread through an `exit_notify` tokio `Notify` through to the
`UnifiedExecSession` so that we can return early if the command
terminates before `yield_time_ms`.

As Codex review correctly pointed out below 🙌 we also need a
`exit_signaled` flag so that commands which finish before we start
waiting can also exit early.

Since the default `yield_time_ms` is now 10s, this means that we don't
have to wait 10s for trivial commands like ls, sed, etc (which are the
majority of agent commands 😅)

---------

Co-authored-by: jif-oai <jif@openai.com>
2025-11-20 13:34:41 +01:00
Michael Bolin
84616324fc merge commit for archive created by Sapling 2025-11-20 00:28:06 -08:00
Michael Bolin
905135c31d feat: waiting for an elicitation should not count against a shell tool timeout 2025-11-20 00:28:01 -08:00
Michael Bolin
62d92e4196 merge commit for archive created by Sapling 2025-11-20 00:20:31 -08:00
Michael Bolin
41d559f5d6 feat: waiting for an elicitation should not count against a shell tool timeout 2025-11-20 00:20:25 -08:00
Michael Bolin
1ac22a0e3d merge commit for archive created by Sapling 2025-11-20 00:08:02 -08:00
Michael Bolin
b309236d00 feat: waiting for an elicitation should not count against a shell tool timeout 2025-11-20 00:07:55 -08:00
Michael Bolin
afcf6e655c merge commit for archive created by Sapling 2025-11-20 00:03:24 -08:00
Michael Bolin
4e9941d4bf feat: waiting for an elicitation should not count against a shell tool timeout 2025-11-20 00:01:35 -08:00
Michael Bolin
bf2e713fe8 merge commit for archive created by Sapling 2025-11-19 23:08:00 -08:00
Michael Bolin
9d420b44f1 feat: update process_exec_tool_call() to take a cancellation token 2025-11-19 23:07:14 -08:00
Michael Bolin
9d2b640442 feat: waiting for an elicitation should not count against a shell tool timeout 2025-11-19 23:07:14 -08:00
Michael Bolin
94bea27841 merge commit for archive created by Sapling 2025-11-19 22:45:25 -08:00
Michael Bolin
229a73bb92 feat: update process_exec_tool_call() to take a cancellation token 2025-11-19 22:45:15 -08:00
Michael Bolin
e19009b43f Merge 6772e869fe into sapling-pr-archive-bolinfest 2025-11-19 22:43:20 -08:00
Michael Bolin
6772e869fe feat: update process_exec_tool_call() to take a cancellation token 2025-11-19 22:43:05 -08:00
Michael Bolin
54e6e4ac32 fix: when displaying execv, show file instead of arg0 (#6966)
After merging https://github.com/openai/codex/pull/6958, I realized that
the `command` I was displaying was not quite right. Since we know it, we
should show the _exact_ program being executed (the first arg to
`execve(3)`) rather than `arg0` to be more precise.

Below is the same command I used to test
https://github.com/openai/codex/pull/6958, but now you can see it shows
`/Users/mbolin/.openai/bin/git` instead of just `git`.

<img width="1526" height="1444" alt="image"
src="https://github.com/user-attachments/assets/428128d1-c658-456e-a64e-fc6a0009cb34"
/>
2025-11-19 22:42:58 -08:00
Michael Bolin
2cbba0850d Merge e7ed3c2884 into sapling-pr-archive-bolinfest 2025-11-19 21:00:17 -08:00
Michael Bolin
e7ed3c2884 fix: when displaying execv, show file instead of arg0 2025-11-19 21:00:07 -08:00