mirror of
https://github.com/openai/codex.git
synced 2026-08-23 13:09:46 +00:00
ci: cross-compile Windows Bazel tests (#20585)
## Status This is the Bazel PR-CI cross-compilation follow-up to #20485. It is intentionally split from the Cargo/cargo-xwin release-build PoC so #20485 can stay as the historical release-build exploration. The unrelated async-utils test cleanup has been moved to #20686, so this PR is focused on the Windows Bazel CI path. The intended tradeoff is now explicit in `.github/workflows/bazel.yml`: pull requests get the fast Windows cross-compiled Bazel test leg, while post-merge pushes to `main` run both that fast cross leg and a fully native Windows Bazel test leg. The native main-only job keeps full V8/code-mode coverage and gets a 40-minute timeout because it is less latency-sensitive than PR CI. All other Bazel jobs remain at 30 minutes. ## Why Windows Bazel PR CI currently does the expensive part of the build on Windows. A native Windows Bazel test job on `main` completed in about 28m12s, leaving very little headroom under the 30-minute job timeout and making Windows the slowest PR signal. #20485 showed that Windows cross-compilation can be materially faster for Cargo release builds, but PR CI needs Bazel because Bazel owns our test sharding, flaky-test retries, and integration-test layout. This PR applies the same high-level shape we already use for macOS Bazel CI: compile with remote Linux execution, then run platform-specific tests on the platform runner. The compromise is deliberately signal-aware: code-mode/V8 changes are rare enough that PR CI can accept losing the direct V8/code-mode smoke-test signal temporarily, while `main` still runs the native Windows job post-merge to catch that class of regression. A follow-up PR should investigate making the cross-built Windows gnullvm V8 archive pass the direct V8/code-mode tests so this tradeoff can eventually go away. ## What Changed - Adds a `ci-windows-cross` Bazel config that targets `x86_64-pc-windows-gnullvm`, uses Linux RBE for build actions, and keeps `TestRunner` actions local on the Windows runner. - Adds explicit Windows platform definitions for `windows_x86_64_gnullvm`, `windows_x86_64_msvc`, and a bridge toolchain that lets gnullvm test targets execute under the Windows MSVC host platform. - Updates the Windows Bazel PR test leg to opt into the cross-compile path via `--windows-cross-compile` and `--remote-download-toplevel`. - Adds a `test-windows-native-main` job that runs only for `push` events on `refs/heads/main`, uses the native Windows Bazel path, includes V8/code-mode smoke tests, and has `timeout-minutes: 40`. - Keeps fork/community PRs without `BUILDBUDDY_API_KEY` on the previous local Windows MSVC-host fallback, including `--host_platform=//:local_windows_msvc` and `--jobs=8`. - Preserves the existing integration-test shape on non-gnullvm platforms, while generating Windows-cross wrapper targets only for `windows_gnullvm`. - Resolves `CARGO_BIN_EXE_*` values from runfiles at test runtime, avoiding hard-coded Cargo paths and duplicate test runfiles. - Extends the V8 Bazel patches enough for the `x86_64-pc-windows-gnullvm` target and Linux remote execution path. - Makes the Windows sandbox test cwd derive from `INSTA_WORKSPACE_ROOT` at runtime when Bazel provides it, because cross-compiled binaries may contain Linux compile-time paths. - Keeps the direct V8/code-mode unit smoke tests out of the Windows cross PR path for now while native Windows CI continues to cover them post-merge. ## Command Shape The fast Windows PR test leg invokes the normal Bazel CI wrapper like this: ```shell ./.github/scripts/run-bazel-ci.sh \ --print-failed-action-summary \ --print-failed-test-logs \ --windows-cross-compile \ --remote-download-toplevel \ -- \ test \ --test_tag_filters=-argument-comment-lint \ --test_verbose_timeout_warnings \ --build_metadata=COMMIT_SHA=${GITHUB_SHA} \ -- \ //... \ -//third_party/v8:all \ -//codex-rs/code-mode:code-mode-unit-tests \ -//codex-rs/v8-poc:v8-poc-unit-tests ``` With the BuildBuddy secret available on Windows, the wrapper selects `--config=ci-windows-cross` and appends the important Windows-cross overrides after rc expansion: ```shell --host_platform=//:rbe --shell_executable=/bin/bash --action_env=PATH=/usr/bin:/bin --host_action_env=PATH=/usr/bin:/bin --test_env=PATH=${CODEX_BAZEL_WINDOWS_PATH} ``` The native post-merge Windows job intentionally omits `--windows-cross-compile` and does not exclude the V8/code-mode unit targets: ```shell ./.github/scripts/run-bazel-ci.sh \ --print-failed-action-summary \ --print-failed-test-logs \ -- \ test \ --test_tag_filters=-argument-comment-lint \ --test_verbose_timeout_warnings \ --build_metadata=COMMIT_SHA=${GITHUB_SHA} \ --build_metadata=TAG_windows_native_main=true \ -- \ //... \ -//third_party/v8:all ``` ## Research Notes The existing macOS Bazel CI config already uses the model we want here: build actions run remotely with `--strategy=remote`, but `TestRunner` actions execute on the macOS runner. This PR mirrors that pattern for Windows with `--strategy=TestRunner=local`. The important Bazel detail is that `rules_rs` is already targeting `x86_64-pc-windows-gnullvm` for Windows Bazel PR tests. This PR changes where the build actions execute; it does not switch the Bazel PR test target to Cargo, `cargo-nextest`, or the MSVC release target. Cargo release builds differ from this Bazel path for V8: the normal Windows Cargo release target is MSVC, and `rusty_v8` publishes prebuilt Windows MSVC `.lib.gz` archives. The Bazel PR path targets `windows-gnullvm`; `rusty_v8` does not publish a prebuilt Windows GNU/gnullvm archive, so this PR builds that archive in-tree. That Linux-RBE-built gnullvm archive currently crashes in direct V8/code-mode smoke tests, which is why the workflow keeps native Windows coverage on `main`. The less obvious Bazel detail is test wrapper selection. Bazel chooses the Windows test wrapper (`tw.exe`) from the test action execution platform, not merely from the Rust target triple. The outer `workspace_root_test` therefore declares the default test toolchain and uses the bridge toolchain above so the test action executes on Windows while its inner Rust binary is built for gnullvm. The V8 investigation exposed a Windows-client gotcha: even when an action execution platform is Linux RBE, Bazel can still derive the genrule shell path from the Windows client. That produced remote commands trying to run `C:\Program Files\Git\usr\bin\bash.exe` on Linux workers. The wrapper now passes `--shell_executable=/bin/bash` with `--host_platform=//:rbe` for the Windows cross path. The same Windows-client/Linux-RBE boundary also affected `third_party/v8:binding_cc`: a multiline genrule command can carry CRLF line endings into Linux remote bash, which failed as `$'\r'`. That genrule now keeps the `sed` command on one physical shell line while using an explicit Starlark join so the shell arguments stay readable. ## Verification Local checks included: ```shell bash -n .github/scripts/run-bazel-ci.sh bash -n workspace_root_test_launcher.sh.tpl ruby -e "require %q{yaml}; YAML.load_file(%q{.github/workflows/bazel.yml}); puts %q{ok}" RUNNER_OS=Linux ./scripts/list-bazel-clippy-targets.sh RUNNER_OS=Windows ./scripts/list-bazel-clippy-targets.sh RUNNER_OS=Linux ./tools/argument-comment-lint/list-bazel-targets.sh RUNNER_OS=Windows ./tools/argument-comment-lint/list-bazel-targets.sh ``` The Linux clippy and argument-comment target lists contain zero `*-windows-cross-bin` labels, while the Windows lists still include 47 Windows-cross internal test binaries. CI evidence: - Baseline native Windows Bazel test on `main`: success in about 28m12s, https://github.com/openai/codex/actions/runs/25206257208/job/73907325959 - Green Windows-cross Bazel run on the split PR before adding the main-only native leg: Windows test 9m16s, Windows release verify 5m10s, Windows clippy 4m43s, https://github.com/openai/codex/actions/runs/25231890068 - The latest SHA adds the explicit PR-vs-main tradeoff in `bazel.yml`; CI is rerunning on that focused diff. ## Follow-Up A subsequent PR should investigate making a cross-built Windows binary work with V8/code-mode enabled. Likely options are either making the Linux-RBE-built `windows-gnullvm` V8 archive correct at runtime, or evaluating whether a Bazel MSVC target/toolchain can reuse the same prebuilt MSVC `rusty_v8` archive shape that Cargo release builds already use.
This commit is contained in:
Michael Bolin
119
defs.bzl
119
defs.bzl
@@ -1,8 +1,8 @@
|
||||
load("@crates//:data.bzl", "DEP_DATA")
|
||||
load("@crates//:defs.bzl", "all_crate_deps")
|
||||
load("@rules_platform//platform_data:defs.bzl", "platform_data")
|
||||
load("@rules_rust//rust:defs.bzl", "rust_binary", "rust_library", "rust_proc_macro", "rust_test")
|
||||
load("@rules_rust//cargo/private:cargo_build_script_wrapper.bzl", "cargo_build_script")
|
||||
load("@rules_rust//rust:defs.bzl", "rust_binary", "rust_library", "rust_proc_macro", "rust_test")
|
||||
|
||||
PLATFORMS = [
|
||||
"linux_arm64_musl",
|
||||
@@ -31,6 +31,16 @@ WINDOWS_RUSTC_LINK_FLAGS = select({
|
||||
"//conditions:default": [],
|
||||
})
|
||||
|
||||
WINDOWS_GNULLVM_INCOMPATIBLE = select({
|
||||
"@rules_rs//rs/experimental/platforms/constraints:windows_gnullvm": ["@platforms//:incompatible"],
|
||||
"//conditions:default": [],
|
||||
})
|
||||
|
||||
WINDOWS_GNULLVM_ONLY = select({
|
||||
"@rules_rs//rs/experimental/platforms/constraints:windows_gnullvm": [],
|
||||
"//conditions:default": ["@platforms//:incompatible"],
|
||||
})
|
||||
|
||||
# libwebrtc uses Objective-C categories from native archives. Any Bazel-linked
|
||||
# macOS binary/test that can pull it in must keep category symbols alive.
|
||||
MACOS_WEBRTC_RUSTC_LINK_FLAGS = select({
|
||||
@@ -64,12 +74,16 @@ def _workspace_root_test_impl(ctx):
|
||||
test_bin = ctx.executable.test_bin
|
||||
workspace_root_marker = ctx.file.workspace_root_marker
|
||||
launcher_template = ctx.file._windows_launcher_template if is_windows else ctx.file._bash_launcher_template
|
||||
runfile_env_exports = _windows_runfile_env_exports(ctx) if is_windows else _bash_runfile_env_exports(ctx)
|
||||
workspace_root_setup = _windows_workspace_root_setup(ctx) if is_windows else _bash_workspace_root_setup(ctx)
|
||||
ctx.actions.expand_template(
|
||||
template = launcher_template,
|
||||
output = launcher,
|
||||
is_executable = True,
|
||||
substitutions = {
|
||||
"__RUNFILE_ENV_EXPORTS__": runfile_env_exports,
|
||||
"__TEST_BIN__": test_bin.short_path,
|
||||
"__WORKSPACE_ROOT_SETUP__": workspace_root_setup,
|
||||
"__WORKSPACE_ROOT_MARKER__": workspace_root_marker.short_path,
|
||||
},
|
||||
)
|
||||
@@ -78,6 +92,22 @@ def _workspace_root_test_impl(ctx):
|
||||
for data_dep in ctx.attr.data:
|
||||
runfiles = runfiles.merge(ctx.runfiles(files = data_dep[DefaultInfo].files.to_list()))
|
||||
runfiles = runfiles.merge(data_dep[DefaultInfo].default_runfiles)
|
||||
for runfile_dep in ctx.attr.runfile_env:
|
||||
executable = runfile_dep[DefaultInfo].files_to_run.executable
|
||||
if executable == None:
|
||||
fail("{} does not provide an executable for runfile_env".format(runfile_dep.label))
|
||||
runfiles = runfiles.merge(ctx.runfiles(files = [executable]))
|
||||
runfiles = runfiles.merge(runfile_dep[DefaultInfo].default_runfiles)
|
||||
|
||||
location_targets = (
|
||||
ctx.attr.data +
|
||||
[ctx.attr.test_bin, ctx.attr.workspace_root_marker] +
|
||||
ctx.attr.runfile_env.keys()
|
||||
)
|
||||
env = {
|
||||
key: ctx.expand_location(value, targets = location_targets)
|
||||
for key, value in ctx.attr.env.items()
|
||||
}
|
||||
|
||||
return [
|
||||
DefaultInfo(
|
||||
@@ -86,18 +116,55 @@ def _workspace_root_test_impl(ctx):
|
||||
runfiles = runfiles,
|
||||
),
|
||||
RunEnvironmentInfo(
|
||||
environment = ctx.attr.env,
|
||||
environment = env,
|
||||
),
|
||||
]
|
||||
|
||||
def _bash_runfile_env_exports(ctx):
|
||||
lines = []
|
||||
for runfile_dep, env_var in ctx.attr.runfile_env.items():
|
||||
executable = runfile_dep[DefaultInfo].files_to_run.executable
|
||||
if executable == None:
|
||||
fail("{} does not provide an executable for runfile_env".format(runfile_dep.label))
|
||||
lines.append('RUNFILE_ENV_ARGS+=("{}=$(resolve_runfile "{}")")'.format(env_var, executable.short_path))
|
||||
return "\n".join(lines)
|
||||
|
||||
def _windows_runfile_env_exports(ctx):
|
||||
lines = []
|
||||
for runfile_dep, env_var in ctx.attr.runfile_env.items():
|
||||
executable = runfile_dep[DefaultInfo].files_to_run.executable
|
||||
if executable == None:
|
||||
fail("{} does not provide an executable for runfile_env".format(runfile_dep.label))
|
||||
lines.append('call :resolve_runfile {} "{}"'.format(env_var, executable.short_path))
|
||||
lines.append("if errorlevel 1 exit /b 1")
|
||||
return "\n".join(lines)
|
||||
|
||||
def _bash_workspace_root_setup(ctx):
|
||||
if not ctx.attr.chdir_workspace_root:
|
||||
return ""
|
||||
return 'export INSTA_WORKSPACE_ROOT="${workspace_root}"\ncd "${workspace_root}"'
|
||||
|
||||
def _windows_workspace_root_setup(ctx):
|
||||
if not ctx.attr.chdir_workspace_root:
|
||||
return ""
|
||||
return """set "INSTA_WORKSPACE_ROOT=%workspace_root%"
|
||||
cd /d "%workspace_root%" || exit /b 1"""
|
||||
|
||||
workspace_root_test = rule(
|
||||
implementation = _workspace_root_test_impl,
|
||||
test = True,
|
||||
toolchains = ["@bazel_tools//tools/test:default_test_toolchain_type"],
|
||||
attrs = {
|
||||
"chdir_workspace_root": attr.bool(
|
||||
default = True,
|
||||
),
|
||||
"data": attr.label_list(
|
||||
allow_files = True,
|
||||
),
|
||||
"env": attr.string_dict(),
|
||||
"runfile_env": attr.label_keyed_string_dict(
|
||||
cfg = "target",
|
||||
),
|
||||
"test_bin": attr.label(
|
||||
cfg = "target",
|
||||
executable = True,
|
||||
@@ -255,6 +322,7 @@ def codex_rust_crate(
|
||||
unit_test_name = name + "-unit-tests"
|
||||
unit_test_binary = name + "-unit-tests-bin"
|
||||
unit_test_shard_count = _test_shard_count(test_shard_counts, unit_test_name)
|
||||
|
||||
# Shard at the workspace_root_test layer. rules_rust's sharding wrapper
|
||||
# expects to run from its own runfiles cwd, while workspace_root_test
|
||||
# deliberately changes cwd so Insta sees Cargo-like snapshot paths.
|
||||
@@ -298,9 +366,11 @@ def codex_rust_crate(
|
||||
|
||||
sanitized_binaries = []
|
||||
cargo_env = {}
|
||||
cargo_env_runfiles = {}
|
||||
for binary, main in binaries.items():
|
||||
#binary = binary.replace("-", "_")
|
||||
sanitized_binaries.append(binary)
|
||||
cargo_env_runfiles[":" + binary] = "CARGO_BIN_EXE_" + binary
|
||||
cargo_env["CARGO_BIN_EXE_" + binary] = "$(rlocationpath :%s)" % binary
|
||||
|
||||
rust_binary(
|
||||
@@ -317,6 +387,7 @@ def codex_rust_crate(
|
||||
for binary_label in extra_binaries:
|
||||
sanitized_binaries.append(binary_label)
|
||||
binary = Label(binary_label).name
|
||||
cargo_env_runfiles[binary_label] = "CARGO_BIN_EXE_" + binary
|
||||
cargo_env["CARGO_BIN_EXE_" + binary] = "$(rlocationpath %s)" % binary_label
|
||||
|
||||
integration_test_kwargs = {}
|
||||
@@ -331,6 +402,7 @@ def codex_rust_crate(
|
||||
test_name = name + "-" + test_file_stem.replace("/", "-")
|
||||
if not test_name.endswith("-test"):
|
||||
test_name += "-test"
|
||||
windows_cross_test_binary = test_name + "-windows-cross-bin"
|
||||
|
||||
test_kwargs = {}
|
||||
test_kwargs.update(integration_test_kwargs)
|
||||
@@ -340,6 +412,9 @@ def codex_rust_crate(
|
||||
test_kwargs["shard_count"] = test_shard_count
|
||||
test_kwargs["flaky"] = True
|
||||
|
||||
# Keep the existing integration test shape on non-gnullvm platforms.
|
||||
# Windows cross tests need workspace_root_test so runfile env vars
|
||||
# resolve to Windows-native absolute paths before the test starts.
|
||||
rust_test(
|
||||
name = test_name,
|
||||
crate_name = test_crate_name,
|
||||
@@ -356,14 +431,48 @@ def codex_rust_crate(
|
||||
"--remap-path-prefix=codex-rs=",
|
||||
],
|
||||
rustc_env = rustc_env,
|
||||
# Important: do not merge `test_env` here. Its unit-test-only
|
||||
# `INSTA_WORKSPACE_ROOT="codex-rs"` is tuned for unit tests that
|
||||
# execute from the repo root and can misplace integration snapshots.
|
||||
env = cargo_env,
|
||||
target_compatible_with = WINDOWS_GNULLVM_INCOMPATIBLE,
|
||||
tags = test_tags,
|
||||
**test_kwargs
|
||||
)
|
||||
|
||||
windows_cross_test_kwargs = {}
|
||||
windows_cross_test_kwargs.update(integration_test_kwargs)
|
||||
if test_shard_count:
|
||||
windows_cross_test_kwargs["shard_count"] = test_shard_count
|
||||
windows_cross_test_kwargs["flaky"] = True
|
||||
|
||||
rust_test(
|
||||
name = windows_cross_test_binary,
|
||||
crate_name = test_crate_name,
|
||||
crate_root = test,
|
||||
srcs = [test],
|
||||
data = native.glob(["tests/**"], allow_empty = True) + sanitized_binaries + test_data_extra,
|
||||
compile_data = native.glob(["tests/**"], allow_empty = True) + integration_compile_data_extra,
|
||||
deps = all_crate_deps(normal = True, normal_dev = True) + maybe_deps + deps_extra,
|
||||
rustc_flags = rustc_flags_extra + WINDOWS_RUSTC_LINK_FLAGS + [
|
||||
"--remap-path-prefix=../codex-rs=",
|
||||
"--remap-path-prefix=codex-rs=",
|
||||
],
|
||||
rustc_env = rustc_env,
|
||||
env = cargo_env,
|
||||
target_compatible_with = WINDOWS_GNULLVM_ONLY,
|
||||
tags = test_tags + ["manual"],
|
||||
)
|
||||
|
||||
workspace_root_test(
|
||||
name = test_name + "-windows-cross",
|
||||
chdir_workspace_root = False,
|
||||
env = cargo_env,
|
||||
runfile_env = cargo_env_runfiles,
|
||||
test_bin = ":" + windows_cross_test_binary,
|
||||
workspace_root_marker = "//codex-rs/utils/cargo-bin:repo_root.marker",
|
||||
target_compatible_with = WINDOWS_GNULLVM_ONLY,
|
||||
tags = test_tags,
|
||||
**windows_cross_test_kwargs
|
||||
)
|
||||
|
||||
def _test_shard_count(test_shard_counts, test_name):
|
||||
shard_count = test_shard_counts.get(test_name)
|
||||
if shard_count == None:
|
||||
|
||||
Reference in New Issue
Block a user