mirror of
https://github.com/openai/codex.git
synced 2026-10-25 18:28:38 +00:00
Tom
c39824c2fd
## Summary - Surface failed GitHub Actions jobs in the PR babysitter watcher so Codex can fetch job logs as soon as a job fails, instead of waiting for the overall workflow run to complete. - Update babysit-pr skill instructions, GitHub API notes, and heuristics to prefer direct job log archives before falling back to `gh run view --log-failed`. - Add guardrails requiring explicit user confirmation before posting replies to human-authored review comments. - Add guardrails preventing Codex from patching unrelated flaky tests, CI infrastructure, runner issues, dependency outages, or other failures not caused by the PR branch. ## Validation - `python3 -m pytest .codex/skills/babysit-pr/scripts/test_gh_pr_watch.py`
3.2 KiB
3.2 KiB
CI / Review Heuristics
CI classification checklist
Treat as branch-related when logs clearly indicate a regression caused by the PR branch:
- Compile/typecheck/lint failures in files or modules touched by the branch
- Deterministic unit/integration test failures in changed areas
- Snapshot output changes caused by UI/text changes in the branch
- Static analysis violations introduced by the latest push
- Build script/config changes in the PR causing a deterministic failure
Treat as likely flaky or unrelated when evidence points to transient or external issues:
- DNS/network/registry timeout errors while fetching dependencies
- Runner image provisioning or startup failures
- GitHub Actions infrastructure/service outages
- Cloud/service rate limits or transient API outages
- Non-deterministic failures in unrelated integration tests with known flake patterns
Do not patch likely flaky/unrelated failures. Use the retry budget for rerunnable failures, wait for pending jobs, or stop and report the blocker when the failure is persistent or infrastructure-owned.
If uncertain, inspect failed logs once before choosing rerun.
Decision tree (fix vs rerun vs stop)
- If PR is merged/closed: stop.
- If there are failed checks:
- Diagnose first.
- If checks are still pending but an individual job has already failed: fetch that job's logs and diagnose now.
- If branch-related: fix locally, commit, push.
- If likely flaky/unrelated and all checks for the current SHA are terminal: rerun failed jobs.
- If likely flaky/unrelated and not safely rerunnable: stop and report the blocker; do not edit unrelated tests, build scripts, CI configuration, dependency pins, or infrastructure code.
- If checks are still pending and no failed job is available yet: wait.
- If flaky reruns for the same SHA reach the configured limit (default 3): stop and report persistent failure.
- Independently, process any new human review comments.
Review comment agreement criteria
Address the comment when:
- The comment is technically correct.
- The change is actionable in the current branch.
- The requested change does not conflict with the user’s intent or recent guidance.
- The change can be made safely without unrelated refactors.
Fix valid human review feedback in code when possible, but do not post a GitHub reply to a human-authored comment/thread unless the user explicitly confirms the exact response.
Do not auto-fix when:
- The comment is ambiguous and needs clarification.
- The request conflicts with explicit user instructions.
- The proposed change requires product/design decisions the user has not made.
- The codebase is in a dirty/unrelated state that makes safe editing uncertain.
- The comment only needs a written answer or disagreement response; propose the reply to the user instead of posting it automatically.
Stop-and-ask conditions
Stop and ask the user instead of continuing automatically when:
- The local worktree has unrelated uncommitted changes.
ghauth/permissions fail.- The PR branch cannot be pushed.
- CI failures persist after the flaky retry budget.
- Reviewer feedback requires a product decision or cross-team coordination.
- A human review comment requires a written GitHub reply instead of a code change.