Skip to content

Back off Codex review triggers on usage limits - #1560

Open
Komzpa wants to merge 1 commit into
mainfrom
fix/codex-label-usage-limit-backoff-20260731
Open

Back off Codex review triggers on usage limits#1560
Komzpa wants to merge 1 commit into
mainfrom
fix/codex-label-usage-limit-backoff-20260731

Conversation

@Komzpa

@Komzpa Komzpa commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • stop the Codex label sync from posting more @codex review comments when the same GitHub comment sender received a recent Codex usage-limit reply
  • allow review requests again only when that same sender account has a newer normal Codex response within the 24-hour window
  • make no-data runs probe once, reread the PR timeline, and suppress the remaining review requests if that probe hits the usage limit

Tests

  • uv run pytest tests/unit/test_sync_codex_ok_labels.py -q
  • uv run ruff check .github/scripts/sync_codex_ok_labels.py tests/unit/test_sync_codex_ok_labels.py
  • uv run ruff format --check .github/scripts/sync_codex_ok_labels.py tests/unit/test_sync_codex_ok_labels.py
  • make test-unit

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits. You can see your limits in the Codex usage dashboard.

@Komzpa
Komzpa force-pushed the fix/codex-label-usage-limit-backoff-20260731 branch from e409b91 to d776d1f Compare July 31, 2026 15:37
@Komzpa
Komzpa force-pushed the fix/codex-label-usage-limit-backoff-20260731 branch from d776d1f to 4cf9616 Compare July 31, 2026 15:47
@Komzpa

Komzpa commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@Soju06

Soju06 commented Aug 4, 2026

Copy link
Copy Markdown
Owner

The backoff logic itself looks sound (per-sender attribution, newer-normal-response unlatch, probe-and-reread), but splitting classification from apply introduced a log misattribution bug:

Stale number in the apply loop.github/scripts/sync_codex_ok_labels.py:1687 and the except handler at :1706 still format f"{repo}#{number}", but the apply loop now iterates for decision in decisions: while number is the leftover from the classification loop (always the last classified PR). With --all-open over PRs 710 and 714, both status lines print apply Soju06/codex-lb#714 ..., and any exception during apply is attributed to #714 regardless of which PR failed. These lines are the script's audit trail, so this actively misleads. Should be decision.number. Note test_main_stops_codex_review_triggers_after_probe_hits_usage_limit passes only because it asserts on #714, the last PR — asserting apply Soju06/codex-lb#710 appears in output would catch it.

Minor: the PR description says no-data runs "probe once", but the sleep+reread at :1678-1682 runs after every posted @codex review, not just the first — with N eligible PRs and no limit hit that's N x 10s plus N extra timeline fetches per run. Consider probing only until the first non-limit observation, or only when latest_normal_response_at is still None.

@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create an environment for this repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants