Respect GitHub rate-limit headers in responses

[ericphanson]

closes #231

written with claude code, though with a lot of hand-holding.

Example of hitting the rate limit in real life:

julia> commits(Repo("JuliaRegistries/RegistryCI.jl"));
┌ Info: GitHub API primary rate limit hit, retrying in 3070.6s
│   method = "GET"
│   url = "https://api.github.com/repositories/208115424/commits?page=27"
│   status = 403
│   limit = "60"
│   remaining = "0"
│   used = "60"
│   reset = "1759013549"
└   resource = "core"
^CERROR: InterruptException:
Stacktrace:
  [1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
    @ Base ./task.jl:958
  [2] wait()
    @ Base ./task.jl:1022
  [3] wait(c::Base.GenericCondition{Base.Threads.SpinLock}; first::Bool)
    @ Base ./condition.jl:130
  [4] wait
    @ ./condition.jl:125 [inlined]
  [5] _trywait(t::Timer)
    @ Base ./asyncevent.jl:145
  [6] wait
    @ ./asyncevent.jl:162 [inlined]
  [7] sleep(sec::Float64)
    @ Base ./asyncevent.jl:272
  [8] with_retries(f::GitHub.var"#21#23"{…}; method::String, url::SubString{…}, max_retries::Int64, verbose::Bool)
    @ GitHub ~/GitHub.jl/src/utils/requests.jl:253
  [9] with_retries
    @ ~/GitHub.jl/src/utils/requests.jl:214 [inlined]
 [10] make_request_with_retries
    @ ~/GitHub.jl/src/utils/requests.jl:331 [inlined]
 [11] github_paged_get(api::GitHub.GitHubWebAPI, endpoint::String; page_limit::Float64, start_page::String, handle_error::Bool, auth::GitHub.AnonymousAuth, headers::Dict{…}, params::Dict{…}, max_retries::Int64, options::@Kwargs{})
    @ GitHub ~/GitHub.jl/src/utils/requests.jl:350
 [12] github_paged_get
    @ ~/GitHub.jl/src/utils/requests.jl:322 [inlined]
 [13] gh_get_paged_json(api::GitHub.GitHubWebAPI, endpoint::String; options::@Kwargs{})
    @ GitHub ~/GitHub.jl/src/utils/requests.jl:368
 [14] gh_get_paged_json
    @ ~/GitHub.jl/src/utils/requests.jl:367 [inlined]
 [15] commits(api::GitHub.GitHubWebAPI, repo::Repo; options::@Kwargs{})
    @ GitHub ~/GitHub.jl/src/repositories/commits.jl:32
 [16] commits
    @ ~/GitHub.jl/src/repositories/commits.jl:31 [inlined]
 [17] commits(repo::Repo)
    @ GitHub ./none:0
 [18] top-level scope
    @ REPL[5]:1
Some type information was truncated. Use `show(err)` to see complete types.

[ericphanson]

I think I should add a limit that if we want to sleep more than say 15 mins, we error instead with a clear error message. Since in many cases it would be unexpected that your task will just sleep for 12 hours or something like that. Super long waits can happen when you use up your quota and it wants to wait for a refresh. This value could be configurable with some “reasonable” default.

edit: done in 416b21e

[DilumAluthge]

@ericphanson Can you also try this functionality out from a non-fork PR, so we can see if there's any behavior difference between a user's PAT and the GitHub Actions GITHUB_TOKEN? I'm hoping there's no difference, but it'd be good to check.

[DilumAluthge]

It looks like CI is failing because we are rate-limited when calling GitHub.authenticate(token).

GitHub.authenticate() calls gh_get():

GitHub.jl/src/utils/auth.jl

Lines 65 to 69 in 942e814

	@api_default function authenticate(api::GitHubAPI, token::AbstractString; options...)
	auth = OAuth2(token)
	gh_get(api, "/"; auth = auth, options...)
	return auth
	end

Which in turn should call github_request():

GitHub.jl/src/utils/requests.jl

Line 78 in 942e814

gh_get(api::GitHubAPI, endpoint = ""; options...) = github_request(api, HTTP.get, endpoint; options...)

@ericphanson Would it be possible to add your rate-limiting logic to github_request() also?

[DilumAluthge]

Could you Dates.canonicalize() the delay_seconds, to make it easier to read?

[ericphanson]

@ericphanson Can you also try this functionality out from a non-fork PR, so we can see if there's any behavior difference between a user's PAT and the GitHub Actions GITHUB_TOKEN? I'm hoping there's no difference, but it'd be good to check.

#249

@ericphanson Would it be possible to add your rate-limiting logic to github_request() also?

It is there! Every API request in this package should be covered. That's the with_retries block in github_request. Where do you see the failure? Send a link if you can

[codecov]

Codecov Report

❌ Patch coverage is 90.10989% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.85%. Comparing base (942e814) to head (a7e1452).

Files with missing lines	Patch %	Lines
src/utils/requests.jl	90.10%	9 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #233      +/-   ##
==========================================
+ Coverage   55.19%   58.85%   +3.66%     
==========================================
  Files          37       37              
  Lines         904      982      +78     
==========================================
+ Hits          499      578      +79     
+ Misses        405      404       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.