🎯 Fix Pagination Loop Termination When Fetching Repository Issues with 'Since' Filter

[alisteuber4ee1]

📝 Description

When querying the GitHub API for repository issues using the go-github client, combining the Since filter (which filters issues updated after a specific timestamp) with ListByRepo causes the pagination mechanism to fail.

Instead of traversing all available pages of issues matching the criteria, the client either terminates after the first page or fails to request subsequent pages. This behavior typically occurs because the pagination loop incorrectly relies on the returned slice length (e.g., checking if len(issues) < PerPage to break the loop) rather than checking the NextPage response header, or because the Since parameter is not being correctly preserved/propagated across paginated requests in the client options.

🎯 Acceptance Criteria

• The pagination logic must successfully retrieve all pages of issues when the Since filter is active.
• The pagination loop must rely on the NextPage value returned in the *github.Response metadata rather than the size of the returned slice.
• The Since timestamp parameter must be preserved across all paginated API calls.
• The implementation must handle edge cases, such as when a page returns zero results but a NextPage link is still present (or vice versa).
• No duplicate issues should be fetched or processed during pagination.

🛠️ Technical Specifications & Context

The issue resides in the GitHub client wrapper, likely located in:

• /pkg/client/github.go or /internal/integration/issues.go (depending on the exact layout of alisteuber4ee1/API-Integration-Pagination).

Expected Implementation Pattern:

Currently, the code might look like this:

opt := &github.IssueListByRepoOptions{
    Since:       sinceTime,
    ListOptions: github.ListOptions{PerPage: 100},
}
var allIssues []*github.Issue
for {
    issues, _, err := client.Issues.ListByRepo(ctx, owner, repo, opt)
    if err != nil {
        return nil, err
    }
    allIssues = append(allIssues, issues...)
    if len(issues) < 100 { // ❌ Bug: GitHub API can return fewer items than PerPage even if more pages exist when filters are applied
        break
    }
    opt.Page++ // ❌ Bug: Manual increment can desync if NextPage is different
}

It should be refactored to use the response metadata:

opt := &github.IssueListByRepoOptions{
    Since:       sinceTime,
    ListOptions: github.ListOptions{PerPage: 100},
}
var allIssues []*github.Issue
for {
    issues, resp, err := client.Issues.ListByRepo(ctx, owner, repo, opt)
    if err != nil {
        return nil, err
    }
    allIssues = append(allIssues, issues...)
    if resp.NextPage == 0 { //  Correct: Rely on GitHub's Link header
        break
    }
    opt.Page = resp.NextPage //  Correct: Use the page number provided by the API response
}

🧪 Verification & Testing

Unit Testing

Create a mock test in /pkg/client/github_test.go using go-github's mock HTTP client or a standard net/http/httptest server:

1. Mock a GitHub API response that returns 3 pages of issues for a ListByRepo request with a since query parameter.
2. Configure the mock server to return:
   • Page 1: 2 issues, Link header pointing to Page 2.
   • Page 2: 1 issue, Link header pointing to Page 3.
   • Page 3: 1 issue, no Link header.
3. Assert that the client performs exactly 3 HTTP requests.
4. Assert that the total number of issues returned is 4.
5. Assert that the since query parameter is present in all 3 requests.

Integration Testing

Run the integration test suite against a test repository with known historical issues:

go test -v ./... -run TestFetchIssuesWithSincePagination

---

---

This repo is using Opire - what does it mean? 👇

💵 Everyone can add rewards for this issue commenting /reward 100 (replace 100 with the amount).
🕵️‍♂️ If someone starts working on this issue to earn the rewards, they can comment /try to let everyone know!
🙌 And when they open the PR, they can comment /claim #1 either in the PR description or in a PR's comment.

🪙 Also, everyone can tip any user commenting /tip 20 @alisteuber4ee1 (replace 20 with the amount, and @alisteuber4ee1 with the user to tip).

📖 If you want to learn more, check out our documentation.

[alisteuber4ee1]

/reward 100

How to earn this $100.00 reward?

💪 Comment /try below and start working on it!
🙌 Once your PR is open, comment /claim #1 either in the PR description or within the comments.

[Codimow]

/claim #1

[Neohu-ceo]

/try

I can fix this pagination loop issue. Will submit a PR shortly. (Lighthouse Analytics)

Check if you're the only one trying 👇

There are already people trying to solve this issue. You're welcome to tackle it independently, but we encourage you to reach out to the contributors that are trying to solve the issue if you'd like to collaborate with them.

The other contributor is: @Codimow.

[0106hyh]

/try

Check if you're the only one trying 👇

There are already people trying to solve this issue. You're welcome to tackle it independently, but we encourage you to reach out to the contributors that are trying to solve the issue if you'd like to collaborate with them.

The other contributors are: @Codimow @Neohu-ceo.

[opirebot]

🎉 The user @0106hyh has claimed all rewards for this issue.
🔍 You can see the PR here.
💸 @alisteuber4ee1 you can pay the related rewards here.

[opirebot]

🎉 The user @lin20070906 has claimed all rewards for this issue.
🔍 You can see the PR here.
💸 @alisteuber4ee1 you can pay the related rewards here.