fix: Improve System Logs pagination with time filters#28
Open
javiermoralestealium wants to merge 1 commit into
Open
fix: Improve System Logs pagination with time filters#28javiermoralestealium wants to merge 1 commit into
javiermoralestealium wants to merge 1 commit into
Conversation
## Problem When using `get_logs()` with `fetch_all=True` and time filters (`since`/`until`), pagination would stop after the first page (100 events) even when more data existed within the time range. This occurred because the Okta SDK's `response.has_next()` method returns `False` when using time-based filters, despite the System Log API returning a `next` link in the HTTP Link header. ## Root Cause The Okta System Log API has unique behavior: - It always returns a `next` link in the Link header (for polling) - When using `since`/`until` filters, the SDK's `response.has_next()` may incorrectly return `False` even when more results exist within the time range - The generic `paginate_all_results()` utility relies on `has_next()` and stops pagination prematurely ## Solution 1. Created `paginate_system_logs()` - a specialized pagination handler for System Logs that: - Doesn't rely solely on `has_next()` for determining if more pages exist - Extracts the cursor from response metadata - Manually constructs subsequent API calls with the cursor - Continues until no more data is returned or max_pages is reached 2. Updated `get_logs()` to use the new pagination handler when `fetch_all=True` 3. Enhanced `create_paginated_response()` to trust the cursor even when `has_next()` returns `False` ## Testing Before fix: `get_logs(fetch_all=True, since="2026-02-17", until="2026-02-24")` returned only 100 events from a ~20 minute window. After fix: Same call should return all events across the 7-day time range, properly paginating through multiple pages. ## Impact - Fixes incomplete log retrieval when using time filters - No breaking changes - existing behavior preserved for non-fetch_all calls - Improves reliability for security monitoring and compliance use cases Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Author
|
Hi @okta We're actively using this MCP server for security operations automation and encountering the pagination bug this PR Impact: When fetching Okta audit logs with time filters ( Question: Is there an estimated timeline for reviewing/merging this PR? If the review timeline is uncertain, we'd like to use a local fork with this fix in the interim. Would you be open Happy to provide any additional context, testing, or adjustments to the PR if needed. Thanks in advance! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When using
get_logs()withfetch_all=Trueand time filters (since/until), pagination would stop after the first page (100 events) even when more data existed within the time range. This occurred because the Okta SDK'sresponse.has_next()method returnsFalsewhen using time-based filters, despite the System Log API returning anextlink in the HTTP Link header.Root Cause
The Okta System Log API has unique behavior:
nextlink in the Link header (for polling)since/untilfilters, the SDK'sresponse.has_next()may incorrectly returnFalseeven when more results exist within the time rangepaginate_all_results()utility relies onhas_next()and stops pagination prematurelySolution
Created
paginate_system_logs()- a specialized pagination handler for System Logs that:has_next()for determining if more pages existUpdated
get_logs()to use the new pagination handler whenfetch_all=TrueEnhanced
create_paginated_response()to trust the cursor even whenhas_next()returnsFalseTesting
Before fix:
get_logs(fetch_all=True, since="2026-02-17", until="2026-02-24")returned only 100 events from a ~20 minute window.After fix: Same call should return all events across the 7-day time range, properly paginating through multiple pages.
Impact