Skip to content

fix: Improve System Logs pagination with time filters#28

Open
javiermoralestealium wants to merge 1 commit into
okta:mainfrom
javiermoralestealium:fix/system-logs-pagination-with-time-filters
Open

fix: Improve System Logs pagination with time filters#28
javiermoralestealium wants to merge 1 commit into
okta:mainfrom
javiermoralestealium:fix/system-logs-pagination-with-time-filters

Conversation

@javiermoralestealium
Copy link
Copy Markdown

Problem

When using get_logs() with fetch_all=True and time filters (since/until), pagination would stop after the first page (100 events) even when more data existed within the time range. This occurred because the Okta SDK's response.has_next() method returns False when using time-based filters, despite the System Log API returning a next link in the HTTP Link header.

Root Cause

The Okta System Log API has unique behavior:

  • It always returns a next link in the Link header (for polling)
  • When using since/until filters, the SDK's response.has_next() may incorrectly return False even when more results exist within the time range
  • The generic paginate_all_results() utility relies on has_next() and stops pagination prematurely

Solution

  1. Created paginate_system_logs() - a specialized pagination handler for System Logs that:

    • Doesn't rely solely on has_next() for determining if more pages exist
    • Extracts the cursor from response metadata
    • Manually constructs subsequent API calls with the cursor
    • Continues until no more data is returned or max_pages is reached
  2. Updated get_logs() to use the new pagination handler when fetch_all=True

  3. Enhanced create_paginated_response() to trust the cursor even when has_next() returns False

Testing

Before fix: get_logs(fetch_all=True, since="2026-02-17", until="2026-02-24") returned only 100 events from a ~20 minute window.

After fix: Same call should return all events across the 7-day time range, properly paginating through multiple pages.

Impact

  • Fixes incomplete log retrieval when using time filters
  • No breaking changes - existing behavior preserved for non-fetch_all calls
  • Improves reliability for security monitoring and compliance use cases

## Problem
When using `get_logs()` with `fetch_all=True` and time filters (`since`/`until`),
pagination would stop after the first page (100 events) even when more data existed
within the time range. This occurred because the Okta SDK's `response.has_next()`
method returns `False` when using time-based filters, despite the System Log API
returning a `next` link in the HTTP Link header.

## Root Cause
The Okta System Log API has unique behavior:
- It always returns a `next` link in the Link header (for polling)
- When using `since`/`until` filters, the SDK's `response.has_next()` may
  incorrectly return `False` even when more results exist within the time range
- The generic `paginate_all_results()` utility relies on `has_next()` and
  stops pagination prematurely

## Solution
1. Created `paginate_system_logs()` - a specialized pagination handler for
   System Logs that:
   - Doesn't rely solely on `has_next()` for determining if more pages exist
   - Extracts the cursor from response metadata
   - Manually constructs subsequent API calls with the cursor
   - Continues until no more data is returned or max_pages is reached

2. Updated `get_logs()` to use the new pagination handler when `fetch_all=True`

3. Enhanced `create_paginated_response()` to trust the cursor even when
   `has_next()` returns `False`

## Testing
Before fix: `get_logs(fetch_all=True, since="2026-02-17", until="2026-02-24")`
returned only 100 events from a ~20 minute window.

After fix: Same call should return all events across the 7-day time range,
properly paginating through multiple pages.

## Impact
- Fixes incomplete log retrieval when using time filters
- No breaking changes - existing behavior preserved for non-fetch_all calls
- Improves reliability for security monitoring and compliance use cases

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@javiermoralestealium
Copy link
Copy Markdown
Author

Hi @okta

We're actively using this MCP server for security operations automation and encountering the pagination bug this PR
addresses.

Impact: When fetching Okta audit logs with time filters (since/until) for security investigations, we only
get the first 100 events, which is causing us to miss critical security events in our analysis workflows.

Question: Is there an estimated timeline for reviewing/merging this PR?

If the review timeline is uncertain, we'd like to use a local fork with this fix in the interim. Would you be open
to that, or do you have any concerns?

Happy to provide any additional context, testing, or adjustments to the PR if needed.

Thanks in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant