Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(URL): Add support to query timestamp by URL #152

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

Henry8192
Copy link
Collaborator

@Henry8192 Henry8192 commented Dec 9, 2024

Description

Implements JSONL & CLPIR findNearestLogEventByTimestamp feature:

  • JSONL implementation is located in src/services/decoders/JsonlDecoder/index.ts.
  • CLPIR findNearestLogEventByTimestamp API is here. We use this API in src/services/decoders/ClpIrDecoder.ts.

Here's an example of searching log events with timestamp Mar 28 2023 04:01:18.594 GMT+0000:
localhost:3010/?filePath=http://localhost:3010/test/example.jsonl#timestamp=1679976078594

Here, the hash parameter #timestamp=1679976078594 is the unix timestamp of our target in miliseconds. This would trigger a search in log events with timestamp closest to 1679976078594. After the search is done, it would jump to that log event, and clear the #timestamp parameter in the URL.

Validation performed

Performed search with timestamp of 1679976078593, 1679976078594, 1679976078595 and 1679976078596 with example.jsonl.zip (please unzip it).

Summary by CodeRabbit

  • New Features

    • Introduced timestamp-based log navigation, allowing users to jump directly to specific events.
    • Enabled URL sharing with a new timestamp parameter for precise log access.
  • Chores

    • Upgraded underlying dependencies to enhance stability and overall performance.

Copy link

coderabbitai bot commented Dec 9, 2024

Walkthrough

This pull request updates multiple components to incorporate a new timestamp parameter for log event handling. The changes include a dependency version bump in the project’s package.json, modifications to state and URL context providers to pass and process the timestamp, service-level updates to handle cursors based on timestamps, and additions to decoder and typing interfaces for precise log event retrieval. The overall update refines the control flow for loading pages and searching for log events based on timestamps.

Changes

File(s) Change Summary
package.json Updated the clp-ffi-js dependency version from ^0.3.4 to ^0.3.5.
src/contexts/StateContextProvider.tsx
src/contexts/UrlContextProvider.tsx
In the context providers, added a new timestamp property and corresponding state/effect hooks. Updated URL hash parsing to include a new timestamp parameter alongside logEventNum.
src/services/LogFileManager/index.ts
src/services/decoders/ClpIrDecoder.ts
src/services/decoders/JsonlDecoder/index.ts
Reworked cursor data handling: replaced switch-case with if-else logic; adjusted to use timestamp for page loading. Added new methods findNearestLogEventByTimestamp in decoders and updated variable type for timestamps to use BigInt where appropriate.
src/typings/decoders.ts
src/typings/logs.ts
src/typings/url.ts
Extended typings by adding the findNearestLogEventByTimestamp method to the Decoder interface, updating INVALID_TIMESTAMP_VALUE to a BigInt, and introducing new enum and interface properties for the timestamp parameter in URL handling.

Sequence Diagram(s)

Loading
sequenceDiagram
    participant U as UrlContextProvider
    participant S as StateContextProvider
    participant LM as LogFileManager
    participant D as Decoder

    U->>S: Provide filePath, logEventNum, timestamp
    S->>LM: Request loadPageByCursor with timestamp cursor
    LM->>D: Invoke findNearestLogEventByTimestamp(timestamp)
    D-->>LM: Return nearest log event index
    LM-->>S: Deliver log page data based on timestamp

Possibly related PRs

Suggested reviewers

  • davemarco
  • junhaoliao
✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Sorry, something went wrong.

@Henry8192 Henry8192 changed the title WIP: feat(URL): Add support to query timestamp by URL feat(URL): Add support to query timestamp by URL Dec 9, 2024
@junhaoliao junhaoliao self-requested a review December 18, 2024 20:29
Copy link
Member

@junhaoliao junhaoliao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall structure looks really good

src/services/decoders/JsonlDecoder/index.ts Outdated Show resolved Hide resolved
src/services/MainWorker.ts Outdated Show resolved Hide resolved
src/contexts/UrlContextProvider.tsx Outdated Show resolved Hide resolved
src/contexts/StateContextProvider.tsx Outdated Show resolved Hide resolved
src/contexts/StateContextProvider.tsx Outdated Show resolved Hide resolved
src/typings/decoders.ts Outdated Show resolved Hide resolved
src/typings/decoders.ts Outdated Show resolved Hide resolved
@Henry8192 Henry8192 requested a review from junhaoliao December 27, 2024 19:32
@junhaoliao junhaoliao removed their request for review December 27, 2024 19:35
@junhaoliao
Copy link
Member

@Henry8192 can we address the linter checks before we proceed further? If the ClpIrDecoder return error is the blocker, we can wait for y-scope/clp-ffi-js#42 to be merged first.

@Henry8192 Henry8192 marked this pull request as ready for review February 16, 2025 06:33
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (5)
src/typings/decoders.ts (1)

102-119: Consider clarifying the return value for no-match cases.

The documentation should explicitly state what value is returned when no matching log event is found. Based on the past review discussion, it would be helpful to document whether it returns null or -1 in such cases.

     /**
      * Finds the log event, L, where if we assume:
      *
      * - the collection of log events is sorted in chronological order;
      * - and we insert a marker log event, M, with timestamp `timestamp` into the collection (if log
      *   events with timestamp `timestamp` already exist in the collection, M should be inserted
      *   after them).
      *
      * L is the event just before M, if M is not the first event in the collection; otherwise L is
      * the event just after M.
      *
      * NOTE: If the collection of log events isn't in chronological order, this method has undefined
      * behaviour.
      *
+     * @return The index of the log event L, or null if no matching log event is found.
      *
-     * @param timestamp
-     * @return The index of the log event L.
+     * @param timestamp The timestamp to search for.
      */
src/services/decoders/JsonlDecoder/index.ts (1)

125-144: Consider adding input validation and improving type safety.

The binary search implementation is efficient, but could benefit from some improvements:

  1. Add validation for negative timestamps
  2. Extract the log event access into a separate constant to improve readability and type safety

Apply this diff to improve the implementation:

 findNearestLogEventByTimestamp (timestamp: number): Nullable<number> {
+    if (timestamp < 0) {
+        return null;
+    }
     let low = 0;
     let high = this.#logEvents.length - 1;
     if (high < low) {
         return null;
     }

     while (low <= high) {
         const mid = Math.floor((low + high) / 2);
-        // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
-        const midTimestamp = this.#logEvents[mid]!.timestamp.valueOf();
+        const midLogEvent = this.#logEvents[mid];
+        const midTimestamp = midLogEvent.timestamp.valueOf();
         if (midTimestamp <= timestamp) {
             low = mid + 1;
         } else {
             high = mid - 1;
         }
     }

     return high;
 }
src/contexts/UrlContextProvider.tsx (1)

208-219: Consider improving readability of the parameter array declaration.

The array declaration could be more readable by using proper line breaks and alignment.

Apply this diff to improve readability:

-    const numberHashParamNames = [HASH_PARAM_NAMES.LOG_EVENT_NUM,
-        HASH_PARAM_NAMES.TIMESTAMP];
+    const numberHashParamNames = [
+        HASH_PARAM_NAMES.LOG_EVENT_NUM,
+        HASH_PARAM_NAMES.TIMESTAMP,
+    ];
src/services/LogFileManager/index.ts (1)

434-462: Consider adding validation and improving error handling.

The cursor data handling for timestamps could benefit from some improvements:

  1. Add validation for negative timestamps
  2. Consider adding error handling for the case when no event is found

Apply this diff to improve the implementation:

     } else {
+        if (args.timestamp < 0) {
+            throw new Error("Timestamp cannot be negative");
+        }
         const eventIdx = this.#decoder.findNearestLogEventByTimestamp(args.timestamp);
         if (null !== eventIdx) {
             eventNum = eventIdx + 1;
+        } else {
+            throw new Error("No log event found for the given timestamp");
         }
     }
src/contexts/StateContextProvider.tsx (1)

488-498: Consider adding error handling for invalid timestamps.

The timestamp effect should handle invalid timestamp values gracefully.

Apply this diff to improve error handling:

     if (null === mainWorkerRef.current || null === timestamp) {
         return;
     }
 
+    if (timestamp < 0) {
+        console.error("Invalid timestamp value:", timestamp);
+        updateWindowUrlHashParams({timestamp: null});
+        return;
+    }
+
     loadPageByCursor(mainWorkerRef.current, {
         code: CURSOR_CODE.TIMESTAMP,
         args: {timestamp: timestamp},
     });
     updateWindowUrlHashParams({timestamp: null});
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 10b7cf5 and a81d570.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (9)
  • package.json (1 hunks)
  • src/contexts/StateContextProvider.tsx (5 hunks)
  • src/contexts/UrlContextProvider.tsx (2 hunks)
  • src/services/LogFileManager/index.ts (1 hunks)
  • src/services/decoders/ClpIrDecoder.ts (1 hunks)
  • src/services/decoders/JsonlDecoder/index.ts (3 hunks)
  • src/typings/decoders.ts (1 hunks)
  • src/typings/logs.ts (1 hunks)
  • src/typings/url.ts (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.{cpp,hpp,java,js,jsx,ts,tsx}`: - Prefer `false ==

**/*.{cpp,hpp,java,js,jsx,ts,tsx}: - Prefer false == <expression> rather than !<expression>.

  • src/typings/logs.ts
  • src/services/decoders/ClpIrDecoder.ts
  • src/typings/url.ts
  • src/typings/decoders.ts
  • src/contexts/UrlContextProvider.tsx
  • src/services/decoders/JsonlDecoder/index.ts
  • src/services/LogFileManager/index.ts
  • src/contexts/StateContextProvider.tsx
🔇 Additional comments (10)
src/typings/url.ts (1)

9-10: LGTM! The changes align well with the PR objectives.

The addition of the TIMESTAMP enum value and the timestamp property in UrlHashParams interface provides proper typing support for querying log events by timestamp via URL hash parameters.

Also applies to: 18-19

src/typings/logs.ts (1)

46-46: LGTM! The constant is properly defined using BigInt.

The use of BigInt for INVALID_TIMESTAMP_VALUE aligns with the timestamp handling in the codebase, particularly with the DecodeResult type that uses BigInt for timestamps.

src/typings/decoders.ts (1)

102-119: LGTM! The method is well-documented and properly typed.

The method signature and documentation provide clear guidance on the chronological order requirement and the marker log event concept.

src/services/decoders/ClpIrDecoder.ts (1)

80-82: LGTM! The implementation is clean and handles type conversion properly.

The method correctly converts the number timestamp to BigInt before passing it to the stream reader, maintaining consistency with the underlying implementation.

src/services/decoders/JsonlDecoder/index.ts (1)

238-256: LGTM! Timestamp type change improves precision.

The change from number to bigint for timestamp handling is a good improvement as it prevents potential precision loss when dealing with large timestamp values.

src/contexts/UrlContextProvider.tsx (1)

32-35: LGTM! Default state properly initialized.

The addition of HASH_PARAM_NAMES.TIMESTAMP to URL_HASH_PARAMS_DEFAULT with a null default value is consistent with the existing pattern.

src/contexts/StateContextProvider.tsx (3)

250-276: LGTM! Proper state management setup.

The addition of timestamp handling in the context and ref setup follows the established patterns.


464-468: LGTM! Consistent ref synchronization.

The useEffect hook for timestamp synchronization follows the same pattern as other ref synchronizations.


541-546: LGTM! Consistent cursor handling.

The addition of timestamp cursor handling in the file loading logic is consistent with the existing event number cursor handling.

package.json (1)

33-33: Dependency Version Bump for clp-ffi-js
The dependency version for clp-ffi-js has been updated from ^0.3.4 to ^0.3.5. This change supports the new timestamp handling features as outlined in the PR objectives. Please ensure that all modules relying on this dependency are tested against this updated version for compatibility.

@Henry8192
Copy link
Collaborator Author

Just woke up for a bug in find nearst log event by timestamp before I go to sleep again: if all log events' timestamps are larger than target timestamp, return high would overflow. Need to check that condition before the return.

@Henry8192 Henry8192 requested a review from junhaoliao February 18, 2025 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants