Skip to content

Only log passing filter#465

Open
antoine-le-calloch wants to merge 1 commit into
mainfrom
only_print_passing_filter
Open

Only log passing filter#465
antoine-le-calloch wants to merge 1 commit into
mainfrom
only_print_passing_filter

Conversation

@antoine-le-calloch

Copy link
Copy Markdown
Contributor

Update filter processing to log "passed filter" only when at least one doc pass the filter.

@antoine-le-calloch antoine-le-calloch self-assigned this May 6, 2026
Copilot AI review requested due to automatic review settings May 6, 2026 16:33
@antoine-le-calloch antoine-le-calloch requested a review from Theodlz May 6, 2026 16:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates ZTF filter processing so the "passed filter" log line is only emitted when at least one document actually passes the filter, reducing misleading/zero-count log noise during normal operations.

Changes:

  • Skip per-filter processing early when out_documents is empty.
  • Move the "passed filter" info! log to occur only after confirming there are output documents to process.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Theodlz

Theodlz commented May 6, 2026

Copy link
Copy Markdown
Collaborator

@antoine-le-calloch one thing to take note of: the throughput test relies on filter logs (which should be the last that are logged in the app) to get the timestamp of the end of processing, so we can compute the wall time.

My suggestion would be to log everytime but at the debug level, and make sure the throughput test sets the log level to debug. That way no problem for production (where I doubt you have the log level set to debug) nor throughput testing/development?

@antoine-le-calloch

Copy link
Copy Markdown
Contributor Author

@antoine-le-calloch one thing to take note of: the throughput test relies on filter logs (which should be the last that are logged in the app) to get the timestamp of the end of processing, so we can compute the wall time.

My suggestion would be to log everytime but at the debug level, and make sure the throughput test sets the log level to debug. That way no problem for production (where I doubt you have the log level set to debug) nor throughput testing/development?

Ohh yeah nice catch, sorry for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants