Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DocDB] Incorrect Aggregation Function for certain xCluster Metrics: Change from Sum to Max #25422

Open
1 task done
yusong-yan opened this issue Dec 24, 2024 · 0 comments
Open
1 task done
Assignees
Labels
area/docdb YugabyteDB core features kind/bug This issue is a bug priority/medium Medium priority issue status/awaiting-triage Issue awaiting triage

Comments

@yusong-yan
Copy link
Contributor

yusong-yan commented Dec 24, 2024

Jira Link: DB-14654

Description

kSum aggregation is currently used for many xCluster metrics, but this leads to overflows and inaccuracies for metrics like last_read_hybridtime, which should instead use maximum aggregation kMax.
For optimal accuracy in the future, we should implement a kLatest aggregation function, which would require metric storing both the latest value and its corresponding timestamp.

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@yusong-yan yusong-yan added area/docdb YugabyteDB core features status/awaiting-triage Issue awaiting triage labels Dec 24, 2024
@yusong-yan yusong-yan self-assigned this Dec 24, 2024
@yugabyte-ci yugabyte-ci added kind/bug This issue is a bug priority/medium Medium priority issue labels Dec 24, 2024
@yusong-yan yusong-yan changed the title [DocDB] Incorrect Aggregation Function for xCluster Metrics: Change from Sum to Max [DocDB] Incorrect Aggregation Function for certain xCluster Metrics: Change from Sum to Max Dec 24, 2024
yusong-yan added a commit that referenced this issue Dec 26, 2024
…etrics from Sum to Max

Summary:
`kSum` aggregation is currently used for many xCluster metrics, but this leads to overflows and inaccuracies for metrics like `last_read_hybridtime`, which should instead use maximum aggregation `kMax`.

Next Step [[ #25437 | #25437 ]]:
For optimal accuracy in the future, we should implement a `kLatest` aggregation function, which would require metric storing both the latest value and its corresponding timestamp.
Jira: DB-14654

Test Plan: Jenkins

Reviewers: hsunder, xCluster, jhe

Reviewed By: jhe

Subscribers: rthallam, ybase, ycdcxcluster, slingam

Differential Revision: https://phorge.dev.yugabyte.com/D40873
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/docdb YugabyteDB core features kind/bug This issue is a bug priority/medium Medium priority issue status/awaiting-triage Issue awaiting triage
Projects
None yet
Development

No branches or pull requests

2 participants