Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validator_monitor_slashed is showing zero until validators loaded #6863

Open
yakimant opened this issue Jan 20, 2025 · 0 comments
Open

validator_monitor_slashed is showing zero until validators loaded #6863

yakimant opened this issue Jan 20, 2025 · 0 comments

Comments

@yakimant
Copy link
Member

yakimant commented Jan 20, 2025

Describe the bug
validator_monitor_slashed is dropped to zero after node resart and then recovers to previous setting.
This makes difficult to catch the slashed validators. Restart causes drop/spike.
We can workaround it, but it's not obvious for other users and causes either delayed or false positive alerts.

To Reproduce

  1. Prepare a node with validator monitoring and slashed validator
  2. Read the validator_monitor_slashed metric (not zero)
  3. Restart the node
  4. Read the validator_monitor_slashed metric (zero)
  5. Give it some time
  6. Read the validator_monitor_slashed metric (not zero)

Screenshots
Image

Possible solutions

  1. Don't provide the metric until all validators are loaded
  2. Reset the counter on node restart completely
@yakimant yakimant changed the title validator_monitor_slashed is validator_monitor_slashed is showing zero until validators loaded Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant