Governance_alerts_reminder repeated notifications.

[iwbinb]

The governance_alerts_reminder is triggering repeated notifications.

1. Even after the validator has already voted, the reminders keep repeating.
2. The reminders appear roughly every 10 minutes.

[gluckzhang]

Hi @iwbinb , thanks for reporting this. That sounds weird indeed. I would like to know more about the exact behavior. Do you receive an alert resolved message after voting, and then the alert is triggered again? Or is it only about receiving reminders for that unvoted proposal?

I used to see one case about this, caused by an RPC that sometimes does not return the vote transactions so that Tenderduty fails to confirm whether the validator has voted or not.

[iwbinb]

Hi @gluckzhang After voting, a ‘resolved’ message is received. Then, after some time, an ‘unvoted’ warning appears again, followed by another ‘resolved’ message — and this cycle keeps repeating.

[gluckzhang]

Hi @gluckzhang After voting, a ‘resolved’ message is received. Then, after some time, an ‘unvoted’ warning appears again, followed by another ‘resolved’ message — and this cycle keeps repeating.

Thanks for the reply. I see! In this case, I think it is most likely related to the case I mentioned: the RPC sometimes gives an empty response when Tenderduty queries a vote transaction of a specific proposal. I will look into this and see how to improve the robustness.

By the way, are you using a public RPC endpoint? Do you mind sharing the related config so that I can also try it locally? You don't need to share the validator address. I can pick up a random one for testing.

[iwbinb]

For the current atom proposal (#1023)
RPC: https://cosmos-rpc.publicnode.com:443
Please check.

[gluckzhang]

For the current atom proposal (#1023) RPC: cosmos-rpc.publicnode.com:443 Please check.

Hi @iwbinb, thanks for the info. I tried it locally with a validator that has already voted on this proposal, using the given endpoint. So far it has been running for 20 minutes but I cannot reproduce the behavior. Do you still have the same issue right now?

I still think the cause is that sometimes Tenderduty fails to query the vote transaction from an rpc endpoint and then self-recovers after several tries. I will see how to avoid these temporary query failures and the corresponding false positive alerts. Will update here later on :)