Skip to content

consul_raft_leader does not reflect correct state of leader #208

Description

@Abhimanyu-Jana

What did you do?
setup monitoring for presence/absence of cluster leader using consul_raft_leader metric

What did you expect to see?
When external queries to consul cluster fail with HTTP 500 or "No cluster leader" error, we expect to see consul_raft_leader value change from 1 to 0

What did you see instead? Under which circumstances?
consul_raft_leader value still remains 1 despite there being obvious issues with cluster health. We can confirm based on logs that show the "No cluster leader" errors, as well as using "consul operator raft list-peers" command

Environment
Linux

  • consul_exporter version:
    0.7.1

  • Consul version:
    Consul v1.8.3

  • Prometheus version:
    N/A

  • Prometheus configuration file:
    N/A

  • Logs:

Error getting peers: Failed to retrieve raft configuration: Unexpected response code: 500 (No cluster leader)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions