Restarting a vault unit results in HostDown lingering alerts

[peppepetra]

Bug Description

After restarting one vault units, we stop receiving metric from it

up{instance="openstack_97c88ed0-2373-4d3f-839a-c993c4eeac89_vault_vault/0", job="juju_openstack_97c88ed0_vault_prometheus_scrape-0", juju_application="vault", juju_charm="vault-k8s", juju_model="openstack", juju_model_uuid="97c88ed0-2373-4d3f-839a-c993c4eeac89", juju_unit="vault/0"}
0
up{instance="openstack_97c88ed0-2373-4d3f-839a-c993c4eeac89_vault_vault/1", job="juju_openstack_97c88ed0_vault_prometheus_scrape-1", juju_application="vault", juju_charm="vault-k8s", juju_model="openstack", juju_model_uuid="97c88ed0-2373-4d3f-839a-c993c4eeac89", juju_unit="vault/1"}
1
up{instance="openstack_97c88ed0-2373-4d3f-839a-c993c4eeac89_vault_vault/2", job="juju_openstack_97c88ed0_vault_prometheus_scrape-2", juju_application="vault", juju_charm="vault-k8s", juju_model="openstack", juju_model_uuid="97c88ed0-2373-4d3f-839a-c993c4eeac89", juju_unit="vault/2"}
1

This results in HostDown alerts

To Reproduce

Deploy Sunbeam Noble/Caracal. Reboot one k8s node.

Environment

vault-k8s 1.18/stable 507
juju 3.6.23
COS HA 2/edge

Relevant log output

NA

Additional context

No response