Skip to content

add check before fencing #10059

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
DaanHoogland wants to merge 3 commits into from
Closed

add check before fencing #10059

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8feecbc
add check befor fencing
leo79901 Dec 4, 2024
50d4cf9
rename logger to prevent merge forward conflict
DaanHoogland Dec 8, 2024
a6c20f4
Merge branch '4.20' into fenchCheck
DaanHoogland Dec 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions plugins/hypervisors/kvm/src/main/java/org/apache/cloudstack/kvm/ha/KVMHAProvider.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
import org.apache.cloudstack.outofbandmanagement.OutOfBandManagement.PowerOperation;
import org.apache.cloudstack.outofbandmanagement.OutOfBandManagementService;
import org.joda.time.DateTime;
import com.cloud.host.Status;

import javax.inject.Inject;

Expand Down Expand Up @@ -84,11 +85,17 @@

@Override
public boolean fence(Host r) throws HAFenceException {

try {
if (outOfBandManagementService.isOutOfBandManagementEnabled(r)){
final OutOfBandManagementResponse resp = outOfBandManagementService.executePowerOperation(r, PowerOperation.OFF, null);
return resp.getSuccess();
// host exists and is managed OOB
if (r != null && outOfBandManagementService.isOutOfBandManagementEnabled(r)) {
// check host status
if (Status.Down.equals(r.getStatus())) {
Copy link
Contributor

@slavkap slavkap Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DaanHoogland, my concern here is that host's status could be Down but in reality to be still running. With this there is an option the virtual machines started to a neighbor host to be left running on the previous one.
Maybe it is better to check the OOBM status rather the host's.
Unfortunately now I don't have environment with IPMI setup and cannot test this change.
Probably here and @rp- could give some advice.

Copy link
Contributor

@rp- rp- Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True what @slavkap said, and now I'm a bit confused what actual the original issue was?
Wouldn't it be not just a noop if the host is already down and fence it again?

Copy link
Contributor Author

@DaanHoogland DaanHoogland Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean to change this condition to something else , @slavkap ?
If it is !Down a power operation will be performed. should it happen here as well?
I am not sure if we have a OOB status retrieval possibility.

Copy link
Contributor

@slavkap slavkap Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rp-, there was a discussion about an issue with the fencing that the Host is Powered off and CS tries to shut it down again. But I've seen cases where the host could be in the down state before the IPMI tries to power it off.
@DaanHoogland, yes, I think there is an option to check the OOBM status and the condition should be changed with the result of it

Copy link
Contributor Author

@DaanHoogland DaanHoogland Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, it is not part of outOfBandManagementService.

logger.info("Host " + r.getName() + " is already down. Returning success.");

Check warning on line 93 in plugins/hypervisors/kvm/src/main/java/org/apache/cloudstack/kvm/ha/KVMHAProvider.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Format specifiers should be used instead of string concatenation. 


See more on https://sonarcloud.io/project/issues?id=apache_cloudstack&issues=AZr-y74J-bON_ald3ZyK&open=AZr-y74J-bON_ald3ZyK&pullRequest=10059
return true;
} else {
final OutOfBandManagementResponse resp = outOfBandManagementService.executePowerOperation(r, PowerOperation.OFF, null);
return resp.getSuccess();
}
} else {
logger.warn("OOBM fence operation failed for this host {}", r);
return false;
Expand Down
Loading