check for active MSses before starting DB upgrade #12140

DaanHoogland · 2025-11-26T14:15:56Z

Description

This PR...

Fixes: #11973

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov · 2025-11-26T14:24:50Z

Codecov Report

❌ Patch coverage is 56.14035% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 16.19%. Comparing base (028dd86) to head (e79dacb).
⚠️ Report is 3 commits behind head on 4.20.

Files with missing lines	Patch %	Lines
...java/com/cloud/upgrade/DatabaseUpgradeChecker.java	56.14%	22 Missing and 3 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               4.20   #12140      +/-   ##
============================================
+ Coverage     16.18%   16.19%   +0.01%     
- Complexity    13298    13304       +6     
============================================
  Files          5657     5657              
  Lines        498470   498511      +41     
  Branches      60493    60505      +12     
============================================
+ Hits          80660    80725      +65     
+ Misses       408830   408800      -30     
- Partials       8980     8986       +6

Flag	Coverage Δ
uitests	`4.00% <ø> (-0.01%)`	⬇️
unittests	`17.04% <56.14%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

Pull request overview

This PR adds a check to prevent database upgrades from running when multiple management servers are active. The implementation verifies that only one management server with status 'UP' exists in the database before proceeding with the upgrade process, addressing issue #11973.

Extracts the upgrade logic into a new doUpgrades() method for better code organization
Introduces checkIfStandalone() to query the management_server table and enforce single-server mode
Throws an exception if multiple active management servers are detected, preventing concurrent upgrade attempts

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java

Co-authored-by: Copilot <[email protected]>

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java

blueorangutan · 2025-12-02T09:23:36Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 15884

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-02T09:31:36Z

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java

+                     ResultSet rs = pstmt.executeQuery()) {
+                    if (rs.next()) {
+                        int count = rs.getInt(1);
+                        return count == 0;


The logic here has a subtle issue. The query counts management servers with state = 'UP', and returns true (standalone) when count is 0. However, this check is being performed by a management server that is currently running.

This means:

If this MS has already registered itself in the mshost table as 'UP', the count would be at least 1, and it would never be considered standalone.

If this MS hasn't registered yet, the count might be 0 even in a clustered environment if other MSes are down or not yet started.

The check should likely exclude the current management server from the count, or check for count <= 1 instead of count == 0, to properly detect if this is the only running MS.

Suggested change

return count == 0;

return count <= 1;

one valid issue is that an MS performing an upgrade is not yet registered as UP. so this fix is better than what we have but certainly not watertight yet.

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java

blueorangutan · 2025-12-02T17:08:43Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15893

check for active MSses before starting DB upgrade

edcbbde

DaanHoogland marked this pull request as draft November 26, 2025 14:16

DaanHoogland requested review from borisstoyanov, Copilot and weizhouapache November 26, 2025 14:16

Copilot started reviewing on behalf of DaanHoogland November 26, 2025 14:16 View session

Copilot finished reviewing on behalf of DaanHoogland November 26, 2025 14:19

Copilot AI reviewed Nov 26, 2025

View reviewed changes

weizhouapache reviewed Nov 27, 2025

View reviewed changes

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java Outdated Show resolved Hide resolved

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java Outdated Show resolved Hide resolved

DaanHoogland added 2 commits November 27, 2025 16:41

fix sql

48af60a

none may be up!

46a6f7a

DaanHoogland requested a review from Copilot November 27, 2025 15:44

Copilot started reviewing on behalf of DaanHoogland November 27, 2025 15:45 View session

Copilot finished reviewing on behalf of DaanHoogland November 27, 2025 15:45

apache deleted a comment from blueorangutan Nov 27, 2025

weizhouapache reviewed Nov 27, 2025

View reviewed changes

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java Outdated Show resolved Hide resolved

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java Outdated Show resolved Hide resolved

Copilot AI reviewed Nov 27, 2025

View reviewed changes

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java Outdated Show resolved Hide resolved

Apply suggestions from code review

2135e1e

Co-authored-by: Copilot <[email protected]>

DaanHoogland commented Nov 28, 2025

View reviewed changes

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java Show resolved Hide resolved

make condition not fatal

42414f4

DaanHoogland commented Dec 1, 2025

View reviewed changes

engine/schema/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java Outdated Show resolved Hide resolved

apache deleted a comment from blueorangutan Dec 1, 2025

apache deleted a comment from blueorangutan Dec 2, 2025

change moment of abort

6d9a6c5

DaanHoogland requested review from Copilot and weizhouapache December 2, 2025 09:21

Copilot started reviewing on behalf of DaanHoogland December 2, 2025 09:21 View session

Copilot finished reviewing on behalf of DaanHoogland December 2, 2025 09:23

Copilot AI reviewed Dec 2, 2025

View reviewed changes

Daan Hoogland added 3 commits December 2, 2025 15:31

more hard exit

0315005

some unit tests

335f144

imports

e79dacb

apache deleted a comment from blueorangutan Dec 2, 2025

check for active MSses before starting DB upgrade #12140

Are you sure you want to change the base?

check for active MSses before starting DB upgrade #12140

Conversation

DaanHoogland commented Nov 26, 2025

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecov bot commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

blueorangutan commented Dec 2, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

DaanHoogland Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

blueorangutan commented Dec 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Nov 26, 2025 •

edited

Loading

DaanHoogland Dec 2, 2025 •

edited

Loading