Boot Delay and CPU Unavailability When Booting with Disabled CPUs in Device Tree

[agicy]

Goal

I am attempting to boot zone0 (root-linux) with a complete device tree that includes all CPUs. To reserve some CPUs for other zones, I have marked them as status = "disabled" in the device tree, rather than deleting their nodes entirely. The intention is to have a complete device tree for the system while isolating these CPUs from zone0.

Observed Behavior

This configuration leads to two primary issues:

1. Significant Boot Delay: During boot, hvisor appears to serially attempt to initialize every CPU listed in the device tree, including those marked as disabled. This results in a long timeout for each disabled CPU, causing a substantial delay in the overall boot process.
2. CPUs Remain Unusable: After zone0 has finished booting, the CPUs that were marked as disabled are not available to be assigned to or used by any other zones.

Expected Behavior

The hypervisor should recognize the status = "disabled" property for the CPU nodes in the device tree. It should skip the initialization attempt for these CPUs, avoiding the boot delay. Subsequently, these disabled (but present) CPUs should be available for assignment to other guest zones.

Question

Is the current behavior — attempting to boot disabled CPUs, leading to a timeout, and rendering them unusable by other zones — an expected feature or a potential bug in the hypervisor?

Note

Error logs will be provided in a few days.

[enkerewpo]

https://github.com/syswonder/hvisor/blob/main/platform/aarch64/qemu-gicv3/board.rs

During boot, hvisor appears to serially attempt to initialize every CPU listed in the device tree

hvisor uses board.rs BOARD_NCPUS to specify "how many CPU hvisor can see"

This results in a long timeout for each disabled CPU

I don't understand this line, do you mean "long timeout for root linux boot" instead of the "long timeout for hvisor boot"? Because we assume hvisor can always boot BOARD_NCPUS cpus.

and hvisor uses

pub const ROOT_ZONE_CPUS: u64 = (1 << 0) | (1 << 1); // CPU0 CPU1

for "which CPUs are issued to root linux". And we require that root linux's DTB should be configured accordingly to ROOT_ZONE_CPUS bitmap.

After zone0 has finished booting, the CPUs that were marked as disabled are not available to be assigned to or used by any other zones.

DTS is used for linux in zone only, and hvisor DO NOT use/parse any DTS for now. If you disable some CPUs(according to ROOT_ZONE_CPUS) on root linux, that doesn't affect hvisor for allocating free CPUs to other zones.

[agicy]

Thanks @enkerewpo for the clarification and the additional suggestions provided offline.

Based on the discussion, I have confirmed that the issue was caused by a mismatch between the hvisor configuration and the Linux Device Tree, as well as an incorrect usage of DT properties for CPU nodes.

Here is the complete solution summary:

1. Hvisor Configuration: As you pointed out, hvisor does not parse the DTB for CPU allocation. We must explicitly define BOARD_NCPUS and the ROOT_ZONE_CPUS bitmap in board.rs to control which CPUs are initialized by the hypervisor and assigned to the root zone.

2. Device Tree Modification: regarding the boot delay and "disabled" status, you rightly suggested that the status property might not be strictly supported or behave as expected for cpu nodes in the kernel bindings (Reference: ARM CPUs Binding).

   Instead of using status = "disabled", the correct approach to hide specific CPUs from the Root Linux (to match the ROOT_ZONE_CPUS configuration) is to remove them entirely using the /delete-node/ syntax.

   Reference: Device Tree Specification - /delete-node/

Result:
By aligning the ROOT_ZONE_CPUS in board.rs and using /delete-node/ in the Root Linux DTS to exclude the reserved CPUs, the boot delay is eliminated, and the system functions as expected.

Closing this issue as resolved.