Skip to content

Rollback CPU pool allocated if memory reservation fails #708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
SSShingne wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Rollback CPU pool allocated if memory reservation fails #708

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions bootstrap/nitro-cli-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ ERR_CLEAR_PAGE=203
ERR_SET_PAGE=204
ERR_ROLLBACK_PAGE=205
ERR_INSUFFICIENT_MEMORY=206
ERR_ROLLBACK_CPU_POOL=207

# Trap any exit condition, including all fatal errors.
trap 'error_handler $? $LINENO' EXIT
Expand Down Expand Up @@ -110,6 +111,9 @@ function fail {
"$ERR_INSUFFICIENT_MEMORY")
echo "Failed to configure entire amount of requested memory. This indicates insufficient system resources."
;;
"$ERR_ROLLBACK_CPU_POOL")
echo "Failed to roll back CPU pool configuration. The CPU pool may be in an inconsistent state."
;;
\?)
echo "An unknown error has occurred: $1"
;;
Expand Down Expand Up @@ -536,9 +540,15 @@ function run_in_driver_dir {
function configure_cpu_pool {
[ -f "$CPU_POOL_FILE" ] || fail "The CPU pool file is missing. Please make sure the Nitro Enclaves driver is inserted."

local prev_val
prev_val=$(cat "$CPU_POOL_FILE" 2>/dev/null || echo "")

print "Configuring the enclave CPU pool..."
sudo_run "echo $1 > $CPU_POOL_FILE" || fail "Failed to configure the CPU pool."
print "Done."

# Return the previous value so caller can roll back later if needed.
echo "$prev_val"
}

# Configure the CPU pool using the provided CPU count.
Expand Down Expand Up @@ -769,10 +779,22 @@ done
# If CPU IDs are requested, memory can only be reserved on the same NUMA node as the CPUs. If a CPU count
# is requested, we need to iterate over all available NUMA nodes to find one which supports both the CPU
# and the memory requirements.
[ -z "$cpu_id_request" ] || configure_cpu_pool "$cpu_id_request"
prev_cpu_pool=""
if [ -n "$cpu_id_request" ]; then
prev_cpu_pool=$(configure_cpu_pool "$cpu_id_request")
fi

if [ -z "$cpu_count_request" ]; then
[ -z "$memory_request" ] || configure_huge_pages "$memory_request" || fail $?
if [ -n "$memory_request" ]; then
if ! configure_huge_pages "$memory_request"; then
# Rollback CPU pool if memory configuration failed
if [ -n "$cpu_id_request" ]; then
print "Memory configuration failed, rolling back CPU pool..."
configure_cpu_pool "$prev_cpu_pool" > /dev/null || fail $ERR_ROLLBACK_CPU_POOL
fi
fail $?
fi
fi
else
configure_cpu_pool_by_cpu_count "$cpu_count_request" "$memory_request"
fi
Expand Down