Skip to content

Commit

Permalink
docs: Remove packer docs and files for azure
Browse files Browse the repository at this point in the history 
The azrue build-image instructions have been updated for mkosi podvms.

Since we don't test packer images for azure anymore, we should remove it
here, it's likely that there is a version drift and the resulting images
won't work as expected anymore.

Signed-off-by: Magnus Kulke <[email protected]>
  • Loading branch information
@mkulke
mkulke committed Jan 13, 2025
1 parent 6e25be3 commit 9f8f3d6
Show file tree
Hide file tree
Showing 8 changed files with 63 additions and 919 deletions.
306 changes: 63 additions & 243 deletions src/cloud-api-adaptor/azure/build-image.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,292 +2,112 @@

This documentation will walk you through building the pod VM image for Azure.

> [!NOTE]
> Run the following commands from the directory `azure/image`.
## Pre-requisites

### Install required tools

- Install tools like `git`, `make` and `curl`.
- Install Azure CLI by following instructions [here](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli).

### Azure login

The image build will use your local credentials, so make sure you have logged into your account via `az login`. Retrieve your Subscription ID and set your preferred region:

```bash
export AZURE_SUBSCRIPTION_ID=$(az account show --query id --output tsv)
export AZURE_REGION="eastus"
```

### Resource group

> [!NOTE]
> Skip this step if you already have a resource group you want to use. Please, export the resource group name in the `AZURE_RESOURCE_GROUP` environment variable.
Create an Azure resource group by running the following command:

```bash
export AZURE_RESOURCE_GROUP="caa-rg-$(date '+%Y%m%b%d%H%M%S')"

az group create \
--name "${AZURE_RESOURCE_GROUP}" \
--location "${AZURE_REGION}"
```

### Shared image gallery

Create a shared image gallery:

```bash
export GALLERY_NAME="caaubntcvmsGallery"
az sig create \
--gallery-name "${GALLERY_NAME}" \
--resource-group "${AZURE_RESOURCE_GROUP}" \
--location "${AZURE_REGION}"
```

Create the "Image Definition" by running the following command:

> [!NOTE]
> The flag `--features SecurityType=ConfidentialVmSupported` allows you to a upload custom image and boot it up as a Confidential Virtual Machine (CVM).
```bash
export GALLERY_IMAGE_DEF_NAME="cc-image"
az sig image-definition create \
--resource-group "${AZURE_RESOURCE_GROUP}" \
--gallery-name "${GALLERY_NAME}" \
--gallery-image-definition "${GALLERY_IMAGE_DEF_NAME}" \
--publisher GreatPublisher \
--offer GreatOffer \
--sku GreatSku \
--os-type "Linux" \
--os-state "Generalized" \
--hyper-v-generation "V2" \
--location "${AZURE_REGION}" \
--architecture "x64" \
--features SecurityType=ConfidentialVmSupported
```

## Build pod-VM image

The Pod-VM image can be built in three ways:

- Customize an existing marketplace image
- Customize an existing marketplace image with pre-built binaries
- Convert and upload a pre-built QCOW2 image

### Option 1: Modifying an existing marketplace image

**Install necessary tools**

- Install the following packages (on Ubuntu):

```bash
sudo apt install \
libdevmapper-dev libgpgme-dev gcc clang pkg-config \
libssl-dev libtss2-dev protobuf-compiler
```

- Install `yq` by following instructions [here](https://mikefarah.gitbook.io/yq/#install).
- Install Golang by following instructions [here](https://go.dev/doc/install).
- Install packer by following [these instructions](https://learn.hashicorp.com/tutorials/packer/get-started-install-cli).

**Build**

- Create a custom Azure VM image based on Ubuntu 22.04 adding kata-agent, agent-protocol-forwarder and other dependencies for Cloud API Adaptor (CAA):

```bash
export PKR_VAR_resource_group="${AZURE_RESOURCE_GROUP}"
export PKR_VAR_location="${AZURE_REGION}"
export PKR_VAR_subscription_id="${AZURE_SUBSCRIPTION_ID}"
export PKR_VAR_use_azure_cli_auth=true
export PKR_VAR_az_gallery_name="${GALLERY_NAME}"
export PKR_VAR_az_gallery_image_name="${GALLERY_IMAGE_DEF_NAME}"
export PKR_VAR_az_gallery_image_version="0.0.1"
export PKR_VAR_offer=0001-com-ubuntu-confidential-vm-jammy
export PKR_VAR_sku=22_04-lts-cvm

export TEE_PLATFORM="az-cvm-vtpm"
export LIBC=gnu
export CLOUD_PROVIDER=azure
PODVM_DISTRO=ubuntu make image
```

Use the `ManagedImageSharedImageGalleryId` field from output of the above command to populate the following environment variable. It's used while deploying cloud-api-adaptor:
This assumes an Ubuntu 24.04 environment for building. When using a different distribution, adjust accordingly:

```bash
# e.g. format: /subscriptions/.../resourceGroups/.../providers/Microsoft.Compute/galleries/.../images/.../versions/../
export AZURE_IMAGE_ID="/subscriptions/${AZURE_SUBSCRIPTION_ID}/resourceGroups/${AZURE_RESOURCE_GROUP}/providers/Microsoft.Compute/galleries/${GALLERY_NAME}/images/${GALLERY_IMAGE_DEF_NAME}/versions/${PKR_VAR_az_gallery_image_version}"
sudo apt-get update -y
sudo apt-get install -y \
alien \
bubblewrap \
build-essential \
dnf \
mtools \
qemu-utils \
systemd-ukify \
uidmap
sudo snap install yq
```

### Option 2: Customize an image using prebuilt binaries via Docker
### Install mkosi

```bash
docker build -t azure-podvm-builder .
```
Clone mkosi and add the bin folder to the PATH:

```bash
docker run --rm \
-v "$HOME/.azure:/root/.azure" \
-e AZURE_SUBSCRIPTION_ID \
-e AZURE_RESOURCE_GROUP \
-e GALLERY_NAME \
-e GALLERY_IMAGE_DEF_NAME \
azure-podvm-builder
MKOSI_VERSION=$(yq -e '.tools.mkosi' versions.yaml)"
git clone -b "$MKOSI_VERSION" https://github.com/systemd/mkosi
export PATH="$PWD/mkosi/bin:$PATH"
mkosi --version
```
If you want to use a different base image, then you'll need to export environment variables: `PUBLISHER`, `OFFER` and `SKU`.

Sometimes using the marketplace image requires accepting a licensing agreement and also using a published plan.
Following [link](https://learn.microsoft.com/en-us/azure/virtual-machines/linux/cli-ps-findimage) provides more detail.

For example using the CentOS 8.5 image from "eurolinux" publisher requires a plan and license agreement.
### Install uplosi
You'll need to first get the Uniform Resource Name (URN):
The tool is required to publish images to an image gallery.
```bash
az vm image list \
--location ${AZURE_REGION} \
--publisher eurolinuxspzoo1620639373013 \
--offer centos-8-5-free \
--sku centos-8-5-free \
--all \
--output table
wget "https://github.com/edgelesssys/uplosi/releases/download/v0.3.0/uplosi_0.3.0_linux_amd64.tar.gz"
tar xzf uplosi_0.3.0_linux_amd64.tar.gz uplosi
sudo mv uplosi mkosi/bin/
```
Then you'll need to accept the agreement:

```bash
az vm image terms accept \
--urn eurolinuxspzoo1620639373013:centos-8-5-free:centos-8-5-free:8.5.5
```

Then you can use the following command line to build the image:

```bash
docker run --rm \
-v "$HOME/.azure:/root/.azure" \
-e AZURE_SUBSCRIPTION_ID \
-e AZURE_RESOURCE_GROUP \
-e PUBLISHER=eurolinuxspzoo1620639373013 \
-e SKU=centos-8-5-free \
-e OFFER=centos-8-5-free \
-e PLAN_NAME=centos-8-5-free \
-e PLAN_PRODUCT=centos-8-5-free \
-e PLAN_PUBLISHER=eurolinuxspzoo1620639373013 \
-e PODVM_DISTRO=centos \
azure-podvm-builder
```

Another example of building Red Hat Enterprise Linux (RHEL) based image:

```bash
docker run --rm \
-v "$HOME/.azure:/root/.azure" \
-e AZURE_SUBSCRIPTION_ID \
-e AZURE_RESOURCE_GROUP \
-e PUBLISHER=RedHat \
-e SKU=9-lvm \
-e OFFER=RHEL \
-e PODVM_DISTRO=rhel \
azure-podvm-builder
```

### Option 3: Using a pre-created QCOW2 image

`quay.io/confidential-containers` hosts pre-created pod-vm images as container images.
### Azure login
- Download QCOW2 image
The image build will use your local credentials, so make sure you have logged into your account via `az login`.
```bash
mkdir -p qcow2-img && cd qcow2-img

export QCOW2_IMAGE="quay.io/confidential-containers/podvm-generic-ubuntu-amd64:latest"
curl -LO https://raw.githubusercontent.com/confidential-containers/cloud-api-adaptor/staging/podvm/hack/download-image.sh

bash download-image.sh $QCOW2_IMAGE . -o podvm.qcow2
export AZURE_SUBSCRIPTION_ID=$(az account show --query id --output tsv)
export AZURE_REGION="eastus"
```
- Convert QCOW2 image to Virtual Hard Disk (VHD) format

You'll need the `qemu-img` tool for conversion.

```bash
qemu-img convert -O vpc -o subformat=fixed,force_size podvm.qcow2 podvm.vhd
```
## Build
- Create Storage Account
### Binaries
Create a storage account if none exists. Otherwise you can use the existing storage account.
Build binaries with support for the Azure CVM TEEs and verify the provenance attestation for the binaries consumed from upstream:
```bash
export AZURE_STORAGE_ACCOUNT=cocosa

az storage account create \
--name $AZURE_STORAGE_ACCOUNT \
--resource-group $AZURE_RESOURCE_GROUP \
--location $AZURE_REGION \
--sku Standard_ZRS \
--encryption-services blob
cd ../podvm-mkosi
TEE_PLATFORM=az-cvm-vtpm VERIFY_PROVENANCE=yes make binaries
```
- Create storage container
### Image
Create a storage container if none exists. Otherwise you can use the existing storage account
You can build a debug image in which you can login as root by providing an SSH key as a build asset, the debug-image will also auto-login on the serial console, and contain some extraneous packages for debugging:
```bash
export AZURE_STORAGE_CONTAINER=vhd
az storage container create \
--account-name $AZURE_STORAGE_ACCOUNT \
--name $AZURE_STORAGE_CONTAINER \
--auth-mode login
cp ~/.ssh/id_rsa.pub podvm-mkosi/resources/authorized_keys
make image-debug
```
- Get storage key
Otherwise, you can build a release image:
```bash
AZURE_STORAGE_KEY=$(az storage account keys list --resource-group $AZURE_RESOURCE_GROUP --account-name $AZURE_STORAGE_ACCOUNT --query "[?keyName=='key1'].{Value:value}" --output tsv)

echo $AZURE_STORAGE_KEY
```
make image
```
- Upload VHD file to Azure Storage
## Publish
```bash
az storage blob upload --container-name $AZURE_STORAGE_CONTAINER --name podvm.vhd --file podvm.vhd
```

- Get the VHD URI
We can upload the built image to an Azure Image Gallery. The resulting image id can be used as `AZURE_IMAGE_ID` param in the [../install/overlays/azure/kustomization.yaml](../install/azure/kustomization.yaml) file.
```bash
AZURE_STORAGE_EP=$(az storage account list -g $AZURE_RESOURCE_GROUP --query "[].{uri:primaryEndpoints.blob} | [? contains(uri, '$AZURE_STORAGE_ACCOUNT')]" --output tsv)

echo $AZURE_STORAGE_EP

export VHD_URI="${AZURE_STORAGE_EP}${AZURE_STORAGE_CONTAINER}/podvm.vhd"
```
SHARING_NAME_PREFIX=sharedcocopodvms # set for a community gallery
IMAGE_VERSION=0.1.0
IMAGE_DEFINITION=podvm
IMAGE_GALLERY=coco
RESOURCE_GROUP=myrg
SUBSCRIPTION_ID=mysub
- Create Azure VM Image Version
cat <<EOF> uplosi.conf
[base]
imageVersion = \"$IMAGE_VERSION\"
name = \"$IMAGE_DEFINITION\"
```bash
az sig image-version create \
--resource-group $AZURE_RESOURCE_GROUP \
--gallery-name $GALLERY_NAME \
--gallery-image-definition $GALLERY_IMAGE_DEF_NAME \
--gallery-image-version 0.0.1 \
--target-regions $AZURE_REGION \
--os-vhd-uri "$VHD_URI" \
--os-vhd-storage-account $AZURE_STORAGE_ACCOUNT
```
[base.azure]
subscriptionID = \"$SUBSCRIPTION_ID\"
location = "eastus"
resourceGroup = \"$RESOURCE_GROUP\"
sharedImageGallery = \"$IMAGE_GALLERY\"
${SHARING_NAME_PREFIX:+sharingNamePrefix = \"$SHARING_NAME_PREFIX\"}
On success, the command will generate the image id. Set this image id as a value of `AZURE_IMAGE_ID` in `peer-pods-cm` Configmap.
[variant.default]
provider = "azure"
You can also use the following command to retrieve the image id:

```bash
AZURE_IMAGE_ID=$(az sig image-version list --resource-group $AZURE_RESOURCE_GROUP --gallery-name $GALLERY_NAME --gallery-image-definition $GALLERY_IMAGE_DEF_NAME --query "[].{Id: id}" --output tsv)
[variant.default.azure]
replicationRegions = ["eastus","eastus2","westeurope","northeurope"]
EOF
echo $AZURE_IMAGE_ID
uplosi upload build/system.raw
```
4 changes: 0 additions & 4 deletions src/cloud-api-adaptor/azure/image/.gitignore
View file

This file was deleted.

Loading

0 comments on commit 9f8f3d6

Please sign in to comment.