Document virtual image conversion to Azure compatibility

[wwarriner]

What would you like to see added?

Azure expects images to be in a fixed-size VHD format. OpenStack cannot export to this format on its own, but with the help of the bundled qemu-img software, images can be converted.

The process involves creating an image from an instance, saving it to a local device, converting it to the appropriate format, and then uploading it to the Azure service. Here we document all but the last step. Note that these three steps can be performed within an instance on OpenStack (but not within the instance being imaged). They do not need to be performed on your local machine.

openstack image create --disk-format==raw --id=$myid
openstack image save --file=myimage.raw $myid
qemu-img convert -f raw -O vpc -o subformat=fixed,force_size myimage.raw myimage.vhd

Note these commands are not final, they need to be double-checked. I wanted to write this down before I forgot about the concept entirely.

Note also that the Azure portion of the process is handled by Client Services in Enterprise IT.

[wwarriner]

It will be helpful to write #970 first and then crosslink from here to there.

[wwarriner]

Table of Content

• Migrate instance from OpenStack to Azure
  • Create snapshot of the server
  • Download the snapshot
  • Convert raw image to vhd image for Azure
  • Upload vhd file to Azure
  • Create disk from the blob we just uploaded
  • Create VM from the disk
• Migrate instance from Azure to OpenStack
  • Download disk from Azure
  • Convert the vhd image to qcow2 for OpenStack
  • Upload the qcow2 to OpenStack image
  • Create data volume from image
  • Launch instance from uploaded image
    Following https://gitlab.rc.uab.edu/rc/openstack-docs/-/blob/main/migrate-instance-using-snaphot.md

Migrate instance from OpenStack to Azure

Create snapshot of the server

$ openstack server image create ubuntu-to-az --name ubuntu-to-az-snap
$ openstack image show <snapshot_id>

Note: If instance only have vdb (data vol) and no vda (root vol), the snapshot of the server will be 0 byte.

Download the snapshot

local_file_name='ubuntu-to-az-snap.raw'
snapshot_id='uuid'
openstack image save --file ${local_file_name} ${snapshot_id}

Convert raw image to vhd image for Azure

azure_file_name='ubuntu-to-az-snap.vhd'
qemu-img convert -f raw -O vpc -o subformat=fixed,force_size ${local_file_name} ${azure_file_name}

Upload vhd file to Azure

From Azure Interface, go to Storage Account -> select/create a storage account -> select/create a container under blob service -> right click on the container and select Generate SAS with write permission, copy the Blob SAS URL, so we can upload image from CLI.

SAS='exampleacctSAS'
azcopy copy ${azure_file_name} $SAS

Create disk from the blob we just uploaded

Go to Disks select Create with following settings:

• Source type: Storage blob
• Source blob: <The blob you just uploaded>
• Size: <The size of your uploaded disk>
  Note: the size usually default to 1024 GB, so make sure you update it to reduce some cost.

Create VM from the disk

Once the managed disk is created, click the managed disk and find Create VM in the page to create a VM from the disk.

Migrate instance from Azure to OpenStack

Before you start, make sure to stop the VM.

Download disk from Azure

Go to the Azure VM overview page of the VM you want to migrate from Azure
From the navigate menu on the left, click Setting -> Disks. You will see OS disk and Data disks. Click on the one you want to download. In the overview page of the disk, click Settings -> Disk Export from the navigate menu on the left, then click Generate URL. You can then use this URL to download the disk as vhd file.

azure_disk_url='examplediskURL'
local_file_vhd='ubuntu-from-az.vhd'
azcopy copy ${azure_disk_url} ${local_file_vhd}

Convert the vhd image to qcow2 for OpenStack

local_file_qcow2='ubuntu-from-az.qcow2'
qemu-img convert -f vpc -O qcow2 ${local_file_vhd} ${local_file_qcow2}

Upload the qcow2 to OpenStack image

os_image_name='ubuntu-from-az'
openstack image create --file ${local_file_qcow2} --container-format bare --disk-format qcow2 ${os_image_name}

Create data volume from image

If you also download data disk from Azure, and want to convert it into volume, you can follow previous step to upload it to image and create volume from the image

local_file_qcow2_data='ubuntu-from-az-data.qcow2'
os_image_name_data='ubuntu-from-az-data'
os_volume_name_data='ubuntu-from-az-data-vol'
volume_size=10       # target volume size in GB
openstack image create --file ${local_file_qcow2_data} --container-format bare --disk-format qcow2 ${os_image_name_data}
# Copy image ID from the output
openstack volume create --image <image_id> --size ${volume_size} ${os_volume_name_data}

Launch instance from uploaded image

Once you have the image uploaded, you can then launch instance with the image, example command could looks like:

openstack server create --image ${os_image_name} --flavor m1.medium --network example-net --key-name my-keypair --security-group my-sec-group ubuntu-from-az-instance

[diedpigs]

Update some instruction to create OS volume before creating instance:

(Optional) Create OS volume from image

We could launch a new instance from image but we might encounter some issue if the image size is big.

os_image_id=<copy from previous output>
volume_size=10
os_volume_name='ubuntu-from-az-vol'
openstack volume create --image ${os_image_id} --size ${volume_size} ${os_volume_name}  

Create data volume from image

If you also download data disk from Azure, and want to convert it into volume, you can follow previous step to upload it to image and create volume from the image

local_file_qcow2_data='ubuntu-from-az-data.qcow2'
os_image_name_data='ubuntu-from-az-data'
os_volume_name_data='ubuntu-from-az-data-vol'
volume_size=10       # target volume size in GB

openstack image create --file ${local_file_qcow2_data} --container-format bare --disk-format qcow2 ${os_image_name_data}
# Copy image ID from the output

openstack volume create --image <image_id_from_previous_output> --size ${volume_size} ${os_volume_name_data}

Launch instance

Option 1: from uploaded image

Once you have the image uploaded, you can then launch instance with the image, example command could looks like:

openstack server create --image ${os_image_name} --flavor m1.medium --network example-net --key-name my-keypair --security-group my-sec-group ubuntu-from-az-instance

Option 2: from created volume

When the image is big, launch directly from image may fail. Refer to Create OS volume from image to create a OS volume first then launch an instance from that volume.

openstack server create --volume ${os_volume_name} --flavor m1.medium --network example-net --key-name my-keypair --security-group my-sec-group ubuntu-from-az-instance

[diedpigs]

The explanation why we do create the volume first. While we could just launch a new instance directly from an image, we had some issue doing so. This is probably because instance creation process will wait for volume to be ready. Maybe it's the image size too big, or maybe it's because we had Ceph house keeping procedure going on during the test. The volume creation took way more time than instance creation expected and eventually timed out. So we ended up creating a volume from an image first, and then launched instance from the volume.