Feature request: tryboot support in pi-gen

[murrellr]

We are using pi-gen to build a customized version of the Raspberry Pi OS Lite (removing some packages, adding others). We want to support the the tryboot_a_b option. This requires me to modify pi-gen to create 5 partitions: two boot (Fat32), two rootfs (ext4), and a data partition. I would need to be able to declare the sizes of the boot and rootfs partitions, leaving the remainder for the data partition. I have already modified the process to take the desired rootfs partition size from the config file, but I don't like doing this. It makes it difficult to upgrade pi-gen, because I have to make sure that my changes have been properly merged into the update.

[XECDesign]

That sounds good. Maybe using the same approach as buildroot uses (mtools and genimage) might simplify export-image and make it easy for the user to provide their own genimage.conf file.

Is your current work to support tryboot_a_b public anywhere?

[murrellr]

No. We are still early in development. Here is what I have changed to set the rootfs size:

export-image/prerun.sh

  BOOT_SIZE="$((256 * 1024 * 1024))"
  if [ "$ROOT_SIZE" = "" ]
  then
    ROOT_SIZE=$(du --apparent-size -s "${EXPORT_ROOTFS_DIR}" --exclude var/cache/apt/archives --exclude boot --block-size=1 | cut -f 1)
    # Add this much space to the calculated file size. This allows for
    # some overhead (since actual space usage is usually rounded up to the
    # filesystem block size) and gives some free space on the resulting
    # image.
    ROOT_MARGIN="$(echo "($ROOT_SIZE * 0.2 + 200 * 1024 * 1024) / 1" | bc)"
  else
    ROOT_MARGIN=0
  fi

I set $ROOT_SIZE in the config file. If it doesn't exist, I fall back to the original setting.

Also, along with creating the whole eMMC image, I extract the rootfs image that can be loaded live:

export-image/05-finalize/01-run.sh:

  make_bootable_image "${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.qcow2" "$IMG_FILE"
fi
 
echo "Creating root image file os_$(basename ${IMG_FILE})"
echo "Creating loop device..."
cnt=0
until LOOP_DEV="$(losetup --show --find --partscan "$IMG_FILE")"; do
  if [ $cnt -lt 5 ]; then
    cnt=$((cnt + 1))
    echo "Error in losetup.  Retrying..."
    sleep 5
  else
    echo "ERROR: losetup failed; exiting"
    exit 1
  fi
done
zerofree -v -f 255 "${LOOP_DEV}p2"
dd if="${LOOP_DEV}p2" of="${DEPLOY_DIR}/os_$(basename ${IMG_FILE})" conv=noerror status=progress
losetup -d "${LOOP_DEV}"
echo "Created root image file os_$(basename ${IMG_FILE})"

case "${DEPLOY_COMPRESSION}" in

To keep the load image small, it only includes partition p1 (fat32 boot) and p2 (rootfs ext4). I have removed the call to "resize2fs" in "stage2/01-sys-tweaks/files/resize2fs_once". Instead, on first run, I create a blank p3 the same size as p2 and format it ext4. Then I create a data drive with the remainder p4 as ext4, and mount that off of root. This supports the old tryboot.txt mechanism. To support tryboot_a_b, I would have to create a second boot partition p3 (fat32) and rootfs partition p4 (ext4), with the remainder for data, p5 (ext4). Note that this is 5 partitions, so the eMMC would have to be GPT instead of MBR partitioning.

[murrellr]

There is a few steps I added to pi-gen to help with implementing tryboot_a_b. You may want to consider these.

export-image/05-finalise/01-run.sh:

• Mount the image to a loop device.
• Extract the boot partition using dd to a file called bootfs.
• Extract the root partition using dd to a file called rootfs.
• targz bootfs and rootfs to the file [image name].tar.gz. The significance of the names bootfs and rootfs will be explained later.

export-image/prerun.sh:
Added export commands to my rc.local using sed to record the partition sizes:

• BOOT_PART_SIZE
• ROOT_PART_START
• ROOT_PART_SIZE

My rc.local, but this could be added to the pi runonce script:

• Using BOOT_PART_SIZE, ROOT_PART_START, and ROOT_PART_SIZE, I create the following:
• /dev/mmcblk0p3 - second boot partition
• /dev/mmcblk0p4 - second root partition
• /dev/mmcblk0p5 - All remainig as data partition
• Create a symbolic link /dev/bootfs to /dev/mmcblk0p3
• Create a symbolic link /dev/rootfs to /dev/mmcblk0p4

We stream the file [image name].tar.gz through our web server. Our update manager pipes the stream to tar by calling popen("tar --overwrite –xvz -C /dev -f -"). The "-C" switch tells tar to run in the /dev directory. The tar file contains two files, bootfs and rootfs. The dev directory has the symbolic links /dev/bootfs and /dev/rootfs, which point to the secondary partitions. So tar is unzipping the files directly to the partitions. If pclose() succeeds, that tells us that the partitions are loaded and intact, since targz does integrity checks. At this point, we do the tryboot. If successful, we change the symbolic links to point to the old partitions to set up for the next update.

[timg236]

@muellR Is your update manager completely custom or based upon SWupdate/other? I'm interested in how people manage synchronizing / schedule updates on APT based systems.

[murrellr]

Our software is ported to run on several platforms with various operating systems, so our update management is custom and unique to each one. To the outside world, you HTTP POST to a special file to perform OS updates.