Build: Add `convert-to-ami` and Multi container image build by charlie3965 · Pull Request #4 · yureutaejin/yob

charlie3965 · 2025-10-08T02:58:41Z

Changes

Add convert-to-ami sub-command to Makefile
- Now we supports AMI disk format
Fix stages of Dockerfile
- Build {core, desktop} image (Just like other OS's distros)
Add docker-bake.hcl
- To manage declarative build configuration
- Simplify multi-image build
- Build consistency and sharing
- Parallel builds
- (Future) To build mutl-platform container image
Change default SELinux status from disabled to permissive
- Build: Add convert-to-ami and Multi container image build #4 (comment)
Made convert-to-{iso,ami} use DinD instead of requiring podman/buildah pre-installation
- No need to pre install podman/buildah anymore
- Build: Add convert-to-ami and Multi container image build #4 (comment)

Tests

build-bootc : https://github.com/yureutaejin/yob/actions/runs/18772878886
convert-to-ami : https://github.com/yureutaejin/yob/actions/runs/18771909470
convert-to-disk : https://github.com/yureutaejin/yob/actions/runs/18772939553

(Misc) Prerequisites for AMI build

Refer to Cloud uploaders section in BIB

Create S3 bucket
Check AWS configuration and where the config file is
Made role vmimport
- Build: Add convert-to-ami and Multi container image build #4 (comment)

charlie3965 · 2025-10-08T06:05:03Z

The way of creating vmimport using AWS cli is as below.

# trust-policy.json
{
   "Version":"2012-10-17",		 	 	 
   "Statement": [
      {
         "Effect": "Allow",
         "Principal": { "Service": "vmie.amazonaws.com" },
         "Action": "sts:AssumeRole",
         "Condition": {
            "StringEquals":{
               "sts:Externalid": "vmimport"
            }
         }
      }
   ]
}

aws iam create-role --role-name vmimport --assume-role-policy-document "file://{YOUR_DIRECTORY}/trust-policy.json"

# role-policy.json
{
   "Version":"2012-10-17",		 	 	 
   "Statement":[
      {
         "Effect": "Allow",
         "Action": [
            "s3:GetBucketLocation",
            "s3:GetObject",
            "s3:ListBucket",
            "s3:PutObject",
            "s3:GetBucketAcl",
            "s3:ListAllMyBuckets",
            "s3:DeleteObject"
         ],
         "Resource": [
            "arn:aws:s3:::{YOUR_BUCKET}",
            "arn:aws:s3:::{YOUR_BUCKET}/*"
         ]
      },
      {
         "Effect": "Allow",
         "Action": [
            "ec2:ModifySnapshotAttribute",
            "ec2:CopySnapshot",
            "ec2:RegisterImage",
            "ec2:Describe*"
         ],
         "Resource": "*"
      }
   ]
}

aws iam put-role-policy --role-name vmimport --policy-name vmimport --policy-document "file://{YOUR_DIRECTORY}/role-policy.json"

All process must be done after
1. You have made your own S3 bucket to upload ami
2. You have installed and inserted access/secret keys to your aws cli
Theoratically, this process can be done on GUI console, but I recommend using aws cli - my attempt using GUI console have occurred bunch of permission errors

yureutaejin

@charlie3965

I added some suggestions and left a few self-reviews.
I just defined belows with dummy values in repository for future ami workflow test
- secrets.AWS_SECRET_ACCESS_KEY
- vars.AWS_ACCESS_KEY_ID
- vars.AWS_REGION

Request

Could you test convert-to-ami with this Makefile in your local environment, and report the result?
Please fix the README.md with your ami set up. (No need to write too much detail)

yureutaejin · 2025-10-09T10:39:11Z

Makefile

+AWS_AMI_NAME ?= immutable-os-bootc-$(GIT_COMMIT_HASH:0:8)
+AWS_S3_BUCKET ?= immutable-os-bootc
+AWS_REGION ?= us-east-1


Add dummy values

yureutaejin · 2025-10-09T10:42:37Z

Makefile

+.PHONY: convert-to-ami
+convert-to-ami:
+	sudo podman load -i image-${GIT_COMMIT_HASH:0:8}.tar
+	sudo docker run --rm \
+	--privileged \
+	--security-opt label=type:unconfined_t \
+	-v /var/lib/containers/storage:/var/lib/containers/storage \
+	--env AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} \
+	--env AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} \
+	${BIB_CONTAINER} \
+	--type ${DISK_FORMAT} \
+	--rootfs ${ROOTFS} \
+	--aws-ami-name ${AWS_AMI_NAME} \
+	--aws-bucket ${AWS_S3_BUCKET} \
+	--aws-region ${AWS_REGION} \
+	${OCI_REGISTRY}/${OCI_IMAGE_REPO}:${OCI_IMAGE_TAG}


@charlie3965
(Please make sure to write a code carefully considering how it works.)

I just removed lines regarding config.toml file
According to official README.md, config.toml is just for anaconda installer

yureutaejin

Self reviews

yureutaejin · 2025-10-17T12:22:23Z

.github/workflows/_build-bootc.yaml

+      - name: Push bootc image to registry
+        if: ${{ github.event_name == 'workflow_dispatch' }}


Prevent pushing OCI registry when merge queue or status checks works

yureutaejin · 2025-10-17T12:26:03Z

docker-bake.hcl

To use multi image build easily

See also

https://docs.docker.com/build/buildkit/

https://docs.docker.com/reference/cli/docker/buildx/bake/

https://docs.docker.com/build/bake/reference/

yureutaejin · 2025-10-17T12:28:50Z

Makefile

I just made user run single make subcommand.
No need to define/install/fix something except Makefile

yureutaejin · 2025-10-17T12:30:22Z

template-ami.toml

File for {ami, qcow2,...}

yureutaejin · 2025-10-24T07:01:01Z

filesystem/etc/selinux/config

Changed SELinux default state to permissive, not disabled.

SELINUX=permissive is just for logging

osbuild and rpm-ostree have some problems in BIB process (SELinux enabled, but no policy found)

BIB project considers iso build process as legacy

It might be helpful to read belows

https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/using_selinux/changing-selinux-states-and-modes_using-selinux

Requires running under rootful podman osbuild/bootc-image-builder#98

when selinux:false, rpm-ostree still looks for selinux policies and raises error coreos/rpm-ostree#3261

yureutaejin · 2025-10-24T07:10:06Z

Makefile

+.PHONY: convert-to-iso
+convert-to-iso: bib-dind-down bib-dind-up
+	cp -rf template-iso.toml config.toml
 	sed -i "s|{DEFAULT_DISK}|${DEFAULT_DISK}|g" config.toml
-	sed -i "s|{DEFAULT_USER_NAME}|${DEFAULT_USER_NAME}|g" config.toml && \
-	sed -i "s|{DEFAULT_USER_PASSWD}|${DEFAULT_USER_PASSWD}|g" config.toml && \
-	sudo docker run --rm \
-	--privileged \
-	--security-opt label=type:unconfined_t \
-	-v ./image-builder-output:/output \
-	-v /var/lib/containers/storage:/var/lib/containers/storage \
-	-v ./config.toml:/config.toml:ro \
-	$(BIB_CONTAINER) \
-	--type $(DISK_FORMAT) \
-	--use-librepo=True \
-	--rootfs $(ROOTFS) \
-	$(OCI_REGISTRY)/$(OCI_IMAGE_REPO):${OCI_IMAGE_TAG}
+	sed -i "s|{DEFAULT_USER_NAME}|${DEFAULT_USER_NAME}|g" config.toml
+	sed -i "s|{DEFAULT_USER_PASSWD}|${DEFAULT_USER_PASSWD}|g" config.toml
+	docker cp config.toml bib-dind:/config.toml
+	[[ "${TARGET_INTERFACE}" == "all" ]] && TARGETS="core desktop" || TARGETS="${TARGET_INTERFACE}"; \
+	for target in $${TARGETS}; do \
+		mkdir -p image-builder-output/$${target}; \
+		docker exec bib-dind /bin/bash -c " \
+			podman load -i container-tarbells/image-${SHORT_COMMIT_HASH}-$${target}.tar.gz; \
+			podman run --rm \
+				--privileged \
+				--security-opt label=type:unconfined_t \
+				-v ./image-builder-output/$${target}:/output \
+				-v /var/lib/containers/storage:/var/lib/containers/storage \
+				-v ./config.toml:/config.toml:ro \
+				${BIB_CONTAINER} \
+				--type iso \
+				--use-librepo=True \
+				--rootfs ${ROOTFS} \
+			${OCI_REGISTRY}/${OCI_IMAGE_REPO}:${OCI_IMAGE_TAG}-$${target}"; \
+	done
+	$(MAKE) bib-dind-down
+
+# See https://github.com/osbuild/bootc-image-builder?tab=readme-ov-file#amazon-machine-images-amis
+.PHONY: convert-to-ami
+convert-to-ami: bib-dind-down bib-dind-up
+	cp -rf template-ami.toml config.toml
+	sed -i "s|{DEFAULT_USER_NAME}|${DEFAULT_USER_NAME}|g" config.toml
+	sed -i "s|{DEFAULT_USER_PASSWD}|${DEFAULT_USER_PASSWD}|g" config.toml
+	docker cp config.toml bib-dind:/config.toml
+	AWS_AMI_NAME=${SHORT_COMMIT_HASH}-core; \
+	docker exec bib-dind /bin/bash -c " \
+		podman load -i container-tarbells/image-${SHORT_COMMIT_HASH}-core.tar.gz; \
+		podman run --rm \
+			--privileged \
+			--security-opt label=type:unconfined_t \
+			-v /var/lib/containers/storage:/var/lib/containers/storage \
+			-v ./config.toml:/config.toml:ro \
+			--env AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} \
+			--env AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} \
+			${BIB_CONTAINER} \
+			--type ami \
+			--rootfs ${ROOTFS} \
+			--aws-ami-name $${AWS_AMI_NAME} \
+			--aws-bucket ${AWS_S3_BUCKET} \
+			--aws-region ${AWS_REGION} \
+		${OCI_REGISTRY}/${OCI_IMAGE_REPO}:${OCI_IMAGE_TAG}-core"
+	$(MAKE) bib-dind-down
+
+.PHONY: bib-dind-up
+bib-dind-up:
+	docker run \
+		-itd \
+		--privileged \
+		--name bib-dind \
+		-v ./container-tarbells:/container-tarbells \
+		-v ./image-builder-output:/image-builder-output \
+	quay.io/containers/podman:latest
+
+.PHONY: bib-dind-down
+bib-dind-down:
+	docker rm -f bib-dind || true


BIB requires /var/lib/containers which podman, buildah usually uses.

So I resolved the issues using quay.io/containers/podman container with DinD.

Now we don't need to require user to install podman/buildah for prerequisites

Additional Info

You might want to read additional comments about SELinux errors

Changed SELinux default state to permissive, not disabled.

SELINUX=permissive is just for logging

osbuild and rpm-ostree have some problems in BIB process (SELinux enabled, but no policy found)

BIB project considers iso build process as legacy

It might be helpful to read belows

https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/using_selinux/changing-selinux-states-and-modes_using-selinux

Requires running under rootful podman osbuild/bootc-image-builder#98

when selinux:false, rpm-ostree still looks for selinux policies and raises error coreos/rpm-ostree#3261

yureutaejin

Final self review

yureutaejin · 2025-10-24T07:21:49Z

.github/workflows/_convert-to-disk.yaml

      - name: Convert OCI image into specified disk format
        run: |
          echo "::group::Convert OCI image into specified disk format:"
-          make convert-to-disk-image
+          make convert-to-${DISK_FORMAT}
          echo "::endgroup::"
        env:
          OCI_REGISTRY: ${{ inputs.oci_registry }}
          OCI_IMAGE_REPO: ${{ inputs.oci_image_repo }}
          OCI_IMAGE_TAG: ${{ inputs.oci_image_tag }}
+          TARGET_INTERFACE: ${{ inputs.target-interface }}
          DISK_FORMAT: ${{ inputs.disk_format }}
          DEFAULT_DISK: ${{ inputs.default_disk }}
-          DEFAULT_USER_NAME: ${{ secrets.DEFAULT_USER_NAME }}
-          DEFAULT_USER_PASSWD: ${{ secrets.DEFAULT_USER_PASSWD }}
          ROOTFS: ${{ inputs.rootfs }}
+          AWS_S3_BUCKET: ${{ inputs.aws_s3_bucket }}
+          AWS_REGION: ${{ vars.AWS_REGION }}
+          AWS_ACCESS_KEY_ID: ${{ vars.AWS_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}


GitHub Actions masked AWS Credential info
-https://github.com/yureutaejin/yob/actions/runs/18771909470/job/53557974734#step:8:95

The merge-base changed after approval.

github-actions · 2025-10-24T08:30:02Z

Dockerfile Lint Result

commit hash : 2bc6267

[
  {
    "code": "DL3041",
    "column": 1,
    "file": "./Dockerfile",
    "level": "info",
    "line": 7,
    "message": "Specify version with `dnf install -y <package>-<version>`."
  },
  {
    "code": "DL3059",
    "column": 1,
    "file": "./Dockerfile",
    "level": "info",
    "line": 35,
    "message": "Multiple consecutive `RUN` instructions. Consider consolidation."
  },
  {
    "code": "DL3040",
    "column": 1,
    "file": "./Dockerfile",
    "level": "info",
    "line": 41,
    "message": "`dnf clean all` missing after dnf command."
  },
  {
    "code": "DL3041",
    "column": 1,
    "file": "./Dockerfile",
    "level": "info",
    "line": 46,
    "message": "Specify version with `dnf install -y <package>-<version>`."
  },
  {
    "code": "DL3041",
    "column": 1,
    "file": "./Dockerfile",
    "level": "info",
    "line": 101,
    "message": "Specify version with `dnf install -y <package>-<version>`."
  },
  {
    "code": "DL3059",
    "column": 1,
    "file": "./Dockerfile",
    "level": "info",
    "line": 111,
    "message": "Multiple consecutive `RUN` instructions. Consider consolidation."
  }
]

yureutaejin changed the title ~~Dev: Adding convert-to-ami in Makefile~~ Build: Add convert-to-ami command in Makefile Oct 8, 2025

yureutaejin assigned charlie3965 Oct 8, 2025

yureutaejin self-requested a review October 8, 2025 09:28

charlie3965 added 2 commits October 9, 2025 17:17

add convert-to-ami in Makefile

902625f

solved missing seperator issue

6b2bd32

yureutaejin force-pushed the build/convert-to-ami branch from f76aa38 to 6b2bd32 Compare October 9, 2025 08:17

Add PR review suggestions

bc3b834

yureutaejin force-pushed the build/convert-to-ami branch from 143d77d to bc3b834 Compare October 9, 2025 10:11

Add condition to prevent OCI registry pushing

fa5d890

yureutaejin marked this pull request as draft October 9, 2025 10:29

yureutaejin requested changes Oct 9, 2025

View reviewed changes

yureutaejin added 2 commits October 11, 2025 09:48

Migrate repository

14b1306

update repo info

cbc77be

yureutaejin force-pushed the build/convert-to-ami branch from 9192b27 to e57b180 Compare October 13, 2025 07:46

Update phony command name

a91d293

yureutaejin force-pushed the build/convert-to-ami branch from e57b180 to a91d293 Compare October 13, 2025 07:54

yureutaejin added 8 commits October 13, 2025 19:55

Add dependency for convert-to-{DISK_FORMAT}

fb1edb6

Fix pipeline diagram

dc8f185

Disable firewalld & selinux

f2d7b50

Divide last stage into core and desktop to provide separated container

283c1c7

Create bake file to build multi target

6060834

Fix Makefile to use bake

756906d

Separate toml for different disk format

1bc1b31

Update workflows

75121c1

yureutaejin force-pushed the build/convert-to-ami branch 5 times, most recently from 2506e3c to bb29364 Compare October 16, 2025 08:35

yureutaejin added 2 commits October 21, 2025 19:25

Test convert-to-ami configured with DinD

b7e9273

Fix convert-to-ami to use DinD

2135799

yureutaejin force-pushed the build/convert-to-ami branch from 725ad91 to 2135799 Compare October 21, 2025 10:25

Remove Podman in prerequisite

3c9bb37

yureutaejin force-pushed the build/convert-to-ami branch from 360d554 to 1b898d5 Compare October 22, 2025 10:01

Set SELinux to permissive by default

dd298b1

yureutaejin force-pushed the build/convert-to-ami branch from 1b898d5 to dd298b1 Compare October 22, 2025 10:02

yureutaejin added 4 commits October 23, 2025 10:26

Fix issue that DinD container is not removed after run

bb6d9da

Refactor few things

19145be

Add step summary for image path

9715928

minor fix in README

3388235

yureutaejin approved these changes Oct 24, 2025

View reviewed changes

yureutaejin previously approved these changes Oct 24, 2025

View reviewed changes

Merge branch 'main' into build/convert-to-ami

13c0188

yureutaejin assigned yureutaejin and charlie3965 and unassigned charlie3965 Oct 24, 2025

remove unexpected newline

2bc6267

Repository owner deleted a comment from github-actions bot Oct 24, 2025

yureutaejin marked this pull request as ready for review October 24, 2025 08:26

yureutaejin marked this pull request as draft October 24, 2025 08:29

yureutaejin marked this pull request as ready for review October 24, 2025 08:29

Repository owner deleted a comment from github-actions bot Oct 24, 2025

yureutaejin merged commit 55d6113 into main Oct 24, 2025
4 checks passed

yureutaejin deleted the build/convert-to-ami branch October 24, 2025 08:32

		- name: Push bootc image to registry
		if: ${{ github.event_name == 'workflow_dispatch' }}

Conversation

charlie3965 commented Oct 8, 2025 • edited by yureutaejin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Tests

(Misc) Prerequisites for AMI build

Uh oh!

charlie3965 commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yureutaejin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Request

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yureutaejin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Additional Info

Uh oh!

yureutaejin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 24, 2025

Dockerfile Lint Result

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

charlie3965 commented Oct 8, 2025 •

edited by yureutaejin

Loading

charlie3965 commented Oct 8, 2025 •

edited

Loading

yureutaejin left a comment •

edited

Loading

yureutaejin left a comment •

edited

Loading