I am trying to boot a VM with fuse-overlayfs as the root filesystem using my custom initramfs scripts here. The scripts currently only work on Debian with the default initramfs generator initramfs-tools, only tested on Debian 13, not yet on its derivatives. I can confirm kernel overlayfs works just fine, however when I switched to fuse-overlayfs I kept getting ENOENT for the init. I have tried init=/bin/bash but still no luck. I have no problem accessing the mount, creating files and chroot-ing on initramfs shell.
# BOOT_IMAGE=/boot/vmlinuz-6.12.86+deb13-cloud-amd64 root=/dev/vda ro overlayroot=/dev/vdb fuseoverlayroot debugoverlayroot console=ttyS0,115200 initrd=/boot/initrd.img-6.12.86+deb13-cloud-amd64
... snipped ...
[ 1.270625] Run /init as init process
Loading, please wait...
Starting systemd-udevd version 257.13-1~deb13u1
[ 1.417765] virtio_blk virtio3: 2/0/0 default/read/poll queues
[ 1.428867] virtio_blk virtio3: [vda] 33554432 512-byte logical blocks (17.2 GB/16.0 GiB)
[ 1.432814] SCSI subsystem initialized
[ 1.435837] virtio_blk virtio4: 2/0/0 default/read/poll queues
[ 1.438273] virtio_blk virtio4: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)
[ 1.444072] virtio_net virtio5 ens18: renamed from eth0
[ 1.466903] scsi host0: ata_piix
[ 1.467892] scsi host1: ata_piix
[ 1.468479] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xf1e0 irq 14 lpm-pol 0
[ 1.469743] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xf1e8 irq 15 lpm-pol 0
[ 1.627761] ata2: found unknown device (class 0)
[ 1.629508] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
[ 1.632101] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5
[ 1.683357] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
[ 1.684548] cdrom: Uniform CD-ROM driver Revision: 3.20
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
Warning: fsck not present, so skipping root file system
[ 1.868889] EXT4-fs (vda): orphan cleanup on readonly fs
[ 1.869899] EXT4-fs (vda): mounted filesystem 3147ef3f-10f9-4e61-8b1f-4b5619ce2b9a ro with ordered data mode. Quota mode: none.
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... Begin: persistent-overlayroot: Setting up overlayfs as rootfs ... [ 1.929557] EXT4-fs (vdb): recovery complete
[ 1.930295] EXT4-fs (vdb): mounted filesystem bdbc7740-4a4d-4eb6-80e6-0d57f15772ba r/w with ordered data mode. Quota mode: none.
Begin: persistent-overlayroot: fuseoverlayroot was set, using fuse-overlayfs instead of kernel overlay ... unknown argument ignored: lazytime
Failure: persistent-overlayroot: Debugging was enabled, initiating crash...
persistent-overlayroot failed! Dropping to a shell...
BusyBox v1.37.0 (Debian 1:1.37.0-6+b8) built-in shell (ash)
Enter 'help' for a list of built-in commands.
(initramfs) mount
rootfs on / type rootfs (rw)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=1995888k,nr_inodes=498972,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=600,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=401656k,mode=755,inode64)
/dev/vda on /root/overlay/lower type ext4 (ro,relatime)
/dev/vdb on /root/overlay type ext4 (rw,relatime)
@fuse-overlayfs on /root type fuse.@fuse-overlayfs (rw,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other)
(initramfs) ^D
Success: persistent-overlayroot: Configured overlayfs as rootfs with /dev/vdb on top of /dev/vda with fuse-overlayfs
done.
[ 2.470876] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.
run-init: can't execute '/sbin/init': No such file or directory
[ 2.473923] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100
[ 2.475289] CPU: 1 UID: 0 PID: 1 Comm: run-init Not tainted 6.12.86+deb13-cloud-amd64 #1 Debian 6.12.86-1
[ 2.476939] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014
[ 2.478747] Call Trace:
[ 2.479160] <TASK>
[ 2.479525] dump_stack_lvl+0x5d/0x80
[ 2.480133] panic+0x118/0x2db
[ 2.480646] do_exit.cold+0x15/0x15
[ 2.481273] do_group_exit+0x30/0x80
[ 2.481866] __x64_sys_exit_group+0x18/0x20
[ 2.482546] x64_sys_call+0x14b4/0x14c0
[ 2.483177] do_syscall_64+0x87/0x180
[ 2.483808] ? common_interrupt+0x13/0xa0
[ 2.484462] ? kvm_clock_get_cycles+0x18/0x30
[ 2.485177] ? ktime_get_mono_fast_ns+0x37/0xb0
[ 2.485917] ? rpm_suspend+0x202/0x5f0
[ 2.486560] ? rpm_suspend+0x17f/0x5f0
[ 2.487181] ? _raw_spin_unlock_irqrestore+0xe/0x40
[ 2.488022] ? __pm_runtime_suspend+0x47/0xc0
[ 2.488738] ? remove_wait_queue+0x24/0x60
[ 2.489405] ? _raw_spin_unlock_irqrestore+0xe/0x40
[ 2.490195] ? n_tty_write+0x3a5/0x500
[ 2.490815] ? _raw_spin_unlock_irqrestore+0xe/0x40
[ 2.491601] ? __wake_up+0x44/0x60
[ 2.492168] ? file_tty_write.isra.0+0x20c/0x2c0
[ 2.492922] ? vfs_write+0x28c/0x440
[ 2.493510] ? arch_exit_to_user_mode_prepare.isra.0+0x16/0xa0
[ 2.494452] ? syscall_exit_to_user_mode+0x37/0x1b0
[ 2.495243] ? do_syscall_64+0x93/0x180
[ 2.495874] ? handle_mm_fault+0x1bb/0x2c0
[ 2.496537] ? do_user_addr_fault+0x36c/0x620
[ 2.497251] ? arch_exit_to_user_mode_prepare.isra.0+0x16/0xa0
[ 2.498193] entry_SYSCALL_64_after_hwframe+0x76/0x7e
[ 2.499012] RIP: 0033:0x7f4c508102d5
[ 2.499641] Code: 41 7b 10 00 f7 d8 bd ff ff ff ff 64 89 02 eb c6 e8 20 f9 03 00 48 8b 35 29 7b 10 00 ba e7 00 00 00 eb 03 66 90 f4 89 d0 0f 05 <48> 3d 00 f0 ff 0
[ 2.502579] RSP: 002b:00007ffe75a66868 EFLAGS: 00000202 ORIG_RAX: 00000000000000e7
[ 2.503793] RAX: ffffffffffffffda RBX: 00007f4c50919fe8 RCX: 00007f4c508102d5
[ 2.504936] RDX: 00000000000000e7 RSI: ffffffffffffffa0 RDI: 0000000000000001
[ 2.506078] RBP: 0000000000000001 R08: 00007ffe75a667f8 R09: 0000000000000000
[ 2.507220] R10: 00007ffe75a666f0 R11: 0000000000000202 R12: 00007f4c50918680
[ 2.508362] R13: 00007f4c50947aa0 R14: 0000000000000001 R15: 00007f4c5091a000
[ 2.509506] </TASK>
[ 2.509991] Kernel Offset: 0x1fe00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 2.511759] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100 ]---
The @ on fuse-overlayfs was done by altering argv[0] to prevent systemd from killing fuse-overlayfs, provided initramfs successfully executed the init. The reason I chose fuse-overlayfs is because I am planning to use virtiofs as the upper mount due to the host storage backed up by rsnapshot that runs every hour (writing to the disk image would quickly fill up the backup disk) and kernel overlayfs won't accept virtiofs as the upper. Tried with virtiofs as well and got the same results.
I am trying to boot a VM with fuse-overlayfs as the root filesystem using my custom initramfs scripts here. The scripts currently only work on Debian with the default initramfs generator initramfs-tools, only tested on Debian 13, not yet on its derivatives. I can confirm kernel overlayfs works just fine, however when I switched to fuse-overlayfs I kept getting ENOENT for the init. I have tried
init=/bin/bashbut still no luck. I have no problem accessing the mount, creating files and chroot-ing on initramfs shell.The @ on fuse-overlayfs was done by altering argv[0] to prevent systemd from killing fuse-overlayfs, provided initramfs successfully executed the init. The reason I chose fuse-overlayfs is because I am planning to use virtiofs as the upper mount due to the host storage backed up by rsnapshot that runs every hour (writing to the disk image would quickly fill up the backup disk) and kernel overlayfs won't accept virtiofs as the upper. Tried with virtiofs as well and got the same results.