Add Darwin support via vfkit hypervisor

README.md

@@ -14,7 +14,7 @@
 </p>
 
 A Nix Flake to build NixOS and run it on one of several Type-2
-Hypervisors on NixOS/Linux. The project is intended to provide a more
+Hypervisors on NixOS/Linux or macOS. The project is intended to provide a more
 isolated alternative to `nixos-container`. You can either build and
 run MicroVMs like Nix packages, or alternatively install them as
 systemd services declaratively in your host's Nix Flake or
@@ -26,8 +26,8 @@ imperatively with the provided `microvm` command.
 
 - MicroVMs are Virtual Machines but use special device interfaces
   (virtio) for high performance.
-- This project runs them on NixOS hosts.
-- You can choose one of five hypervisors for each MicroVM.
+- This project runs them on NixOS/Linux hosts and macOS (via vfkit).
+- You can choose from eight hypervisors for each MicroVM.
 - MicroVMs have a fixed RAM allocation (default: 512 MB) but can be
   shrunk using `microvm-balloon`
 - MicroVMs have a read-only root disk with either a prepopulated
@@ -40,20 +40,21 @@ imperatively with the provided `microvm` command.
   a block device, or alternatively as a shared directory hierarchy
   through *9p* or *virtiofs*.
 - Zero, one, or more virtual tap ethernet network interfaces can be
-  attached to a MicroVM. `qemu` and `kvmtool` also support *user*
+  attached to a MicroVM. `qemu`, `kvmtool`, and `vfkit` also support *user*
   networking which requires no additional setup on the host.
 
 ## Hypervisors
 
-| Hypervisor                                                              | Language | Restrictions                             |
-|-------------------------------------------------------------------------|----------|------------------------------------------|
-| [qemu](https://www.qemu.org/)                                           | C        |                                          |
-| [cloud-hypervisor](https://www.cloudhypervisor.org/)                    | Rust     | no 9p shares                             |
-| [firecracker](https://firecracker-microvm.github.io/)                   | Rust     | no 9p/virtiofs shares                    |
-| [crosvm](https://chromium.googlesource.com/chromiumos/platform/crosvm/) | Rust     | 9p shares broken                         |
-| [kvmtool](https://github.com/kvmtool/kvmtool)                           | C        | no virtiofs shares, no control socket    |
-| [stratovirt](https://github.com/openeuler-mirror/stratovirt)            | Rust     | no 9p/virtiofs shares, no control socket |
-| [alioth](https://github.com/google/alioth)                              | Rust     | no virtiofs shares, no control socket    |
+| Hypervisor                                                              | Language | Restrictions                                          |
+|-------------------------------------------------------------------------|----------|-------------------------------------------------------|
+| [qemu](https://www.qemu.org/)                                           | C        |                                                       |
+| [cloud-hypervisor](https://www.cloudhypervisor.org/)                    | Rust     | no 9p shares                                          |
+| [firecracker](https://firecracker-microvm.github.io/)                   | Rust     | no 9p/virtiofs shares                                 |
+| [crosvm](https://chromium.googlesource.com/chromiumos/platform/crosvm/) | Rust     | 9p shares broken                                      |
+| [kvmtool](https://github.com/kvmtool/kvmtool)                           | C        | no virtiofs shares, no control socket                 |
+| [stratovirt](https://github.com/openeuler-mirror/stratovirt)            | Rust     | no 9p/virtiofs shares, no control socket              |
+| [alioth](https://github.com/google/alioth)                              | Rust     | no virtiofs shares, no control socket                 |
+| [vfkit](https://github.com/crc-org/vfkit)                               | Go       | macOS only, no 9p shares, no tap/bridge networking    |
 
 
 ## Installation
@@ -85,6 +86,9 @@ nix run microvm#cloud-hypervisor-example
 nix run microvm#crosvm-example
 nix run microvm#kvmtool-example
 nix run microvm#stratovirt-example
+
+# On macOS only:
+nix run microvm#vfkit-example
 ```
 
 ### Run a MicroVM example with nested MicroVMs on 5 different Hypervisors
@@ -96,12 +100,15 @@ nix run microvm#vm
 Check `networkctl status virbr0` for the DHCP leases of the nested
 MicroVMs. They listen for ssh with an empty root password.
 
-### Experimental: run graphical applications in cloud-hypervisor with Wayland forwarding
+### Experimental: run graphical applications with graphics support
 
+On Linux with cloud-hypervisor and Wayland forwarding:
 ```shell
 nix run microvm#graphics neverball
 ```
 
+On macOS with vfkit, enable graphics with `microvm.graphics.enable = true`.
+
 ## Commercial support
 
 Accelerate your operations and secure your infrastructure with support from a

doc/src/SUMMARY.md

@@ -8,6 +8,7 @@
   - [Device pass-through](./devices.md)
   - [CPU emulation](./cpu-emulation.md)
   - [Output options](./output-options.md)
+  - [Using Rosetta with vfkit (macOS)](./vfkit-rosetta.md)
   - [MicroVM options reference ⚙️](./microvm-options.md)
 - [Running a MicroVM as a package](./packages.md)
 - [Preparing a host for declarative MicroVMs](./host.md)

doc/src/interfaces.md

@@ -20,12 +20,15 @@ configuration:
 
 ## `type = "user"`
 
-User-mode networking is only provided by qemu and kvmtool, providing
+User-mode networking is provided by qemu, kvmtool, and vfkit, providing
 outgoing connectivity to your MicroVM without any further setup.
 
 As kvmtool seems to lack a built-in DHCP server, additional static IP
 configuration is necessary inside the MicroVM.
 
+**Note:** vfkit (macOS) only supports user-mode networking. TAP and bridge
+networking are not available.
+
 ## `type = "tap"`
 
 Use a virtual tuntap Ethernet interface. Its name is the value of

doc/src/intro.md

@@ -1,9 +1,9 @@
 # Intro
 
 **microvm.nix** is a Flake to run lightweight NixOS virtual machines
-on NixOS. Starting with the reasons why for the remainder of this
+on NixOS and macOS. Starting with the reasons why for the remainder of this
 chapter, this handbook guides you through the provisioning of MicroVMs
-on your NixOS machine.
+on your NixOS or macOS machine.
 
 ## Compartmentalization
 
@@ -47,4 +47,5 @@ overhead has been reduced a lot by replacing emulated devices with
 This Flake offers you to run your MicroVMs not only on QEMU but with
 other Hypervisors that have been explicitly authored for
 *virtio*. Some of them are written in Rust, a programming language
-that is renowned for being safer than C.
+that is renowned for being safer than C. On macOS, vfkit leverages
+Apple's native Virtualization.framework for running Linux VMs.

doc/src/options.md

@@ -15,6 +15,9 @@ available for customization. These are the most important ones:
 | `microvm.socket`               | Control socket for the Hypervisor so that a MicroVM can be shutdown cleanly                         |
 | `microvm.user`                 | (qemu only) User account which Qemu will switch to when started as root                             |
 | `microvm.forwardPorts`         | (qemu user-networking only) TCP/UDP port forwarding                                                 |
+| `microvm.vfkit.extraArgs`      | (vfkit only) Extra arguments to pass to vfkit                                                       |
+| `microvm.vfkit.logLevel`       | (vfkit only) Log level: "debug", "info", or "error" (default: "info")                              |
+| `microvm.vfkit.rosetta.enable` | (vfkit only) Enable Rosetta for running x86_64 binaries on ARM64 (Apple Silicon only)              |
 | `microvm.kernelParams`         | Like `boot.kernelParams` but will not end up in `system.build.toplevel`, saving you rebuilds        |
 | `microvm.storeOnDisk`          | Enables the store on the boot squashfs even in the presence of a share with the host's `/nix/store` |
 | `microvm.writableStoreOverlay` | Optional string of the path where all writes to `/nix/store` should go to.                          |

doc/src/shares.md

@@ -10,7 +10,7 @@ In `microvm.shares` elements the `proto` field allows either of two
 values:
 
 - `9p` (default) is built into many hypervisors, allowing you to
-  quickly share a directory tree
+  quickly share a directory tree. Not supported by vfkit on macOS.
 
 - `virtiofs` requires a separate virtiofsd service which is started as
   a prerequisite when you start MicroVMs through a systemd service
@@ -21,6 +21,9 @@ values:
 
   Expect `virtiofs` to yield better performance over `9p`.
 
+  **Note:** vfkit (macOS) has built-in virtiofs support and does not
+  require a separate virtiofsd service.
+
 ```nix
 microvm.shares = [ {
   proto = "virtiofs";

doc/src/vfkit-rosetta.md

@@ -0,0 +1,81 @@
+# Using Rosetta with vfkit on Apple Silicon
+
+Rosetta support enables running x86_64 (Intel) binaries in your ARM64 Linux VM on Apple Silicon Macs. This is useful for running legacy applications or development tools that haven't been ported to ARM yet.
+
+## Requirements
+
+- Apple Silicon (M1/M2/M3/etc.) Mac
+- macOS with Rosetta installed
+- vfkit hypervisor
+
+## Configuration
+
+Enable Rosetta in your MicroVM configuration:
+
+```nix
+{
+  microvm = {
+    hypervisor = "vfkit";
+
+    vfkit.rosetta = {
+      enable = true;
+      # Optional: install Rosetta automatically if missing
+      install = true;
+    };
+  };
+}
+```
+
+## Guest Setup
+
+After enabling Rosetta, you need to mount the share and configure binfmt in your guest:
+
+```nix
+{
+  # Mount the Rosetta share
+  fileSystems."/mnt/rosetta" = {
+    device = "rosetta";
+    fsType = "virtiofs";
+  };
+
+  # Configure binfmt to use Rosetta for x86_64 binaries
+  boot.binfmt.registrations.rosetta = {
+    interpreter = "/mnt/rosetta/rosetta";
+    magicOrExtension = ''\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00'';
+    mask = ''\xff\xff\xff\xff\xff\xfe\xfe\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'';
+  };
+}
+```
+
+## Testing
+
+Once configured, you can verify Rosetta is working:
+
+```bash
+# Inside the VM
+uname -m
+# Should show: aarch64
+
+# Try running an x86_64 binary (if you have one)
+file /path/to/x86_64/binary
+# Should show: ELF 64-bit LSB executable, x86-64
+
+/path/to/x86_64/binary
+# Should run successfully via Rosetta
+```
+
+## Options Reference
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `microvm.vfkit.rosetta.enable` | bool | `false` | Enable Rosetta support |
+| `microvm.vfkit.rosetta.mountTag` | string | `"rosetta"` | Mount tag for the virtiofs share |
+| `microvm.vfkit.rosetta.install` | bool | `false` | Auto-install Rosetta if missing |
+| `microvm.vfkit.rosetta.ignoreIfMissing` | bool | `false` | Continue if Rosetta unavailable |
+
+## Limitations
+
+- Only works on Apple Silicon Macs (M-series chips)
+- vfkit will fail to start on Intel Macs if Rosetta is enabled
+- Performance is slower than native ARM64 execution
+- Not all x86_64 binaries may work perfectly

flake.nix

@@ -106,18 +106,24 @@
             };
           } //
           # wrap self.nixosConfigurations in executable packages
-          builtins.foldl' (result: systemName:
-            let
-              nixos = self.nixosConfigurations.${systemName};
-              name = builtins.replaceStrings [ "${system}-" ] [ "" ] systemName;
-              inherit (nixos.config.microvm) hypervisor;
-            in
-              if nixos.pkgs.stdenv.hostPlatform.system == nixpkgs.lib.replaceString "-darwin" "-linux" system
-              then result // {
-                "${name}" = nixos.config.microvm.runner.${hypervisor};
-              }
-              else result
-          ) {} (builtins.attrNames self.nixosConfigurations);
+          nixpkgs.lib.listToAttrs (
+            nixpkgs.lib.concatMap (configName:
+              let
+                config = self.nixosConfigurations.${configName};
+                packageName = builtins.replaceStrings [ "${system}-" ] [ "" ] configName;
+                # Check if this config's guest system matches our current build system
+                # (accounting for darwin hosts building linux guests)
+                guestSystem = config.pkgs.stdenv.hostPlatform.system;
+                buildSystem = nixpkgs.lib.replaceString "-darwin" "-linux" system;
+              in
+              if guestSystem == buildSystem
+              then [{
+                name = packageName;
+                value = config.config.microvm.runner.${config.config.microvm.hypervisor};
+              }]
+              else []
+            ) (builtins.attrNames self.nixosConfigurations)
+          );
 
         # Takes too much memory in `nix flake show`
         # checks = import ./checks { inherit self nixpkgs system; };
@@ -156,7 +162,22 @@
               # currently broken:
               # "crosvm"
             ];
-            hypervisorsWithUserNet = [ "qemu" "kvmtool" ];
+            hypervisorsWithUserNet = [ "qemu" "kvmtool" "vfkit" ];
+            hypervisorsDarwinOnly = [ "vfkit" ];
+            # Hypervisors that work on darwin (qemu via HVF, vfkit natively)
+            hypervisorsOnDarwin = [ "qemu" "vfkit" ];
+            hypervisorsWithTap = builtins.filter
+              # vfkit supports networking, but does not support tap
+              (hv: hv != "vfkit")
+              self.lib.hypervisorsWithNetwork;
+
+            isDarwinOnly = hypervisor: builtins.elem hypervisor hypervisorsDarwinOnly;
+            isDarwinSystem = system: nixpkgs.lib.hasSuffix "-darwin" system;
+            hypervisorSupportsSystem = hypervisor: system:
+              if isDarwinSystem system
+              then builtins.elem hypervisor hypervisorsOnDarwin
+              else !(isDarwinOnly hypervisor);
+
             makeExample = { system, hypervisor, config ? {} }:
               nixpkgs.lib.nixosSystem {
                 system = nixpkgs.lib.replaceString "-darwin" "-linux" system;
@@ -172,12 +193,21 @@
                     nixpkgs.overlays = [ self.overlay ];
                     microvm = {
                       inherit hypervisor;
-                      # share the host's /nix/store if the hypervisor can do 9p
-                      shares = lib.optional (builtins.elem hypervisor hypervisorsWith9p) {
-                        tag = "ro-store";
-                        source = "/nix/store";
-                        mountPoint = "/nix/.ro-store";
-                      };
+                      # share the host's /nix/store if the hypervisor supports it
+                      shares =
+                        if builtins.elem hypervisor hypervisorsWith9p then [{
+                          tag = "ro-store";
+                          source = "/nix/store";
+                          mountPoint = "/nix/.ro-store";
+                          proto = "9p";
+                        }]
+                        else if hypervisor == "vfkit" then [{
+                          tag = "ro-store";
+                          source = "/nix/store";
+                          mountPoint = "/nix/.ro-store";
+                          proto = "virtiofs";
+                        }]
+                        else [];
                       # writableStoreOverlay = "/nix/.rw-store";
                       # volumes = [ {
                       #   image = "nix-store-overlay.img";
@@ -206,22 +236,28 @@
                   config
                 ];
               };
-          in
-            (builtins.foldl' (results: system:
-              builtins.foldl' ({ result, n }: hypervisor: {
-                result = result // {
-                  "${system}-${hypervisor}-example" = makeExample {
-                    inherit system hypervisor;
-                  };
-                } //
-                nixpkgs.lib.optionalAttrs (builtins.elem hypervisor self.lib.hypervisorsWithNetwork) {
-                  "${system}-${hypervisor}-example-with-tap" = makeExample {
+
+            basicExamples = nixpkgs.lib.flatten (
+              builtins.map (system:
+                builtins.map (hypervisor: {
+                  name = "${system}-${hypervisor}-example";
+                  value = makeExample { inherit system hypervisor; };
+                  shouldInclude = hypervisorSupportsSystem hypervisor system;
+                }) self.lib.hypervisors
+              ) systems
+            );
+
+            tapExamples = nixpkgs.lib.flatten (
+              builtins.map (system:
+                nixpkgs.lib.imap1 (idx: hypervisor: {
+                  name = "${system}-${hypervisor}-example-with-tap";
+                  value = makeExample {
                     inherit system hypervisor;
                     config = _: {
                       microvm.interfaces = [ {
                         type = "tap";
                         id = "vm-${builtins.substring 0 4 hypervisor}";
-                        mac = "02:00:00:01:01:0${toString n}";
+                        mac = "02:00:00:01:01:0${toString idx}";
                       } ];
                       networking = {
                         interfaces.eth0.useDHCP = true;
@@ -233,9 +269,14 @@
                       };
                     };
                   };
-                };
-                n = n + 1;
-              }) results self.lib.hypervisors
-            ) { result = {}; n = 1; } systems).result;
+                  shouldInclude = builtins.elem hypervisor hypervisorsWithTap
+                    && hypervisorSupportsSystem hypervisor system;
+                }) self.lib.hypervisors
+              ) systems
+            );
+
+            included = builtins.filter (ex: ex.shouldInclude) (basicExamples ++ tapExamples);
+          in
+            builtins.listToAttrs (builtins.map ({ name, value, ... }: { inherit name value; }) included);
       };
 }

lib/default.nix

@@ -8,6 +8,7 @@ rec {
     "kvmtool"
     "stratovirt"
     "alioth"
+    "vfkit"
   ];
 
   hypervisorsWithNetwork = hypervisors;