Add documentation for shadow ioeventfd

docs/ioregionfd.md

@@ -4,32 +4,35 @@ ioregionfd is a mechanism that speeds up ioeventfds:
 https://lore.kernel.org/kvm/[email protected]/. In the
 author's original words: "ioregionfd is a KVM dispatch mechanism which can be
 used for handling MMIO/PIO accesses over file descriptors without returning
-from ioctl(KVM_RUN).".
+from `ioctl(KVM_RUN)`.". It hasn't yet been merged upstream and it's not
+supported by libvfio-user.
 
-libvfio-user currently supports an experimental variant of this mechanism
-called shadow ioeventfd. A shadow ioeventfd is a normal ioeventfd where the
-vfio-user server passes another piece of memory (called the _shadow_ memory)
-via an additional file descriptor when configuring the ioregionfd, which then
-QEMU memory maps and passes this address to KVM. This shadow memory is never
-exposed to the guest. When the guest writes to the trapped memory, KVM writes
-the value to the shadow memory instread of discarding it, and then proceeds
-kicking the eventfd as normal.
+# shadow ioeventfd
+
+libvfio-user currently supports a similar mechanism called shadow ioeventfd.
+A shadow ioeventfd is a normal ioeventfd where the vfio-user server passes
+another piece of memory (called the _shadow_ memory) via an additional file
+descriptor when configuring the ioregionfd, which then QEMU memory maps and
+passes this address to KVM. This shadow memory is never exposed to the guest.
+When the guest writes to the trapped memory, KVM writes the value to the shadow
+memory instead of discarding it, and then proceeds kicking the eventfd as
+normal.
 
 To use shadow ioeventfd, the kernel and QEMU need to be patched. The QEMU patch
 is designed specifically for SPDK's doorbells (one ioregionfd of 4K in BAR0);
 it should be trivial to extend.
 
 The list of patches:
 * kernel: https://gist.github.com/tmakatos/532afd092a8df2175120d3dbfcd719ef
-* QEMU: https://gist.github.com/tmakatos/57755d2a37a6d53c9ff392e7c34470f6
+* QEMU: https://github.com/tmakatos/qemu/tree/shadow-ioeventfd
 * SPDK: https://gist.github.com/tmakatos/f6c10fdaff59c9d629f94bd8e44a53bc
 
 shadow ioeventfd sample
 -----------------------
 
-samples/shadow_ioeventfd_server.c implements a vfio-user server that allows a
+`samples/shadow_ioeventfd_server.c` implements a vfio-user server that allows a
 part of its BAR0 to be accessed via a shadow ioeventfd.
-shadow_ioeventfd_speed_test.c is run in the guest. It compares peformance of
+`shadow_ioeventfd_speed_test.c` is run in the guest. It compares performance of
 shadow ioeventfd vs. vfio-user messages by repeatedly writing to the part of
 the BAR0 that is handled by shadow ioeventfd and to the part not handled by
 shadow ioeventfd.
@@ -38,7 +41,7 @@ To run the sample:
 * Patch and build the kernel and QEMU using above patches.
 * Enable support for shadow ioeventfd in libvfio-user (set `shadow-ieoventfd`
   to `true` in meson_options.txt and then build libvfio-user.
-* Run samples/shadow_ioeventfd_server, e.g.
+* Run `samples/shadow_ioeventfd_server`, e.g.
   ```
   build/samples/shadow_ioeventfd_server /var/run/cntrl
   ```

docs/shadow_ioeventfd.md

@@ -0,0 +1,15 @@
+# Shadow ioeventfd
+
+Shadow ioeventfd is mechanism that reduces the cost of MMIO writes in
+vfio-user. In a nutshell, it eliminates the involvement of the VMM by allowing
+KVM to write the MMIO value in a piece of memory provided by the device
+emulation task and then notifying it by kicking the ioeventfd. The device
+emulation task can then find the value in a known location.
+
+This mechanism is especially important for cases where the MMIO value is
+required by the device protocol, which is the case for NVMe.
+
+This functionality requires patching the kernel (KVM) and QEMU:
+- kernel patches: https://github.com/tmakatos/linux/tree/shadow-ioeventfd
+- QEMU patches: https://github.com/tmakatos/qemu/tree/shadow-ioeventfd
+