Different adjustments and improvements across the board

CHANGES.rst

@@ -6,6 +6,12 @@ Racker changelog
 in progress
 ===========
 
+- Increase resources assigned to Vagrant environment.
+  Now, it gets 4 VCPUs and 4096 MB RAM.
+- Improve platform guards and naming things
+- Improve central command invocation function
+- Improve documentation
+
 
 2022-05-20 0.2.0
 ================

README.rst

@@ -40,9 +40,9 @@ Racker
     :target: https://pypi.org/project/racker/
     :alt: License
 
-.. image:: https://img.shields.io/pypi/dm/racker.svg?label=PyPI%20downloads
+.. image:: https://pepy.tech/badge/racker/month
     :target: https://pypi.org/project/racker/
-    :alt: PyPI downloads
+    :alt: PyPI downloads / month
 
 
 ----
@@ -52,44 +52,61 @@ Racker
 About
 *****
 
-An experimental harness tool based on `systemd`_ and `systemd-nspawn`_, to run
-`operating system containers <OS containers_>`_, in the spirit of addressing some details of
-`Docker Considered Harmful`_ and `Systemd vs. Docker`_.
 
-At the same time, it is a tribute to the authors and contributors of GNU, Linux,
-systemd, Python, VirtualBox, Vagrant, Docker, and more.
+At a glance
+===========
 
-Most people running Linux probably want to use `Podman`_ these days. For more
-background, enjoy reading `Container wars`_ and `Container Tools Guide`_.
+Racker is an experimental harness tool for provisioning and running operating
+system containers.
 
-Racker is ...
-=============
+For launching Linux operating systems, it is based on `systemd`_ and
+`systemd-nspawn`_, to run `operating system containers <OS containers_>`_,
+in the spirit of addressing some details of `Docker Considered Harmful`_ and
+`Systemd vs. Docker`_.
 
-- A runtime harness for testing software packages and similar purposes, in
+
+Details
+=======
+
+With Racker, you can ...
+
+- Launch an interactive command prompt within a Linux environment or invoke
+  programs non-interactively.
+
+- Use the runtime harness for testing software packages and similar purposes, in
   different environments, mostly run headless and non-interactively.
 
+Racker is ...
+
 - A lightweight wrapper around ``systemd-nspawn`` to provide and launch
   container environments for/with ``systemd``.
 
 - A lightweight wrapper around ``vagrant`` to provide convenient access to all
   things needing a full VM, like running Windows on Linux or macOS.
 
+- A tribute to the authors and contributors of GNU, Linux, systemd, Python,
+  VirtualBox, Vagrant, Docker, Windows, Windows Docker Machine and countless
+  others.
+
 
 Comparison with similar tools
 =============================
 
-The aims of Racker are very similar to `Distrobox`_ and `Toolbox`_. However,
-there are also some differences.
+The aims of Racker are very similar to `Docker`_, `Podman`_, `Distrobox`_ and
+`Toolbox`_. However, there are also some differences.
 
-- Racker is currently based on `systemd-nspawn`_ instead of `Docker`_ or
-  `Podman`_.
-- Racker can invoke any kind of container payload, but strongly focuses on
-  running `OS containers`_ aka. `OS-level virtualization`_, using `systemd`_
-  as init process.
-- Racker aims to provide concise usability by folding its usage into a single
-  command.
+Most people running Linux probably want to use `Podman`_ these days. For more
+background, enjoy reading `Container wars`_ and `Container Tools Guide`_.
+
+- Racker is currently based on `systemd-nspawn`_ and `Vagrant`_ instead of
+  `Docker`_ or `Podman`_.
+- Racker's focus is to provide easy provisioning and launching `OS containers`_
+  aka. `OS-level virtualization`_, using `systemd`_ as init process.
 - The acquisition and provisioning of operating system images does not need any
   special preparation steps, those are handled by Racker on the fly.
+- Racker aims to provide concise usability by folding its usage into a single
+  command.
+- Racker is written in Python instead of Golang or Bash.
 
 See also `Comparison with similar tools - more details`_.
 
@@ -398,6 +415,7 @@ Troubleshooting
 
 .. _autopkgtest: https://www.freedesktop.org/wiki/Software/systemd/autopkgtest/
 .. _Changing Roots: http://0pointer.de/blog/projects/changing-roots.html
+.. _Chocolatey: https://chocolatey.org/
 .. _Comparison with similar tools - more details: https://github.com/cicerops/racker/blob/main/doc/comparison.rst
 .. _Container Tools Guide: https://github.com/containers/buildah/tree/main/docs/containertools
 .. _Container wars: https://github.com/cicerops/racker/blob/main/doc/research/container-wars.rst
@@ -421,6 +439,8 @@ Troubleshooting
 .. _Systemd vs. Docker: https://lwn.net/Articles/676831/
 .. _Toolbox: https://containertoolbx.org/
 .. _umoci: https://github.com/opencontainers/umoci
+.. _Vagrant: https://www.vagrantup.com/
+.. _Windows Docker Machine: https://github.com/StefanScherer/windows-docker-machine
 
 .. _Containers without a Container Manager, with systemd: https://invidious.fdn.fr/watch?v=sqhojVPr7xM
 .. _Lennart Poettering und Kay Sievers über Systemd: https://invidious.fdn.fr/watch?v=6Q_iTG6_EF4

Vagrantfile

@@ -30,12 +30,18 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
     # Configure host specifications
     machine.vm.provider :virtualbox do |v|
 
-      v.memory = 2048
-      v.cpus = 2
+      v.memory = 4096
+      v.cpus = 4
 
       v.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
-      #v.customize ["modifyvm", :id, "--memory", 1024]
       v.customize ["modifyvm", :id, "--name", "rackerhost-debian11"]
+      #v.customize ["modifyvm", :id, "--memory", 4096]
+
+      # Turn on nested virtualization.
+      v.customize ["modifyvm", :id, "--nested-hw-virt", "on"]
+
+      # https://forums.virtualbox.org/viewtopic.php?f=1&t=59379
+      v.customize ["modifyvm", :id, "--vtxux", "on"]
     end
 
     machine.vm.provision :shell, inline: <<-SHELL

doc/backlog.rst

@@ -7,62 +7,54 @@ Racker backlog
     Those are just random notes about ideas and more.
 
 
-***********
-Iteration 2
-***********
-
-- [x] Check if software tests can be invoked on CI/GHA.
-- [x] Improve software tests.
-- [x] Split functionality between ``racker`` and ``postroj``.
-
-  - ``racker {run,ps,pull,logs}``
-  - ``postroj {list-images,pkgprobe,selftest}``
-  - ``pronto opensuse/tumbleweed hostnamectl`` (``pronto.hexagon`` (hx), ``pronto.kaxon`` (kx))
-
-- [x] Continue renaming to ``racker``.
-- [x] Improve error messages, see "Compatibility" section
-  Compare with ``docker run --rm -it debian:bullseye-slim foo``.
-
-  - ``racker run -it --rm debian-stretch hostnamectl``
-  - ``racker run -it --rm debian-stretch /bin/hostnamectl``
-  - ``racker run -it --rm debian-stretch /usr/bin/hostnamectl``
-
-- [x] Provide more advanced and generic image (label) resolution.
-  From docker.io, ghcr.io, registry.access.redhat.com, etc.
-- [x] Improve documentation
-- [x] Is it possible to run RHEL and SLES?
-
-  - registry.access.redhat.com/rhel7/rhel
-  - registry.suse.com/bci/bci-base
-  - https://registry.suse.com/
-
-
 ***********
 Iteration 3
 ***********
 
+- [o] Support for Windows. ``windows/nanoserver:1809`` and ``windows/servercore:ltsc2019``.
+- [o] Support for Windows 11 and Windows Server 2022
+
+  - https://github.com/StefanScherer/packer-windows
+  - https://app.vagrantup.com/StefanScherer/boxes/windows_2022
+  - ``mcr.microsoft.com/windows/servercore:ltsc2022-amd64``
+- [o] Provide curated image types per ``os://`` or ``vagrant://``.
 - [o] Tests: Wait for container to properly shut down before moving on.
 
   - Q: When running ``postroj selftest``, why is there more output from
     containers shutting down, while the program is finished already?
   - Q: Why are the tests failing when trying to subsequently spin up / tear down
     the same container?
+- [o] Use/integrate with ``mkosi``.
+
+  - https://github.com/systemd/mkosi
+  - http://0pointer.net/blog/mkosi-a-tool-for-generating-os-images.html
+  - https://lwn.net/Articles/726655/
+  - https://github.com/asiffer/netspot/search?q=mkosi
+
+
+*************
+Iteration 3.5
+*************
+
+- [o] Windows support leftovers
+
+  - --cpus=8 --memory=8192M
+  - https://github.com/docker/machine/issues/531#issuecomment-73938730
+  - --mount type=git,src=https://github.com/crate/crate,dst=C:/src
+  - --repository=https://github.com/crate/crate
+  - https://github.com/StefanScherer/packer-windows
+  - Make piping from STDIN possible
+  - Display machine resources (VCPUs, memory) before starting the command/job.
 
 - [o] Address issues in ``bugs.rst``
+- [o] When using ``apt-get``, use ``DEBIAN_FRONTEND=noninteractive``
 
 - [o] Invoke arbitrary Docker containers, even when they don't contain an OS root directory.
 
   - Directory ``/var/lib/postroj/archive/hello-world.img/rootfs`` doesn't look like an OS root directory (os-release file is missing). Refusing.
   - Directory ``/var/lib/postroj/archive/hello-world.img/rootfs`` doesn't look like it has an OS root directory. Refusing.
   - ``unshare --fork --pid --mount-proc --root=/var/lib/postroj/images/hello-world ./hello``
 
-- [o] Use/integrate with ``mkosi``.
-
-  - https://github.com/systemd/mkosi
-  - http://0pointer.net/blog/mkosi-a-tool-for-generating-os-images.html
-  - https://lwn.net/Articles/726655/
-  - https://github.com/asiffer/netspot/search?q=mkosi
-
 - [o] Improve timeout behaviors across the board,
   using ``with stopit.ThreadingTimeout(timeout) as to_ctx_mgr``.
 - [o] Tests: ``racker pull foo``. Also remove ``/var/lib/postroj/archive/foo`` again.
@@ -71,6 +63,9 @@ Iteration 3
 - [o] ``racker --verbose run -it --rm https://cloud-images.ubuntu.com/minimal/daily/focal/current/focal-minimal-cloudimg-amd64-root.tar.xz /bin/bash``
 - [o] TODO: Introduce appropriate exception classes.
 - [o] Maybe use ``ubi8/ubi-init`` instead of ``ubi8/ubi``?
+- [o] How to integrate with other tools from https://github.com/topics/systemd-nspawn
+- [o] https://github.com/NixOS/nixops
+
 
 
 ***********

doc/bugs.rst

@@ -26,3 +26,84 @@ See::
 stdout/stderr redirection
 =========================
 Most probably, since c5266940/a721345, the progress bar of ``skopeo`` isn't displayed anymore.
+
+
+VirtualBox on macOS with nested virtualization
+==============================================
+::
+
+    ==> 2019-box: Booting VM...
+    There was an error while executing `VBoxManage`, a CLI used by Vagrant
+    for controlling VirtualBox. The command and stderr is shown below.
+
+    Command: ["startvm", "5f422042-d72f-46c5-9bcf-ec27eeee2e06", "--type", "headless"]
+
+    Stderr: VBoxManage: error: Cannot enable nested VT-x/AMD-V without nested-paging and unrestricted guest execution!
+    VBoxManage: error:  (VERR_CPUM_INVALID_HWVIRT_CONFIG)
+    VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component ConsoleWrap, interface IConsole
+    2022-06-07 15:58:59,573 [racker.cli        ] CRITICAL: Launching container failed. Command 'vagrant up --provider=virtualbox 2019-box' returned non-zero exit status 1.
+
+
+
+Using the CrateDB RPM package on openSUSE
+=========================================
+
+| A: Workaround applied by running ``rpm --install --nodeps``.
+| TODO: Maybe adjust the RPM dependencies?
+
+::
+
+    zypper --non-interactive install crate-4.7.2-1.x86_64.rpm
+    Loading repository data...
+    Reading installed packages...
+    Resolving package dependencies...
+
+    Problem: nothing provides 'systemd-units' needed by the to be installed crate-4.7.2-1.x86_64
+     Solution 1: do not install crate-4.7.2-1.x86_64
+     Solution 2: break crate-4.7.2-1.x86_64 by ignoring some of its dependencies
+
+    Choose from above solutions by number or cancel [1/2/c/d/?] (c): c
+
+::
+
+    rpm -i crate-4.7.2-1.x86_64.rpm
+    warning: crate-4.7.2-1.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 06f6eaeb: NOKEY
+    error: Failed dependencies:
+        shadow-utils is needed by crate-4.7.2-1.x86_64
+        systemd-units is needed by crate-4.7.2-1.x86_64
+
+
+Spawning a container from the CrateDB Docker image
+==================================================
+
+Currently, as of 2022-06, the CrateDB Docker image is based on CentOS 7, which
+has a systemd installation that is too old for being able to invoke nested
+systemd environments.
+
+::
+
+    racker run -it --rm crate/crate:nightly bash
+
+::
+
+    Failed to start transient service unit: Cannot set property AddRef, or unknown property.
+
+=> systemd too old.
+
+
+Docker context on Windows VM not reachable
+==========================================
+
+https://github.com/docker/machine/issues/531
+
+::
+
+    $ docker --context=2019-box ps
+    error during connect: Get "https://192.168.59.90:2376/v1.24/containers/json": x509: certificate is valid for 169.254.232.221, 172.30.112.1, 10.0.2.15, 127.0.0.1, not 192.168.59.90
+
+    $ docker --context=2019-box ps
+    error during connect: Get "https://192.168.59.90:2376/v1.24/containers/json": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "Docker TLS Root")
+
+Solution::
+
+    docker context rm 2019-box

doc/comparison.rst

@@ -63,7 +63,8 @@ invoking `systemd-nspawn`_ without using the ``--boot`` option.
 Notes
 *****
 
-There is also `Boombox`_, `CoreOS toolbox`_, `microos-toolbox`_ and `tlbx`_.
+There is also `Boombox`_, `CoreOS toolbox`_, `docker-coreos-toolbox-ubuntu`_,
+`microos-toolbox`_ and `tlbx`_.
 
 
 .. _Boombox: https://github.com/anthr76/boombox
@@ -72,6 +73,7 @@ There is also `Boombox`_, `CoreOS toolbox`_, `microos-toolbox`_ and `tlbx`_.
 .. _Distrobox: https://github.com/89luca89/distrobox
 .. _Distrobox - Containers distribution support: https://distrobox.privatedns.org/compatibility.html#containers-distros
 .. _distrobox-init: https://distrobox.privatedns.org/usage/distrobox-init.html
+.. _docker-coreos-toolbox-ubuntu: https://github.com/wallneradam/docker-coreos-toolbox-ubuntu
 .. _microos-toolbox: https://github.com/openSUSE/microos-toolbox
 .. _systemd-nspawn: https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html
 .. _tlbx: https://gitlab.com/uppercat/tlbx