make a docker version of the dev environment

README.md

@@ -1,3 +1,16 @@
+##  Kubos-Dev
+
+This repo aims to contain various dev and build environment setup instructions.
+
+It includes:
+- the kubos vagrant environment mentioned in the kubos docs (`kubos-dev`)
+- docker images for CI builds (`docker/kubos-build`) that have been relocated from the main repo
+- a docker image that aims to be an equivalent to the vagrant image to promote reuse and space savings by deduplicating many things with the CI build image (`docker/kobos-sdk`) 
+
+
+------
+
+
 For usage instructions checkout the official [Kubos docs](https://docs.kubos.com/latest/sdk-docs/index.html)
 
 ## Testing Vagrant Changes:

docker/.dockerignore

@@ -0,0 +1 @@
+!.git/

docker/README.md

@@ -0,0 +1,48 @@
+This folder contains a docker environment for developing KubOS. 
+
+It consists of a couple docker containers and a script of some kind (currently docker compose) to make the experience as similar as possible to "vagrant up".
+
+## Containers
+There are three containers at play here to keep things organized:
+
+**kubos-base** - a base image containing dependencies that must be installed from a priviliged account (root)
+
+**kubos-build** - this image is intended to be a replica of what is currently being used for CI builds
+
+**kubos-sdk** - this image is intended to emulate the KubOS SDK vagrant envrionment as closely as possible and installs many dependencies under a regular user account
+
+## Setup
+### Building the images
+
+**TL;DR** to build any of the docker containers, use the command `docker build -t <image_name> -f <image_folder>/Dockerfile . ` while inside this `docker` folder. if you are just looking to build the docker container for the SDK, theres also a `build-sdk.sh` script.
+
+This slightly different command from the usual `docker build -t <name> .` is needed because the various containers need to copy some fonfiguration files from the `bin` folder throughout the process, so this folder also needs to be part of dockers build context.
+
+The build context is the directory passed to `docker build`, which is where it looks for the `Dockerfile` by default. Because this `bin` folder and the `Dockerfile` are in separate directories, the build context should be set to the `docker` folder and the Dockerfile needs to be manually specified because it differs from the default.
+
+#### Image names
+Most of the images rely on the `kubos-base` image, so it is recommended to keep the image names the same as the directory of their `Dockerfile` so images like `kubos-sdk` that depend on this base image are still able to build correctly.
+### Configuring the container
+
+Because the docker compose file is designed to be run from the directory you want to mount (similar to the Vagrantfile), it should be relocated there.
+
+If you want a one-time setup, use this copy command from this directory to create or replace your docker compose in your desired location: `cp ./docker-compose.yml /path/to/your/Vagrantfile`
+
+If you want to automatically keep things in sync, use a symlink: `ln -s /full/path/to/docker/docker-compose.yml ./path/to/your/Vagrantfile`
+
+### Running the container
+To get a shell into the container similar to vagrant ssh, use `docker-compose run --rm kubos-sdk` from the directory where you copoed the compose file to.
+
+The docker container will auto-mount this directory into the container at the mount point /vagrant just like the KubOS Vagrant SDK.
+
+To add additional directory mounts, such as for mounting something to `/home/kubos`, feel free to edit the `docker-compose.yml`
+
+## Transferring data from the VM
+Vagrant offers a plugin that can be used to exfiltrate the data you want to transfer to the docker environment. This can be set up using `vagrant plugin install vagrant-scp` ([source](https://stackoverflow.com/questions/16704059/easiest-way-to-copy-a-single-file-from-host-to-vagrant-guest#28359455))
+
+
+## Moving the generated image to an external drive
+For space-constrained host PC's, its possible to move the docker images to an external drive:
+- Macos: https://stackoverflow.com/questions/38205735/store-docker-image-files-on-external-drive-in-macos#40114330
+- linux:
+

docker/build-sdk.sh

@@ -0,0 +1,4 @@
+docker build -t kubos-base -f kubos-base/Dockerfile . 
+docker build -t kubos-sdk -f kubos-sdk/Dockerfile . 
+
+

docker/cargo_config

@@ -0,0 +1,5 @@
+[target.arm-unknown-linux-gnueabihf]
+linker = "/usr/bin/bbb_toolchain/usr/bin/arm-linux-gcc"
+
+[target.armv5te-unknown-linux-gnueabi]
+linker = "/usr/bin/iobc_toolchain/usr/bin/arm-linux-gcc"

docker/docker-compose.yml

@@ -0,0 +1,12 @@
+version: "3.9"  # optional since v1.27.0
+services:
+  kubos-sdk:
+    image: kubos-sdk
+    volumes:
+      - .:/vagrant
+    entrypoint: "bash"
+    stdin_open: true
+    tty: true
+    # if you have a device you want to make available in the VM, uncomment this
+    # devices:
+    # - "/dev/ttyUSB0:/dev/FTDI"

docker/kubos-base/Dockerfile

@@ -0,0 +1,49 @@
+FROM ubuntu:18.04
+
+LABEL org.opencontainers.image.authors="[email protected], [email protected]" \
+	version="1.20.1" \
+	com.example.vendor="The KubOS Community"
+
+RUN apt-get update -y
+
+RUN apt-get upgrade --no-install-recommends -y python3.7
+RUN apt-get install --no-install-recommends -y pkg-config build-essential git cmake unzip wget sqlite3 libsqlite3-dev libssl-dev curl ssh
+
+# Linux build dependencies
+RUN apt-get install --no-install-recommends -y python file rsync bc cpio ncurses-dev libc6-i386 lib32stdc++6 lib32z1
+
+#Tools to generate docs
+RUN apt-get install --no-install-recommends -y doxygen graphviz plantuml
+
+# resolvconf is no longer included by default as of Ubuntu 18.04
+# apt-get install -y resolvconf
+# rm -f /etc/resolv.conf
+# # There's something wrong with the default resolv.conf symlink. This fixes it
+# ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf
+
+
+# Install pip for Python2 and Python3
+RUN apt-get install --no-install-recommends -y python3-pip python3-setuptools
+
+# So that we have bdist_wheel available when installing other packages
+RUN pip3 install wheel
+
+#Kubos Linux setup
+RUN echo "Installing Kubos Linux Toolchain"
+
+RUN wget https://s3.amazonaws.com/kubos-world-readable-assets/iobc_toolchain.tar.gz && tar -xf ./iobc_toolchain.tar.gz -C /usr/bin && rm ./iobc_toolchain.tar.gz
+
+RUN wget https://s3.amazonaws.com/kubos-world-readable-assets/bbb_toolchain.tar.gz && tar -xf ./bbb_toolchain.tar.gz -C /usr/bin && rm ./bbb_toolchain.tar.gz
+
+# Install NOS3 dependencies
+RUN apt-get --no-install-recommends install -y software-properties-common
+# This was originally a cz.archive domain not us.
+RUN add-apt-repository 'deb http://us.archive.ubuntu.com/ubuntu xenial main universe'
+RUN apt-get install -y libboost-system1.58.0 libboost-program-options1.58.0 libxerces-c3.1
+RUN wget https://github.com/nasa/nos3/raw/v1_04_00/support/packages/ubuntu/itc-common-cxx11-Release_1.9.1_amd64.deb && \
+apt-get install -y ./itc-common-cxx11-Release_1.9.1_amd64.deb && rm itc-common-cxx11-Release_1.9.1_amd64.deb
+
+RUN wget https://github.com/nasa/nos3/raw/v1_04_00/support/packages/ubuntu/nos-engine-cxx11-Release_1.4.0_amd64.deb && \
+apt-get install -y ./nos-engine-cxx11-Release_1.4.0_amd64.deb && rm nos-engine-cxx11-Release_1.4.0_amd64.deb
+
+ENV PATH "$PATH:/usr/bin/iobc_toolchain/usr/bin:/usr/bin/bbb_toolchain/usr/bin"

docker/kubos-build/Dockerfile

@@ -0,0 +1,15 @@
+FROM kubos-base
+
+# Install all Kubos Python dependencies
+RUN git clone https://github.com/kubos/kubos --depth 1 && pip3 install -r kubos/requirements.txt && rm -r kubos
+
+# Setup rust stuff
+ENV PATH "$PATH:/root/.cargo/bin"
+RUN curl https://sh.rustup.rs -sSf | sh -s -- -y && rustup toolchain uninstall stable-x86_64-unknown-linux-gnu
+RUN rustup default 1.39.0 && rm -rf /root/.rustup/toolchains/*/share/doc
+RUN rustup target install arm-unknown-linux-gnueabihf
+RUN rustup target install armv5te-unknown-linux-gnueabi
+RUN rustup component add clippy
+RUN rustup component add rustfmt
+RUN /root/.cargo/bin/cargo install --git https://github.com/kubos/cargo-kubos
+COPY ../bin/cargo_config /root/.cargo/config

docker/kubos-sdk/Dockerfile

@@ -0,0 +1,83 @@
+FROM kubos-base
+
+# Install more priviliged dependencies that only seem to be needed in the SDK environment
+
+RUN apt-get install -y software-properties-common
+
+# Install kernel additions for better USB device recognition
+RUN apt-get install -y linux-image-extra-virtual
+
+RUN apt-get install -y libffi-dev libhidapi-hidraw0 gdb ninja-build python-dev libffi-dev 
+
+
+# tools for user friendliness
+RUN apt-get install -y sshpass nano sudo
+
+
+# Documentation tools
+# latex stuff takes up a fair bit of space. lets wait and see how necessary this is
+# RUN apt-get install -y texlive-latex-recommended texlive-latex-extra texlive-fonts-recommended latexmk texlive-fonts-extra
+
+# KubOS Linux setup
+# echo "Installing KubOS Linux Toolchains"
+
+RUN apt-get install -y minicom
+
+# Utilities for building KubOS Linux
+RUN apt-get install -y mtools
+
+
+# Copy configs
+COPY bin/minirc.kubos /etc/minicom/minirc.kubos
+COPY bin/kubos-usb.rules /etc/udev/rules.d/kubos-usb.rules
+
+# Create user accounts
+RUN useradd --create-home --shell /bin/bash kubos
+RUN adduser kubos sudo
+RUN adduser kubos dialout
+
+RUN echo 'kubos:Kubos123' | chpasswd
+
+#Vagrant commands may act funny without password-less sudo
+RUN echo "kubos ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
+
+USER kubos
+WORKDIR /home/kubos
+
+RUN mkdir -p /home/kubos/.kubos
+# Install all Kubos Python dependencies
+
+RUN git clone https://github.com/kubos/kubos /home/kubos/.kubos/kubos --depth 1 && pip3 install -r /home/kubos/.kubos/kubos/requirements.txt
+#&& rm -r kubos
+RUN chown -R kubos /home/kubos/.kubos
+
+# Setup rust stuff
+ENV PATH "$PATH:/home/kubos/.cargo/bin"
+RUN curl https://sh.rustup.rs -sSf | sh -s -- -y && rustup toolchain uninstall stable-x86_64-unknown-linux-gnu
+RUN rustup default 1.39.0 && rm -rf /home/kubos/.rustup/toolchains/*/share/doc
+RUN rustup target install arm-unknown-linux-gnueabihf
+RUN rustup target install armv5te-unknown-linux-gnueabi
+RUN rustup component add clippy
+RUN rustup component add rustfmt
+RUN /home/kubos/.cargo/bin/cargo install --git https://github.com/kubos/cargo-kubos
+COPY bin/cargo_config /home/kubos/.cargo/config
+
+
+RUN /home/kubos/.cargo/bin/cargo install --bin kubos-file-client --path /home/kubos/.kubos/kubos/clients/kubos-file-client/
+# Install shell-client
+RUN /home/kubos/.cargo/bin/cargo install --bin kubos-shell-client --path /home/kubos/.kubos/kubos/clients/kubos-shell-client/
+# Install example UART comms client
+RUN /home/kubos/.cargo/bin/cargo install --bin uart-comms-client --path /home/kubos/.kubos/kubos/clients/uart-comms-client/
+# Cleanup temporary build files
+RUN rm -rf /home/kubos/.kubos/kubos/target
+
+# Install app-api python module
+RUN cd /home/kubos/.kubos/kubos/apis/app-api/python && python3 -m pip install .
+
+# Install i2c python module
+RUN cd /home/kubos/.kubos/kubos/hal/python-hal/i2c && python3 -m pip install .
+
+# Install kubos-service python module
+RUN cd /home/kubos/.kubos/kubos/libs/kubos-service && python3 -m pip install .
+
+

kubos-dev/bin

@@ -0,0 +1 @@
+../docker/bin