Experimental environment for emulating a device shell

from a root file system image.

Changelog: Experimental device emulation environment

Signed-off-by: Eystein Måløy Stenberg <eystein.maloy.stenberg@northern.tech>

device-image-shell/.dockerignore

@@ -0,0 +1 @@
+output/

device-image-shell/Dockerfile

@@ -0,0 +1,13 @@
+FROM ubuntu:18.04
+
+ARG MENDER_ARTIFACT_VERSION=2.3.0
+
+RUN apt-get update && apt-get install -y \
+    wget \
+    qemu-user-static
+
+RUN wget -q -O /usr/bin/mender-artifact https://d1b0l86ne08fsf.cloudfront.net/mender-artifact/$MENDER_ARTIFACT_VERSION/mender-artifact \
+    && chmod +x /usr/bin/mender-artifact
+
+COPY docker-entrypoint.sh /usr/local/bin/
+ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]

device-image-shell/README.md

@@ -0,0 +1,64 @@
+Device Image Shell
+==================
+
+This directory contains a tool that takes the root file system of your device as input and emulates a shell session on your device.
+Any commands you run (e.g. apt update, apt upgrade) will behave as if you were logged into your actual device!
+
+The purpose of the tool is to create a new root file system image and Mender Artifact that can be deployed to a fleet of devices in the field.
+
+
+## Docker environment
+
+To ensure dependencies are correctly set up and it is portable, a docker environment is used.
+
+You need to [install Docker Engine](https://docs.docker.com/install) to use this tool.
+
+
+### Build the device-image-shell container image
+
+To build a container based on Ubuntu 18.04 with the required dependencies, copy this directory to your workstation and change the current directory to it.
+
+Then run
+
+```bash
+./docker-build
+```
+
+This will create a container image `device-image-shell`.
+
+
+### Use the device-image-shell container image
+
+The also assumes your device is based on the ARM architecture, which is the most common (e.g. Raspberry Pi, BeagleBoard, etc.).
+
+You need a root file system image (usually with .ext4 extension) for your device as a starting point, such as one output by [mender-convert](https://github.com/mendersoftware/mender-convert).
+
+You can now enter a shell in your device root file system image by running `docker-device-image-shell` with the desired arguments:
+
+1. path to your existing root file system image
+2. desired name for the generated Mender Artifact
+3. device type, which Mender uses to ensure compatibility between devices and software
+
+For example, if you are using a Raspberry Pi 3, you can run:
+
+```bash
+./docker-device-image-shell ../output/2018-11-13-raspbian-stretch-lite.ext4 2018-11-13-raspbian-stretch-lite-aptupgrade raspberrypi3
+```
+
+You should now see a shell prompt. You are in an emulated environment, so any commands run here will behave as if you ran them on your device! In addition, any changes you make will be preserved in the output root file system image and Mender Artifact.
+
+For example, to update to the latest packages run:
+
+```bash
+apt update
+apt upgrade
+```
+
+When you are done, press `Ctrl+D` or run the `exit` command. Generating the Mender Artifact will take a few more minutes, depending on the size of the input image and resources available on your workstation.
+
+After it finishes you can find your new `.ext4` and `.mender` files in the `output/` directory. For devices where Mender is installed, you can use the Mender Artifact (`.mender` file) to deploy the changes you made in the shell to all your devices!
+
+
+### Use caution
+
+Please note that since this tool is using an emulated environment (based on `qemu`) and you are not properly logged in to your device, some things may not work as expected. Look for any relevant errors in  commands you run and make sure to test your changes before deploying to production devices!

device-image-shell/docker-build

@@ -0,0 +1,7 @@
+#!/bin/sh
+
+set -e
+
+DOCKER_IMAGE_NAME=device-image-shell
+
+docker build . -t ${DOCKER_IMAGE_NAME}

device-image-shell/docker-device-image-shell

@@ -0,0 +1,58 @@
+#!/bin/sh
+
+set -e
+
+DOCKER_IMAGE_NAME=device-image-shell
+OUTPUT_DIR="$(pwd)/output"
+
+show_usage() {
+  echo "Usage:"
+  echo "$0 <rootfs input file> <output Artifact name> <device type>"
+}
+
+if [ "$#" -ne 3 ]; then
+  echo "ERROR: 3 parameters required."
+  show_usage
+  exit 1
+fi
+
+ROOTFS_INPUT_FILE=$1
+ARTIFACT_NAME=$2
+DEVICE_TYPE=$3
+
+if [ ! -f "$ROOTFS_INPUT_FILE" ]; then
+  echo "ERROR: File passed as first argument is not accessible."
+  echo "Got ROOTFS_INPUT_FILE=\"$ROOTFS_INPUT_FILE\""
+  show_usage
+  exit 1
+fi
+
+
+mkdir -p $OUTPUT_DIR
+
+ROOTFS_INPUT_FILE_NAME="$(basename -- $ROOTFS_INPUT_FILE)"
+ROOTFS_INPUT_FILE_EXTENSION="${ROOTFS_INPUT_FILE_NAME##*.}"
+
+ROOTFS_OUTPUT_FILE_NAME="$ARTIFACT_NAME.$ROOTFS_INPUT_FILE_EXTENSION"
+
+echo "Copying rootfs input file..."
+rsync -h --progress $ROOTFS_INPUT_FILE $OUTPUT_DIR/$ROOTFS_OUTPUT_FILE_NAME
+
+docker run \
+       -ti \
+       --privileged=true \
+       --mount type=bind,source=$OUTPUT_DIR,target=/root_images \
+       $DOCKER_IMAGE_NAME $ROOTFS_OUTPUT_FILE_NAME $ARTIFACT_NAME $DEVICE_TYPE
+
+
+# Output Artifact gets root owner and group, change to current logged in
+CURRENT_USER=$(id -u -n)
+CURRENT_GROUP=$(id -g -n)
+
+echo "Changing ownership of Mender Artifact (may ask you to authenticate)"
+sudo chown $CURRENT_USER:$CURRENT_GROUP $OUTPUT_DIR/$ARTIFACT_NAME.mender
+
+echo "Image generation complete!"
+/bin/echo -e "The new root file system is at:\n\t$OUTPUT_DIR/$ROOTFS_OUTPUT_FILE_NAME"
+/bin/echo -e "The new Mender Artifact you can upload to your Mender server to deploy to your devices is at:\
+         \n\t$OUTPUT_DIR/$ARTIFACT_NAME.mender"

device-image-shell/docker-entrypoint.sh

@@ -0,0 +1,38 @@
+#!/bin/sh
+set -e
+
+ROOTFS_OUTPUT_FILE_NAME=$1
+ARTIFACT_NAME=$2
+DEVICE_TYPE=$3
+
+mkdir /root_system
+
+mount /root_images/$ROOTFS_OUTPUT_FILE_NAME /root_system
+
+if [ -f /root_system/usr/bin/qemu-arm-static ]; then
+  echo "WARNING: /usr/bin/qemu-arm-static already exists in image. Using this but may be of incompatible version."
+  QEMU_STATIC_COPIED=false
+else
+  # trick to make chroot into ARM image work
+  cp /usr/bin/qemu-arm-static /root_system/usr/bin
+  QEMU_STATIC_COPIED=true
+fi
+
+echo "Entering emulated shell in device image. All commands are run as the root user of the device image."
+echo "Make changes (e.g. apt update, apt upgrade, wget ...) and press Ctrl-D when done."
+
+# Using bash for command completion support and other conveniences
+chroot /root_system /bin/bash
+
+# Mender Artifact name must also be present inside
+echo artifact_name=$ARTIFACT_NAME > /root_system/etc/mender/artifact_info
+
+if [ "$QEMU_STATIC_COPIED" = true ]; then
+  rm /root_system/usr/bin/qemu-arm-static
+fi
+
+umount /root_system
+
+echo "Creating Mender Artifact. This may take a few minutes..."
+mender-artifact write rootfs-image -t $DEVICE_TYPE -n $ARTIFACT_NAME -u /root_images/$ROOTFS_OUTPUT_FILE_NAME -o /root_images/$ARTIFACT_NAME.mender
+sync