OTA Updates (#32)

Add the functionality to allow the user to update their Umbrel installation on the go, without having to manually install new releases or re-flash Umbrel OS image.
5 years ago · 7e4fb3413e
10 changed files with 342 additions and 2 deletions
--- a/bin/README.md
+++ b/bin/README.md
@ -0,0 +1,68 @@
+# Over-The-Air (OTA) Updates
+How over-the-air updates work on Umbrel.
+
+## Execution Flow
+
+1. New developments across the any/entire fleet of Umbrel's services (bitcoind, lnd, dashboard, middleware, etc) are made, which maintain their own independent version-control and release-schedule. Subsequently, their new docker images are built, tagged and pushed to Docker Hub.
+
+2. The newly built and tagged images are updated in the main repository's (i.e. this repo) [`docker-compose.yml`](https://github.com/getumbrel/umbrel/blob/master/docker-compose.yml) file.
+
+3. Any new developments to the main repository (i.e. this repo) are made, eg. adding a new directory or a new config file.
+
+4. To prepare a new release of Umbrel, called `vX.Y.Z`, a PR is opened that updates the [`info.json`](https://github.com/getumbrel/umbrel/blob/master/info.json) file to:
+
+```json
+{
+    "version": "X.Y.Z",
+    "name": "Umbrel vX.Y.Z",
+    "notes": "This release contains a number of bug fixes and new features.",
+    "requires": ">=A.B.C"
+}
+```
+
+5. Once the PR is merged, the master branch is immediately tagged `vX.Y.Z` and released on GitHub.
+
+6. Thus the new `info.json` will automatically be available at `https://raw.githubusercontent.com/getumbrel/umbrel/master/info.json`. This is what triggers the OTA update.
+
+6. When the user opens his [`umbrel-dashboard`](https://github.com/getumbrel/umbrel-dashboard), it periodically polls [`umbrel-manager`](https://github.com/getumbrel/umbrel-manager) to check for new updates.
+
+7. `umbrel-manager` fetches the latest `info.json` from umbrel's main repo's master branch using `GET https://raw.githubusercontent.com/getumbrel/umbrel/master/info.json`, compares it's `version` with the `version` of the local `$UMBREL_ROOT/info.json` file, and exits if both the versions are same.
+
+8. If fetched `version` > local `version`, `umbrel-manager` checks if local `version` satisfies the `requires` condition in the fetched `info.json`.
+
+9. If not, `umbrel-manager` computes the minimum satisfactory version, called `L.M.N`, required for update. Eg, for `"requires": ">=1.2.2"` the minimum satisfactory version would be `1.2.2`. `umbrel-manager` then makes a `GET` request to `https://raw.githubusercontent.com/getumbrel/umbrel/vL.M.N/info.json` and repeats step 8 and 9 until local `version` < fetched `version` **AND** local `version` fulfills the fetched `requires` condition.
+
+10. `umbrel-manager` then returns the satisfying `info.json` to `umbrel-dashboard`.
+
+11. `umbrel-dashboard` then alerts the user regarding the available update, and after the user consents, it makes a `POST` request to `umbrel-manager` to start the update process.
+
+12. `umbrel-manager` adds the `updateTo` key to `$UMBREL_ROOT/statuses/update-status.json` (a file used to continuosly update the user with the update status and progress) with the update release tag.
+
+```json
+{
+    ...
+    "updateTo": "vX.Y.Z"
+    ...
+}
+```
+
+13. `umbrel-manager` then creates an update signal file on the mounted host OS volume (`$UMBREL_ROOT/events/signals/update`) and returns `OK` to the `umbrel-dashboard`.
+
+14. [`karen`](https://github.com/getumbrel/umbrel/blob/master/karen) is triggered (obviously) as soon as `$UMBREL_ROOT/events/signals/update` is touched/updated, and immediately runs the `update` trigger script [`$UMBREL_ROOT/events/triggers/update`](https://github.com/getumbrel/umbrel/blob/master/events/triggers/update) as root.
+
+15. `$UMBREL_ROOT/events/triggers/update` clones release `vX.Y.Z` from github in `$UMBREL_ROOT/.umbrel-vX.Y.Z`.
+
+16. `$UMBREL_ROOT/events/triggers/update` then executes all of the following update scripts from the new release `$UMBREL_ROOT/.umbrel-vX.Y.Z` one-by-one:
+
+- [`$UMBREL_ROOT/.umbrel-vX.Y.Z/bin/update/00-run.sh`](https://github.com/getumbrel/umbrel/blob/master/bin/update/00-run.sh): Pre-update preparation script (does things like making a backup)
+- [`$UMBREL_ROOT/.umbrel-vX.Y.Z/bin/update/01-run.sh`](https://github.com/getumbrel/umbrel/blob/master/bin/update/01-run.sh): Install update script (installs the update)
+- [`$UMBREL_ROOT/.umbrel-vX.Y.Z/bin/update/02-run.sh`](https://github.com/getumbrel/umbrel/blob/master/bin/update/02-run.sh): Post-update script (used to run unit-tests to make sure the update was successfully installed)
+- [`$UMBREL_ROOT/.umbrel-vX.Y.Z/bin/update/03-run.sh`](https://github.com/getumbrel/umbrel/blob/master/bin/update/03-run.sh): Success script (runs after the updated has been successfully downloaded and installed)
+
+All of the above scripts continuously update `$UMBREL_ROOT/statuses/update-status.json` with the progress of update, which the dashboard periodically fetches every 2s via `umbrel-manager` to keep the user updated.
+
+### Further improvements
+
+- OTA updates should not trust GitHub, they should verify signed checksums before installing
+- Catch any error during the update and restore from the backup
+- Restore from backup on power-failure
--- a/bin/update/.updateignore
+++ b/bin/update/.updateignore
@ -0,0 +1,7 @@
+.*
+bitcoin
+db
+lnd
+secrets
+statuses
+tor
--- a/bin/update/00-run.sh
+++ b/bin/update/00-run.sh
@ -0,0 +1,41 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+RELEASE=$1
+UMBREL_ROOT=$2
+UMBREL_USER=$3
+
+echo
+echo "======================================="
+echo "============= OTA UPDATE =============="
+echo "======================================="
+echo "========= Stage: Pre-update ==========="
+echo "======================================="
+echo
+
+# Make sure any previous backup doesn't exist
+if [[ -d "$UMBREL_ROOT"/.umbrel-backup ]]; then
+    echo "Cannot install update. A previous backup already exists at $UMBREL_ROOT/.umbrel-backup"
+    echo "This can only happen if the previous update installation wasn't successful"
+    exit 1
+fi
+
+echo "Installing Umbrel $RELEASE at $UMBREL_ROOT"
+
+# Update status file
+cat <<EOF > "$UMBREL_ROOT"/statuses/update-status.json
+{"state": "installing", "progress": 20, "description": "Backing up", "updateTo": "$RELEASE"}
+EOF
+
+# Fix permissions
+echo "Fixing permissions"
+chown -R 1000:1000 "$UMBREL_ROOT"/
+
+# Backup
+echo "Backing up existing directory tree"
+
+rsync -av "$UMBREL_ROOT"/ \
+    --exclude-from="$UMBREL_ROOT/.umbrel-$RELEASE/bin/update/.updateignore" \
+    "$UMBREL_ROOT"/.umbrel-backup/
+
+echo "Successfully backed up to $UMBREL_ROOT/.umbrel-backup"
--- a/bin/update/01-run.sh
+++ b/bin/update/01-run.sh
@ -0,0 +1,86 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+RELEASE=$1
+UMBREL_ROOT=$2
+UMBREL_USER=$3
+
+echo
+echo "======================================="
+echo "============= OTA UPDATE =============="
+echo "======================================="
+echo "=========== Stage: Install ============"
+echo "======================================="
+echo
+
+cat <<EOF > "$UMBREL_ROOT"/statuses/update-status.json
+{"state": "installing", "progress": 33, "description": "Configuring settings", "updateTo": "$RELEASE"}
+EOF
+
+# Checkout to the new release
+cd "$UMBREL_ROOT"/.umbrel-"$RELEASE"
+
+# Update RPC Password in docker-compose.yml
+# Get gnu sed
+gnused=sed
+if [[ "$(uname)" == "Darwin" ]]; then
+  if ! command -v gsed >/dev/null 2>&1; then
+    echo "Error: This script requires gnu-sed!"
+    echo "Install it with:"
+    echo "  brew install gnu-sed"
+    exit 1
+  fi
+  gnused=gsed
+fi
+
+echo "Updating RPC Password in docker-compose.yml"
+RPCPASS=$(cat "$UMBREL_ROOT"/secrets/rpcpass.txt)
+$gnused -i "s/RPCPASS/${RPCPASS}/g;" docker-compose.yml
+
+# echo "Setting regtest"
+# $gnused -i 's/mainnet/regtest/g; ' docker-compose.yml
+# $gnused -i "s/RPCPORT/18443/g;" docker-compose.yml
+
+echo "Setting mainnet"
+$gnused -i "s/RPCPORT/8332/g;" docker-compose.yml
+
+if [[ "$HOSTNAME" != "umbrel" ]]; then
+  echo "Changing hostname to http://$HOSTNAME.local"
+  $gnused -i "s/umbrel.local/${HOSTNAME}.local/g;" docker-compose.yml
+fi
+
+
+# Pull new images
+echo "Pulling new images"
+cat <<EOF > "$UMBREL_ROOT"/statuses/update-status.json
+{"state": "installing", "progress": 40, "description": "Downloading new Docker images", "updateTo": "$RELEASE"}
+EOF
+cd "$UMBREL_ROOT"/.umbrel-"$RELEASE"
+docker-compose pull
+
+# Stop existing containers
+echo "Stopping existing containers"
+cat <<EOF > "$UMBREL_ROOT"/statuses/update-status.json
+{"state": "installing", "progress": 70, "description": "Removing old containers", "updateTo": "$RELEASE"}
+EOF
+cd "$UMBREL_ROOT"
+docker-compose down
+
+# Overlay home dir structure with new dir tree
+echo "Overlaying $UMBREL_ROOT/ with new directory tree"
+rsync -av "$UMBREL_ROOT"/.umbrel-"$RELEASE"/ \
+    --exclude-from="$UMBREL_ROOT/.umbrel-$RELEASE/bin/update/.updateignore" \
+    "$UMBREL_ROOT"/
+
+# Fix permissions
+echo "Fixing permissions"
+chown -R 1000:1000 "$UMBREL_ROOT"/
+chmod -R 700 "$UMBREL_ROOT"/tor/data/*
+
+# Start updated containers
+echo "Starting new containers"
+cat <<EOF > "$UMBREL_ROOT"/statuses/update-status.json
+{"state": "installing", "progress": 80, "description": "Starting new containers", "updateTo": "$RELEASE"}
+EOF
+cd "$UMBREL_ROOT"
+docker-compose up --detach --remove-orphans
--- a/bin/update/02-run.sh
+++ b/bin/update/02-run.sh
@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+RELEASE=$1
+UMBREL_ROOT=$2
+
+echo
+echo "======================================="
+echo "============= OTA UPDATE =============="
+echo "======================================="
+echo "========= Stage: Post-update =========="
+echo "======================================="
+echo
+
+# Nothing here for now
--- a/bin/update/03-run.sh
+++ b/bin/update/03-run.sh
@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+RELEASE=$1
+UMBREL_ROOT=$2
+
+echo
+echo "======================================="
+echo "============= OTA UPDATE =============="
+echo "======================================="
+echo "=========== Stage: Success ============"
+echo "======================================="
+echo
+
+# Delete previous (unused) images
+echo "Deleting previous images"
+cat <<EOF > "$UMBREL_ROOT"/statuses/update-status.json
+{"state": "installing", "progress": 90, "description": "Deleting previous images", "updateTo": "$RELEASE"}
+EOF
+docker image prune --all --force
+
+# Cleanup
+echo "Removing backup"
+cat <<EOF > "$UMBREL_ROOT"/statuses/update-status.json
+{"state": "installing", "progress": 95, "description": "Removing backup"}
+EOF
+[[ -d "$UMBREL_ROOT"/.umbrel-backup ]] && rm -rf "$UMBREL_ROOT"/.umbrel-backup
+
+echo "Successfully installed Umbrel $RELEASE"
+cat <<EOF > "$UMBREL_ROOT"/statuses/update-status.json
+{"state": "success", "progress": 100, "description": "Successfully installed Umbrel $RELEASE", "updateTo": ""}
+EOF
--- a/docker-compose.yml
+++ b/docker-compose.yml
@ -13,6 +13,7 @@ services:
        tor:
                container_name: tor
                image: getumbrel/tor:v0.4.1.9
+                user: toruser
                restart: unless-stopped
                logging: *default-logging
                volumes:
@ -69,7 +70,7 @@ services:
                        ipv4_address: 10.11.1.2
        dashboard:
                container_name: dashboard
-                image: getumbrel/dashboard:v0.2.3
+                image: getumbrel/dashboard:v0.3.0
                logging: *default-logging
                restart: unless-stopped
                stop_grace_period: 1m30s
@ -78,15 +79,17 @@ services:
                        ipv4_address: 10.11.0.3
        manager:
                container_name: manager
-                image: getumbrel/manager:v0.1.3
+                image: getumbrel/manager:v0.2.0
                logging: *default-logging
                depends_on: [ tor ]
                restart: unless-stopped
                stop_grace_period: 5m30s
                volumes:
                        - ${PWD}:${PWD}
+                        - ${PWD}/info.json:/info.json
                        - ${PWD}/db:/db
                        - ${PWD}/events/signals:/signals
+                        - ${PWD}/statuses:/statuses
                        - ${PWD}/tor/data:/var/lib/tor/
                        - /var/run/docker.sock:/var/run/docker.sock
                        - /usr/bin/docker:/usr/bin/docker
@ -104,6 +107,11 @@ services:
                    UMBREL_DASHBOARD_HIDDEN_SERVICE_FILE: "/var/lib/tor/web/hostname"
                    SHUTDOWN_SIGNAL_FILE: "/signals/shutdown"
                    REBOOT_SIGNAL_FILE: "/signals/reboot"
+                    GITHUB_REPO: "getumbrel/umbrel"
+                    UMBREL_VERSION_FILE: "/info.json"
+                    UPDATE_STATUS_FILE: "/statuses/update-status.json"
+                    UPDATE_SIGNAL_FILE: "/signals/update"
+                    UPDATE_LOCK_FILE: "/statuses/update-in-progress"
                networks:
                    net:
                        ipv4_address: 10.11.2.1
--- a/events/triggers/update
+++ b/events/triggers/update
@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+
+check_dependencies () {
+  for cmd in "$@"; do
+    if ! command -v "$cmd" >/dev/null 2>&1; then
+      echo "This script requires \"${cmd}\" to be installed"
+      exit 1
+    fi
+  done
+}
+
+check_dependencies jq wget git rsync
+
+# UMBREL_ROOT=/home/umbrel
+UMBREL_ROOT=$(dirname $(readlink -f "${BASH_SOURCE[0]}"))/../..
+RELEASE=$(cat "$UMBREL_ROOT"/statuses/update-status.json | jq .updateTo -r)
+
+echo
+echo "======================================="
+echo "============= OTA UPDATE =============="
+echo "======================================="
+echo "========== Stage: Download ============"
+echo "======================================="
+echo
+
+# Make sure an update is not in progres
+if [[ -f "$UMBREL_ROOT/statuses/update-in-progress" ]]; then
+    echo "An update is already in progress. Exiting now."
+    exit 2
+fi
+
+echo "Creating lock"
+touch "$UMBREL_ROOT"/statuses/update-in-progress
+
+# Cleanup just in case there's temp stuff lying around from previous update
+echo "Cleaning up any previous mess"
+[[ -d "$UMBREL_ROOT"/.umbrel-"$RELEASE" ]] && rm -rf "$UMBREL_ROOT"/.umbrel-"$RELEASE"
+
+# Update status file
+cat <<EOF > "$UMBREL_ROOT"/statuses/update-status.json
+{"state": "installing", "progress": 10, "description": "Downloading Umbrel $RELEASE", "updateTo": "$RELEASE"}
+EOF
+
+# Clone new release
+echo "Downloading Umbrel $RELEASE"
+mkdir -p "$UMBREL_ROOT"/.umbrel-"$RELEASE"
+cd "$UMBREL_ROOT"/.umbrel-"$RELEASE"
+wget -qO- "https://raw.githubusercontent.com/getumbrel/umbrel/$RELEASE/install-box.sh" | sh
+
+# Run update scripts
+echo "Running update install scripts of the new release"
+cd bin/update
+UPDATE_INSTALL_SCRIPTS=$(ls *-run.sh)
+for script in $UPDATE_INSTALL_SCRIPTS; do
+    if [[ -x $script ]]; then
+        echo
+        echo "== Begin Update Script $script =="
+        ./"$script" "$RELEASE" "$UMBREL_ROOT"
+        echo "== End Update Script $script =="
+        echo
+    fi
+done
+
+# Delete cloned repo
+echo "Deleting cloned repository"
+[[ -d "$UMBREL_ROOT"/.umbrel-"$RELEASE" ]] && rm -rf "$UMBREL_ROOT"/.umbrel-"$RELEASE"
+
+echo "Removing lock"
+rm -f "$UMBREL_ROOT"/statuses/update-in-progress
+
+exit 0
--- a/info.json
+++ b/info.json
@ -0,0 +1,6 @@
+{
+    "version": "0.1.4-beta",
+    "name": "Umbrel v0.1.4 Beta",
+    "requires": ">=0.1.3-beta",
+    "notes": ""
+}
--- a/statuses/update-status.json
+++ b/statuses/update-status.json
@ -0,0 +1,6 @@
+{
+    "state": "success",
+    "progress": 100,
+    "description": "",
+    "updateTo": ""
+}