Browse Source

mac build: conform to macOS 10.15 Gatekeeper requirements

fixes #6128

some of this is based on:
e1354632d2/scripts/package/macos-notarize-app.sh
1eb8b71e7d
24e44e9784
5abec73eee
master
SomberNight 5 years ago
parent
commit
095464b620
No known key found for this signature in database GPG Key ID: B33B5F232C6271E9
  1. 63
      contrib/osx/README.md
  2. 23
      contrib/osx/base.sh
  3. 19
      contrib/osx/entitlements.plist
  4. 62
      contrib/osx/make_osx
  5. 77
      contrib/osx/notarize_app.sh

63
contrib/osx/README.md

@ -1,5 +1,5 @@
Building Mac OS binaries
========================
Building macOS binaries
=======================
✗ _This script does not produce reproducible output (yet!).
Please help us remedy this._
@ -7,36 +7,47 @@ Building Mac OS binaries
This guide explains how to build Electrum binaries for macOS systems.
## 1. Building the binary
## Building the binary
This needs to be done on a system running macOS or OS X. We use El Capitan (10.11.6) as building it
on High Sierra (or later)
makes the binaries [incompatible with older versions](https://github.com/pyinstaller/pyinstaller/issues/1191).
This needs to be done on a system running macOS or OS X.
Another factor for the minimum supported macOS version is the
[bundled Qt version](https://github.com/spesmilo/electrum/issues/3685).
Notes about compatibility with different macOS versions:
- In general the binary is not guaranteed to run on an older version of macOS
than what the build machine has. This is due to bundling the compiled Python into
the [PyInstaller binary](https://github.com/pyinstaller/pyinstaller/issues/1191).
- The [bundled version of Qt](https://github.com/spesmilo/electrum/issues/3685) also
imposes a minimum supported macOS version.
- If you want to build binaries that conform to the macOS "Gatekeeper", so as to
minimise the warnings users get, the binaries need to be codesigned with a
certificate issued by Apple, and starting with macOS 10.15 the binaries also
need to be notarized by Apple's central server. The catch is that to be able to build
binaries that Apple will notarise (due to the requirements on the binaries themselves,
e.g. hardened runtime) the build machine needs at least macOS 10.14.
See [#6128](https://github.com/spesmilo/electrum/issues/6128).
We currently build the release binaries on macOS 10.14.6, and these seem to run on
10.13 or newer.
Before starting, make sure that the Xcode command line tools are installed (e.g. you have `git`).
#### 1.1a Get Xcode
#### 1.a Get Xcode
Building the QR scanner (CalinsQRReader) requires full Xcode (not just command line tools).
The last Xcode version compatible with El Capitan is Xcode 8.2.1
Get it from [here](https://developer.apple.com/download/more/).
Unfortunately, you need an "Apple ID" account.
(note: the last Xcode that runs on macOS 10.14.6 is Xcode 11.3.1)
After downloading, uncompress it.
Make sure it is the "selected" xcode (e.g.):
sudo xcode-select -s $HOME/Downloads/Xcode.app/Contents/Developer/
#### 1.1b Build QR scanner separately on newer Mac
#### 1.b Build QR scanner separately on another Mac
Alternatively, you can try building just the QR scanner on newer macOS.
Alternatively, you can try building just the QR scanner on another Mac.
On newer Mac, run:
@ -46,27 +57,17 @@ On newer Mac, run:
Move `prebuilt_qr` to El Capitan: `contrib/osx/CalinsQRReader/prebuilt_qr`.
#### 1.2 Build Electrum
#### 2. Build Electrum
cd electrum
./contrib/osx/make_osx
This creates both a folder named Electrum.app and the .dmg file.
If you want the binaries codesigned for MacOS and notarised by Apple's central server,
provide these env vars to the `make_osx` script:
## 2. Building the image deterministically (WIP)
The usual way to distribute macOS applications is to use image files containing the
application. Although these images can be created on a Mac with the built-in `hdiutil`,
they are not deterministic.
Instead, we use the toolchain that Bitcoin uses: genisoimage and libdmg-hfsplus.
These tools do not work on macOS, so you need a separate Linux machine (or VM).
Copy the Electrum.app directory over and install the dependencies, e.g.:
apt install libcap-dev cmake make gcc faketime
Then you can just invoke `package.sh` with the path to the app:
cd electrum
./contrib/osx/package.sh ~/Electrum.app/
CODESIGN_CERT="Developer ID Application: Electrum Technologies GmbH (L6P37P7P56)" \
APPLE_ID_USER="me@email.com" \
APPLE_ID_PASSWORD="1234" \
./contrib/osx/make_osx

23
contrib/osx/base.sh

@ -1,23 +0,0 @@
#!/usr/bin/env bash
. $(dirname "$0")/../build_tools_util.sh
function DoCodeSignMaybe { # ARGS: infoName fileOrDirName codesignIdentity
infoName="$1"
file="$2"
identity="$3"
deep=""
if [ -z "$identity" ]; then
# we are ok with them not passing anything; master script calls us unconditionally even if no identity is specified
return
fi
if [ -d "$file" ]; then
deep="--deep"
fi
if [ -z "$infoName" ] || [ -z "$file" ] || [ -z "$identity" ] || [ ! -e "$file" ]; then
fail "Argument error to internal function DoCodeSignMaybe()"
fi
info "Code signing ${infoName}..."
codesign -f -v $deep -s "$identity" "$file" || fail "Could not code sign ${infoName}"
}

19
contrib/osx/entitlements.plist

@ -0,0 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<!-- These are required for binaries built by PyInstaller -->
<!-- see pyinstaller/pyinstaller#4629 -->
<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
<true/>
<key>com.apple.security.cs.disable-library-validation</key>
<true/>
<!-- These are required for USB HID access (hw wallets). -->
<!-- see https://github.com/Electron-Cash/Electron-Cash/commit/5abec73eee0cdeb725e3c5a989621ec4ccfb92a0 -->
<key>com.apple.security.cs.allow-dyld-environment-variables</key>
<true/>
<key>com.apple.security.cs.allow-jit</key>
<true/>
</dict>
</plist>

62
contrib/osx/make_osx

@ -5,11 +5,12 @@ PYTHON_VERSION=3.7.6
BUILDDIR=/tmp/electrum-build
PACKAGE=Electrum
GIT_REPO=https://github.com/spesmilo/electrum
LIBSECP_VERSION="b408c6a8b287003d1ade5709e6f7bc3c7f1d5be7"
export GCC_STRIP_BINARIES="1"
. $(dirname "$0")/base.sh
. $(dirname "$0")/../build_tools_util.sh
CONTRIB_OSX="$(dirname "$(realpath "$0")")"
CONTRIB="$CONTRIB_OSX/.."
@ -24,26 +25,46 @@ which brew > /dev/null 2>&1 || fail "Please install brew from https://brew.sh/ t
which xcodebuild > /dev/null 2>&1 || fail "Please install Xcode and xcode command line tools to continue"
# Code Signing: See https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html
APP_SIGN=""
if [ -n "$1" ]; then
if [ -n "$CODESIGN_CERT" ]; then
# Test the identity is valid for signing by doing this hack. There is no other way to do this.
cp -f /bin/ls ./CODESIGN_TEST
codesign -s "$1" --dryrun -f ./CODESIGN_TEST > /dev/null 2>&1
codesign -s "$CODESIGN_CERT" --dryrun -f ./CODESIGN_TEST > /dev/null 2>&1
res=$?
rm -f ./CODESIGN_TEST
if ((res)); then
fail "Code signing identity \"$1\" appears to be invalid."
fail "Code signing identity \"$CODESIGN_CERT\" appears to be invalid."
fi
unset res
APP_SIGN="$1"
info "Code signing enabled using identity \"$APP_SIGN\""
info "Code signing enabled using identity \"$CODESIGN_CERT\""
else
warn "Code signing DISABLED. Specify a valid macOS Developer identity installed on the system as the first argument to this script to enable signing."
warn "Code signing DISABLED. Specify a valid macOS Developer identity installed on the system to enable signing."
fi
function DoCodeSignMaybe { # ARGS: infoName fileOrDirName
infoName="$1"
file="$2"
deep=""
if [ -z "$CODESIGN_CERT" ]; then
# no cert -> we won't codesign
return
fi
if [ -d "$file" ]; then
deep="--deep"
fi
if [ -z "$infoName" ] || [ -z "$file" ] || [ ! -e "$file" ]; then
fail "Argument error to internal function DoCodeSignMaybe()"
fi
hardened_arg="--entitlements=${CONTRIB_OSX}/entitlements.plist -o runtime"
info "Code signing ${infoName}..."
codesign -f -v $deep -s "$CODESIGN_CERT" $hardened_arg "$file" || fail "Could not code sign ${infoName}"
}
info "Installing Python $PYTHON_VERSION"
export PATH="~/.pyenv/bin:~/.pyenv/shims:~/Library/Python/3.7/bin:$PATH"
if [ -d "~/.pyenv" ]; then
if [ -d "${HOME}/.pyenv" ]; then
pyenv update
else
curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash > /dev/null 2>&1
@ -109,7 +130,7 @@ rm -fr build
# prefer building using xcode ourselves. otherwise fallback to prebuilt binary
xcodebuild || cp -r prebuilt_qr build || fail "Could not build CalinsQRReader"
popd
DoCodeSignMaybe "CalinsQRReader.app" "${d}/build/Release/CalinsQRReader.app" "$APP_SIGN" # If APP_SIGN is empty will be a noop
DoCodeSignMaybe "CalinsQRReader.app" "${d}/build/Release/CalinsQRReader.app"
info "Installing requirements..."
@ -131,7 +152,7 @@ for d in ~/Library/Python/ ~/.pyenv .; do
done
info "Building binary"
APP_SIGN="$APP_SIGN" pyinstaller --noconfirm --ascii --clean --name $VERSION contrib/osx/osx.spec || fail "Could not build binary"
APP_SIGN="$CODESIGN_CERT" pyinstaller --noconfirm --ascii --clean --name $VERSION contrib/osx/osx.spec || fail "Could not build binary"
info "Adding bitcoin URI types to Info.plist"
plutil -insert 'CFBundleURLTypes' \
@ -139,14 +160,23 @@ plutil -insert 'CFBundleURLTypes' \
-- dist/$PACKAGE.app/Contents/Info.plist \
|| fail "Could not add keys to Info.plist. Make sure the program 'plutil' exists and is installed."
DoCodeSignMaybe "app bundle" "dist/${PACKAGE}.app" "$APP_SIGN" # If APP_SIGN is empty will be a noop
DoCodeSignMaybe "app bundle" "dist/${PACKAGE}.app"
if [ ! -z "$CODESIGN_CERT" ]; then
if [ ! -z "$APPLE_ID_USER" ]; then
info "Notarizing .app with Apple's central server..."
"${CONTRIB_OSX}/notarize_app.sh" "dist/${PACKAGE}.app" || fail "Could not notarize binary."
else
warn "AppleID details not set! Skipping Apple notarization."
fi
fi
info "Creating .DMG"
hdiutil create -fs HFS+ -volname $PACKAGE -srcfolder dist/$PACKAGE.app dist/electrum-$VERSION.dmg || fail "Could not create .DMG"
DoCodeSignMaybe ".DMG" "dist/electrum-${VERSION}.dmg" "$APP_SIGN" # If APP_SIGN is empty will be a noop
DoCodeSignMaybe ".DMG" "dist/electrum-${VERSION}.dmg"
if [ -z "$APP_SIGN" ]; then
if [ -z "$CODESIGN_CERT" ]; then
warn "App was built successfully but was not code signed. Users may get security warnings from macOS."
warn "Specify a valid code signing identity as the first argument to this script to enable code signing."
warn "Specify a valid code signing identity to enable code signing."
fi

77
contrib/osx/notarize_app.sh

@ -0,0 +1,77 @@
#!/usr/bin/env bash
# from https://github.com/metabrainz/picard/blob/e1354632d2db305b7a7624282701d34d73afa225/scripts/package/macos-notarize-app.sh
if [ -z "$1" ]; then
echo "Specify app bundle as first parameter"
exit 1
fi
if [ -z "$APPLE_ID_USER" ] || [ -z "$APPLE_ID_PASSWORD" ]; then
echo "You need to set your Apple ID credentials with \$APPLE_ID_USER and \$APPLE_ID_PASSWORD."
exit 1
fi
APP_BUNDLE=$(basename "$1")
APP_BUNDLE_DIR=$(dirname "$1")
cd "$APP_BUNDLE_DIR" || exit 1
# Package app for submission
echo "Generating ZIP archive ${APP_BUNDLE}.zip..."
ditto -c -k --rsrc --keepParent "$APP_BUNDLE" "${APP_BUNDLE}.zip"
# Submit for notarization
echo "Submitting $APP_BUNDLE for notarization..."
RESULT=$(xcrun altool --notarize-app --type osx \
--file "${APP_BUNDLE}.zip" \
--primary-bundle-id org.electrum.electrum \
--username $APPLE_ID_USER \
--password @env:APPLE_ID_PASSWORD \
--output-format xml)
if [ $? -ne 0 ]; then
echo "Submitting $APP_BUNDLE failed:"
echo "$RESULT"
exit 1
fi
REQUEST_UUID=$(echo "$RESULT" | xpath \
"//key[normalize-space(text()) = 'RequestUUID']/following-sibling::string[1]/text()" 2> /dev/null)
if [ -z "$REQUEST_UUID" ]; then
echo "Submitting $APP_BUNDLE failed:"
echo "$RESULT"
exit 1
fi
echo "$(echo "$RESULT" | xpath \
"//key[normalize-space(text()) = 'success-message']/following-sibling::string[1]/text()" 2> /dev/null)"
# Poll for notarization status
echo "Submitted notarization request $REQUEST_UUID, waiting for response..."
sleep 60
while :
do
RESULT=$(xcrun altool --notarization-info "$REQUEST_UUID" \
--username "$APPLE_ID_USER" \
--password @env:APPLE_ID_PASSWORD \
--output-format xml)
STATUS=$(echo "$RESULT" | xpath \
"//key[normalize-space(text()) = 'Status']/following-sibling::string[1]/text()" 2> /dev/null)
if [ "$STATUS" = "success" ]; then
echo "Notarization of $APP_BUNDLE succeeded!"
break
elif [ "$STATUS" = "in progress" ]; then
echo "Notarization in progress..."
sleep 20
else
echo "Notarization of $APP_BUNDLE failed:"
echo "$RESULT"
exit 1
fi
done
# Staple the notary ticket
xcrun stapler staple "$APP_BUNDLE"
Loading…
Cancel
Save