Browse Source

mac build: conform to macOS 10.15 Gatekeeper requirements

fixes #6128

some of this is based on:
e1354632d2/scripts/package/macos-notarize-app.sh
1eb8b71e7d
24e44e9784
5abec73eee
master
SomberNight 5 years ago
parent
commit
095464b620
No known key found for this signature in database GPG Key ID: B33B5F232C6271E9
  1. 63
      contrib/osx/README.md
  2. 23
      contrib/osx/base.sh
  3. 19
      contrib/osx/entitlements.plist
  4. 62
      contrib/osx/make_osx
  5. 77
      contrib/osx/notarize_app.sh

63
contrib/osx/README.md

@ -1,5 +1,5 @@
Building Mac OS binaries Building macOS binaries
======================== =======================
✗ _This script does not produce reproducible output (yet!). ✗ _This script does not produce reproducible output (yet!).
Please help us remedy this._ Please help us remedy this._
@ -7,36 +7,47 @@ Building Mac OS binaries
This guide explains how to build Electrum binaries for macOS systems. This guide explains how to build Electrum binaries for macOS systems.
## 1. Building the binary ## Building the binary
This needs to be done on a system running macOS or OS X. We use El Capitan (10.11.6) as building it This needs to be done on a system running macOS or OS X.
on High Sierra (or later)
makes the binaries [incompatible with older versions](https://github.com/pyinstaller/pyinstaller/issues/1191).
Another factor for the minimum supported macOS version is the Notes about compatibility with different macOS versions:
[bundled Qt version](https://github.com/spesmilo/electrum/issues/3685). - In general the binary is not guaranteed to run on an older version of macOS
than what the build machine has. This is due to bundling the compiled Python into
the [PyInstaller binary](https://github.com/pyinstaller/pyinstaller/issues/1191).
- The [bundled version of Qt](https://github.com/spesmilo/electrum/issues/3685) also
imposes a minimum supported macOS version.
- If you want to build binaries that conform to the macOS "Gatekeeper", so as to
minimise the warnings users get, the binaries need to be codesigned with a
certificate issued by Apple, and starting with macOS 10.15 the binaries also
need to be notarized by Apple's central server. The catch is that to be able to build
binaries that Apple will notarise (due to the requirements on the binaries themselves,
e.g. hardened runtime) the build machine needs at least macOS 10.14.
See [#6128](https://github.com/spesmilo/electrum/issues/6128).
We currently build the release binaries on macOS 10.14.6, and these seem to run on
10.13 or newer.
Before starting, make sure that the Xcode command line tools are installed (e.g. you have `git`). Before starting, make sure that the Xcode command line tools are installed (e.g. you have `git`).
#### 1.1a Get Xcode #### 1.a Get Xcode
Building the QR scanner (CalinsQRReader) requires full Xcode (not just command line tools). Building the QR scanner (CalinsQRReader) requires full Xcode (not just command line tools).
The last Xcode version compatible with El Capitan is Xcode 8.2.1
Get it from [here](https://developer.apple.com/download/more/). Get it from [here](https://developer.apple.com/download/more/).
Unfortunately, you need an "Apple ID" account. Unfortunately, you need an "Apple ID" account.
(note: the last Xcode that runs on macOS 10.14.6 is Xcode 11.3.1)
After downloading, uncompress it. After downloading, uncompress it.
Make sure it is the "selected" xcode (e.g.): Make sure it is the "selected" xcode (e.g.):
sudo xcode-select -s $HOME/Downloads/Xcode.app/Contents/Developer/ sudo xcode-select -s $HOME/Downloads/Xcode.app/Contents/Developer/
#### 1.1b Build QR scanner separately on newer Mac #### 1.b Build QR scanner separately on another Mac
Alternatively, you can try building just the QR scanner on newer macOS. Alternatively, you can try building just the QR scanner on another Mac.
On newer Mac, run: On newer Mac, run:
@ -46,27 +57,17 @@ On newer Mac, run:
Move `prebuilt_qr` to El Capitan: `contrib/osx/CalinsQRReader/prebuilt_qr`. Move `prebuilt_qr` to El Capitan: `contrib/osx/CalinsQRReader/prebuilt_qr`.
#### 1.2 Build Electrum #### 2. Build Electrum
cd electrum cd electrum
./contrib/osx/make_osx ./contrib/osx/make_osx
This creates both a folder named Electrum.app and the .dmg file. This creates both a folder named Electrum.app and the .dmg file.
If you want the binaries codesigned for MacOS and notarised by Apple's central server,
provide these env vars to the `make_osx` script:
## 2. Building the image deterministically (WIP) CODESIGN_CERT="Developer ID Application: Electrum Technologies GmbH (L6P37P7P56)" \
The usual way to distribute macOS applications is to use image files containing the APPLE_ID_USER="me@email.com" \
application. Although these images can be created on a Mac with the built-in `hdiutil`, APPLE_ID_PASSWORD="1234" \
they are not deterministic. ./contrib/osx/make_osx
Instead, we use the toolchain that Bitcoin uses: genisoimage and libdmg-hfsplus.
These tools do not work on macOS, so you need a separate Linux machine (or VM).
Copy the Electrum.app directory over and install the dependencies, e.g.:
apt install libcap-dev cmake make gcc faketime
Then you can just invoke `package.sh` with the path to the app:
cd electrum
./contrib/osx/package.sh ~/Electrum.app/

23
contrib/osx/base.sh

@ -1,23 +0,0 @@
#!/usr/bin/env bash
. $(dirname "$0")/../build_tools_util.sh
function DoCodeSignMaybe { # ARGS: infoName fileOrDirName codesignIdentity
infoName="$1"
file="$2"
identity="$3"
deep=""
if [ -z "$identity" ]; then
# we are ok with them not passing anything; master script calls us unconditionally even if no identity is specified
return
fi
if [ -d "$file" ]; then
deep="--deep"
fi
if [ -z "$infoName" ] || [ -z "$file" ] || [ -z "$identity" ] || [ ! -e "$file" ]; then
fail "Argument error to internal function DoCodeSignMaybe()"
fi
info "Code signing ${infoName}..."
codesign -f -v $deep -s "$identity" "$file" || fail "Could not code sign ${infoName}"
}

19
contrib/osx/entitlements.plist

@ -0,0 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<!-- These are required for binaries built by PyInstaller -->
<!-- see pyinstaller/pyinstaller#4629 -->
<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
<true/>
<key>com.apple.security.cs.disable-library-validation</key>
<true/>
<!-- These are required for USB HID access (hw wallets). -->
<!-- see https://github.com/Electron-Cash/Electron-Cash/commit/5abec73eee0cdeb725e3c5a989621ec4ccfb92a0 -->
<key>com.apple.security.cs.allow-dyld-environment-variables</key>
<true/>
<key>com.apple.security.cs.allow-jit</key>
<true/>
</dict>
</plist>

62
contrib/osx/make_osx

@ -5,11 +5,12 @@ PYTHON_VERSION=3.7.6
BUILDDIR=/tmp/electrum-build BUILDDIR=/tmp/electrum-build
PACKAGE=Electrum PACKAGE=Electrum
GIT_REPO=https://github.com/spesmilo/electrum GIT_REPO=https://github.com/spesmilo/electrum
LIBSECP_VERSION="b408c6a8b287003d1ade5709e6f7bc3c7f1d5be7"
export GCC_STRIP_BINARIES="1" export GCC_STRIP_BINARIES="1"
. $(dirname "$0")/base.sh
. $(dirname "$0")/../build_tools_util.sh
CONTRIB_OSX="$(dirname "$(realpath "$0")")" CONTRIB_OSX="$(dirname "$(realpath "$0")")"
CONTRIB="$CONTRIB_OSX/.." CONTRIB="$CONTRIB_OSX/.."
@ -24,26 +25,46 @@ which brew > /dev/null 2>&1 || fail "Please install brew from https://brew.sh/ t
which xcodebuild > /dev/null 2>&1 || fail "Please install Xcode and xcode command line tools to continue" which xcodebuild > /dev/null 2>&1 || fail "Please install Xcode and xcode command line tools to continue"
# Code Signing: See https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html # Code Signing: See https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html
APP_SIGN="" if [ -n "$CODESIGN_CERT" ]; then
if [ -n "$1" ]; then
# Test the identity is valid for signing by doing this hack. There is no other way to do this. # Test the identity is valid for signing by doing this hack. There is no other way to do this.
cp -f /bin/ls ./CODESIGN_TEST cp -f /bin/ls ./CODESIGN_TEST
codesign -s "$1" --dryrun -f ./CODESIGN_TEST > /dev/null 2>&1 codesign -s "$CODESIGN_CERT" --dryrun -f ./CODESIGN_TEST > /dev/null 2>&1
res=$? res=$?
rm -f ./CODESIGN_TEST rm -f ./CODESIGN_TEST
if ((res)); then if ((res)); then
fail "Code signing identity \"$1\" appears to be invalid." fail "Code signing identity \"$CODESIGN_CERT\" appears to be invalid."
fi fi
unset res unset res
APP_SIGN="$1" info "Code signing enabled using identity \"$CODESIGN_CERT\""
info "Code signing enabled using identity \"$APP_SIGN\""
else else
warn "Code signing DISABLED. Specify a valid macOS Developer identity installed on the system as the first argument to this script to enable signing." warn "Code signing DISABLED. Specify a valid macOS Developer identity installed on the system to enable signing."
fi fi
function DoCodeSignMaybe { # ARGS: infoName fileOrDirName
infoName="$1"
file="$2"
deep=""
if [ -z "$CODESIGN_CERT" ]; then
# no cert -> we won't codesign
return
fi
if [ -d "$file" ]; then
deep="--deep"
fi
if [ -z "$infoName" ] || [ -z "$file" ] || [ ! -e "$file" ]; then
fail "Argument error to internal function DoCodeSignMaybe()"
fi
hardened_arg="--entitlements=${CONTRIB_OSX}/entitlements.plist -o runtime"
info "Code signing ${infoName}..."
codesign -f -v $deep -s "$CODESIGN_CERT" $hardened_arg "$file" || fail "Could not code sign ${infoName}"
}
info "Installing Python $PYTHON_VERSION" info "Installing Python $PYTHON_VERSION"
export PATH="~/.pyenv/bin:~/.pyenv/shims:~/Library/Python/3.7/bin:$PATH" export PATH="~/.pyenv/bin:~/.pyenv/shims:~/Library/Python/3.7/bin:$PATH"
if [ -d "~/.pyenv" ]; then if [ -d "${HOME}/.pyenv" ]; then
pyenv update pyenv update
else else
curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash > /dev/null 2>&1 curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash > /dev/null 2>&1
@ -109,7 +130,7 @@ rm -fr build
# prefer building using xcode ourselves. otherwise fallback to prebuilt binary # prefer building using xcode ourselves. otherwise fallback to prebuilt binary
xcodebuild || cp -r prebuilt_qr build || fail "Could not build CalinsQRReader" xcodebuild || cp -r prebuilt_qr build || fail "Could not build CalinsQRReader"
popd popd
DoCodeSignMaybe "CalinsQRReader.app" "${d}/build/Release/CalinsQRReader.app" "$APP_SIGN" # If APP_SIGN is empty will be a noop DoCodeSignMaybe "CalinsQRReader.app" "${d}/build/Release/CalinsQRReader.app"
info "Installing requirements..." info "Installing requirements..."
@ -131,7 +152,7 @@ for d in ~/Library/Python/ ~/.pyenv .; do
done done
info "Building binary" info "Building binary"
APP_SIGN="$APP_SIGN" pyinstaller --noconfirm --ascii --clean --name $VERSION contrib/osx/osx.spec || fail "Could not build binary" APP_SIGN="$CODESIGN_CERT" pyinstaller --noconfirm --ascii --clean --name $VERSION contrib/osx/osx.spec || fail "Could not build binary"
info "Adding bitcoin URI types to Info.plist" info "Adding bitcoin URI types to Info.plist"
plutil -insert 'CFBundleURLTypes' \ plutil -insert 'CFBundleURLTypes' \
@ -139,14 +160,23 @@ plutil -insert 'CFBundleURLTypes' \
-- dist/$PACKAGE.app/Contents/Info.plist \ -- dist/$PACKAGE.app/Contents/Info.plist \
|| fail "Could not add keys to Info.plist. Make sure the program 'plutil' exists and is installed." || fail "Could not add keys to Info.plist. Make sure the program 'plutil' exists and is installed."
DoCodeSignMaybe "app bundle" "dist/${PACKAGE}.app" "$APP_SIGN" # If APP_SIGN is empty will be a noop DoCodeSignMaybe "app bundle" "dist/${PACKAGE}.app"
if [ ! -z "$CODESIGN_CERT" ]; then
if [ ! -z "$APPLE_ID_USER" ]; then
info "Notarizing .app with Apple's central server..."
"${CONTRIB_OSX}/notarize_app.sh" "dist/${PACKAGE}.app" || fail "Could not notarize binary."
else
warn "AppleID details not set! Skipping Apple notarization."
fi
fi
info "Creating .DMG" info "Creating .DMG"
hdiutil create -fs HFS+ -volname $PACKAGE -srcfolder dist/$PACKAGE.app dist/electrum-$VERSION.dmg || fail "Could not create .DMG" hdiutil create -fs HFS+ -volname $PACKAGE -srcfolder dist/$PACKAGE.app dist/electrum-$VERSION.dmg || fail "Could not create .DMG"
DoCodeSignMaybe ".DMG" "dist/electrum-${VERSION}.dmg" "$APP_SIGN" # If APP_SIGN is empty will be a noop DoCodeSignMaybe ".DMG" "dist/electrum-${VERSION}.dmg"
if [ -z "$APP_SIGN" ]; then if [ -z "$CODESIGN_CERT" ]; then
warn "App was built successfully but was not code signed. Users may get security warnings from macOS." warn "App was built successfully but was not code signed. Users may get security warnings from macOS."
warn "Specify a valid code signing identity as the first argument to this script to enable code signing." warn "Specify a valid code signing identity to enable code signing."
fi fi

77
contrib/osx/notarize_app.sh

@ -0,0 +1,77 @@
#!/usr/bin/env bash
# from https://github.com/metabrainz/picard/blob/e1354632d2db305b7a7624282701d34d73afa225/scripts/package/macos-notarize-app.sh
if [ -z "$1" ]; then
echo "Specify app bundle as first parameter"
exit 1
fi
if [ -z "$APPLE_ID_USER" ] || [ -z "$APPLE_ID_PASSWORD" ]; then
echo "You need to set your Apple ID credentials with \$APPLE_ID_USER and \$APPLE_ID_PASSWORD."
exit 1
fi
APP_BUNDLE=$(basename "$1")
APP_BUNDLE_DIR=$(dirname "$1")
cd "$APP_BUNDLE_DIR" || exit 1
# Package app for submission
echo "Generating ZIP archive ${APP_BUNDLE}.zip..."
ditto -c -k --rsrc --keepParent "$APP_BUNDLE" "${APP_BUNDLE}.zip"
# Submit for notarization
echo "Submitting $APP_BUNDLE for notarization..."
RESULT=$(xcrun altool --notarize-app --type osx \
--file "${APP_BUNDLE}.zip" \
--primary-bundle-id org.electrum.electrum \
--username $APPLE_ID_USER \
--password @env:APPLE_ID_PASSWORD \
--output-format xml)
if [ $? -ne 0 ]; then
echo "Submitting $APP_BUNDLE failed:"
echo "$RESULT"
exit 1
fi
REQUEST_UUID=$(echo "$RESULT" | xpath \
"//key[normalize-space(text()) = 'RequestUUID']/following-sibling::string[1]/text()" 2> /dev/null)
if [ -z "$REQUEST_UUID" ]; then
echo "Submitting $APP_BUNDLE failed:"
echo "$RESULT"
exit 1
fi
echo "$(echo "$RESULT" | xpath \
"//key[normalize-space(text()) = 'success-message']/following-sibling::string[1]/text()" 2> /dev/null)"
# Poll for notarization status
echo "Submitted notarization request $REQUEST_UUID, waiting for response..."
sleep 60
while :
do
RESULT=$(xcrun altool --notarization-info "$REQUEST_UUID" \
--username "$APPLE_ID_USER" \
--password @env:APPLE_ID_PASSWORD \
--output-format xml)
STATUS=$(echo "$RESULT" | xpath \
"//key[normalize-space(text()) = 'Status']/following-sibling::string[1]/text()" 2> /dev/null)
if [ "$STATUS" = "success" ]; then
echo "Notarization of $APP_BUNDLE succeeded!"
break
elif [ "$STATUS" = "in progress" ]; then
echo "Notarization in progress..."
sleep 20
else
echo "Notarization of $APP_BUNDLE failed:"
echo "$RESULT"
exit 1
fi
done
# Staple the notary ticket
xcrun stapler staple "$APP_BUNDLE"
Loading…
Cancel
Save