|
|
|
|
#!/bin/bash
|
|
|
|
|
# This script installs our pure python dependencies into the 'packages' folder.
|
|
|
|
|
|
|
|
|
|
set -e
|
|
|
|
|
|
|
|
|
|
CONTRIB="$(dirname "$(readlink -e "$0")")"
|
|
|
|
|
PROJECT_ROOT="$CONTRIB"/..
|
|
|
|
|
PACKAGES="$PROJECT_ROOT"/packages/
|
|
|
|
|
|
|
|
|
|
test -n "$CONTRIB" -a -d "$CONTRIB" || exit
|
|
|
|
|
|
|
|
|
|
if [ -d "$PACKAGES" ]; then
|
|
|
|
|
rm -r "$PACKAGES"
|
|
|
|
|
fi
|
|
|
|
|
|
build: android reprod: "pip install" needs "--no-build-isolation"
maybe fixes https://github.com/spesmilo/electrum/issues/7640
Looks like by default pip is ignoring the locally available setuptools and wheel,
and downloading the latest ones from the internet at build time...
https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/?highlight=no-build-isolation#disabling-build-isolation
https://stackoverflow.com/a/62889268
> When making build requirements available, pip does so in an isolated environment. That is, pip does not install those requirements into the user’s site-packages, but rather installs them in a temporary directory which it adds to the user’s sys.path for the duration of the build. This ensures that build requirements are handled independently of the user’s runtime environment. For example, a project that needs a recent version of setuptools to build can still be installed, even if the user has an older version installed (and without silently replacing that version).
>
> In certain cases, projects (or redistributors) may have workflows that explicitly manage the build environment. For such workflows, build isolation can be problematic. If this is the case, pip provides a --no-build-isolation flag to disable build isolation. Users supplying this flag are responsible for ensuring the build environment is managed appropriately (including ensuring that all required build dependencies are installed).
If only it were that easy!
If we add the "--no-build-isolation" flag, it becomes our responsibility to install *all* build time deps,
hence we now have "requirements-build-makepackages.txt".
3 years ago
|
|
|
# create virtualenv
|
|
|
|
|
# note: venv path needs to be deterministic as some produced files will contain it
|
|
|
|
|
venv_dir="$CONTRIB/.venv_make_packages/"
|
|
|
|
|
rm -rf "$venv_dir"
|
|
|
|
|
python3 -m venv "$venv_dir"
|
|
|
|
|
source "$venv_dir"/bin/activate
|
|
|
|
|
|
|
|
|
|
# installing pinned build-time requirements, such as pip/wheel/setuptools
|
|
|
|
|
python -m pip install --no-build-isolation --no-dependencies --no-warn-script-location \
|
|
|
|
|
-r "$CONTRIB"/deterministic-build/requirements-build-base.txt
|
build: android reprod: "pip install" needs "--no-build-isolation"
maybe fixes https://github.com/spesmilo/electrum/issues/7640
Looks like by default pip is ignoring the locally available setuptools and wheel,
and downloading the latest ones from the internet at build time...
https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/?highlight=no-build-isolation#disabling-build-isolation
https://stackoverflow.com/a/62889268
> When making build requirements available, pip does so in an isolated environment. That is, pip does not install those requirements into the user’s site-packages, but rather installs them in a temporary directory which it adds to the user’s sys.path for the duration of the build. This ensures that build requirements are handled independently of the user’s runtime environment. For example, a project that needs a recent version of setuptools to build can still be installed, even if the user has an older version installed (and without silently replacing that version).
>
> In certain cases, projects (or redistributors) may have workflows that explicitly manage the build environment. For such workflows, build isolation can be problematic. If this is the case, pip provides a --no-build-isolation flag to disable build isolation. Users supplying this flag are responsible for ensuring the build environment is managed appropriately (including ensuring that all required build dependencies are installed).
If only it were that easy!
If we add the "--no-build-isolation" flag, it becomes our responsibility to install *all* build time deps,
hence we now have "requirements-build-makepackages.txt".
3 years ago
|
|
|
|
|
|
|
|
# opt out of compiling C extensions
|
|
|
|
|
# FIXME aiohttp opt-out is not released yet: https://github.com/aio-libs/aiohttp/pull/3828
|
|
|
|
|
export AIOHTTP_NO_EXTENSIONS=1
|
|
|
|
|
export YARL_NO_EXTENSIONS=1
|
|
|
|
|
export MULTIDICT_NO_EXTENSIONS=1
|
|
|
|
|
|
|
|
|
|
# if we end up having to compile something, at least give reproducibility a fighting chance
|
|
|
|
|
export LC_ALL=C
|
|
|
|
|
export TZ=UTC
|
|
|
|
|
export SOURCE_DATE_EPOCH="$(git log -1 --pretty=%ct)"
|
|
|
|
|
export PYTHONHASHSEED="$SOURCE_DATE_EPOCH"
|
|
|
|
|
export BUILD_DATE="$(LC_ALL=C TZ=UTC date +'%b %e %Y' -d @$SOURCE_DATE_EPOCH)"
|
|
|
|
|
export BUILD_TIME="$(LC_ALL=C TZ=UTC date +'%H:%M:%S' -d @$SOURCE_DATE_EPOCH)"
|
|
|
|
|
|
|
|
|
|
# FIXME aiohttp will compile some .so files using distutils
|
|
|
|
|
# (until https://github.com/aio-libs/aiohttp/pull/4079 gets released),
|
|
|
|
|
# which are not reproducible unless using at least python 3.9
|
|
|
|
|
# (as it needs https://github.com/python/cpython/commit/0d30ae1a03102de07758650af9243fd31211325a).
|
|
|
|
|
# Hence "aiohttp-*.dist-info/" is not reproducible either.
|
|
|
|
|
# All this means that downstream users of this script, such as the sdist build
|
|
|
|
|
# and the android apk build need to make sure these files get excluded.
|
build: android reprod: "pip install" needs "--no-build-isolation"
maybe fixes https://github.com/spesmilo/electrum/issues/7640
Looks like by default pip is ignoring the locally available setuptools and wheel,
and downloading the latest ones from the internet at build time...
https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/?highlight=no-build-isolation#disabling-build-isolation
https://stackoverflow.com/a/62889268
> When making build requirements available, pip does so in an isolated environment. That is, pip does not install those requirements into the user’s site-packages, but rather installs them in a temporary directory which it adds to the user’s sys.path for the duration of the build. This ensures that build requirements are handled independently of the user’s runtime environment. For example, a project that needs a recent version of setuptools to build can still be installed, even if the user has an older version installed (and without silently replacing that version).
>
> In certain cases, projects (or redistributors) may have workflows that explicitly manage the build environment. For such workflows, build isolation can be problematic. If this is the case, pip provides a --no-build-isolation flag to disable build isolation. Users supplying this flag are responsible for ensuring the build environment is managed appropriately (including ensuring that all required build dependencies are installed).
If only it were that easy!
If we add the "--no-build-isolation" flag, it becomes our responsibility to install *all* build time deps,
hence we now have "requirements-build-makepackages.txt".
3 years ago
|
|
|
# note: --no-build-isolation is needed so that pip uses the locally available setuptools and wheel,
|
|
|
|
|
# instead of downloading the latest ones
|
|
|
|
|
python3 -m pip install --no-build-isolation --no-compile --no-dependencies --no-binary :all: \
|
|
|
|
|
-r "$CONTRIB"/deterministic-build/requirements.txt -t "$CONTRIB"/../packages
|
