And add secp_recovery to headers, while we're at it.
```
./wire/wire.h:7:10: fatal error: secp256k1_recovery.h: No such file or directory
#include <secp256k1_recovery.h>
^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Makefile:254: recipe for target 'connectd/test/run-initiator-success.o' failed
make: *** [connectd/test/run-initiator-success.o] Error 1
m
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
make: *** No rule to make target 'external/gheap/gheap.h', needed by 'bitcoin/chainparams.o'. Stop.
make: *** Waiting for unfinished jobs....
```
Just simplify the Makefile to make all the external headers we use
dependent on submodcheck.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Includes:
psbt: Use renamed functions for new wally version
psbt: Set the transaction directly to avoid script workarounds
psbt: Use low-S grinding when computing signatures
tx: Use wally_tx_clone from libwally now that its exported
Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
Quote marks are not special to make: as it can't find
external/"x86_64-linux-gnu"/libwally-core-build/src/libwallycore.la
it always insists on rebuilding it (which rebuilds the world).
If we have spaces in TARGET_DIR, we're in trouble already.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None
Without this gcc 4.8 defaults to an older standard and cannot compile
the code:
```
gcc-4.8 -DHAVE_CONFIG_H -I. -I../../libwally-core/src -I../../libwally-core -I../../libwally-core/src/ccan -DWALLY_CORE_BUILD=1 -Wall -Wextra -Wpedantic -Wcast-align -Wnested-externs -Wshadow -Wstrict-prototypes -Wformat -Wformat-security -Wformat-nonliteral -O0 -ggdb -DBUILD_ELEMENTS=1 -flax-vector-conversions -Wno-unused-function -Wno-long-long -Wno-overlength-strings -Wno-variadic-macros -fvisibility=hidden -g -O2 -MT libwallycore_la-elements.lo -MD -MP -MF .deps/libwallycore_la-elements.Tpo -c ../../libwally-core/src/elements.c -o libwallycore_la-elements.o
../../libwally-core/src/elements.c: In function ‘wally_asset_pak_whitelistproof’:
../../libwally-core/src/elements.c:629:5: error: ‘for’ loop initial declarations are only allowed in C99 mode
for (size_t i = 0; i < num_keys; ++i) {
^
../../libwally-core/src/elements.c:629:5: note: use option -std=c99 or -std=gnu99 to compile your code
```
Changelog-None
We specify `PYTHON_VERSION=3` to prevent libwally's ./configure from searchin
for python2, which some distros have started removing, and we were requiring
it only for the configuration step anyway.
Changelog-Changed: dependencies: We no longer depend on python2 which has reached end-of-life
On CI it is nice to cache the external dependencies. However if we
always compile them in the same folder we cannot cache for multiple
different architectures. After this commit native compile targets will
still live in `external` but cross compiled versions will live in
`external/<arch>`.
Requesting them is done by setting BUILD=<target_arch>
Otherwise autotools (used by external dependencies like
libsodium) is not happy with setting cross-compilation
variables and may possibly lead to unexpected results.
On master I get a crash and the following from valgrind, which
seems like a libwally bug. Upgrading made it go away.
==45076== Invalid read of size 1
==45076== at 0x1191A1: topo_add_utxos (chaintopology.c:657)
==45076== by 0x11935F: add_tip (chaintopology.c:681)
==45076== by 0x1197E6: have_new_block (chaintopology.c:761)
==45076== by 0x114BB0: process_rawblock (bitcoind.c:480)
==45076== by 0x1140A0: bcli_finished (bitcoind.c:227)
==45076== by 0x1A4CA2: destroy_conn (poll.c:244)
==45076== by 0x1A4CC2: destroy_conn_close_fd (poll.c:250)
==45076== by 0x1B28CD: notify (tal.c:235)
==45076== by 0x1B2DBC: del_tree (tal.c:397)
==45076== by 0x1B3148: tal_free (tal.c:481)
==45076== by 0x1A34C1: io_close (io.c:450)
==45076== by 0x1A53DD: io_loop (poll.c:449)
==45076== Address 0x59fc658 is 4 bytes after a block of size 4 alloc'd
==45076== at 0x483874F: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==45076== by 0x1BD3A5: tx_from_bytes (in /home/rusty/lightning-ltest/lightningd/lightningd)
==45076== by 0x1715C3: pull_bitcoin_tx (tx.c:411)
==45076== by 0x16B25A: bitcoin_block_from_hex (block.c:54)
==45076== by 0x114B45: process_rawblock (bitcoind.c:473)
==45076== by 0x1140A0: bcli_finished (bitcoind.c:227)
==45076== by 0x1A4CA2: destroy_conn (poll.c:244)
==45076== by 0x1A4CC2: destroy_conn_close_fd (poll.c:250)
==45076== by 0x1B28CD: notify (tal.c:235)
==45076== by 0x1B2DBC: del_tree (tal.c:397)
==45076== by 0x1B3148: tal_free (tal.c:481)
==45076== by 0x1A34C1: io_close (io.c:450)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
* remove libbase58, use base58 from libwally
This removes libbase58 and uses libwally instead.
It allocates and then frees some memory, we may want to
add a function in wally that doesn't or override
wally_operations to use tal.
Signed-off-by: Lawrence Nahum lawrence@greenaddress.it
This fixes block parsing on testnet; specifically, non-standard tx versions.
We hit a type bug in libwally (wallt_get_secp_context()) which I had to
work around for the moment, and the updated libsecp adds an optional hash
function arg to the ECDH function.
Fixes: #2563
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Also one less headache for reproducible builds. But unlike
libsodium, this only seems common in Ubuntu.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
jsmn would accept invalid JSON objects. This is bad because it would
set ->size incorrectly: we expect to have at least size * 2 tokens (in
pairs). We want to rely on ->size, but this would create an exploitable
buffer overflow!
Fortunately, this is fixed upstream, so we add a test and upgrade to v1.0.0.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we change an upstream URL, all submodules break. Users would need
to run 'git submodule sync'. Note that the libbacktrace fix was merged
upstream so this is no longer necessary, but it's good for future changes.
Also, stress-testing reveals that git submodule fails locking
'.git/config' when run in paralell. It also segfaults and other
problems.
This is my final attempt to fix submodules; I've wasted far too many
days on obscure problems it creates: I've already lost one copy of my
repo to apparently unfixable submodule preoblems. The next "fix" will
be to simply import the source code so it works properly.
Reported-by: @jsarenikFixes: #1543
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Proposed fix for issue #1231 - FreeBSD (tcsh) build problem due
to HOST environment variable.
The variable is used for cross-compilation. The process may be
improved even further in the future. So far this hot fix.
This is how the error before the fix looked like on Arch Linux with
GNU libtool 2.4.6.40-6ca5-dirty:
$ make installcheck
...
make[4]: Entering directory '/home/jasan/lightning/external/libwally-core/src/secp256k1'
/usr/bin/mkdir -p 'testinstall/home/jasan/lightning/external/libwally-core/..'
/bin/sh ./libtool --mode=install /usr/bin/install -c libsecp256k1.la 'testinstall/home/jasan/lightning/external/libwally-core/..'
Usage: /home/jasan/lightning/external/libwally-core/src/secp256k1/libtool [OPTION]... [MODE-ARG]...
Try 'libtool --help' for more information.
libtool: error: 'testinstall/home/jasan/lightning/external/libwally-core/..' must be an absolute directory name
make[4]: *** [Makefile:910: install-libLTLIBRARIES] Error 1
make[4]: Leaving directory '/home/jasan/lightning/external/libwally-core/src/secp256k1'
make[3]: *** [Makefile:1253: install-exec-recursive] Error 1
make[3]: Leaving directory '/home/jasan/lightning/external/libwally-core/src'
make[2]: *** [Makefile:429: install-exec-recursive] Error 1
make[2]: Leaving directory '/home/jasan/lightning/external/libwally-core'
make[1]: *** [external/Makefile:41: external/libwallycore.a] Error 2
rm external/libwally-core/src/secp256k1/libsecp256k1.la
make[1]: Leaving directory '/home/jasan/lightning'
make: *** [Makefile:430: installcheck] Error 2
Then there was linking error which needed to be fixed by correcting
the idea of installing external libraries to a particular directory.
We can't rely on the existence or not of a header, as Carl's
delete-and-make-submodule change showed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This doesn't cover external libs in general (which *could* be effected by
CFLAGS), but at least all our own objects are rebuilt.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell
niftynei
Jon Griffiths
Wladimir J. van der Laan
Niklas Claesson
Dr. Maxim Orlovsky
Vasil Dimov
Christian Decker
Ján Sáreník
GreenAddress
Lawrence Nahum
Jan Sarenik
Jan Sarenik
conanoc
Igor Cota
Carl Dong
rvandermeer