common/onion is going to need to use this for the case where it finds a blinding
seed inside the TLV. But how it does ecdh is daemon-specific.
We already had this problem for devtools/gossipwith, which supplied a
special hsm_do_ecdh(). This just makes it more general.
So we create a generic ecdh() interface, with a specific implementation
which subdaemons and lightningd can use.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
e.g.
$ PUBKEY1=0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
$ PRIVKEY1=41bfd2660762506c9933ade59f1debf7e6495b10c14a92dbcd2d623da2507d3d
$ PUBKEY2=022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59
$ PRIVKEY1=c4a813f81ffdca1da6864db81795ad2d320add274452cafa1fb2ac2d07d062bd
# First line is blinding, second is contents and nodeids for onion.
$ ./devtools/blindedpath create $PUBKEY1 $PUBKEY2
03f006a18d5653c4edf5391ff23a61f03ff83d237e880ee61187fa9f379a028e0a
0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518/350633c340f28bc69cbc86f568b7b9e99fa41eb581452d066fcd70dd53c43ace14d034eebfbe472a2b9901b11c268d2cc2034a77928a 0326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911/00
# Generate the onion
$ /devtools/onion generate 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518/350633c340f28bc69cbc86f568b7b9e99fa41eb581452d066fcd70dd53c43ace14d034eebfbe472a2b9901b11c268d2cc2034a77928a 0326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911/00 > /tmp/onion.dat
# First node unwraps it, gives next blinding and onion
$ ./devtools/blindedpath --first-node unwrap $PRIVKEY1 `cat /tmp/onion.dat` 03f006a18d5653c4edf5391ff23a61f03ff83d237e880ee61187fa9f379a028e0a
Contents: 04210326f31ff78e584461420e5026fe72374af2ef853e65c47a3f2406348b7c6c0911
Next blinding: 021295ce94fcadc42c3e5187a12dd80122214c8f9da61635163cddb63282f1ee9b
Next onion: 0003c8fa9d4997ebd049480db14f0a90db211ec2b9f062e569419ee2c2b528a2d7adc63aa9e7b74997c2c122c4c1923e2f4587fc62532965666dbe55a76aa1ea903cfe6c498c6b7e80c14609d3c0f98f86a30f94b0b8a405067ee3801aab03420922cdc847d6f2fc359655408047a8d7d7892a595f630cdf114d1cc0d1164dc8099378042cfba7a13711dff64000356aac29726a6039bf938b81cc0dcc7f65dc126ae58838c0446d6492b6381f0402a33318a9ae71486bcb4b58f66c9a056fca306668655c11f7d7d0cd447e4162100565369629ca8b705b2b999a40ad5493953cb70b35f382e6acdc04e5a933783f9c5859fb0beeaa9c54e5220f5de3b107813d33148501aebcf67e190d3dcf10553714d4de8a1643b519cd124da9a345e2da0d669954a10fce9c1e7795572fb2ac8fe6de4db856bdbd327f0c4ae3cb11f6e1422f663423ad57891d069bbc5bdc7613c742a2227d3789d9039dcdcbddc2703835002dd176004c56cc497d88deae2328ad1376877f4582c71a7fa1eba4ae4e6696782bd97d7b362f41e81335b47273a74c983c3bc80499069a08c6b7ffa32cc77f54a98d8bc2f80f38c370c98edd8f6d6f95c6bbf5c8040296f68560de3b50c3450de4fcceae41469bea6a24c83141c92956fa4c4087f11e9c26b282e9c3974fadc8f9bb9fb9c3fbd2ead0cf4ceaba452eb8791828a159ace7a2e1e8ff5c69704a821c6c898a3c38439149862f14b7ed34afb93640c3ae61b089011ef698c9e26dc16b8a5a8ec66fce702b7bdf04a21cf9a2ebff6b89b29904e7e3e6a98088c2848951c0d6915249a3e1199c2affe4635ac6ef3a16ddfefc5790435a14067b24d5ecd16a26d2f7dbd8065b8e5b86f73f878cf55ca0c9f12104d861d03ac5c4b3dcbd0e30ad7ba888ddcc2e89acb3c04be2ef4bce3ef8c8878fef5be65664f1ec288f91dbd1748e2e53bcbd7dc9cecd75a246992e76a844ab122c5e179f97531190c7d91586289410c9ddad33eb156ab7312c82e55f3d643f3c12468ce79f7221051da608dd17ceb235b7df89f3c4b9aa9448bf36206b2db7bb97f544d062d6aa1b1706376fb6e3e8ef1ac293b9adca478458e9e51845dc7b554c70a91e32c331962968f98db26faa5b10a39bc778b0aab5a9fd11fdbab7b456db286049e584b7e4d1c76d6c3c6249b567aa357982ceead2ad8d5113a866818997b24018059e93eb5cafd293942efec3cf4a43bf322cd444e8370dc2cf1f1164c5147b30a791e262acadc15c30f1e169af4bff5e6c098acf95534b0b59517e3168413134984d50c8242590c8729fe34190d20d7f88505747b919e0bf8b41ed4ec9146743339c6885dcb770dcac627a1570dc145f6f61a976d87d16473195d5f5ce39347ab040c34fe0888b498f0ba25686a9bae51f6e5973d15f10d1c7dbe5fbfa2c7fe05cdd52d8eca8150914b4cb2e75e52010bb4b9241160d7337f47bbebd3bea58ddcf320a0464c34dbfd3d3ea7ffaee966c36064b2cc77babbf4613a7d5d65b3931dc42f91ed902207c57509a46738d31671c439d052c16db5ce3e613c5b37f77e574e9a847ebab20159130f33186557d16ddd1d765e7a9adedb253b755128e1af58da9e8b0fe6ed5834ddf1537c0ee78cd9803ee06031c3c1331d405a94a54f06d1147b26d0788179320d882f57ee9c63ab7a76fb9572eb813fe55369b5061b0a242b07a72095754a9d5699534a46829398204e76f1eda1d65e98fab1e8f3cf7c85257c8e2da0546fed215c3db38d231a637fd7a4e6f10b786d15534137489c662a0f289da824ca8dfef335bdcc623d636d231e002cc32febbe6683046ee54702dbeb55a70911505844c79c0be4630cf49456fba55aec9218d3ba449540370d407fc653007fdf59b4711d1da3c2e569eec4de8cee7b05d248ad0
# Feed that onion and blinding to second node
$ ./devtools/blindedpath unwrap $PRIVKEY2 0003c8fa9d4997ebd049480db14f0a90db211ec2b9f062e569419ee2c2b528a2d7adc63aa9e7b74997c2c122c4c1923e2f4587fc62532965666dbe55a76aa1ea903cfe6c498c6b7e80c14609d3c0f98f86a30f94b0b8a405067ee3801aab03420922cdc847d6f2fc359655408047a8d7d7892a595f630cdf114d1cc0d1164dc8099378042cfba7a13711dff64000356aac29726a6039bf938b81cc0dcc7f65dc126ae58838c0446d6492b6381f0402a33318a9ae71486bcb4b58f66c9a056fca306668655c11f7d7d0cd447e4162100565369629ca8b705b2b999a40ad5493953cb70b35f382e6acdc04e5a933783f9c5859fb0beeaa9c54e5220f5de3b107813d33148501aebcf67e190d3dcf10553714d4de8a1643b519cd124da9a345e2da0d669954a10fce9c1e7795572fb2ac8fe6de4db856bdbd327f0c4ae3cb11f6e1422f663423ad57891d069bbc5bdc7613c742a2227d3789d9039dcdcbddc2703835002dd176004c56cc497d88deae2328ad1376877f4582c71a7fa1eba4ae4e6696782bd97d7b362f41e81335b47273a74c983c3bc80499069a08c6b7ffa32cc77f54a98d8bc2f80f38c370c98edd8f6d6f95c6bbf5c8040296f68560de3b50c3450de4fcceae41469bea6a24c83141c92956fa4c4087f11e9c26b282e9c3974fadc8f9bb9fb9c3fbd2ead0cf4ceaba452eb8791828a159ace7a2e1e8ff5c69704a821c6c898a3c38439149862f14b7ed34afb93640c3ae61b089011ef698c9e26dc16b8a5a8ec66fce702b7bdf04a21cf9a2ebff6b89b29904e7e3e6a98088c2848951c0d6915249a3e1199c2affe4635ac6ef3a16ddfefc5790435a14067b24d5ecd16a26d2f7dbd8065b8e5b86f73f878cf55ca0c9f12104d861d03ac5c4b3dcbd0e30ad7ba888ddcc2e89acb3c04be2ef4bce3ef8c8878fef5be65664f1ec288f91dbd1748e2e53bcbd7dc9cecd75a246992e76a844ab122c5e179f97531190c7d91586289410c9ddad33eb156ab7312c82e55f3d643f3c12468ce79f7221051da608dd17ceb235b7df89f3c4b9aa9448bf36206b2db7bb97f544d062d6aa1b1706376fb6e3e8ef1ac293b9adca478458e9e51845dc7b554c70a91e32c331962968f98db26faa5b10a39bc778b0aab5a9fd11fdbab7b456db286049e584b7e4d1c76d6c3c6249b567aa357982ceead2ad8d5113a866818997b24018059e93eb5cafd293942efec3cf4a43bf322cd444e8370dc2cf1f1164c5147b30a791e262acadc15c30f1e169af4bff5e6c098acf95534b0b59517e3168413134984d50c8242590c8729fe34190d20d7f88505747b919e0bf8b41ed4ec9146743339c6885dcb770dcac627a1570dc145f6f61a976d87d16473195d5f5ce39347ab040c34fe0888b498f0ba25686a9bae51f6e5973d15f10d1c7dbe5fbfa2c7fe05cdd52d8eca8150914b4cb2e75e52010bb4b9241160d7337f47bbebd3bea58ddcf320a0464c34dbfd3d3ea7ffaee966c36064b2cc77babbf4613a7d5d65b3931dc42f91ed902207c57509a46738d31671c439d052c16db5ce3e613c5b37f77e574e9a847ebab20159130f33186557d16ddd1d765e7a9adedb253b755128e1af58da9e8b0fe6ed5834ddf1537c0ee78cd9803ee06031c3c1331d405a94a54f06d1147b26d0788179320d882f57ee9c63ab7a76fb9572eb813fe55369b5061b0a242b07a72095754a9d5699534a46829398204e76f1eda1d65e98fab1e8f3cf7c85257c8e2da0546fed215c3db38d231a637fd7a4e6f10b786d15534137489c662a0f289da824ca8dfef335bdcc623d636d231e002cc32febbe6683046ee54702dbeb55a70911505844c79c0be4630cf49456fba55aec9218d3ba449540370d407fc653007fdf59b4711d1da3c2e569eec4de8cee7b05d248ad0 021295ce94fcadc42c3e5187a12dd80122214c8f9da61635163cddb63282f1ee9b
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Header from folded patch 'fixup':
fixup! devtool/blindedpath: primitive tool to make blinded onions.
On decode, don't mess with op.ephemeralkey, since it will be used to derive
the next hop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now that we have json_stream in common/, we can move all the related
helpers from lightningd/json to common/json. This way everyone can
benefit of them (including libplugin, the plugins themselves,
potentially lightning-cli), not lightningd alone!
Note that the Makefile of the common/test/ had to be modified, because
the new helpers make use of common/wireaddr... Which turns out to
\#include <lightingd/lightningd.h> ! So we couldnt just include the .c
and add mocks if we redefined some structs (hello run-param).
This makes it clear we're dealing with a message which is a wrapped error
reply (needing unwrap_onionreply), not an already-wrapped one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is an intermediary step: we still don't save it to the database,
but we do use the fee_states struct to track it internally.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now "raw_payload" is always the complete string (including realm or length
bytes at the front).
This has several effects:
1. We can receive an decrypt an onion which is grossly malformed.
2. We can still hand this to the htlc_accepted hook.
3. We then fail it unless the htlc_accepted accepts it manually.
4. The createonion API now takes the raw payload, and does not know
anything about "style".
The only caveat is that the sphinx code needs to know the payload
length: we have a call for that, which simply tells it to copy the
entire onion (and treat us as the final node) if it's invalid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This encoding scheme is no longer just used for short_channel_ids, so make
the names more generic.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rather than reaching into data structures, let them register their own
callbacks. This avoids us having to expose "memleak_remove_xxx"
functions, and call them manually.
Under the hood, this is done by having a specially-named tal child of
the thing we want to assist, containing the callback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The `runtest` command takes a JSON onion spec, creates the onion and decodes
it with the provided private keys. It is fully configurable and can be used
for the test-vectors in the spec.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
We need to hand -s to both header and body generation, or neither:
wire/gen_peer_wire.c:53:13: error: static declaration of ‘towire_channel_update_timestamps’ follows non-static declaration
In file included from wire/gen_peer_wire.c:5:
./wire/gen_peer_wire.h:78:6: note: previous declaration of ‘towire_channel_update_timestamps’ was here
We also need it for printwire, otherwise we get static unused functions for subtypes:
devtools/gen_print_wire.c:155:13: error: ‘printwire_channel_update_checksums’ defined but not used [-Werror=unused-function]
static void printwire_channel_update_checksums(const char *fieldname, const u8 **cursor, size_t *plen)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
devtools/gen_print_wire.c:133:13: error: ‘printwire_channel_update_timestamps’ defined but not used [-Werror=unused-function]
static void printwire_channel_update_timestamps(const char *fieldname, const u8 **cursor, size_t *plen)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This allows for complete channel simulation, including HTLC
transactions, but means we use higher-level primitives to
make the easy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These utilities allow us to create valid test txs and information given both
sides' complete set of secrets.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
updates the bolt version to 6639cef095a2ecc7b8f0c48c6e7f2f906fbfbc58.
this requires us to use the new bolt parser at generate-bolt.py
and updates to all of the type specifications (ie. from u8 -> byte)
Encapsulating the peer state was a win for lightningd; not surprisingly,
it's even more of a win for the other daemons, especially as we want
to add a little gossip information.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
added sanity check to make sure scid of csv is the same as scid in gossip.
Revised style, mem allocation, and error checks
[ Minor fixups, and updated benchmark script -- RR ]
With data.tar.gz: 456609740 Apr 2 12:33
store_load_msec:35300-42354(37118.2+/-2.7e+03)
vsz_kb:582832
store_rewrite_sec:12.700000-13.430000(12.988+/-0.27)
listnodes_sec:3.000000-3.160000(3.076+/-0.057)
listchannels_sec:30.790000-31.690000(31.03+/-0.34)
routing_sec:0.00
peer_write_all_sec:63.640000-67.860000(66.294+/-1.4)
I tried to just do gossipd, but it was uncontainable, so this ended up being
a complete sweep.
We didn't get much space saving in gossipd, even though we should save
24 bytes per node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
They're generally used pass-by-copy (unusual for C structs, but
convenient they're basically u64) and all possibly problematic
operations return WARN_UNUSED_RESULT bool to make you handle the
over/underflow cases.
The new #include in json.h means we bolt11.c sees the amount.h definition
of MSAT_PER_BTC, so delete its local version.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This requires a tweak to generate-wire.py too, since it always called the
top-level routine 'print_message'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This way the object file correctly depends on external headers. Currently
a parallel build on a clean tree can give:
```
In file included from ./common/sphinx.h:6:0,
from devtools/onion.c:5:
./bitcoin/pubkey.h:8:10: fatal error: secp256k1.h: No such file or directory
#include <secp256k1.h>
^~~~~~~~~~~~~
compilation terminated.
<builtin>: recipe for target 'devtools/onion.o' failed
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
These were so far only used for bolt11 construction, but we'll need them for the
DNS seed as well, so here we just pull them out into their own unit and prefix
them.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Matt Whitlock
Rusty Russell
darosior
Christian Decker
lisa neigut
Joe Netti
ueno