memmem is also O(n^2), though it's faster. Now we have infrastructure,
let's do incremental parsing.
time lightning-cli -R --network=regtest --lightning-dir /tmp/ltests-k8jhvtty/test_pay_stress_1/lightning-1/ listpays > /dev/null
Before:
real 0m13.674s
user 0m0.131s
sys 0m0.024s
After:
real 0m12.447s
user 0m0.143s
sys 0m0.008s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This doesn't make any difference, since lightningd generally sends us
short commands (command responses are via the rpc loop, which is
already done), but it's harmless.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The jsmn parser is a beautiful piece of code. In particular, you can parse
part of a string, then continue where you left off.
We don't take advantage of this, however, meaning for large JSON objects
we parse them multiple times before finally having enough to complete.
Expose the parser state and tokens through the API, so the caller can pass
them in repeatedly. For the moment, every caller is allocates each time
(except the unit tests).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We're going to change the API on the more complete JSON parser, so
make and use a simple API for the easy cases.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Tested on a test node which had made 50,000 payment, with no optimization.
For comparison, time for 'listsendpays' was 0.983s.
time lightning-cli -R --network=regtest --lightning-dir /tmp/ltests-k8jhvtty/test_pay_stress_1/lightning-1/ listpays > /dev/null
Before:
real 0m52.415s
user 0m0.127s
sys 0m0.044s
After:
real 0m42.741s
user 0m0.149s
sys 0m0.016s
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: libplugin: significant speedups for reading large JSON replies (e.g. calling listsendpays on large nodes, or listchannels / listnodes).
Changelog-Fixed: plugin: `bcli` no longer logs a harmless warning about being unable to connect to the JSON-RPC interface.
Changelog-Added: plugin: Plugins can opt out of having an RPC connection automatically initialized on startup.
The documentation was wrong, and I copied my mistake to `libplugin` where it
was then ignored instead of ORed into the node's featurebits. This fixes both.
They now use -fno-common by default, so duplicated variables cause
a link error:
/usr/bin/ld: common/utils.o:(.bss+0x10): multiple definition of `chainparams'; plugins/libplugin.o:(.bss+0x0): first defined here
collect2: error: ld returned 1 exit status
make: *** [Makefile:408: plugins/autoclean] Error 1
This was introduced in 9ebfdf0b8c.
Fixes: #3597
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Multiple definition of chainparams on Fedora (or other really recent gcc)
It's almost always "their_features" and "our_features" respectively, so
make those names clear.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Spark does this, for example:
{"method":"pay","params":["lnbc..."],"id":22}
Which doesn't have a jsonrpc field. The result is that the command
doesn't terminate, there is nothing in the logs, stderr contains
"pay: JSON-RPC message does not contain "jsonrpc" field", and
from then on "Unknown command 'pay'".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We are going to initialize a plugin before its creation, so log as
UNUSUAL instead.
Also, `pay` and `fundchannel` inits are using rpc_delve(), so we need to
io_new_conn() (which sets the socket as non blocking) after calling the
plugin's init.
We don't take the callback result into account, so it can better be void.
Having a general callback parameter is handy, because for bcli we want
to pass it the struct bcli.
As a separated commit because it was pre-existent (changelog + xfail test).
This also fix a logical problem in lightningd/plugin_control: we were
assuming a plugin started with 'plugin start' but which did not comport
a 'dynamic' entry in its manifest to be dynamic, though it should have
been treated as static.
Changelog-fixed: plugins: Dynamic C plugins can now be managed when lightningd is up
This adds helpers to start and send a jsonrpc request using json_stream
in order to benefit from the helpers.
This then simplifies existing plugins RPC requests by using json_stream
helpers.
This pass to json_stream helpers for commands outputs, but keeps
compatibility with existing plugins which use jout as of now, by not
starting/closing the "result"/"error" objects.
Now we have streams and a global object, we can use them for io_plans.
This makes the logic closer from lightningd/plugin (it has been mostly
stolen from here), and simplifies the main.
This also allows plugins to use io_plans (thanks to the io_loop) for
asynchronous connections.
This commit only handle incoming commands.
GCC 10 defaults to `-fno-common`. no longer automatically sharing
global variable definitions, which makes it important to define
them in only one place (otherwise there will be duplicate definition
errors). Add `extern` qualifiers where (I think) is the best place for
them.
Before this patch we used `int` for error codes. The problem with
`int` is that we try to pass it to/from wire and the size of `int` is
not defined by the standard. So a sender with 4-byte `int` would write
4 bytes to the wire and a receiver with 2-byte `int` (for example) would
read just 2 bytes from the wire.
To resolve this:
* Introduce an error code type with a known size:
`typedef s32 errcode_t`.
* Change all error code macros to constants of type `errcode_t`.
Constants also play better with gdb - it would visualize the name of
the constant instead of the numeric value.
* Change all functions that take error codes to take the new type
`errcode_t` instead of `int`.
* Introduce towire / fromwire functions to send / receive the newly added
type `errcode_t` and use it instead of `towire_int()`.
In addition:
* Remove the now unneeded `towire_int()`.
* Replace a hardcoded error code `-2` with a new constant
`INVOICE_EXPIRED_DURING_WAIT` (903).
Changelog-Changed: The waitinvoice command would now return error code 903 to designate that the invoice expired during wait, instead of the previous -2
The fundchannel plugin needs to know how to build a transaction, so we need to
tell it which chainparams to use. Also adds `chainparams` as a global, since
that seems to be the way to do things in plugins.
We now hand around struct json_out members, rather than using formatted
strings, so plugins need to construct them properly.
There's no automatic conversion between ' and " any more, so those
are eliminated too. pay still uses some manual construction of elements.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Since this handles escaping for us, this automatically fixes our previous
escaping issued.
Fixes: #2612
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rusty Russell
Christian Decker
ZmnSCPxj jxPCSnmZ
darosior
Wladimir J. van der Laan
Vasil Dimov
trueptolemy
lisa neigut