From 3ae0c20026a84f2e987b35ab81db5322987db61d Mon Sep 17 00:00:00 2001 From: Rusty Russell Date: Fri, 1 Feb 2019 16:23:39 +1030 Subject: [PATCH] getroute: change definition (and pay default) for riskfactor. Up until now, riskfactor was useless due to implementation bugs, and also the default setting is wrong (too low to have an effect on reasonable payment scenarios). Let's simplify the definition (by assuming that P(failure) of a node is 1), to make it a simple percentage. I examined the current network fees to see what would work, and under this definition, a default of 10 seems reasonable (equivalent to 1000 under the old definition). It is *this* change which finally fixes our test case! The riskfactor is now 40msat (1500000 * 14 * 10 / 5259600 = 39.9), comparable with worst-case fuzz is 50msat (1001 * 0.05 = 50). Signed-off-by: Rusty Russell --- CHANGELOG.md | 2 + doc/lightning-getroute.7 | 357 ++++++++++++++++++++++++++++++----- doc/lightning-getroute.7.txt | 74 +++++--- doc/lightning-pay.7 | 6 +- doc/lightning-pay.7.txt | 2 +- gossipd/routing.c | 2 +- plugins/pay.c | 2 +- tests/test_pay.py | 1 - 8 files changed, 359 insertions(+), 87 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 47e7a8efb..cf44d1ace 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. - JSON API: `listpeers` now has a `direction` field in `channels`. - JSON API: `listchannels` now takes a `source` option to filter by node id. - JSON API: New command `paystatus` gives detailed information on `pay` commands. +- JSON API: `getroute` `riskfactor` argument is simplified; `pay` now defaults to setting it to 10. ### Changed @@ -40,6 +41,7 @@ changes. - Fixed occasional deadlock with peers when exchanging huge amounts of gossip. - You can no longer make giant unpayable "wumbo" invoices. - CLTV of total route now correctly evaluated when finding best route. +- `riskfactor` arguments to `pay` and `getroute` now have an effect. ### Security diff --git a/doc/lightning-getroute.7 b/doc/lightning-getroute.7 index af94a0aa3..3d9de8675 100644 --- a/doc/lightning-getroute.7 +++ b/doc/lightning-getroute.7 @@ -2,12 +2,12 @@ .\" Title: lightning-getroute .\" Author: [see the "AUTHOR" section] .\" Generator: DocBook XSL Stylesheets v1.79.1 -.\" Date: 01/23/2019 +.\" Date: 02/01/2019 .\" Manual: \ \& .\" Source: \ \& .\" Language: English .\" -.TH "LIGHTNING\-GETROUTE" "7" "01/23/2019" "\ \&" "\ \&" +.TH "LIGHTNING\-GETROUTE" "7" "02/01/2019" "\ \&" "\ \&" .\" ----------------------------------------------------------------- .\" * Define some portability stuff .\" ----------------------------------------------------------------- @@ -36,9 +36,9 @@ lightning-getroute \- Command for routing a payment (low\-level)\&. .sp The \fBgetroute\fR RPC command attempts to find the best route for the payment of \fImsatoshi\fR to lightning node \fIid\fR, such that the payment will arrive at \fIid\fR with \fIcltv\fR\-blocks to spare (default 9)\&. .sp -There are two considerations for how good a route is: how low the fees are, and how long your payment will get stuck if a node goes down during the process\&. The \fIriskfactor\fR floating\-point field controls this tradeoff; it is the annual cost of your funds being stuck (as a percentage), multiplied by the percentage chance of each node failing\&. +There are two considerations for how good a route is: how low the fees are, and how long your payment will get stuck if a node goes down during the process\&. The \fIriskfactor\fR floating\-point field controls this tradeoff; it is the annual cost of your funds being stuck (as a percentage)\&. .sp -For example, if you thought there was a 1% chance that a node would fail, and it would cost you 20% per annum if that happened, \fIriskfactor\fR would be 20\&. +For example, if you thought the inconvenience of having funds stuck was worth 20% per annum interest, \fIriskfactor\fR would be 20\&. .sp If you didn\(cqt care about risk, \fIriskfactor\fR would be zero\&. .sp @@ -55,15 +55,13 @@ The formula used is the following approximation: .RS 4 .\} .nf -hop\-risk = num\-hops x per\-hop\-risk -timeout\-cost = blocks\-timeout x per\-block\-cost -risk\-fee = amount x hop\-risk x timeout\-cost +risk\-fee = amount x blocks\-timeout x per\-block\-cost .fi .if n \{\ .RE .\} .sp -We are given a \fIriskfactor\fR; expressed as two multiplied percentages is the same as fractions multiplied by 10000\&. There are 52596 blocks per year, thus \fIper\-block\-cost\fR x \fIper\-hop\-risk\fR is riskfactor\*(Aq divided by 5,259,600,000\&. +We are given a \fIriskfactor\fR expressed as a percentage\&. There are 52596 blocks per year, thus \fIper\-block\-cost\fR is \fIriskfactor\fR divided by 5,259,600\&. .sp The final result is: .sp @@ -71,26 +69,26 @@ The final result is: .RS 4 .\} .nf -risk\-fee = amount x num\-hops x blocks\-timeout x riskfactor / 5259600000 +risk\-fee = amount x blocks\-timeout x riskfactor / 5259600 .fi .if n \{\ .RE .\} .sp -Here are the risk fees as a percentage of the amount sent, using various parameters\&. For comparison with actual fees, we assume nodes charge 0\&.05%: +Here are the risk fees in millisatoshis, using various parameters\&. I assume a channel charges the default of 1000 millisatoshis plus 1 part\-per\-million\&. Common delay values on the network at 14 and 144\&. .TS allbox tab(:); ltB ltB ltB ltB ltB. T{ -Riskfactor +Amount (msat) T}:T{ -Nodes +Riskfactor T}:T{ -Delay per node +Delay T}:T{ -Risk Fee % +Risk Fee T}:T{ -Route fee % +Route fee T} .T& lt lt lt lt lt @@ -101,160 +99,417 @@ lt lt lt lt lt lt lt lt lt lt lt lt lt lt lt lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt +lt lt lt lt lt lt lt lt lt lt. T{ .sp -0\&.001 +10,000 T}:T{ .sp -5 +1 T}:T{ .sp -6 +14 T}:T{ .sp 0 T}:T{ .sp -0\&.25 +1001 T} T{ .sp -1 +10,000 T}:T{ .sp -5 +10 T}:T{ .sp -6 +14 T}:T{ .sp 0 T}:T{ .sp -0\&.25 +1001 +T} +T{ +.sp +10,000 +T}:T{ +.sp +100 +T}:T{ +.sp +14 +T}:T{ +.sp +2 +T}:T{ +.sp +1001 T} T{ .sp +10,000 +T}:T{ +.sp 1000 T}:T{ .sp -5 +14 T}:T{ .sp -6 +26 T}:T{ .sp -0\&.0029 +1001 +T} +T{ +.sp +1,000,000 T}:T{ .sp -0\&.25 +1 +T}:T{ +.sp +14 +T}:T{ +.sp +2 +T}:T{ +.sp +1001 T} T{ .sp -0\&.001 +1,000,000 T}:T{ .sp 10 T}:T{ .sp -72 +14 T}:T{ .sp -0 +26 T}:T{ .sp -0\&.5 +1001 T} T{ .sp +1,000,000 +T}:T{ +.sp +100 +T}:T{ +.sp +14 +T}:T{ +.sp +266 +T}:T{ +.sp +1001 +T} +T{ +.sp +1,000,000 +T}:T{ +.sp +1000 +T}:T{ +.sp +14 +T}:T{ +.sp +2661 +T}:T{ +.sp +1001 +T} +T{ +.sp +100,000,000 +T}:T{ +.sp 1 T}:T{ .sp +14 +T}:T{ +.sp +266 +T}:T{ +.sp +1100 +T} +T{ +.sp +100,000,000 +T}:T{ +.sp 10 T}:T{ .sp -72 +14 +T}:T{ +.sp +2661 +T}:T{ +.sp +1100 +T} +T{ +.sp +100,000,000 +T}:T{ +.sp +100 T}:T{ .sp -0\&.0001 +14 T}:T{ .sp -0\&.5 +26617 +T}:T{ +.sp +1100 T} T{ .sp +100,000,000 +T}:T{ +.sp 1000 T}:T{ .sp +14 +T}:T{ +.sp +266179 +T}:T{ +.sp +1100 +T} +T{ +.sp +10,000 +T}:T{ +.sp +1 +T}:T{ +.sp +144 +T}:T{ +.sp +0 +T}:T{ +.sp +1001 +T} +T{ +.sp +10,000 +T}:T{ +.sp 10 T}:T{ .sp -72 +144 T}:T{ .sp -0\&.1369 +2 T}:T{ .sp -0\&.5 +1001 T} T{ .sp -0\&.001 +10,000 T}:T{ .sp -20 +100 T}:T{ .sp -1008 +144 T}:T{ .sp -0 +27 T}:T{ .sp -1\&.0 +1001 T} T{ .sp +10,000 +T}:T{ +.sp +1000 +T}:T{ +.sp +144 +T}:T{ +.sp +273 +T}:T{ +.sp +1001 +T} +T{ +.sp +1,000,000 +T}:T{ +.sp 1 T}:T{ .sp -20 +144 +T}:T{ +.sp +27 +T}:T{ +.sp +1001 +T} +T{ +.sp +1,000,000 +T}:T{ +.sp +10 +T}:T{ +.sp +144 +T}:T{ +.sp +273 +T}:T{ +.sp +1001 +T} +T{ +.sp +1,000,000 +T}:T{ +.sp +100 T}:T{ .sp -1008 +144 T}:T{ .sp -0\&.0077 +2737 T}:T{ .sp -1\&.0 +1001 T} T{ .sp +1,000,000 +T}:T{ +.sp 1000 T}:T{ .sp -20 +144 +T}:T{ +.sp +27378 +T}:T{ +.sp +1001 +T} +T{ +.sp +100,000,000 +T}:T{ +.sp +1 +T}:T{ +.sp +144 +T}:T{ +.sp +2737 +T}:T{ +.sp +1100 +T} +T{ +.sp +100,000,000 +T}:T{ +.sp +10 T}:T{ .sp -1008 +144 T}:T{ .sp -7\&.6660 +27378 T}:T{ .sp -1\&.0 +1100 +T} +T{ +.sp +100,000,000 +T}:T{ +.sp +100 +T}:T{ +.sp +144 +T}:T{ +.sp +273785 +T}:T{ +.sp +1100 +T} +T{ +.sp +100,000,000 +T}:T{ +.sp +1000 +T}:T{ +.sp +144 +T}:T{ +.sp +2737850 +T}:T{ +.sp +1100 T} .TE .sp 1 .SH "RECOMMENDED RISKFACTOR VALUES" .sp -0\&.001 is a value for tie\-breaking in favor of shorter routes, but not really costing in any risk\&. +The default \fIfuzz\fR factor is 5%, so as you can see from the table above, that tends to overwhelm the effect of \fIriskfactor\fR less than about 5\&. .sp 1 is a conservative value for a stable lightning network with very few failures\&. .sp 1000 is an aggressive value for trying to minimize timeouts at all costs\&. +.sp +The default for lightning\-pay(7) is 10, which starts to become a major factor for larger amounts, and is basically ignored for tiny ones\&. .SH "RETURN VALUE" .sp On success, a "route" array is returned\&. Each array element contains \fIid\fR (the node being routed through), \fImsatoshi\fR (the millisatoshis sent), and \fIdelay\fR (the number of blocks to timeout at this node)\&. diff --git a/doc/lightning-getroute.7.txt b/doc/lightning-getroute.7.txt index 388072f14..771b95ca5 100644 --- a/doc/lightning-getroute.7.txt +++ b/doc/lightning-getroute.7.txt @@ -21,11 +21,10 @@ There are two considerations for how good a route is: how low the fees are, and how long your payment will get stuck if a node goes down during the process. The 'riskfactor' floating-point field controls this tradeoff; it is the annual cost of your funds being stuck (as a -percentage), multiplied by the percentage chance of each node failing. +percentage). -For example, if you thought there was a 1% chance that a node would -fail, and it would cost you 20% per annum if that happened, -'riskfactor' would be 20. +For example, if you thought the inconvenience of having funds stuck was +worth 20% per annum interest, 'riskfactor' would be 20. If you didn't care about risk, 'riskfactor' would be zero. @@ -46,45 +45,59 @@ for the purposes of comparing routes. The formula used is the following approximation: ---- -hop-risk = num-hops x per-hop-risk -timeout-cost = blocks-timeout x per-block-cost -risk-fee = amount x hop-risk x timeout-cost +risk-fee = amount x blocks-timeout x per-block-cost ---- -We are given a 'riskfactor'; expressed as two multiplied percentages -is the same as fractions multiplied by 10000. There are 52596 blocks -per year, thus 'per-block-cost' x 'per-hop-risk' is riskfactor' -divided by 5,259,600,000. +We are given a 'riskfactor' expressed as a percentage. There are 52596 blocks +per year, thus 'per-block-cost' is 'riskfactor' divided by 5,259,600. The final result is: ---- -risk-fee = amount x num-hops x blocks-timeout x riskfactor / 5259600000 +risk-fee = amount x blocks-timeout x riskfactor / 5259600 ---- -Here are the risk fees as a percentage of the amount sent, using -various parameters. For comparison with actual fees, we assume nodes -charge 0.05%: +Here are the risk fees in millisatoshis, using various parameters. I +assume a channel charges the default of 1000 millisatoshis plus 1 +part-per-million. Common delay values on the network at 14 and 144. [options="header"] |======================= -|Riskfactor |Nodes | Delay per node |Risk Fee % |Route fee % -|0.001 |5 | 6 |0 |0.25 -|1 |5 | 6 |0 |0.25 -|1000 |5 | 6 |0.0029 |0.25 - -|0.001 |10 | 72 |0 |0.5 -|1 |10 | 72 |0.0001 |0.5 -|1000 |10 | 72 |0.1369 |0.5 - -|0.001 |20 | 1008 |0 |1.0 -|1 |20 | 1008 |0.0077 |1.0 -|1000 |20 | 1008 |7.6660 |1.0 +|Amount (msat) |Riskfactor | Delay |Risk Fee |Route fee +|10,000 |1 | 14 |0 |1001 +|10,000 |10 | 14 |0 |1001 +|10,000 |100 | 14 |2 |1001 +|10,000 |1000 | 14 |26 |1001 + +|1,000,000 |1 | 14 |2 |1001 +|1,000,000 |10 | 14 |26 |1001 +|1,000,000 |100 | 14 |266 |1001 +|1,000,000 |1000 | 14 |2661 |1001 + +|100,000,000 |1 | 14 |266 |1100 +|100,000,000 |10 | 14 |2661 |1100 +|100,000,000 |100 | 14 |26617 |1100 +|100,000,000 |1000 | 14 |266179 |1100 + +|10,000 |1 | 144 |0 |1001 +|10,000 |10 | 144 |2 |1001 +|10,000 |100 | 144 |27 |1001 +|10,000 |1000 | 144 |273 |1001 + +|1,000,000 |1 | 144 |27 |1001 +|1,000,000 |10 | 144 |273 |1001 +|1,000,000 |100 | 144 |2737 |1001 +|1,000,000 |1000 | 144 |27378 |1001 + +|100,000,000 |1 | 144 |2737 |1100 +|100,000,000 |10 | 144 |27378 |1100 +|100,000,000 |100 | 144 |273785 |1100 +|100,000,000 |1000 | 144 |2737850 |1100 |======================= RECOMMENDED RISKFACTOR VALUES ----------------------------- -0.001 is a value for tie-breaking in favor of shorter routes, but not really -costing in any risk. +The default 'fuzz' factor is 5%, so as you can see from the table above, +that tends to overwhelm the effect of 'riskfactor' less than about 5. 1 is a conservative value for a stable lightning network with very few failures. @@ -92,6 +105,9 @@ failures. 1000 is an aggressive value for trying to minimize timeouts at all costs. +The default for lightning-pay(7) is 10, which starts to become a major +factor for larger amounts, and is basically ignored for tiny ones. + RETURN VALUE ------------ diff --git a/doc/lightning-pay.7 b/doc/lightning-pay.7 index 1983dbab3..b4193205b 100644 --- a/doc/lightning-pay.7 +++ b/doc/lightning-pay.7 @@ -2,12 +2,12 @@ .\" Title: lightning-pay .\" Author: [see the "AUTHOR" section] .\" Generator: DocBook XSL Stylesheets v1.79.1 -.\" Date: 11/26/2018 +.\" Date: 02/01/2019 .\" Manual: \ \& .\" Source: \ \& .\" Language: English .\" -.TH "LIGHTNING\-PAY" "7" "11/26/2018" "\ \&" "\ \&" +.TH "LIGHTNING\-PAY" "7" "02/01/2019" "\ \&" "\ \&" .\" ----------------------------------------------------------------- .\" * Define some portability stuff .\" ----------------------------------------------------------------- @@ -34,7 +34,7 @@ lightning-pay \- Command for sending a payment to a BOLT11 invoice \fBpay\fR \fIbolt11\fR [\fImsatoshi\fR] [\fIdescription\fR] [\fIriskfactor\fR] [\fImaxfeepercent\fR] [\fIretry_for\fR] [\fImaxdelay\fR] [\fIexemptfee\fR] .SH "DESCRIPTION" .sp -The \fBpay\fR RPC command attempts to find a route to the given destination, and send the funds it asks for\&. If the \fIbolt11\fR does not contain an amount, \fImsatoshi\fR is required, otherwise if it is specified it must be \fInull\fR\&. If \fIbolt11\fR contains a description hash (\fIh\fR field) \fIdescription\fR is required, otherwise it is unused\&. The \fIriskfactor\fR is described in detail in lightning\-getroute(7), and defaults to 1\&.0\&. The \fImaxfeepercent\fR limits the money paid in fees, and defaults to 0\&.5\&. The maxfeepercent\*(Aq is a percentage of the amount that is to be paid\&. The `exemptfee option can be used for tiny payments which would be dominated by the fee leveraged by forwarding nodes\&. Setting exemptfee allows the maxfeepercent check to be skipped on fees that are smaller than exemptfee (default: 5000 millisatoshi)\&. +The \fBpay\fR RPC command attempts to find a route to the given destination, and send the funds it asks for\&. If the \fIbolt11\fR does not contain an amount, \fImsatoshi\fR is required, otherwise if it is specified it must be \fInull\fR\&. If \fIbolt11\fR contains a description hash (\fIh\fR field) \fIdescription\fR is required, otherwise it is unused\&. The \fIriskfactor\fR is described in detail in lightning\-getroute(7), and defaults to 10\&. The \fImaxfeepercent\fR limits the money paid in fees, and defaults to 0\&.5\&. The maxfeepercent\*(Aq is a percentage of the amount that is to be paid\&. The `exemptfee option can be used for tiny payments which would be dominated by the fee leveraged by forwarding nodes\&. Setting exemptfee allows the maxfeepercent check to be skipped on fees that are smaller than exemptfee (default: 5000 millisatoshi)\&. .sp The response will occur when the payment fails or succeeds\&. Once a payment has succeeded, calls to \fBpay\fR with the same \fIbolt11\fR will succeed immediately\&. .sp diff --git a/doc/lightning-pay.7.txt b/doc/lightning-pay.7.txt index 04e830f65..fab59a867 100644 --- a/doc/lightning-pay.7.txt +++ b/doc/lightning-pay.7.txt @@ -18,7 +18,7 @@ and send the funds it asks for. If the 'bolt11' does not contain an amount, 'msatoshi' is required, otherwise if it is specified it must be 'null'. If 'bolt11' contains a description hash ('h' field) 'description' is required, otherwise it is unused. The 'riskfactor' is described in detail -in lightning-getroute(7), and defaults to 1.0. +in lightning-getroute(7), and defaults to 10. The 'maxfeepercent' limits the money paid in fees, and defaults to 0.5. The `maxfeepercent' is a percentage of the amount that is to be paid. diff --git a/gossipd/routing.c b/gossipd/routing.c index 5fcf2a2b8..e77c2e606 100644 --- a/gossipd/routing.c +++ b/gossipd/routing.c @@ -1528,7 +1528,7 @@ struct route_hop *get_route(const tal_t *ctx, struct routing_state *rstate, } route = find_route(ctx, rstate, source, destination, msatoshi, - riskfactor / BLOCKS_PER_YEAR / 10000, + riskfactor / BLOCKS_PER_YEAR / 100, fuzz, &base_seed, max_hops, &fee); /* Now restore the capacity. */ diff --git a/plugins/pay.c b/plugins/pay.c index 5910e2f99..352c25f7c 100644 --- a/plugins/pay.c +++ b/plugins/pay.c @@ -833,7 +833,7 @@ static struct command_result *handle_pay(struct command *cmd, p_req("bolt11", param_string, &b11str), p_opt("msatoshi", param_u64, &msatoshi), p_opt("description", param_string, &pc->desc), - p_opt_def("riskfactor", param_double, &riskfactor, 1.0), + p_opt_def("riskfactor", param_double, &riskfactor, 10), p_opt_def("maxfeepercent", param_percent, &maxfeepercent, 0.5), p_opt_def("retry_for", param_number, &retryfor, 60), p_opt_def("maxdelay", param_number, &maxdelay, diff --git a/tests/test_pay.py b/tests/test_pay.py index c2bbffd7d..4bad1387d 100644 --- a/tests/test_pay.py +++ b/tests/test_pay.py @@ -1341,7 +1341,6 @@ def test_pay_routeboost(node_factory, bitcoind): assert [h['channel'] for h in attempts[2]['routehint']] == [r['short_channel_id'] for r in routel3l5] -@pytest.mark.xfail(strict=True) def test_pay_direct(node_factory, bitcoind): """Check that we prefer the direct route. """