From 22c1a68bfbb4186b1e8684421939ca927c194300 Mon Sep 17 00:00:00 2001 From: jl777 Date: Mon, 16 Jul 2018 01:11:54 -1100 Subject: [PATCH 01/10] Update prices --- iguana/exchanges/auto_chipsbtc | 2 +- iguana/exchanges/auto_chipskmd | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/iguana/exchanges/auto_chipsbtc b/iguana/exchanges/auto_chipsbtc index 35b9319e4..40e61d94b 100755 --- a/iguana/exchanges/auto_chipsbtc +++ b/iguana/exchanges/auto_chipsbtc @@ -1,2 +1,2 @@ source userpass -curl --url "http://127.0.0.1:7783" --data "{\"minprice\":0.00006,\"maxprice\":0.0002,\"userpass\":\"$userpass\",\"method\":\"autoprice\",\"base\":\"CHIPS\",\"rel\":\"BTC\",\"margin\":0.05,\"refbase\":\"chips\",\"refrel\":\"coinmarketcap\"}" +curl --url "http://127.0.0.1:7783" --data "{\"minprice\":0.00002,\"maxprice\":0.0002,\"userpass\":\"$userpass\",\"method\":\"autoprice\",\"base\":\"CHIPS\",\"rel\":\"BTC\",\"margin\":0.05,\"refbase\":\"chips\",\"refrel\":\"coinmarketcap\"}" diff --git a/iguana/exchanges/auto_chipskmd b/iguana/exchanges/auto_chipskmd index 5a5794c86..fd2c7d6ca 100755 --- a/iguana/exchanges/auto_chipskmd +++ b/iguana/exchanges/auto_chipskmd @@ -1,2 +1,2 @@ source userpass -curl --url "http://127.0.0.1:7783" --data "{\"minprice\":0.15,\"maxprice\":0.4,\"userpass\":\"$userpass\",\"method\":\"autoprice\",\"base\":\"CHIPS\",\"rel\":\"KMD\",\"margin\":0.05,\"refbase\":\"chips\",\"refrel\":\"coinmarketcap\"}" +curl --url "http://127.0.0.1:7783" --data "{\"minprice\":0.05,\"maxprice\":0.4,\"userpass\":\"$userpass\",\"method\":\"autoprice\",\"base\":\"CHIPS\",\"rel\":\"KMD\",\"margin\":0.05,\"refbase\":\"chips\",\"refrel\":\"coinmarketcap\"}" From 0f967bb57323255547a848b675df0d4d48d798f9 Mon Sep 17 00:00:00 2001 From: Artem Date: Wed, 25 Jul 2018 17:41:56 +0300 Subject: [PATCH 02/10] Rewrite goals I've gathered some information about the possible goals behind the MM rewrite. Documenting the goals (the vision) is often important for the project success, it affects future design decisions and helps team communication by establishing shared values. --- iguana/exchanges/mm2.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 iguana/exchanges/mm2.md diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md new file mode 100644 index 000000000..bb144a8ee --- /dev/null +++ b/iguana/exchanges/mm2.md @@ -0,0 +1,29 @@ +# MarketMaker2 + +This document will help us track the information related to the MarketMaker Rust rewrite. + +## Rewrite goals + +Rewrites and ports +[are costly](http://nibblestew.blogspot.com/2017/04/why-dont-you-just-rewrite-it-in-x.html). +We tend to think that porting is simple, and on some level this intuition is true +because we can skip some high-level design decisions and focus on translating the existing logic, +but it still takes a lot of time +and though we don't need to make some of the new design decisions, +we might spend no less effort to reverse-engineer and understand the old ones. + +So why the rewrite then? + +Carol, in her talk about rewrites, offers some possible reasons: + +"*If* you have some code in C or another language, and need to change it, or it’s slow, or it crashes a +lot, or no one understands it anymore, THEN maybe a rewrite in Rust would be a good fit. +I would also posit that more people are *able* to write production Rust than production C, so if your +team *is* willing to learn Rust, it might actually expand the number of maintainers." +- https://github.com/carols10cents/rust-out-your-c-talk, https://www.youtube.com/watch?v=SKGVItFlK3w + +And we have some of those: + +* We *need to change* MarketMaker: A more approachable and reliable API. Ability to embed the MarketMaker in the GUI applications. Ways to more easily deploy it at home by running it from small computers like on a spare mobile phone or on a Raspberry Pi 3. + +* The MarketMaker *crashes a lot*, to quote hyperDEX: "The biggest issue with the MM right now, is bobs crash or does not have the orders in users orderbook, or when users try to do a order it doesnt work or goes unmatched or other random stuff". From 191ff53715a5bc39fde433c70d7eccbd47b9c0b1 Mon Sep 17 00:00:00 2001 From: Artem Date: Wed, 25 Jul 2018 17:47:09 +0300 Subject: [PATCH 03/10] Minor text fixes after looking it over on GitHub --- iguana/exchanges/mm2.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md index bb144a8ee..4c8add732 100644 --- a/iguana/exchanges/mm2.md +++ b/iguana/exchanges/mm2.md @@ -1,4 +1,4 @@ -# MarketMaker2 +# Market Maker 2 This document will help us track the information related to the MarketMaker Rust rewrite. @@ -19,11 +19,11 @@ Carol, in her talk about rewrites, offers some possible reasons: "*If* you have some code in C or another language, and need to change it, or it’s slow, or it crashes a lot, or no one understands it anymore, THEN maybe a rewrite in Rust would be a good fit. I would also posit that more people are *able* to write production Rust than production C, so if your -team *is* willing to learn Rust, it might actually expand the number of maintainers." -- https://github.com/carols10cents/rust-out-your-c-talk, https://www.youtube.com/watch?v=SKGVItFlK3w +team *is* willing to learn Rust, it might actually expand the number of +maintainers." - https://github.com/carols10cents/rust-out-your-c-talk, https://www.youtube.com/watch?v=SKGVItFlK3w. -And we have some of those: +And we have some of these: -* We *need to change* MarketMaker: A more approachable and reliable API. Ability to embed the MarketMaker in the GUI applications. Ways to more easily deploy it at home by running it from small computers like on a spare mobile phone or on a Raspberry Pi 3. +* We *need to change* the MarketMaker: A more approachable and reliable API. Ability to embed the MarketMaker in the GUI applications. Ways to more easily deploy it at home by running it from small computers like on a spare mobile phone or on a Raspberry Pi 3. * The MarketMaker *crashes a lot*, to quote hyperDEX: "The biggest issue with the MM right now, is bobs crash or does not have the orders in users orderbook, or when users try to do a order it doesnt work or goes unmatched or other random stuff". From 9e731e7183130361060ff39c63261960645c5be0 Mon Sep 17 00:00:00 2001 From: Artem Date: Wed, 25 Jul 2018 17:55:11 +0300 Subject: [PATCH 04/10] Clarify the second goal --- iguana/exchanges/mm2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md index 4c8add732..198edca72 100644 --- a/iguana/exchanges/mm2.md +++ b/iguana/exchanges/mm2.md @@ -26,4 +26,4 @@ And we have some of these: * We *need to change* the MarketMaker: A more approachable and reliable API. Ability to embed the MarketMaker in the GUI applications. Ways to more easily deploy it at home by running it from small computers like on a spare mobile phone or on a Raspberry Pi 3. -* The MarketMaker *crashes a lot*, to quote hyperDEX: "The biggest issue with the MM right now, is bobs crash or does not have the orders in users orderbook, or when users try to do a order it doesnt work or goes unmatched or other random stuff". +* The MarketMaker *crashes a lot*, to quote hyperDEX: "The biggest issue with the MM right now, is bobs crash or does not have the orders in users orderbook, or when users try to do a order it doesnt work or goes unmatched or other random stuff". And we want it to be stable and reliable instead. From f57cd9fb7ef0de9ebbac9ca37f1bf2f987839d90 Mon Sep 17 00:00:00 2001 From: Artem Date: Wed, 25 Jul 2018 20:31:01 +0300 Subject: [PATCH 05/10] The Purely functional core RFC --- iguana/exchanges/mm2.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md index 198edca72..ab98f81c6 100644 --- a/iguana/exchanges/mm2.md +++ b/iguana/exchanges/mm2.md @@ -27,3 +27,28 @@ And we have some of these: * We *need to change* the MarketMaker: A more approachable and reliable API. Ability to embed the MarketMaker in the GUI applications. Ways to more easily deploy it at home by running it from small computers like on a spare mobile phone or on a Raspberry Pi 3. * The MarketMaker *crashes a lot*, to quote hyperDEX: "The biggest issue with the MM right now, is bobs crash or does not have the orders in users orderbook, or when users try to do a order it doesnt work or goes unmatched or other random stuff". And we want it to be stable and reliable instead. + +## Purely functional core + +One of our goals is to make the MarketMaker 2 more +[stable and reliable](https://softwareengineering.stackexchange.com/questions/158054/stability-vs-reliability). +We want it to crash less often. +If there was a failure, we want to simplify and preferably automate recovery. +And we want to reduce the time between a failure and a fix. + +We'll make a big step towards these goals if the core of the MarketMaker is purely functional. +That is, if we can untangle the *state* of the MarketMaker from the code operating on that state. + +The benefits we want to reap from this are: +* Transparency. Some bugs are hard to fix because we don't have enough information about them. We might be lucky to have been running the program in a debugger or finding the necessary bits it verbose logs, but more often than not this is not the case: we know that a failure has happened, but have no idea where and on what input. Separating the state from the code allows the state to be easily shared with a developer, which means much faster roundtrips between discovering a failure and fixing it. +* Replayability. Having a separate state allows us to easily replay any operation. If a failure occured in the middle of a transaction, it should be possible to try a new version of the code without manually repeating all the steps that were necessary to initiate the transaction. And the updated code will run exactly on the failing transaction, not on some other transaction initiated at a later time, which means that users will benefit from less friction and developers will have a better chance to fix the hard-to-reproduce bugs. +* Testability. Stateless code is much easier to test and according to Kent Beck is often a natural result of a Test-Driven development process. +* Portability. Separating the state from the code allows us to more easily use the stateless parts from the sandboxed environments, such as when running under the Web Assembly (WASM). We only need to port the state-managing layer, fully reusing the stateless code. +* Hot reloading. When the code is separated from state, it's trivial to reload it, both with the shared libraries in CPU-native environments (dlopen) and with WASM in GUI environments. This might positively affect the development cycle, reducing the round-trip time from a failure to a fix. + +Implementation might consist of two layers. +A layer that is ported to the host environment (native, JS, Java, Objective-C, cross-compiled Raspberry Pi 3, etc) and implements the TCP/IP communication, state management, hot reloading all things that won't fit into the WASM sandbox. +And a layer that implements the core logic in a stateless manner and which is compiled to a native shared library or WASM. + +The feasibility of this approach is yet to be evaluated, but we can move gradually towards it +by separating the code into the stateful and stateless layers while working on the basic Rust port. \ No newline at end of file From 5d24afdc473db1ea6e8359896a7387ade119a215 Mon Sep 17 00:00:00 2001 From: Artem Date: Wed, 25 Jul 2018 20:38:43 +0300 Subject: [PATCH 06/10] Minor text fixes --- iguana/exchanges/mm2.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md index ab98f81c6..31be25d6c 100644 --- a/iguana/exchanges/mm2.md +++ b/iguana/exchanges/mm2.md @@ -47,8 +47,8 @@ The benefits we want to reap from this are: * Hot reloading. When the code is separated from state, it's trivial to reload it, both with the shared libraries in CPU-native environments (dlopen) and with WASM in GUI environments. This might positively affect the development cycle, reducing the round-trip time from a failure to a fix. Implementation might consist of two layers. -A layer that is ported to the host environment (native, JS, Java, Objective-C, cross-compiled Raspberry Pi 3, etc) and implements the TCP/IP communication, state management, hot reloading all things that won't fit into the WASM sandbox. -And a layer that implements the core logic in a stateless manner and which is compiled to a native shared library or WASM. +A layer that is ported to the host environment (native, JS, Java, Objective-C, cross-compiled Raspberry Pi 3, etc) and implements the TCP/IP communication, state management, hot reloading all, things that won't fit into the WASM sandbox. +And a layer that implements the core logic in a stateless manner and which is compiled into a native shared library or WASM. The feasibility of this approach is yet to be evaluated, but we can move gradually towards it by separating the code into the stateful and stateless layers while working on the basic Rust port. \ No newline at end of file From c10a15b2b1d19c5a03906b4495045f13a5efb64a Mon Sep 17 00:00:00 2001 From: Artem Date: Wed, 25 Jul 2018 20:52:14 +0300 Subject: [PATCH 07/10] Handling the sensitive data --- iguana/exchanges/mm2.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md index 31be25d6c..62ea08a06 100644 --- a/iguana/exchanges/mm2.md +++ b/iguana/exchanges/mm2.md @@ -50,5 +50,11 @@ Implementation might consist of two layers. A layer that is ported to the host environment (native, JS, Java, Objective-C, cross-compiled Raspberry Pi 3, etc) and implements the TCP/IP communication, state management, hot reloading all, things that won't fit into the WASM sandbox. And a layer that implements the core logic in a stateless manner and which is compiled into a native shared library or WASM. +Parts of the state might be marked as sensitive. +This will give the users an option to share only the information that can be freely shared, +without a risk of loosing money that is. +Even without the sensitive information a state snapshot might provide the developer with enough data to quickly triage and/or fix the failure, therefore improving the roundtrip time before a failure and a fix. +Plus users will more easily share their problems when it's quick, automatic and doesn't pose a monetary risk. + The feasibility of this approach is yet to be evaluated, but we can move gradually towards it by separating the code into the stateful and stateless layers while working on the basic Rust port. \ No newline at end of file From 7cdf02f9713a4f5687bb9cac6b1346054c26d6f1 Mon Sep 17 00:00:00 2001 From: Artem Date: Wed, 25 Jul 2018 20:55:09 +0300 Subject: [PATCH 08/10] A minor typo fix --- iguana/exchanges/mm2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md index 62ea08a06..b04be0919 100644 --- a/iguana/exchanges/mm2.md +++ b/iguana/exchanges/mm2.md @@ -47,7 +47,7 @@ The benefits we want to reap from this are: * Hot reloading. When the code is separated from state, it's trivial to reload it, both with the shared libraries in CPU-native environments (dlopen) and with WASM in GUI environments. This might positively affect the development cycle, reducing the round-trip time from a failure to a fix. Implementation might consist of two layers. -A layer that is ported to the host environment (native, JS, Java, Objective-C, cross-compiled Raspberry Pi 3, etc) and implements the TCP/IP communication, state management, hot reloading all, things that won't fit into the WASM sandbox. +A layer that is ported to the host environment (native, JS, Java, Objective-C, cross-compiled Raspberry Pi 3, etc) and implements the TCP/IP communication, state management, hot reloading, all things that won't fit into the WASM sandbox. And a layer that implements the core logic in a stateless manner and which is compiled into a native shared library or WASM. Parts of the state might be marked as sensitive. From 565f447b26c8b6f682fbaa5581e6c2fb8fe3c1f4 Mon Sep 17 00:00:00 2001 From: Artem Date: Thu, 26 Jul 2018 02:54:12 +0300 Subject: [PATCH 09/10] Found new items for the changes and crashes lists --- iguana/exchanges/mm2.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md index 198edca72..ed7b51e10 100644 --- a/iguana/exchanges/mm2.md +++ b/iguana/exchanges/mm2.md @@ -24,6 +24,14 @@ maintainers." - https://github.com/carols10cents/rust-out-your-c-talk, https://w And we have some of these: -* We *need to change* the MarketMaker: A more approachable and reliable API. Ability to embed the MarketMaker in the GUI applications. Ways to more easily deploy it at home by running it from small computers like on a spare mobile phone or on a Raspberry Pi 3. - -* The MarketMaker *crashes a lot*, to quote hyperDEX: "The biggest issue with the MM right now, is bobs crash or does not have the orders in users orderbook, or when users try to do a order it doesnt work or goes unmatched or other random stuff". And we want it to be stable and reliable instead. +* We *need to change* the MarketMaker: +A more approachable and reliable API. +Ability to embed the MarketMaker in the GUI applications. +Ways to more easily deploy it at home by running it from small computers like on a spare mobile phone or on a Raspberry Pi 3. +Ability to process multiple API calls in parallel. +A faster version of the `swapstatus` API call. + +* The MarketMaker *crashes a lot*, +to quote hyperDEX: "The biggest issue with the MM right now, is bobs crash or does not have the orders in users orderbook, or when users try to do a order it doesnt work or goes unmatched or other random stuff" +and lukechilds: "We've frequently experienced crashes while querying all swaps with swapstatus". +We want it to be stable and reliable instead. From a3b0e276a0e3ddedb86bba4e70c86aba22d2fbfd Mon Sep 17 00:00:00 2001 From: jl777 Date: Wed, 25 Jul 2018 20:11:25 -1100 Subject: [PATCH 10/10] Prevent old style addresses in withdraw of BCH --- iguana/exchanges/LP_rpc.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/iguana/exchanges/LP_rpc.c b/iguana/exchanges/LP_rpc.c index 615942e0a..5f4a246df 100644 --- a/iguana/exchanges/LP_rpc.c +++ b/iguana/exchanges/LP_rpc.c @@ -401,6 +401,8 @@ int32_t LP_address_isvalid(char *symbol,char *address) int32_t isvalid = 0; cJSON *retjson; if ( symbol == 0 || symbol[0] == 0 ) return(0); + if ( strcmp(symbol,"BCH") == 0 && (address[0] == '1' || address[0] == '3') ) + return(-1); if ( (retjson= LP_validateaddress(symbol,address)) != 0 ) { if ( jobj(retjson,"isvalid") != 0 && is_cJSON_True(jobj(retjson,"isvalid")) != 0 )