From 1b6c7d499df8f1b67c7d013b18bb6fb7f68b7908 Mon Sep 17 00:00:00 2001 From: Artem Date: Mon, 30 Jul 2018 20:00:39 +0300 Subject: [PATCH 1/5] Clarify the Purely functional docu Add improved Concurrency as one of the desired benefits. Clarify that WASM isn't an immediate goal. Explain what practical steps we're going to take during the port. --- iguana/exchanges/mm2.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md index a9315a438..66680691c 100644 --- a/iguana/exchanges/mm2.md +++ b/iguana/exchanges/mm2.md @@ -53,10 +53,11 @@ The benefits we want to reap from this are: * Testability. Stateless code is much easier to test and according to Kent Beck is often a natural result of a Test-Driven development process. * Portability. Separating the state from the code allows us to more easily use the stateless parts from the sandboxed environments, such as when running under the Web Assembly (WASM). We only need to port the state-managing layer, fully reusing the stateless code. * Hot reloading. When the code is separated from state, it's trivial to reload it, both with the shared libraries in CPU-native environments (dlopen) and with WASM in GUI environments. This might positively affect the development cycle, reducing the round-trip time from a failure to a fix. +* Concurrency. MarketMaker can currently only perform a single API operation at the time. The more stateless code we have the easier it should be to introduce the parallel execution of API requests in the future. Implementation might consist of two layers. A layer that is ported to the host environment (native, JS, Java, Objective-C, cross-compiled Raspberry Pi 3, etc) and implements the TCP/IP communication, state management, hot reloading, all things that won't fit into the WASM sandbox. -And a layer that implements the core logic in a stateless manner and which is compiled into a native shared library or WASM. +And a layer that implements the core logic in a stateless manner and which is compiled into a native shared library or, in the future, to WASM. Parts of the state might be marked as sensitive. This will give the users an option to share only the information that can be freely shared, @@ -65,4 +66,9 @@ Even without the sensitive information a state snapshot might provide the develo Plus users will more easily share their problems when it's quick, automatic and doesn't pose a monetary risk. The feasibility of this approach is yet to be evaluated, but we can move gradually towards it -by separating the code into the stateful and stateless layers while working on the basic Rust port. \ No newline at end of file +by separating the code into the stateful and stateless layers while working on the basic Rust port. + +During the initial Rust port we're going to +a) Mark the ported functions as purely functional or stateful, allowing us to more easily focus on the state management code in the future. +b) Where possible, take a low-hanging fruit and try to refactor the functions towards being stateless. +c) Probably mark as stateful the `sleep`ing functions, because `sleep` can be seen as changing the global state (moving the time forwards) and might negatively affect Transparency (we have no idea what's going on while a function is sleeping), Testability (sleeping tests might kill the TDD development cycle), Portability (sleeps are not compatible with WASM), Hot reloading and Concurrency (let's say we want to load new version of the code, but the old version is still sleeping somewhere). \ No newline at end of file From c90f3b0c8afc10cef6c611fb1f8a7a30bafa537a Mon Sep 17 00:00:00 2001 From: Artem Date: Mon, 30 Jul 2018 21:38:18 +0300 Subject: [PATCH 2/5] Gradual rewrite RFC --- iguana/exchanges/mm2.md | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md index 66680691c..00c530742 100644 --- a/iguana/exchanges/mm2.md +++ b/iguana/exchanges/mm2.md @@ -71,4 +71,32 @@ by separating the code into the stateful and stateless layers while working on t During the initial Rust port we're going to a) Mark the ported functions as purely functional or stateful, allowing us to more easily focus on the state management code in the future. b) Where possible, take a low-hanging fruit and try to refactor the functions towards being stateless. -c) Probably mark as stateful the `sleep`ing functions, because `sleep` can be seen as changing the global state (moving the time forwards) and might negatively affect Transparency (we have no idea what's going on while a function is sleeping), Testability (sleeping tests might kill the TDD development cycle), Portability (sleeps are not compatible with WASM), Hot reloading and Concurrency (let's say we want to load new version of the code, but the old version is still sleeping somewhere). \ No newline at end of file +c) Probably mark as stateful the `sleep`ing functions, because `sleep` can be seen as changing the global state (moving the time forwards) and might negatively affect Transparency (we have no idea what's going on while a function is sleeping), Testability (sleeping tests might kill the TDD development cycle), Portability (sleeps are not compatible with WASM), Hot reloading and Concurrency (let's say we want to load new version of the code, but the old version is still sleeping somewhere). + +## Gradual rewrite + +Above in the [Rewrite goals](#rewrite-goals) section we have identified some of the goals that we pursue with this rewrite. +These goals constitute the Value (in the Lean Production terms) that we are going to create. + +For a project to suceed it is usually important to make shorter the path the Value takes to the users. +(Inventory is waste. If we have created the Value but the users can't get their hands on it, we're wasting that Value). + +Hence we're going to start with a gradual rewrite. Keeping the version under rewrite immediately avaliable to the users willing to experiment with it. + +Let's list the good things that should come out of the gradual rewrite: +* Transparency. With the second version of the MarketMaker being immediately available we can always check the Value we're getting. Is it more stable? Does it have new functions? Or did we hit the wall? What's going on with the code and how can we help? Gradual rewrite is critical for transparency because the change is available in small increments. We can more easily see what function has changed or what new functionality was added when we aren't being uprooted from the familiar context. +* Risk reduction. It comes with transparency, as we can now more easily measure the progress, identify the roadblocks as they occur, see certain problems when they occur and not months after. Plus a gradual rewrite will by default follow the outline of the original project. We have a working system and we're improving it piece by piece, having the original design to fall back to. This makes it less likely for the rewrite to suffer from far-reaching redesign flaws (cf. [Second-system effect](https://en.wikipedia.org/wiki/Second-system_effect)) and creative blocks (cf. [Pantsing](https://www.wikiwrimo.org/wiki/Pantsing)). +* Feedback. Incorporating user feedback is critical for most projects out there, allowing us to iteratively improve the functionality in the right direction (cf. [Fail faster](https://www.youtube.com/watch?v=rDjrOaoHz9s), [PDIA](https://www.youtube.com/watch?v=ZKdjBbiGjao)). The more early we get the feedback, the more time we have to react, and at a lesser cost. +* Motivation. Feedback is not only important to guide us, but also to show us that our work is meaningful and changes lives to the better. It is the cornerstone of Agile (["Build projects around motivated individuals"](https://www.agilealliance.org/agile101/12-principles-behind-the-agile-manifesto/)) and affects our performance on all levels, down to the physical health. + +The plan so far is to by default use the C functions as the atomic units of rewrite. +Rust FFI allows us to swap any C function with a similar Rust function. +Porting on this level we +a) reuse the function-level modularity of the C language; +b) preserve the code meta-data (Git history will show a nice diff between the C and Rust functions, we'll be able to easily investigate the evolution of the code back to its roots); +c) avoid the complexity and slow-downs associated with adding RPC/networking layers or drawing new lines of abstraction; +d) have a good indicator of the porting progress (how many functions were ported, how many remains). + +Focusing on the function call chains that are a common part of a failure/crash or touch on the new functionality +will allow us to leverage the [Pareto principle](https://en.wikipedia.org/wiki/Pareto_principle), +advancing on 80% of desired Value (stability, functionality) with 20% of effort. \ No newline at end of file From 9ee5fd3698cd0dce437b6a6de8774f60ad352489 Mon Sep 17 00:00:00 2001 From: Artem Date: Mon, 30 Jul 2018 21:44:05 +0300 Subject: [PATCH 3/5] Break the C FFI notes to their respective lines for readability --- iguana/exchanges/mm2.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md index 00c530742..718209486 100644 --- a/iguana/exchanges/mm2.md +++ b/iguana/exchanges/mm2.md @@ -91,12 +91,12 @@ Let's list the good things that should come out of the gradual rewrite: The plan so far is to by default use the C functions as the atomic units of rewrite. Rust FFI allows us to swap any C function with a similar Rust function. -Porting on this level we -a) reuse the function-level modularity of the C language; -b) preserve the code meta-data (Git history will show a nice diff between the C and Rust functions, we'll be able to easily investigate the evolution of the code back to its roots); -c) avoid the complexity and slow-downs associated with adding RPC/networking layers or drawing new lines of abstraction; +Porting on this level we +a) reuse the function-level modularity of the C language; +b) preserve the code meta-data (Git history will show a nice diff between the C and Rust functions, we'll be able to easily investigate the evolution of the code back to its roots); +c) avoid the complexity and slow-downs associated with adding RPC/networking layers or drawing new lines of abstraction; d) have a good indicator of the porting progress (how many functions were ported, how many remains). Focusing on the function call chains that are a common part of a failure/crash or touch on the new functionality will allow us to leverage the [Pareto principle](https://en.wikipedia.org/wiki/Pareto_principle), -advancing on 80% of desired Value (stability, functionality) with 20% of effort. \ No newline at end of file +advancing on 80% of desired Value (stability, functionality) with 20% of initial effort. \ No newline at end of file From 717c923e08bfe2198aacb725e1e37622f867b4a6 Mon Sep 17 00:00:00 2001 From: Artem Date: Mon, 30 Jul 2018 21:46:24 +0300 Subject: [PATCH 4/5] Bullets seems to look nicer after all --- iguana/exchanges/mm2.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/iguana/exchanges/mm2.md b/iguana/exchanges/mm2.md index 718209486..b88dfb815 100644 --- a/iguana/exchanges/mm2.md +++ b/iguana/exchanges/mm2.md @@ -91,11 +91,11 @@ Let's list the good things that should come out of the gradual rewrite: The plan so far is to by default use the C functions as the atomic units of rewrite. Rust FFI allows us to swap any C function with a similar Rust function. -Porting on this level we -a) reuse the function-level modularity of the C language; -b) preserve the code meta-data (Git history will show a nice diff between the C and Rust functions, we'll be able to easily investigate the evolution of the code back to its roots); -c) avoid the complexity and slow-downs associated with adding RPC/networking layers or drawing new lines of abstraction; -d) have a good indicator of the porting progress (how many functions were ported, how many remains). +Porting on this level we +* reuse the function-level modularity of the C language; +* preserve the code meta-data (Git history will show a nice diff between the C and Rust functions, we'll be able to easily investigate the evolution of the code back to its roots); +* avoid the complexity and slow-downs associated with adding RPC/networking layers or drawing new lines of abstraction; +* have a good indicator of the porting progress (how many functions were ported, how many remains). Focusing on the function call chains that are a common part of a failure/crash or touch on the new functionality will allow us to leverage the [Pareto principle](https://en.wikipedia.org/wiki/Pareto_principle), From 896749df625a751037870f14cb0aa1b4c2e1764d Mon Sep 17 00:00:00 2001 From: Mihail Fedorov Date: Fri, 10 Aug 2018 02:30:42 +0300 Subject: [PATCH 5/5] Testing IP list dependency bug --- iguana/m_notary_run | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/iguana/m_notary_run b/iguana/m_notary_run index 770e30e6f..1ef0844fa 100755 --- a/iguana/m_notary_run +++ b/iguana/m_notary_run @@ -18,7 +18,9 @@ sleep 4 curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"SuperNET\",\"method\":\"myipaddr\",\"ipaddr\":\"$myip\"}" sleep 3 curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"addnotary\",\"ipaddr\":\"78.47.196.146\"}" +curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"addnotary\",\"ipaddr\":\"95.213.238.98\"}" curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"addnotary\",\"ipaddr\":\"82.202.193.98\"}" +curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"addnotary\",\"ipaddr\":\"82.202.193.100\"}" curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"addnotary\",\"ipaddr\":\"54.95.68.31\"}" curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"addnotary\",\"ipaddr\":\"142.54.164.114\"}" curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"addnotary\",\"ipaddr\":\"103.6.12.111\"}" @@ -27,7 +29,7 @@ curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"ad curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"addnotary\",\"ipaddr\":\"139.99.149.41\"}" curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"addnotary\",\"ipaddr\":\"209.58.169.65\"}" curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"addnotary\",\"ipaddr\":\"5.189.232.34\"}" - +curl --url "http://127.0.0.1:7776" --data "{\"agent\":\"iguana\",\"method\":\"addnotary\",\"ipaddr\":\"46.4.33.66\"}" #tests/addnotarys_7776 coins/btc_7776