Light-client: Light client on low-end devices

[manuelwedler]

Description

This issue is for documenting what I have done to get the light client running on low.js and why it is failing on our NanoPi. The initial goal for this project was to get a better transaction speed for our train demo and being able to run the light client on small devices and for that on an IoT JavaScript engine.

Process

In the beginning I tried to use IoT.js (built on JerryScript engine), which adds some functionality Node.js-like functionality. But it showed that Node.js-like is not enough, as a lot of the light client's dependecies use Node.js' standard library. So we need a framework that provides most of Node.js' functionality to get all dependencies running.
I came across low.js, which states to be a port of Node and has currently implemented 70% Node's library. A list what is implemented is documented here. Low.js is built on the Duktape engine.

I could get Express running on it pretty quickly. But still there are some differences and missing functionalities to Node's library, so I needed to add some patches to fix that.
First I identified which dependencies of the light client are causing issues. Then I created patches for these dependecies, so that they are working on low.js. I am using Webpack to apply these patches and to add some polyfills. It is also transpiling the code to ES5, as it is the supported JS version. See the lowjs-patch repository below for details.

In the end I was able to use the light client on low.js flawlessly on my local machine. Unfortunately not on the Nanopi.

Problem

Sooner or later the light client runs into a segmentation fault on the NanoPI (ARM v7 architecture). We tried different operating systems and also a RaspberryPi (ARM v6), but there was always this segmentation fault. I also tried different light client versions, v0.1.1 and current master.

We didn't try debugging the segmentation fault itself as it can be very hard and we don't know if it would be successful in any reasonable time. The reason for the segmentation fault could be anything from an incompatible library to a bug in low.js itself.

Besides that I was able to make a transfer on the NanoPi one time and it took very long (at least a minute), before it ran into the segmentation fault. But that was using the old version v0.1.1 and I could not make a transfer again until now. I think as long as we have the segmentation fault it does not make sense to investigate that further.

Further steps

• [ ] @eorituz ordered another board (ARM v8 / 64bit). We will check if a different architecture changes anything
• [ ] create an issue in the low.js repository

Resulting repositories

1. https://github.com/raiden-network/light-client-rest-api
2. https://github.com/raiden-network/light-client-lowjs-patch

1 is an express app providing a REST API on top of the light client. It can be further developed to mimic the Raiden full client API and may be used in the scenario player then. At the moment it only has basic functionality.
2 are the patches and the Webpack configuration for running the express app on low.js.

Tasks

• [ ] See if we can close this issue

[christianbrb]

@manuelwedler Thanks for your awesome work :)

We should do a feasibility assessment first and not try to execute it end to end

[christianbrb]

Discussed with @eorituz to timebox it and looking at it starting 19.12.2019

[christianbrb]

We invest 1-2 days in investigating the issue

[andrevmatos]

As @eorituz and me talked, one of my bets is that some architecture/library/compilation issue is causing the segfault, so he would try (and already did) to compile low.js from sources on the NanoPi with the same OS and libs.
Now we can test again on the new compilation and see if it works better.
Meanwhile, I'm going to develop a small script using the SDK to quickly perform operations from a given account, provider, etc on a single call and then exit. This should give us a quick tool to both test the SDK and more easily pinpoint the reason of any issue/segfault, as well as a smoketest cli and E2E tool to integrate with other tests.

[andrevmatos]

Update: I managed to create a standalone bundle JS ES5 script using the SDK & webpack, which doesn't require any external dependency, registers and syncs given account (todo: add basic functionality to perform some actual operations, like open channel & transfer).
It packs everything in 2.4MB, and peak node memory usage is around 55MB at runtime, which points it should be good enough to run with full Node.js even on quite constrained environments.
Low.js unfortunatelly isn't working with the current version of it, giving some parsing error, even though everything got properly transpiled to ES5 JS.

[christianbrb]

We have decided to finalize the script within this iteration.

Open Tasks:

• [x] Extend the standalone script functionality
  
  
  
  • [x] Open Channel
  
  
  • [x] Deposit
  
  
  • [x] Make a transfer
  
  
• [x] Expose the tasks through Rest-API

[andrevmatos]

Work on this is being done here https://github.com/raiden-network/light-client/tree/cli/raiden-cli
bundle.js transpilation (~3.2MB) runs on ES5 without dependencies beyond node itself. Tests demonstrates, even with dependencies for the script (yargs, express, etc..), memory usage stays around 100MB of whole process, which is still acceptable for low-end devices.
It'd be nice to see it running on low.js. It's failing right now with some syntax error, either webpack is doing something weird with the es5 bundling or low.js doesn't comply properly with it. Although it seems Node.js isn't the bottleneck here and architecture/transport/libs optimizations would make it run better than actually changing the interpreter.

[manuelwedler]

Good Job here! To get it running on low.js you could try to apply the patches I created.

[andrevmatos]

I'd argue as it was demonstrated the cli app running on the NanoPi, we should/close this issue.
It's still taking 10-20s per transfer, due to some factors we can try to workaround (performance improvements we're doing thanks to work intiated on this issue) and some we can't (protocol/transport limitations we need to work with for now).
@nephix is doing some tests to and profiling to identify more optimization targets, but it is running, and we could track this work on the remaining issues. @manuelwedler , it'd be great if you could test the current solution again on low.js . Thank you for all you've done so far, let's see how this progresses.

[christianbrb]

@nephix Did you document the profiling results somewhere?

[nephix]

@christianbrb I spoke with @andrevmatos and it was basically what had already mentioned: Matrix is the bottleneck

[christianbrb]

Performance analysis from @nephix on normal hardware: https://github.com/raiden-network/light-client/issues/876#issuecomment-574045285

[manuelwedler]

I could get the script parsed correctly by low.js by using babel. Unfortunately some dependencies are making problems again with TypeErrors. It would take me some time to fix it and I don't think it's worth it.

[christianbrb]

Next step: @eorituz to test the CLI script described in https://github.com/raiden-network/light-client/issues/551#issuecomment-572226618

[nephix]

CLI has been tested in #876