Status-react: [Perf] Test hypothesis 3: Whisper network overhead

[oskarth]

Identified Behavior

@churik observed that Status takes up a lot of bandwidth when doing common operations, in total two orders of magnitude more than Telegram.

Specifically, creating an account takes ~30mb vs <1mb for Telegram.

• [x] User story part of MVP scope
• [x] Identified end user behavior

Hypothesis

Whisper overhead is cause of x30+ data for creating an account when using Ropsten RPC.

(Hypothesis 2: Blockchain is still being synced when using Ropsten RPC.)

UPDATE 2018-01-20: Hypothesis 3: p2p layer is causing network overhead.

Assumptions

1) Assuming this data usage is repeatable and corresponds roughly 1-1 to specified action
2) Assuming this is a critical problem for end user
3) Assuming this is reasonably fixable (order of magnitude of TG)

Experiment

• [ ] Repeat action and confirm data usage looks the same
• [ ] Profile data usage and see where 80% of it comes from

Acceptance criteria

• [ ] Numbers on where majority of data usage comes from
• [ ] Creation of issue with proposal on fix to get within order of magnitude of TG
• [ ] Basic understanding of relation to findings in this issue and Chu's other findings

Notes

• https://github.com/status-im/status-react/issues/2931

[oskarth]

@divan do you want to look into this? I know you mentioned it a bit.

[oskarth]

@annadanchenko pointed out it might also be a good idea to test most common user action, which is probably:

chatting for 1 min (emoji/text) | ~0,2 MB | ~3-10 MB | messages contains emojis, text, long messages pasted from clipboard

It'd also be OK to replace specific hypothesis with one related to this. The general idea is to use a specific scenario and spike test, as there's likely to be overlap in solution. If there isn't it'd be good to have a hunch as to why for future experiments.

[divan]

@oskarth I'm working on the code that will allow testing this hypothesis. If I'm going to do the verification myself — cannot answer right now.

[oskarth]

@divan Gotcha, thanks. Probably worth checking with @JekaMas so there is no overlap in effort, he is doing something similar I believe.

[JekaMas]

@divan @oskarth I've done with whisper network overhead. I've tested network overhead of clean geth without status app to get a bias.

Test env: Geth/v1.8.0-unstable-3f40b22d/linux-amd64/go1.9.2
Metodology: each case ran for 10 minutes on two independent nodes at the same time
Script: https://gist.github.com/JekaMas/bd726dbc563864998d1df191b6976e1e

IDLE

With whisper

Stats (bytes/s):

| | Receive | Transmit | Total |
|---|---|---|---|
| mean | 370813 | 370085 | 740899 |
| std | 218636 | 211305 | 429757 |
| min | 506 | 132 | 638 |
| 25% | 179029 | 186036 | 365065 |
| 50% | 374748 | 377393 | 752141 |
| 75% | 552994 | 550450 | 1103444 |
| max | 799617 | 739831 | 1539448 |

Traffic sum(bytes):

| | |
|---|---|
| Receive | 222487946 |
| Transmit | 222051570 |
| Total | 444539516 |

Without whisper

Stats (bytes/s):

| | Receive | Transmit | Total |
|---|---|---|---|
| mean | 231629 | 354676 | 586305 |
| std | 128071 | 201885 | 329736 |
| min | 506 | 3226 | 3732 |
| 25% | 120942 | 179424 | 300366 |
| 50% | 250381 | 364511 | 614892 |
| 75% | 334190 | 525168 | 859358 |
| max | 473914 | 706034 | 1179948 |

Traffic sum(bytes):

| | |
|---|---|
| Receive | 138977652 |
| Transmit | 212805692 |
| Total | 351783344 |

Create account

With whisper

Stats (bytes/s):

| | Receive | Transmit | Total |
|---|---|---|---|
| mean | 27126 | 39463 | 66589 |
| std | 19041 | 22432 | 41296 |
| min | 506 | 3226 | 3732 |
| 25% | 8793 | 19411 | 28205 |
| 50% | 27531 | 40407 | 67938 |
| 75% | 36440 | 56470 | 92911 |
| max | 69192 | 80766 | 149958 |

Traffic sum(bytes):

| | |
|---|---|
| Receive | 1627560 |
| Transmit | 2367788 |
| Total | 3995348 |

Without whisper

Stats (bytes/s):

| | Receive | Transmit | Total |
|---|---|---|---|
| mean | 25840 | 38924 | 64765 |
| std | 17559 | 24042 | 41487 |
| min | 506 | 330 | 836 |
| 25% | 9923 | 18095 | 28019 |
| 50% | 24173 | 37437 | 61610 |
| 75% | 34852 | 56276 | 91128 |
| max | 61951 | 82216 | 144167 |

Traffic sum(bytes):

| | |
|---|---|
| Receive | 1550436 |
| Transmit | 2335474 |
| Total | 3885910 |

1-to-1 chat

With whisper

Stats (bytes/s):

| | Receive | Transmit | Total |
|---|---|---|---|
| mean | 419388 | 386979 | 806367 |
| std | 276086 | 245251 | 521201 |
| min | 426 | 3652 | 4078 |
| 25% | 157804 | 165446 | 323250 |
| 50% | 408736 | 374733 | 783469 |
| 75% | 638422 | 594332 | 1232754 |
| max | 907451 | 819841 | 1727292 |

Traffic sum(bytes):

| | |
|---|---|
| Receive | 251633287 |
| Transmit | 232187419 |
| Total | 483820706 |

Without whisper

Stats (bytes/s):

| | Receive | Transmit | Total |
|---|---|---|---|
| mean | 268071 | 364106 | 632177 |
| std | 201343 | 235094 | 436076 |
| min | 426 | 330 | 756 |
| 25% | 85817 | 150765 | 236583 |
| 50% | 244263 | 348307 | 592570 |
| 75% | 422895 | 561594 | 984490 |
| max | 639356 | 781904 | 1421260 |

Traffic sum(bytes):

| | |
|---|---|
| Receive | 160843101 |
| Transmit | 218463610 |
| Total | 379306711 |

Network overhead

| | Whisper | W/o Whisper | Diff |
|---|---|---|---|
| IDLE | 444539516 | 351783344 | +20.87% |
| Create account | 3995348 | 3885910 | +2.74% |
| 1-to-1 chat | 483820706 | 379306711 | +21.6% |

Result: Whisper adds 20% of traffic in idle and chatting however this overhead relates to geth themself and we can't quickly fix this issue. So i suggest to monitor this metrics from time to time and consentrace on performance issues.

[oskarth]

@JekaMas Would you mind making the main metrics on the same format as @churik's investigation? So it is more clear what the difference is in terms of magnitude.

[feuGeneA]

@oskarth , I took a quick stab at trying to correlate @JekaMas ' results above with @churik 's results in #2931 . In short, I didn't find much of a correlation.

I made the following assumptions:

• churik had whisper enabled, implying her data should be compared to Jeka's "With whisper" data.
• churik's data consumption numbers are a sum of both receive and transmit totals.
• churik's "chatting for 1 min" observation was in a 1:1 chat scenario.

The 1:1 chat action is the most relatable. churik observed between 3 and 10 MB of traffic for 1 minute of chatting, which equates to between 52,429 and 174,763 bytes per second. Compare this to Jeka's mean bytes per second of 806,637. The vast difference in those measurements may be due to the volume of chat messages being transmitted (churik said "messages contains emojis, text, long messages pasted from clipboard" ... not sure what sort of data Jeka was using). Or it may be that churik's sample size (1 minute) was not enough to infer a realistic mean (compared to Jeka's 10 minutes). Or, of course, it may be something else entirely.

As for account creation, Jeka observed a total of 3 MB bytes of data consumption (reported as 3,995,348 bytes), while churik observed 30 MB. Of potential importance here is Jarrad's comment suggesting that the data flows for this action are for blockchain synchronization, not whisper overhead.

And as for idle time, it's not clear whether the numbers are comparable, as churik's data was recorded while the app was in the background, and Jeka's was (presumably) recorded while idle in foreground. In any case, churik observed between 0.01 and 0.1 MB/s, and Jeka observed an average of 0.7 MB (reported as 740,899 bytes) per second.

What do these imply about the next steps for the Experiment at hand?

[dshulyak]

How is it relevant to measure network traffic when account is being created?
As far as I understand it won't have any impact on it. Neither on whisper, nor on les/eth. This is just a coincidence that when the account is being created new blocks or envelopes were received by a node.

[oskarth]

@JekaMas Who is currently owning this issue? If it is deemed to no longer be relevantly / badly specified, where is the continuation?

[status-github-bot[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[status-github-bot[bot]]

This issue has been automatically closed. Please re-open if this issue is important to you.