Parity-ethereum: feature request: allow to send already, before chain is fully synced

[drandreaskrueger]

repurposed for feature request:

see below https://github.com/paritytech/parity/issues/7801#issuecomment-373923355

---

old post:

• Parity/v1.8.7-stable-e3d32a4ab-20180123/x86_64-linux-gnu/rustc1.23.0
• Linux
• gdebi .deb
• fully synchronized: no
• network ethereum
• restart the node? yes

Incredibly slow syncing

when after many hours it was still not synced, I measured it:

syncing
from (#5002747) Jan-31-2018 12:47:45 AM
to (#5002765) Jan-31-2018 12:54:28 AM
i.e. blockchain time of 06:43 minutes

took from Sat 3 Feb 19:50:38 GMT 2018
to Sat 3 Feb 19:56:46 GMT 2018
so 06:08 minutes.

so syncing takes 91% of the blockchain time ? ? ?

To catch up those 3 days ... needs 2.73 days, and by then the goal will have moved by 2.73 days.

It feels like Zeno's Paradox

In my despair, I have now deleted the huge (35GB) db folder 906a34e69aec8c0d, and have started with these switches:

parity --no-ancient-blocks --no-serve-light --db-compaction=ssd ui

does any of those actually influence the syncing speed?

What else can I do to speed up parity?

Are all ethereum clients seeing this problem at the moment?

A new way to think about unscalable systems, I suppose - make syncing slower than mining.

[drandreaskrueger]

Alright, so that worked.

started warp restore at   Sat  3 Feb 20:04:16 GMT 2018
ended  warp restore  at  Sat  3 Feb 20:35:09 GMT 2018

remaining blocks:
#5020001 at   Sat  3 Feb 20:35:09 GMT 2018
#5025314 at   Sat  3 Feb 21:08:30 GMT 2018

current size of 906a34e69aec8c0d = 13.3 GB

so for the future, when I have not synced for a few days, should I rather always delete the then old database, and always start from scratch?

what was the reason why it was so slow before? does the rocksDB get slower above a certain size?

[5chdn]

It's Ethereum growing so fast. Not much we can do about it. We will prepare a light client soon.

[drandreaskrueger]

It's Ethereum growing so fast. Not much we can do about it.

Ouch.

If I have not only seen an odd artifact on my one machine (which back then went away after a complete deletion of the chain, and resyncing from last snapshot), but a general behavior of Ethereum, then it sounds as if ... the 15s block time needs to be increased, or the gas price, or the gas limit decreased, no? --> Less EVM calculations per second.

[drandreaskrueger]

Is the following the shortest possible routine each time I want to start parity?

parity db kill
parity --warp --no-ancient-blocks ui

any other switches that can be useful for this situation?

Looks like the only way now to have a synced Ethereum wallet is to keep a node running 24/7, no?

[drandreaskrueger]

@5chdn please reopen, until we have fully understood & documented what to do, thanks

[drandreaskrueger]

what is going wrong here?

2018-03-17 13:04:33 UTC Syncing snapshot 801/807        #0   25/25 peers   8 KiB chain 3 MiB db 0 bytes queue 10 KiB sync  RPC:  7 conn,  2 req/s,  82 µs
2018-03-17 13:04:38 UTC Syncing snapshot 806/807        #0   25/25 peers   8 KiB chain 3 MiB db 0 bytes queue 10 KiB sync  RPC:  7 conn,  2 req/s,  84 µs
2018-03-17 13:04:48 UTC Syncing #4880001 3f40…f815     0 blk/s   20 tx/s   0 Mgas/s     11+   76 Qed  #4880095   24/25 peers   945 KiB chain 2 MiB db 16 MiB queue 2 MiB sync  RPC:  7 conn,  7 req/s,  68 µs
2018-03-17 13:04:53 UTC Syncing #4880005 68e1…328e     0 blk/s  189 tx/s   6 Mgas/s      0+  118 Qed  #4880127   24/25 peers   1 MiB chain 10 MiB db 23 MiB queue 8 MiB sync  RPC:  7 conn,  3 req/s,  69 µs
2018-03-17 13:05:03 UTC Syncing #4880016 0b17…007e     1 blk/s  225 tx/s   8 Mgas/s      0+  110 Qed  #4880127   24/25 peers   2 MiB chain 29 MiB db 22 MiB queue 28 MiB sync  RPC:  7 conn,  2 req/s,  69 µs
2018-03-17 13:05:13 UTC Syncing #4880029 8a1e…256f     1 blk/s  299 tx/s  10 Mgas/s      0+   94 Qed  #4880127   24/25 peers   2 MiB chain 56 MiB db 19 MiB queue 47 MiB sync  RPC:  7 conn,  5 req/s,  82 µs
2018-03-17 13:05:24 UTC Syncing #4880044 7b7d…dd9b     1 blk/s  301 tx/s  11 Mgas/s   1027+  185 Qed  #4881651   24/25 peers   3 MiB chain 71 MiB db 109 MiB queue 13 MiB sync  RPC:  7 conn,  2 req/s,  78 µs

but the current block is 5271351.

that is 5271351 - 4881651 = 389,700 blocks in the future.

It would take weeks to catch up with that.

[drandreaskrueger]

If the above was the right way, i.e. there is nothing I can do now ...

then parity (or probably also geth for that matter, or is geth syncing cleverer, faster, differently?) have become completely useless (unless for 24/7 nodes). Not your fault of course, it's the curse of popularity, I assume.

But ME, I need a solution now. How to move my money?

Am I right in this? --> parity does not seem to allow sending funds until the chain is synced. Which is pretty stupid. Because via e.g. etherscan I can see my balance, and thus can decide myself whether I can actually send funds. No need for parity blocking me to do that, no?

Please create one more option "allow to send even before chain is synced" - thanks.

Second workaround now:

--> What is the best light client (which supports tokens) to import all my parity accounts into?

Then I could let go of asking more annoying questions, because I simply stop using it. Thanks.

[drandreaskrueger]

feature request 2: snapshots more often

why would I need 400k blocks to catch up?

[drandreaskrueger]

I am trying again now, without the --no-ancient-blocks

so

parity db kill
parity --warp ui

but now it does this:

2018-03-17 14:24:10 UTC Syncing     #405 9b12…379a    40 blk/s    0 tx/s   0 Mgas/s     98+    0 Qed      #508    7/25 peers   465 KiB chain 5 MiB db 206 KiB queue 330 KiB sync  RPC:  1 conn, 26 req/s, 190 µs
2018-03-17 14:24:15 UTC Syncing    #2286 13d2…9c77   376 blk/s    0 tx/s   0 Mgas/s      0+    0 Qed     #2286    8/25 peers   3 MiB chain 13 MiB db 0 bytes queue 3 MiB sync  RPC:  1 conn, 27 req/s, 2435 µs
2018-03-17 14:24:25 UTC Syncing    #3800 fa8a…ef60   151 blk/s    0 tx/s   0 Mgas/s   1659+    0 Qed     #5465   16/25 peers   5 MiB chain 23 MiB db 3 MiB queue 11 MiB sync  RPC:  1 conn, 15 req/s, 7067 µs
2018-03-17 14:24:30 UTC Syncing    #6813 ebe6…2c04   602 blk/s    0 tx/s   0 Mgas/s  14528+    5 Qed    #21353   17/25 peers   8 MiB chain 31 MiB db 23 MiB queue 6 MiB sync  RPC:  1 conn, 21 req/s, 49712 µs

which is completely bonkers.

Months from now, I would have a synced chain.

Is geth better able to solve this problem?

[drandreaskrueger]

found some more switches, using this now:

parity db kill
rm -rf ~/.local/share/io.parity.ethereum/chains/ethereum/db/906a34e69aec8c0d/
parity --warp --pruning=fast --tracing=off --db-compaction ssd --cache-size 4096 --mode active --no-ancient-blocks

without the --no-ancient-blocks it would sync the chain since genesis, see bonkers above. Trying again:

2018-03-17 14:35:50 UTC Syncing snapshot 0/807        #0   21/25 peers   8 KiB chain 3 MiB db 0 bytes queue 10 KiB sync  RPC:  0 conn,  0 req/s,   0 µs
...

by the way:

parity --version
version Parity/v1.8.9-stable-1952d05d9-20180201

[drandreaskrueger]

third workaround:

any 24/7 full node, please go into your ~/.local/share/io.parity.ethereum/chains/ethereum/db folder, and zip the 906a34e69aec8c0d folder, and upload it somewhere please.

Then I download that, and use the age-old Bitcoin type of bootstrapping via blockchain file method to catch up.

Thanks!

[drandreaskrueger]

newest version is worse

sudo gdebi ware/parity_1.9.4_debian_amd64.deb
parity --version
  version Parity/v1.9.4-beta-6f21a32b2-20180228

parity db kill
parity --warp --pruning=fast --tracing=off --db-compaction ssd --cache-size 4096 --mode active --no-ancient-blocks

Option '--warp' does nothing. It's on by default.

2018-03-17 14:55:41 UTC Starting Parity/v1.9.4-beta-6f21a32b2-20180228/x86_64-linux-gnu/rustc1.23.0
2018-03-17 14:55:41 UTC Keys path /home/andreas/.local/share/io.parity.ethereum/keys/Foundation
2018-03-17 14:55:41 UTC DB path /home/andreas/.local/share/io.parity.ethereum/chains/ethereum/db/906a34e69aec8c0d
2018-03-17 14:55:41 UTC Path to dapps /home/andreas/.local/share/io.parity.ethereum/dapps
2018-03-17 14:55:41 UTC State DB configuration: fast
2018-03-17 14:55:41 UTC Operating mode: active
2018-03-17 14:55:41 UTC Configured for Foundation using Ethash engine
2018-03-17 14:55:42 UTC Updated conversion rate to Ξ1 = US$581.90 (204584300 wei/gas)
2018-03-17 14:55:46 UTC Public node URL: enode://c389a4a1c3ce98f075d941dcd8fc566ad6f35af951a3160e5c2e9221d2fdba8ee671aac95cb3541c0f5a889b376b67804e170a557474bfeb507c12eacbd9ed0c@192.168.1.141:30303
2018-03-17 14:55:51 UTC Syncing       #0 d4e5…8fa3     0 blk/s    0 tx/s   0 Mgas/s      0+    0 Qed        #0    5/25 peers   8 KiB chain 3 MiB db 0 bytes queue 19 KiB sync  RPC:  0 conn,  0 req/s,   0 µs
2018-03-17 14:56:01 UTC Syncing       #0 d4e5…8fa3     0 blk/s    0 tx/s   0 Mgas/s      0+    0 Qed        #0    6/25 peers   8 KiB chain 3 MiB db 0 bytes queue 19 KiB sync  RPC:  0 conn,  0 req/s,   0 µs
2018-03-17 14:56:06 UTC Syncing       #0 d4e5…8fa3     0 blk/s    0 tx/s   0 Mgas/s      0+    0 Qed        #0    6/25 peers   8 KiB chain 3 MiB db 0 bytes queue 19 KiB sync  RPC:  0 conn,  0 req/s,   0 µs
2018-03-17 14:56:16 UTC Syncing    #2808 e45b…bf48   280 blk/s    0 tx/s   0 Mgas/s   3278+    5 Qed     #6097   10/25 peers   3 MiB chain 14 MiB db 6 MiB queue 6 MiB sync  RPC:  0 conn,  0 req/s,   0 µs
2018-03-17 14:56:26 UTC Syncing    #9816 d495…5127   701 blk/s    0 tx/s   0 Mgas/s   8360+    0 Qed    
...

ooooohhhh nooooo.

That one now goes direectly into full bonkers mode.

why is it ignoring the --no-ancient-blocks switch, to choose only snapshot mode??

Is there a list of snapshot servers perhaps, that I can manually add, or how else can I force parity to sync from last snapshot?

[drandreaskrueger]

What am I doing wrong?

What are you doing wrong?

What is wrong with the network?

[drandreaskrueger]

third workaround:

any 24/7 full node, please go into your ~/.local/share/io.parity.ethereum/chains/ethereum/db folder, and zip the 906a34e69aec8c0d folder, and upload it somewhere please.

Then I download that, and use the age-old Bitcoin type of bootstrapping via blockchain file method to catch up.

Thanks!