Go-ethereum: What is the upper bound of "imported new state entries"? When will it end?
System information
Geth version: 1.7.2
OS & Version: OSX
Expected behaviour
geth --fast should finish soon.
Actual behaviour
it run for 3days and print “Imported new state entries count=384 elapsed=26.970ms processed=50023987 pending=33074 retry=0 duplicate=19087 unexpected=47765” constantly
Steps to reproduce the behaviour
run geth --fast in console
Backtrace
INFO [12-06|07:08:00] Imported new state entries count=1259 elapsed=12.971ms processed=50014526 pending=34891 retry=0 duplicate=19087 unexpected=47765
INFO [12-06|07:08:23] Imported new state entries count=774 elapsed=8.950ms processed=50015300 pending=34311 retry=0 duplicate=19087 unexpected=47765
INFO [12-06|07:08:31] Imported new state entries count=1125 elapsed=9.513ms processed=50016425 pending=33428 retry=0 duplicate=19087 unexpected=47765
INFO [12-06|07:08:39] Imported new state entries count=1061 elapsed=11.198ms processed=50017486 pending=32566 retry=0 duplicate=19087 unexpected=47765
INFO [12-06|07:08:49] Imported new state entries count=1314 elapsed=12.041ms processed=50018800 pending=31248 retry=0 duplicate=19087 unexpected=47765
INFO [12-06|07:09:10] Imported new state entries count=1028 elapsed=10.446ms processed=50019828 pending=30496 retry=0 duplicate=19087 unexpected=47765
INFO [12-06|07:09:25] Imported new state entries count=1241 elapsed=10.423ms processed=50021069 pending=30088 retry=0 duplicate=19087 unexpected=47765
INFO [12-06|07:09:37] Imported new state entries count=777 elapsed=6.224ms processed=50021846 pending=29851 retry=26 duplicate=19087 unexpected=47765
idotial
All 154 comments
I don't exactly know "how long" it would take, but it's suggested to let it run for as long as it takes. It took me 12-15 hours on my 8-core i7 machine with --cache set to 10 GB of RAM.
asgs
on 6 Dec 2017
Similar issues https://github.com/ethereum/go-ethereum/issues/14647 and https://github.com/ethereum/go-ethereum/issues/15001
asgs
on 6 Dec 2017
@asgs I have read issues before, they don't make any sense. I just want to know how long I have to wait, and should I stop it. yesterday eth.syncing return false, and few minutes later, it print like "{
currentBlock: 4685483,
highestBlock: 4685675,
knownStates: 51479249,
pulledStates: 51462166,
startingBlock: 4681817
}" again. should I stop it, and update geth and mist? I see there is a new version 1.7.3.
idotial
on 7 Dec 2017
@idotial how long have u been waiting and do u have an ssd/fast internet? No u probably shouldnt stop it, but you can try... it took me like 6 hours on importing the states part yesterday. also how much ram do u have? maybe thats an issue
ugotjelly
on 7 Dec 2017
i have the same problem i am using Version: 1.7.3-stable on OSX i used geth syncmode="light" in terminal.
ive got 251 GB Flash Storage and 8gb of ram
here is theeth.syncing so far
currentBlock: 4762485,
highestBlock: 4762629,
knownStates: 14708481,
pulledStates: 14675786,
startingBlock: 0
The problem is i dont have enough disk space or i have 7GB and im not sure weather to throw away my old chain.data and trust it will be ok. Because if it doesnt work i will have to do it again and it still might not work and i wont have any chain.data All this to get my DAO withdrawal contract to work.
How much space did you need to finish the sync and did it work?
keznerve
on 20 Dec 2017
Did it ever finish?
xstalpha
on 28 Dec 2017
@xstalpha It's still running, but it start to Imported new chain segment, so I guest it's alright.
idotial
on 29 Dec 2017
@keznerve just stop it. you can add an Portable Hard Disk, and rerun it.
idotial
on 29 Dec 2017
@ugotjelly I stop it, and restart it few times. and now the log is 'Imported new chain segment', so maybe it's will finish in several days.
idotial
on 29 Dec 2017
Hi yes it finished ...it went up to 38GB on the 19th ofDec it got me to clear out all the crap on my Mac, to save space ; the Import new chain segment lasted a a few hours it all made sense in the end , it had more warnings towards the finish but i didnt stop it. The whole thing took around 15hrs . I havent done it since. I did get my ETH from the withdraw DAO contract.
keznerve
on 29 Dec 2017
@keznerve congratulation! It seems you have better network
idotial
on 2 Jan 2018
In my case millions of Imported new state entries happen at the ens of fast sync. The import works fine for some time, then slows down to almost stop while continuously dropping peers Stalling state sync, dropping peer.
I have tried to serve light peers, but in that case import of the state always fails after some time with error Node data write error. So I disabled --lightserv and ended up with the issue described above.
I run dedicated node in cloud, with SSD and fast internet. Cache is 2GB (--cache 2048).
GoodMirek
on 13 Jan 2018
I was also seeing this with current master (1.8.0). 1.7.3 is pulling in chain state properly however is too slow to catch up on non-SSD disks.
I think this PR is going to be a big win.
ekryski
on 19 Jan 2018
I just finished fast syncing with Geth 1.7.3 64-bit for Windows using an SSD and wanted to leave this data point for others. It took a little more than 2 days.
eth.syncing
{
currentBlock: 4999894,
highestBlock: 4999897,
knownStates: 80999148,
pulledStates: 80999148,
startingBlock: 4999894
}
Yep, finished syncing just short of block 5,000,000. That's about 81 million known states for 5 million blocks. The database size on disk is 66,323,816,699. I kept getting crashes using the 1.7.2 that came with the wallet even going to an SSD. Parity in warp mode was also crashing.
100Jason
on 30 Jan 2018
thanks much
jackoelv
on 6 Feb 2018
81 million known states for 5 million blocks 😱 😨
Currently at currentBlock=5071336 and knownStates=66991785
At least I know there's some way to go yet...
Been at this for ~8 hours on an Amazon EC2 m5.4xlarge.
hynese
on 11 Feb 2018
@hynese did it finish?
rhaps107
on 12 Feb 2018
Haven't had a chance to check yet. I was up over 80 million state at midnight last night (15 hours ago) and was still at eth.blockNumber=0 then.
hynese
on 12 Feb 2018
Yes, I'm in sync now.
Finally...
Several hours to go from 0 to ~85 million knownStates
hynese
on 12 Feb 2018
Appreciate you guys letting us know how many ironically named "known states" there are :-D
martindye
on 15 Feb 2018
@martindye it finished for (took 15 hours). Unfortunately after download that metrics are hidden, but i think last number of states i saw was 100M
rhaps107
on 16 Feb 2018
On a machine with slow disk subsystem and the new trie cache (merged in v1.8.0) it took six days to perform fast sync. Before the new cache has been implemented, the node was not able to stay in sync.
State entries import stopped just before 100M mark:
INFO [02-17|18:26:09] Imported new state entries count=221 elapsed=3.390ms processed=99338822 pending=0 retry=0 duplicate=33987 unexpected=139311
INFO [02-17|18:26:09] Imported new block receipts count=0 elapsed=343.488µs number=5081106 hash=bae2fd…909c39 size=0.00B ignored=1
INFO [02-17|18:26:09] Committed new head block number=5081106 hash=bae2fd…909c39
is still being updated...
{
currentBlock: 5204440,
highestBlock: 5211269,
knownStates: 100913700,
pulledStates: 100893264,
startingBlock: 0
}
torinat
on 7 Mar 2018
i downloaded the Geth 1.8.1 on ethereum wallet and got locked out...unfortunately i didnt have any luck after December running fast client. I have had to ditch the Node for now. Hope it all works out soon .
keznerve
on 7 Mar 2018
INFO [04-17|10:02:46] Imported new state entries count=0 elapsed=729.752ms processed=121292080 pending=2995 retry=16 duplicate=26888 unexpected=32477
INFO [04-17|10:02:46] Imported new state entries count=200 elapsed=17.6µs processed=121292280 pending=2995 retry=16 duplicate=26888 unexpected=32477
INFO [04-17|10:02:50] Imported new state entries count=0 elapsed=164.846ms processed=121292280 pending=3186 retry=92 duplicate=26981 unexpected=32477
INFO [04-17|10:02:51] Imported new block headers count=0 elapsed=86.325µs number=5457364 hash=3054b0…74d7ae ignored=1
WARN [04-17|10:03:00] Synchronisation failed, retrying err="block body download canceled (requested)"
INFO [04-17|10:03:34] Imported new block headers count=2 elapsed=7.211ms number=5457378 hash=fb48cd…4ba488 ignored=79
INFO [04-17|10:03:35] Imported new block headers count=2 elapsed=5.819ms number=5457380 hash=67b961…83eceb ignored=0
what is hapening?
johnkevinbarrera
on 17 Apr 2018
approximately 130 millions, three weeks ago(state entries)
torinat
on 17 Apr 2018
Just finished today, the last knownStates I saw is 130622327
tobernguyen
on 4 May 2018
As for 8th May 2018, with geth 1.8.7-stable you'll need 87GB and have downloaded 134526609 "knownStates" for a t2.medium instance on AWS with their default SSD. The fast sync completed in about 42hours. Hope that helps some people !
erickhun
on 8 May 2018
Just wanted to say thanks to the last 3 responders for actually giving a number! Every other thread on this topic just tells people to run in light mode, use MEW, etc, etc. At least I know what I'm in for here (I'm chugging along at 129mm states at the moment, hoping the target is about 140mm now). I'll endeavor to post the most recent state number when complete. Would be nice if this number could be found somewhere (like etherscan).
The answer: As of May 18, it finally completed at 140,936,000
TS2Git
on 19 May 2018
just want to know say for an instance of 8Gb of memory, what the cache setting should be? 4096?
my current eth.syncing result is
{
currentBlock: 5637459,
highestBlock: 5637534,
knownStates: 51360598,
pulledStates: 51336172,
startingBlock: 4363788
}
already running for 20 hours. and yesterday, the disk IO was 180/170 write/read per second, the cpu went up 60-70%, now it's almost like it's doing nothing, everything is low.
so when it's pulling states (instead of blocks), it almost seems like the current geth (1.8.8 stable) is not trying very hard to complete the job...
process:
geth --syncmode fast --cache 2048 --rpc
hardware:
ec2, t2.large instance which has 8gb of memory and 2 cores.
Update on 5/20/2018:
I've restarted the geth process and it'd improved, at least the disk IO shows it's active now.
eth syncing returns
{
currentBlock: 5643662,
highestBlock: 5643742,
knownStates: 133445120,
pulledStates: 133438316,
startingBlock: 5643648
}
I think it's almost done now, the knownstates should be around 14xxxxxxx
Update:
sync achieved, finally. I however can't really tell the pulledstates value because now eth.syncing returns false. anybody can tell me how to get pulledstates after you achieve sync status? the last saw was 14xxxxxxx. but now I don't know how to get pulledstates count anymore.
shawhu
on 19 May 2018
Geth is leaking memory in my setup, have ready a wrapper script to restart
it upon failure. T2 will run out of CPU credits unless you switch on T2
unlimited. Cache 2048 is fine. Very important is EBS throughput, you will
also run out of EBS credits with geth on gp2 volume.
@shawhu As you have indicated it is doing almost nothing now, I guess you might hit some of the limits mentioned above.
GoodMirek
on 19 May 2018
05/28 Finished a --syncmode "fast" after 7 1/2 days using geth 1.8.8
currentBlock: 5691828,
highestBlock: 5691837,
knownStates: 153542316,
pulledStates: 153542316,
93.4GB on Disk in my /roaming/ethereum folder
Uranium75
on 28 May 2018
i made a quick script to run in your geth console to check how fast your update is happening
https://gist.github.com/Beasta/695612bfc856450353cd6710dbfe22bb
I'm running a 24 core, 128 gb ram digital ocean droplet and getting about roughly 5000-6000 states/sec update speed (roughly 8 hours to sync if 150 million states). Seems like its rarely breaking 50% ram usage so if I was to do it again, I'd do a droplet with 64 gb of ram. Once its synced, i'll pass the chaindata folder to a smaller machine.
I did actually have it fill all 128 gb once and then crash ... memory leak maybe?
On a local machine with 4 cores and 8 gb ram and a hard disk (not SSD) i'm getting 10 states/sec update speed :/
Beasta
on 6 Jun 2018
Just give some information:
I started the syncing from May 18, It just ended minutes ago.
BlockNumber: 5740903
pulledStates: 191247645
TooBug
on 6 Jun 2018
@Beasta Thanks! Curious,
1/ Do you do the switch to the smaller machine manually ?
2/ What happen when you run out of space? Do you restart the process again?
erickhun
on 6 Jun 2018
@erickhun
I add an external volume thats 250 gb. Once its synced on the big machine, I detach the volume and move it to the small machine. I just use --datadir command to point geth to the external volume. I have yet to overfill the 250 gb drive but when it happens i'll probably just move it to a larger volume.
Using the script i posted above to calculate states/second update speed, my single core 1 gig of ram digital ocean droplet is doing roughly 400 sates/sec.
Beasta
on 7 Jun 2018
The upper bound is 196408701.
Ended today, july 5 2018.
The chaindata folder size is 115 GB.
colin-i
on 5 Jul 2018
Just give some information:
The upper bound is 181012156
Ended today, July 18 2018
The chaindata folder size is 108GB
Satoshi-Kusumoto
on 18 Jul 2018
maybe because i synced with more than one version and stopped a few months ago and restarted these days. Maybe this is the reason i have different state entries and chain data folder.
colin-i
on 18 Jul 2018
I am running --fast --cache 12288 on dedicated datacenter server for 8 days
currently
block height 6191990
state entries 204 411 181
I hope it will be synced soon
nklak
on 22 Aug 2018
@nklak any luck? running 1wk, up to block 6203435 states 203,927,275
stelloxx
on 24 Aug 2018
No, currently on 209,815,176 state entries...it is slowing down so I guess we are close.
Anybody recently synced?, can you give us a number...
nklak
on 24 Aug 2018
...2 days later... block 6216181, knownStates 209902865 ...still crunching
stelloxx
on 26 Aug 2018
I am on 217.237.711..no luck so far...
nklak
on 26 Aug 2018
insane.
stelloxx
on 26 Aug 2018
Insane is when you try to run geth with --syncmode "full" flag. On block 2.6kk the data saving slower, than blocks creating.
Snitovets
on 27 Aug 2018
the mythical "fully sync'ed ethereum node". 215,817,528 knownstates and going...
stelloxx
on 29 Aug 2018
I am on 225.538.765 and it is obvious that it is having problem keeping up. I am running the node on Virtual machine on 2CPU and 24GB RAM on 2x6TB enterprise SATA 7k2 HDD in RAID1.
In meanwhile I created cloud instance with 4vCPU, 32GB RAM and 240GB nvme SSD, and after running for 6 hours I am already on 100.000.000 entries and it can bee seen that is much much faster . If nodes on HDD have problem to keep up, I will turn it to SSD, but I am limited in size. If fast sync node is about 110-120GB now, how will it grow after sync? I know that full sync is over 800GB now. Can anybody advise?
nklak
on 29 Aug 2018
@nklak how long have you been running geth to get to 225? can you specify your command line params as well
stelloxx
on 29 Aug 2018
On server with HDD I am running geth for 18 days now...currently at 226.000.000. It takes now about one
day for 4-5 millions entries.
On server with SSD I am running for 12h now and I am already at 156.000.000 entries...It is obvious that with SSD is much much faster. When I hit the max number of state entries on SSD I will post the number and total time needed.
nklak
on 29 Aug 2018
geth --syncmode "fast" --cache 12288
that is 12GB for geth cache, but it is never over 30% of memory use.
nklak
on 29 Aug 2018
Just wanted to post this followup from my message back in January. About 2 months after being in sync and running 24 hours a day, I noticed it'd get behind and have to start syncing again. After a few hours it'd catch up and be in sync again, but eventually it hit the point where it was losing ground and not getting in sync anymore. This was despite having a good internet connection and an SSD on a computer that didn't do anything else. I got tired of hosting that huge database when it couldn't even keep it in sync anymore, so I gave up and figured out how to get the light mode working so I could at least get into my wallet. Haven't tried fast mode since.
100Jason
on 29 Aug 2018
Is 18+ days to build a "--fast" node norm? Looking around various comments it seems like either people sync-up within a week or not at all. Running almost 2wks on 200Mbit+ bandwidth, 100% dedicated hardware PowerEdge T110, Quad Xeon E3-1245 V2 @ 3.40GHz, 16GB RAM, w/ 500GB raid1 SATA (not SSD). Perpetually ~100 blocks behind with 217,857,984 knownStates so far. Running geth 1.8.13-stable-225171a4
stelloxx
on 29 Aug 2018
OK, with SSD machine geth synced in less than 24h. I can not tell the exact number of state entries as I can not find geth log?!?
But main thing here is that it seems that HDDs can not keep up, and to use geth you need to run geth on SSDs. When using fast sync blockchain database is 120GB and I would like to know how will it grow after fast sync is finished
nklak
on 30 Aug 2018
That's what I suspected, thanks! I'll try pushing geth data to an SSD and see how quickly it wraps up.
stelloxx
on 1 Sep 2018
nklak: After I was in sync back on January 30 I did a database capture (February 13th). That capture is 120 GB (so it grew from about 62 GB to 120 GB in 2 weeks). But I was running 1.7.3 at the time and it didn't have any sort of pruning. That was introduced with Iceberg. From reading the blog, it seems that the database will still grow, but how fast will be dependent upon the memory cache. I'd love to hear what size your database is in a couple of weeks.
100Jason
on 1 Sep 2018
100Jason, as I see it now, it is growing about 200-300MB a day...I will post more details later.
nklak
on 4 Sep 2018
7 hours and i am on 69245052 🍶
digiuzman
on 7 Sep 2018
Successfully synced, it took us 3-4 days, fast sync, Digital Ocean 8 GB Memory / 160 GB SSD Disk + 1000 GB HDD - Ubuntu 16.04.4 x64 , last log entry was:
Imported new state entries count=331 elapsed=4.832ms processed=211975531 pending=0 retry=0 duplicate=46561 unexpected=178269
andrewshvv
on 7 Sep 2018
Sync'ed in a few hours on SSD drive. Was over 230M known states at the time.
stelloxx
on 11 Sep 2018
Since there is no public information about current count of state entries (and I have no idea how to get that data from synced node), I'll post my latest SSD fast sync - took 24 hours, latest known state entries count was close to 240 million, geth version is 1.8.16-stable
pavel-main
on 10 Nov 2018
Centralization inside data centers is not good for the network. I fear Ethereum has had its day. We need next-gen ledgers (e.g. DAGs) like never before.
hynese
on 10 Nov 2018
hynese:
Implying any non-State actor can run a "data center" good enough to get close to a 51% attack on the Ethereum network, let alone pull it off successfully and without notice
Lols
cryptify
on 10 Nov 2018
Hi, with an SSD drive mounted on /mnt/crypto/ and the following command:
geth --syncmode "fast" --cache 2048 --datadir "/mnt/crypto/ethereum"
I'm waiting more than one week to fast sync. Any idea why?. People is reporting 24h sync with SSD drives. Any way to improve my sync? High bandwidth and dedicated server with 4Gb Ram, dual core CPU.
I'm still at 224429484 states. and more than 240000000 are reported.
top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
31312 xxx 20 0 2866744 1,662g 62608 S 48,8 44,5 23:39.21 geth
I had to restart geth several times.
rarigita
on 27 Nov 2018
I was able to sync a Bitcoin node with the snapshot here much faster than allowing it to download the data by itself, but can't find something like that for Ethereum.
Does anyone know where to find the fast sync snapshot for Ethereum?
rraallvv
on 28 Nov 2018
Still at 240000000 and growing...
INFO [11-28|10:44:19.906] Imported new state entries count=507 elapsed=5.315ms processed=240236665 pending=3470 retry=0 duplicate=4627 unexpected=28978
rarigita
on 28 Nov 2018
Still syncing:
> eth.syncing
{
currentBlock: 6793621,
highestBlock: 6793734,
knownStates: 261049222,
pulledStates: 261040316,
startingBlock: 6788824
}
rarigita
on 29 Nov 2018
@rarigita Could you please post the final number of pulledStates?
rraallvv
on 1 Dec 2018
Finally got synced at 265036678 know states and block number 6815039
rraallvv
on 3 Dec 2018
My SSD drive went Read only itself, so I had to unmount, fsck, remount and rerun geth. Now I'm at:
{
currentBlock: 6800559,
highestBlock: 6817840,
knownStates: 262895563,
pulledStates: 262895563,
startingBlock: 6798915
}
rarigita
on 3 Dec 2018
@rarigita Thanks.
rraallvv
on 3 Dec 2018
It took me around 3 days and 4 hours to synchronize a ETH Geth fast node:
Settings: --syncmode fast --gcmode full --cache 4096
Start: 2018-12-15 10:00 UTC
End: 2018-12-18 14:08 UTC
State entries: 260172278
Disk usage: 149G
Hardware: Virtual machine, 4 cores, 8 GB memory, 300 GB SSD
grittibaenz
on 18 Dec 2018
01-02 10:36:56 --> 01-05 19:20:32 UTC+8
tencent cloud hk: C2.LARGE8 - 4C8GB 600GB HDD
final entry: 267880510
disk usage: 150G
sync mode: fast
cache: 2048
downpouring
on 5 Jan 2019
For some reason mine has 217G disk usage, should I be worried about that?
rraallvv
on 5 Jan 2019
It took me around 5 days to synchronize a ETH Geth fast node:
State entries: 274079704
Disk usage: 156G
Mr-zhangQi
on 16 Jan 2019
Got it fast synced in around 4-5 days
Last pulledStates was over 279164294, db size 161GB
Needed two forced restarts because geth got into hung states (no new states coming in) for long periods (>10min).
This was using a SSD and a residential 100Mbps connection.
jdrowell
on 13 Feb 2019
Fast sync cache 2048 - 34ish hours sync.
i7 6600k
16gb ram
m.2 NVME 1TB
Used 171.1GB - measured a number of hours after sync completed.
Final State entries number - 277295920
mtj151
on 13 Feb 2019
In our case, after a long time and many #||@#!"|@ geth is synced!
It took almost a week.
SSD 50 GB
8GB RAM
i5
Used 165GB of disk space.
Final state entries number - 291739903
It is anoying, but at least it is synced!!
e-scavo
on 25 Feb 2019
Count as of 3/20/2019 is 292,494,140.
Rate of state entry import for the last 20M entries on my machine was 187M entries _per 24 hours_
https://github.com/ethereum/go-ethereum/issues/16558#issuecomment-475073726
https://github.com/ethereum/go-ethereum/issues/16558#issuecomment-475104906
deke997
on 21 Mar 2019
It would be great if the state database could be "on demand" instead of downloading the entire thing. Because I probably only care about < 1% of the total state information. Perhaps a new "light" mode that in addition to tracking block headers it downloads and stores state nodes "on demand" and compares state diffs only against state nodes my client has bothered to download.
Not sure if possible. But would be nice :)
andrewheadricke
on 22 Mar 2019
4/2/2019
303.6M
172G
mstrokin
on 2 Apr 2019
My VM has been syncing for days :(
eth.syncing
{
currentBlock: 7550331,
highestBlock: 7550402,
knownStates: 314681698,
pulledStates: 314681256,
startingBlock: 7545995
}
, any update on the current state of a synced client ?
TuanAnh207
on 12 Apr 2019
I have a linode 4GB (2 CPU cores, 4GB ram - https://www.linode.com/pricing) with an extra attached "high speed storage" device for geth.
I started geth with geth --syncmode=fast and I have been running it uninterrupted for 29 days so far.
$ geth attach --exec="eth.syncing"
{
currentBlock: 7553668,
highestBlock: 7553745,
knownStates: 241426253,
pulledStates: 241425376,
startingBlock: 0
}
I guess I still have a few weeks to go before my _fast_ sync completes.
At this point I just want to keep it running to see how long it takes until I sync all 314681698+ states... but clearly linodes are useless for running geth :(
cobordism
on 12 Apr 2019
On node with 16GB RAM and SSD storage fast sync takes about 2 days to complete. (more if your internet speed is slow). For GETH imperative is SSD, plain HDDs and even SAS can not keep up, so SSDs are only option if you want to run ethereum node.
nklak
on 13 Apr 2019
I have a linode 4GB (2 CPU cores, 4GB ram - https://www.linode.com/pricing) with an extra attached "high speed storage" device for geth.
I started
gethwithgeth --syncmode=fastand I have been running it uninterrupted for 29 days so far.$ geth attach --exec="eth.syncing" { currentBlock: 7553668, highestBlock: 7553745, knownStates: 241426253, pulledStates: 241425376, startingBlock: 0 }I guess I still have a few weeks to go before my _fast_ sync completes.
At this point I just want to keep it running to see how long it takes until I sync all 314681698+ states... but clearly linodes are useless for running geth :(
web3.eth.syncing
{
currentBlock: 7557761,
highestBlock: 7561051,
knownStates: 291739903,
pulledStates: 291739903,
startingBlock: 7554538
}
web3.eth.syncing
{
currentBlock: 7557762,
highestBlock: 7561051,
knownStates: 291739903,
pulledStates: 291739903,
startingBlock: 7554538
}
web3.eth.syncing
{
currentBlock: 7557764,
highestBlock: 7561051,
knownStates: 291739903,
pulledStates: 291739903,
startingBlock: 7554538
}
web3.eth.syncing
{
currentBlock: 7557767,
highestBlock: 7561051,
knownStates: 291739903,
pulledStates: 291739903,
startingBlock: 7554538
}
e-scavo
on 13 Apr 2019
When syncing, make sure you have enough peers you're connected to, e.g. in @e-scavo message we see that knownStates didn't change at all (not sure how long he/she waited between commands). From my experience, bottleneck is not HDD/SDD, but the network itself - sometimes it syncs states really fast, and sometimes it struggles to find a peer with enough states and gets stuck for hours.
Check your disk usage using iostat & iotop to see if geth is busy doing anything (as sometimes sync slows down due to db compression, which is expected) -- if you don't see any reads/writes for geth, restart the geth.
mstrokin
on 13 Apr 2019
I know I am not constructive, but it's just nuts to see how inefficient the whole Ethereum blockchain / apps (Geth and Parity) are designed. I don't understand why anyone would invest into this project if you have something like Bitcoin (which runs on a Raspberry Pi with dusty old spinning disks).
At the time of writing the ETH synced with Parity consumes >2.3 Terrabyte of data. I don't have a fully synced Geth node to give the numbers here, but it's probably even more. Anyone knows?
¯\_(ツ)_/¯
grittibaenz
on 15 Apr 2019
Geth node needs some 24-48h to fast-sync with SSD and good network connection. Currently it takes some 200GB of space. There is no need for anyone to run full sync so you are just wasting 2+ TB of data.
When running geth make sure that you define larger buffer for database (cache option). This is my geth startup.
geth -rpc -rpcapi eth,personal,net,web3 -cache 16384
Best regards
nklak
on 15 Apr 2019
There are business cases where you need the full blockchain to analyze the history (AML...)
grittibaenz
on 15 Apr 2019
grittibaenz please stop. It is getting tiresome how often people post this misinformation.
A full node contains all the data (~200GB).
See here for proof:
https://medium.com/@marcandrdumas/are-ethereum-full-nodes-really-full-an-experiment-b77acd086ca7
@karalabe or @fjl If you know of an online resource that shows the current number of knownStates, then I suggest we post the link here and close the issue. (and also post at issue 14647)
The only node-stats sites I can find list blocknumber and diskspace, but not number of state entries.
[or can we add the number of known state to ethstats.net?]
cobordism
on 15 Apr 2019
Update on 4/17/2019
The number of pulled states is around 305M just before it is synced
Disk: 176 GB
Running geth with larger cache (mine is 4096) would speed up the synchronization.
rexey3s
on 17 Apr 2019
Update on 4/19/2019
The number of pulled states is around 318532338 when it is synced
Disk: 178G
wh0am111
on 19 Apr 2019
1) pulledStates=306397423 / fast sync from 04-19 09:32, complete on 04-21 08:42:03 / storage: 180GB / memory 16GB / CPU: AMD Athlon(tm) X4 870K Quad Core Processor / WDC WDS100T2B0A SSD/Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz /geth --cache=4096 --syncmode "fast"
2) pulledStates=310416897 / fast sync from: 04-19 09:32, complete on: 04-22 19:55:27 / storage: 180GB / memory: 8G B/ cpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz / SSD / Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz / geth --cache=4096 --syncmode "fast"
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Upgrading chain index type=bloombits percentage=99
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Imported new block headers count=1 elapsed=7.829ms number=7617253 hash=5519b7…b14245 age=1m43s
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.213] Imported new block headers count=2 elapsed=14.470ms number=7617255 hash=1b2c47…cb5ae0 age=1m7s
Apr 22 19:56:44 eth-001 geth[2012]: INFO [04-22|19:56:44.592] Imported new chain segment blocks=1 txs=205 mgas=7.991 elapsed=1.372s mgasps=5.822 number=7617252 hash=5efb08…180758 age=2m48s cache=87.82mB
Apr 22 19:56:46 eth-001 geth[2012]: INFO [04-22|19:56:46.449] Finished upgrading chain index type=bloombits
Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new chain segment blocks=3 txs=362 mgas=23.964 elapsed=3.453s mgasps=6.938 number=7617255 hash=1b2c47…cb5ae0 age=1m12s cache=91.15mB
Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new block headers count=3 elapsed=4.827s number=7617258 hash=15ffda…93cff9
Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.078] Imported new chain segment blocks=3 txs=276 mgas=17.896 elapsed=2.016s mgasps=8.875 number=7617258 hash=15ffda…93cff9 cache=93.46mB
Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.079] Fast sync complete, auto disabling
Apr 22 19:56:53 eth-001 geth[2012]: INFO [04-22|19:56:53.150] Imported new chain segment blocks=1 txs=3 mgas=8.000 elapsed=1.762s mgasps=4.538 number=7617259 hash=6279e1…69d3ff cache=93.95mB
aihua
on 22 Apr 2019
- pulledStates=306397423 / fast sync from 04-19 09:32, complete on 04-21 08:42:03 / storage: 180GB / memory 16GB / CPU: AMD Athlon(tm) X4 870K Quad Core Processor / WDC WDS100T2B0A SSD/Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz /geth --cache=4096 --syncmode "fast"
- pulledStates=310416897 / fast sync from: 04-19 09:32, complete on: 04-22 19:55:27 / storage: 180GB / memory: 8G B/ cpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz / SSD / Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz / geth --cache=4096 --syncmode "fast"
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Upgrading chain index type=bloombits percentage=99
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Imported new block headers count=1 elapsed=7.829ms number=7617253 hash=5519b7…b14245 age=1m43s
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.213] Imported new block headers count=2 elapsed=14.470ms number=7617255 hash=1b2c47…cb5ae0 age=1m7s
Apr 22 19:56:44 eth-001 geth[2012]: INFO [04-22|19:56:44.592] Imported new chain segment blocks=1 txs=205 mgas=7.991 elapsed=1.372s mgasps=5.822 number=7617252 hash=5efb08…180758 age=2m48s cache=87.82mB
Apr 22 19:56:46 eth-001 geth[2012]: INFO [04-22|19:56:46.449] Finished upgrading chain index type=bloombits
Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new chain segment blocks=3 txs=362 mgas=23.964 elapsed=3.453s mgasps=6.938 number=7617255 hash=1b2c47…cb5ae0 age=1m12s cache=91.15mB
Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new block headers count=3 elapsed=4.827s number=7617258 hash=15ffda…93cff9
Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.078] Imported new chain segment blocks=3 txs=276 mgas=17.896 elapsed=2.016s mgasps=8.875 number=7617258 hash=15ffda…93cff9 cache=93.46mB
Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.079] Fast sync complete, auto disabling
Apr 22 19:56:53 eth-001 geth[2012]: INFO [04-22|19:56:53.150] Imported new chain segment blocks=1 txs=3 mgas=8.000 elapsed=1.762s mgasps=4.538 number=7617259 hash=6279e1…69d3ff cache=93.95mB
180GB？ I think is not enough
wh0am111
on 23 Apr 2019
- pulledStates=306397423 / fast sync from 04-19 09:32, complete on 04-21 08:42:03 / storage: 180GB / memory 16GB / CPU: AMD Athlon(tm) X4 870K Quad Core Processor / WDC WDS100T2B0A SSD/Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz /geth --cache=4096 --syncmode "fast"
- pulledStates=310416897 / fast sync from: 04-19 09:32, complete on: 04-22 19:55:27 / storage: 180GB / memory: 8GB/ cpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz / SSD / Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz / geth --cache=4096 --syncmode "fast"
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Upgrading chain index type=bloombits percentage=99
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Imported new block headers count=1 elapsed=7.829ms number=7617253 hash=5519b7…b14245 age=1m43s
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.213] Imported new block headers count=2 elapsed=14.470ms number=7617255 hash=1b2c47…cb5ae0 age=1m7s
Apr 22 19:56:44 eth-001 geth[2012]: INFO [04-22|19:56:44.592] Imported new chain segment blocks=1 txs=205 mgas=7.991 elapsed=1.372s mgasps=5.822 number=7617252 hash=5efb08…180758 age=2m48s cache=87.82mB
Apr 22 19:56:46 eth-001 geth[2012]: INFO [04-22|19:56:46.449] Finished upgrading chain index type=bloombits
Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new chain segment blocks=3 txs=362 mgas=23.964 elapsed=3.453s mgasps=6.938 number=7617255 hash=1b2c47…cb5ae0 age=1m12s cache=91.15mB
Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new block headers count=3 elapsed=4.827s number=7617258 hash=15ffda…93cff9
Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.078] Imported new chain segment blocks=3 txs=276 mgas=17.896 elapsed=2.016s mgasps=8.875 number=7617258 hash=15ffda…93cff9 cache=93.46mB
Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.079] Fast sync complete, auto disabling
Apr 22 19:56:53 eth-001 geth[2012]: INFO [04-22|19:56:53.150] Imported new chain segment blocks=1 txs=3 mgas=8.000 elapsed=1.762s mgasps=4.538 number=7617259 hash=6279e1…69d3ff cache=93.95mB180GB？ I think is not enough
~180GB is used when completing fast sync, ssd storage is 1TB.
aihua
on 23 Apr 2019
Ooh，nice
发送自 Windows 10 版邮件https://go.microsoft.com/fwlink/?LinkId=550986应用
发件人: Aihua notifications@github.com
发送时间: Tuesday, April 23, 2019 9:24:04 AM
收件人: ethereum/go-ethereum
抄送: wh0am111; Comment
主题: Re: [ethereum/go-ethereum] What is the upper bound of "imported new state entries"? When will it end? (#15616)
- pulledStates=306397423 / fast sync from 04-19 09:32, complete on 04-21 08:42:03 / storage: 180GB / memory 16GB / CPU: AMD Athlon(tm) X4 870K Quad Core Processor / WDC WDS100T2B0A SSD/Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz /geth --cache=4096 --syncmode "fast"
- pulledStates=310416897 / fast sync from: 04-19 09:32, complete on: 04-22 19:55:27 / storage: 180GB / memory: 8G B/ cpu: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz / SSD / Ubuntu 18.04.2 LTS 64bit server / geth-alltools-linux-amd64-1.8.26-cdae1c59.tar.gz / geth --cache=4096 --syncmode "fast"
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Upgrading chain index type=bloombits percentage=99
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.198] Imported new block headers count=1 elapsed=7.829ms number=7617253 hash=5519b7…b14245 age=1m43s
Apr 22 19:56:43 eth-001 geth[2012]: INFO [04-22|19:56:43.213] Imported new block headers count=2 elapsed=14.470ms number=7617255 hash=1b2c47…cb5ae0 age=1m7s
Apr 22 19:56:44 eth-001 geth[2012]: INFO [04-22|19:56:44.592] Imported new chain segment blocks=1 txs=205 mgas=7.991 elapsed=1.372s mgasps=5.822 number=7617252 hash=5efb08…180758 age=2m48s cache=87.82mB
Apr 22 19:56:46 eth-001 geth[2012]: INFO [04-22|19:56:46.449] Finished upgrading chain index type=bloombits
Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new chain segment blocks=3 txs=362 mgas=23.964 elapsed=3.453s mgasps=6.938 number=7617255 hash=1b2c47…cb5ae0 age=1m12s cache=91.15mB
Apr 22 19:56:48 eth-001 geth[2012]: INFO [04-22|19:56:48.047] Imported new block headers count=3 elapsed=4.827s number=7617258 hash=15ffda…93cff9
Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.078] Imported new chain segment blocks=3 txs=276 mgas=17.896 elapsed=2.016s mgasps=8.875 number=7617258 hash=15ffda…93cff9 cache=93.46mB
Apr 22 19:56:50 eth-001 geth[2012]: INFO [04-22|19:56:50.079] Fast sync complete, auto disabling
Apr 22 19:56:53 eth-001 geth[2012]: INFO [04-22|19:56:53.150] Imported new chain segment blocks=1 txs=3 mgas=8.000 elapsed=1.762s mgasps=4.538 number=7617259 hash=6279e1…69d3ff cache=93.95mB
180GB？ I think is not enough
~180GB is used when completing fast sync, ssd storage is 1TB.
―
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/ethereum/go-ethereum/issues/15616#issuecomment-485607035, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKUHGT2RU4WWZ2ILP5X6AWLPRZQLJANCNFSM4EG7I5CQ.
wh0am111
on 23 Apr 2019
My statistics:
2019-05-06 01:00:32 avg: 1827 max: 1938 min: 1378 states/s remain: 136604075 states 4 peers eta@ 20:46:28.165828
2019-05-06 01:00:37 avg: 1864 max: 1938 min: 1378 states/s remain: 136595500 states 3 peers eta@ 20:21:14.951050
2019-05-06 01:00:42 avg: 1791 max: 1938 min: 1378 states/s remain: 136583359 states 3 peers eta@ 21:11:16.481006
2019-05-06 01:00:48 avg: 1742 max: 1938 min: 1378 states/s remain: 136580287 states 3 peers eta@ 21:46:35.797305
2019-05-06 01:00:53 avg: 1721 max: 1938 min: 1378 states/s remain: 136575694 states 3 peers eta@ 22:03:01.154434
2019-05-06 01:00:58 avg: 1682 max: 1938 min: 1378 states/s remain: 136569043 states 4 peers eta@ 22:33:15.402442
2019-05-06 01:01:03 avg: 1698 max: 1938 min: 1378 states/s remain: 136564293 states 3 peers eta@ 22:20:27.458747
PS. I wrote a tiny python script to overview the process. It's here https://github.com/hayorov/ethereum-sync-mertics
hayorov
on 5 May 2019
Roughly 368,609,457 State entries @ Block 8,432,213
Chaindata = 70GiB
Ancient = 93GiB
Really wish there was a syncmode = selfish where I can run a light node but keep up to date state/event information for just accounts / contracts I interact with.
andrewheadricke
on 27 Aug 2019
10/11/2019
379,265,938 state entries
187.5 GB of disk space used for fast sync
deke997
on 12 Oct 2019
12-25-2019
Geth version: 1.9.9
Highest block: 9160798
Known states: 475519190
momaee
on 25 Dec 2019
Will there ever be an API to get the number of states from a synced geth node to provide a reference?
nyetwurk
on 15 Jan 2020
My statistics:
2019-05-06 01:00:32 avg: 1827 max: 1938 min: 1378 states/s remain: 136604075 states 4 peers eta@ 20:46:28.165828 2019-05-06 01:00:37 avg: 1864 max: 1938 min: 1378 states/s remain: 136595500 states 3 peers eta@ 20:21:14.951050 2019-05-06 01:00:42 avg: 1791 max: 1938 min: 1378 states/s remain: 136583359 states 3 peers eta@ 21:11:16.481006 2019-05-06 01:00:48 avg: 1742 max: 1938 min: 1378 states/s remain: 136580287 states 3 peers eta@ 21:46:35.797305 2019-05-06 01:00:53 avg: 1721 max: 1938 min: 1378 states/s remain: 136575694 states 3 peers eta@ 22:03:01.154434 2019-05-06 01:00:58 avg: 1682 max: 1938 min: 1378 states/s remain: 136569043 states 4 peers eta@ 22:33:15.402442 2019-05-06 01:01:03 avg: 1698 max: 1938 min: 1378 states/s remain: 136564293 states 3 peers eta@ 22:20:27.458747PS. I wrote a tiny python script to overview the process. It's here https://github.com/hayorov/ethereum-sync-mertics
Unfortunately this has a hardcoded reference state count.
nyetwurk
on 15 Jan 2020
Jan 17 2020
Known states: 432391041
nyetwurk
on 17 Jan 2020
nyetwurk
on 27 Jan 2020
@nyetwurk thanks! I was wondering, I synced blocks extremely quickly (less than a day) and thought I must have picked the best machine and configuration for the job, but alas, I currently only report 262 million known states, so this will take a while
1blockologist
on 31 Jan 2020
15 Feb 2020
446266045
INFO [02-15|22:49:59.641] Imported new state entries count=484 elapsed=11.987ms processed=446266045 pending=0 retry=0 duplicate=50314 unexpected=640943
23 Feb 2020
449369291
199 GB
INFO [02-23|14:42:57.521] Imported new state entries count=794 elapsed=22.895ms processed=449369291 pending=0 retry=0 duplicate=7272 unexpected=23183
34x4p08
on 23 Feb 2020
449369291
NUM_OF_STATES_AS_OF = 455300013 # Actual for Feb, 15 2020
This small script can be helpful if you anticipate about sync process https://github.com/hayorov/ethereum-sync-metrics (written in Python)
# python metrics.py
2019-05-06 01:00:32 avg: 1827 max: 1938 min: 1378 states/s remain: 136604075 states 4 peers eta@ 20:46:28.165828
2019-05-06 01:00:37 avg: 1864 max: 1938 min: 1378 states/s remain: 136595500 states 3 peers eta@ 20:21:14.951050
2019-05-06 01:00:42 avg: 1791 max: 1938 min: 1378 states/s remain: 136583359 states 3 peers eta@ 21:11:16.481006
April 09, 2020
475800759
220 GB
INFO [04-09|08:44:12.951] Imported new state entries count=417 elapsed=16.310ms processed=475800759 pending=0 retry=0 duplicate=15060 unexpected=83332
pluracoin
on 9 Apr 2020
April 20, 2020
483782179
MischaWeerwag
on 20 Apr 2020
April 20, 2020
updated https://github.com/hayorov/ethereum-sync-metrics
hayorov
on 1 May 2020
Does anyone have the current number of state entries for Ropsten?
DanielHH
on 7 May 2020
Updated bound for mainnet:
INFO [05-10|23:51:14.924] Imported new state entries count=147 elapsed=7.432ms processed=504700513 pending=0 retry=0 duplicate=1115 unexpected=4208
INFO [05-10|23:51:14.968] Imported new block receipts count=1 elapsed=37.733ms number=10040872 hash=27a688…d5f8f4 age=28m12s size=23.45KiB
INFO [05-10|23:51:14.978] Committed new head block number=10040872 hash=27a688…d5f8f4
INFO [05-10|23:51:14.982] Deallocated fast sync bloom items=504344987 errorrate=0.202
INFO [05-10|23:51:25.223] Imported new chain segment blocks=3 txs=407 mgas=29.972 elapsed=10.224s mgasps=2.932 number=10040875 hash=4cb642…c3d687 age=27m45s dirty=3.67MiB
I.e. 504344987 state entries at 10 May 2020. For fast sync this is about 240GB of data stored on the SSD.
Using a RPi v4, 4GB RAM, externally powered Samsung T5 1TB SSD to sync. Pro tip for Raspberry Pi users: use Ubuntu Server 64bit. Raspbian currently does not natively use 64bit. You can manually "allow" this via some kernel settings but I still ran into out of memory issues. I might have done something wrong there. However, on Ubuntu, I have never had an out of memory error so far since syncing. I have to note that I allocated 8GB swap on the SSD and using --cache=512 as Geth parameter. Also make sure your 30303 port is open, so external nodes can connect (and you can find peers!).
Sadly, I have not written down when I started syncing. However, I know it took less than a week. It is somewhere between 5-6 days I think.
jochem-brouwer
on 11 May 2020
@jochem-brouwer Hey I am syncing now and is showing:
INFO [05-11|08:18:49.103] Initializing fast sync bloom items=12537911 errorrate=0.000 elapsed=13m44.944s
INFO [05-11|08:18:53.745] Imported new state entries count=384 elapsed=3.677µs processed=491362414 pending=99287 retry=0 duplicate=0 unexpected=0
INFO [05-11|08:18:57.107] Initializing fast sync bloom items=12627357 errorrate=0.000 elapsed=13m52.948s
INFO [05-11|08:19:05.109] Initializing fast sync bloom items=12711112 errorrate=0.000 elapsed=14m0.950s
INFO [05-11|08:19:13.113] Initializing fast sync bloom items=12802400 errorrate=0.000 elapsed=14m8.954s
with processed 491362414. Does this mean I am in the end of the process or its still has other processes left?
Also right now my RAM usage is only 8GB whereas my capacity is 30GB. And my CPU usage is also prettly low. Is there a way to increase the performance?
sssubik
on 11 May 2020
@sssubik Yes you are almost there! About 13 million states.
The RAM is fine. My node was using about 2GB most of the time.
I dont know how to increase performance though.
jochem-brouwer
on 11 May 2020
NFO [05-13|13:17:39.185] Imported new block headers count=1 elapsed=7.321ms number=10057490 hash=c3dce2…dc2732
INFO [05-13|13:17:39.443] Imported new block headers count=1 elapsed=6.425ms number=10057491 hash=1d75f2…cf1b86
INFO [05-13|13:17:46.280] Imported new state entries count=0 elapsed=16.717µs processed=492850530 pending=255 retry=2 duplicate=2655 unexpected=32505
INFO [05-13|13:17:46.280] Imported new state entries count=0 elapsed=31.72µs processed=492850530 pending=250 retry=0 duplicate=2656 unexpected=32505
INFO [05-13|13:17:46.886] Imported new state entries count=0 elapsed=94.498µs processed=492850530 pending=250 retry=229 duplicate=2657 unexpected=32506
Hey @jochem-brouwer I am still in processed=492850530 state. Its already been 2 days and its pace is very low. The block headers that is being imported is actually the current block thats being mined.
Is this normal to take this long for the state trie to get downloaded? I am having very little progress
sssubik
on 13 May 2020
Hey @sssubik, you are very close to getting synced. However it seems like your node isn't really importing new state entries. Did you try the classic "turn it off and on again"? (I.e. kill your node and restart it). Do you have port 30303 open as well (for peer discovery)?
jochem-brouwer
on 13 May 2020
@jochem-brouwer how close? I am in similar situation as sssubik. Am I couple of days close or couple of hours? is there is a way to even approximate that somehow?
kot-begemot
on 17 May 2020
May 18, 2020 knownStates = 581,839,369
WyseNynja
on 18 May 2020
I'm doing an experiment with "fast" sync with a server rented in Hetzner. 4 10TB disks in RAID10.
21 days from the beginning and my current stats are:
{
currentBlock: 10333881,
highestBlock: 10333971,
knownStates: 530115529,
pulledStates: 530103862,
startingBlock: 10301219
}
Close to WyseNynja's 581mln, but it still has very slow progress.
I think such huge "fast sync" times could be easily avoided if Ethereum project simply posted the chainstate archive every month or so. Otherwise everyone should wait for at least a month to get it synced.
AdminAnticaptcha
on 25 Jun 2020
You need SSD disks to run geth, it will never sync to 100% and even if it
does it will lag behind after some time
On Thu, 25 Jun 2020 at 12:07, AdminAnticaptcha notifications@github.com
wrote:
I'm doing an experiment with "fast" sync with a server rented in Hetzner.
4 10TB disks in RAID10.
21 days from the beginning and my current stats are:
{ currentBlock: 10333881, highestBlock: 10333971, knownStates: 530115529,
pulledStates: 530103862, startingBlock: 10301219 }
Close to WyseNynja's 581mln, but it still has very slow progress.I think such huge "fast sync" times could be easily avoided if Ethereum
project simply posted the chainstate archive every month or so. Otherwise
everyone should wait for at least a month to get it synced.—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ethereum/go-ethereum/issues/15616#issuecomment-649407072,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEJ2SDN46NY44QBNQQQL6TRYMHURANCNFSM4EG7I5CQ
.
mstrokin
on 25 Jun 2020
Hi, I was wondering if anyone has succeeded recently at syncing on a rpi4. I followed the tutorial here but after three weeks it seems to be stuck at downloading state entries. I've also tried restarting, rebooting the rpi..
These are my logs:
Imported new state entries count=437 elapsed=5.679ms processed=616671456 pending=3538 retry=0 duplicate=9385 unexpected=58338
Imported new state entries count=608 elapsed=9.139ms processed=616672064 pending=3419 retry=0 duplicate=9385 unexpected=58338
Imported new block headers count=2 elapsed=25.017ms number=10390929 hash="0bf3e0…99c55e" age=1m3s
Imported new state entries count=552 elapsed=11.419ms processed=616672616 pending=3195 retry=0 duplicate=9385 unexpected=58338
Imported new state entries count=353 elapsed=4.867ms processed=616672969 pending=3169 retry=0 duplicate=9385 unexpected=58338
> eth.syncing
{
currentBlock: 10390879,
highestBlock: 10390954,
knownStates: 616693498,
pulledStates: 616690561,
startingBlock: 10364969
}
> eth.syncing.highestBlock - eth.syncing.currentBlock
77
> eth.syncing.knownStates - eth.syncing.pulledStates
7911
> eth.syncing.currentBlock * 100 / eth.syncing.highestBlock
99.99924933320578
yosoyubik
on 4 Jul 2020
Checking in, fast sync just finished up at 545 million states.
Which is very odd given that @WyseNynja pulled 581 million states. Is it normal for 41 million states to have been deleted since May 18?
ditchfieldcaleb
on 5 Jul 2020
Checking in, fast sync just finished up at 545 million states.
That's... a bit more than the 616693498 I can see in my case. Maybe I need to start syncing again from scratch...
yosoyubik
on 6 Jul 2020
557918041 states
drhashes
on 26 Jul 2020
processed=598602988 pending=0
HaffASE
on 31 Jul 2020
What are states now? My current pulledStates is 634149016
gr8den
on 11 Sep 2020
Done syncing at 663339026 states. Size on disk: 328G
gr8den
on 13 Sep 2020
block 10936777
states 593892151
size 265G
34x4p08
on 26 Sep 2020
processed=767339105 and still increasing.
Block=10980245
2020/10/3
Any idea of the current state count? A website with state count would be super useful.
Incrediblez7
on 2 Oct 2020
Finally synced.
processed=785813881
Block=10980323
Incrediblez7
on 3 Oct 2020
Yikes! Scalability on ETH2.0 needed ASAP.
Gave up on blockchain. Run all my Solidity smart contracts on Hedera hashgraph now.
hynese
on 7 Oct 2020
Took me just over 9 days to full sync on a Pi. No sweat!
SirLancelot-OG
on 7 Oct 2020
Why do some of the nodes have 785M entries, but others have 603M?
My node just completed with 603101157, but just 5 days ago @Incrediblez7 had 785813881. I restarted my node just a couple minutes into the sync to change some config, but that wouldn't explain such a large difference, would it?
I started my sync at Oct 5 10:57 AM and it ended at Oct 6 12:28 PM (1 day 1 hour 31 minutes)
INFO [10-06|12:27:36.623] Imported new state entries count=2053 elapsed=31.145ms processed=603101100 pending=135 trieretry=0 coderetry=0 duplicate=69447 unexpected=264947
INFO [10-06|12:27:36.652] Imported new state entries count=57 elapsed=1.330ms processed=603101157 pending=0 trieretry=0 coderetry=0 duplicate=69447 unexpected=264947
INFO [10-06|12:27:36.655] Imported new block receipts count=1 elapsed=1.692ms number=11003934 hash="f9cdf7…743976" age=25m28s size=82.56KiB
INFO [10-06|12:27:36.658] Committed new head block number=11003934 hash="f9cdf7…743976"
INFO [10-06|12:27:36.722] Deallocated fast sync bloom items=602129406 errorrate=0.000
INFO [10-06|12:27:45.152] Imported new chain segment blocks=26 txs=3848 mgas=311.621 elapsed=8.427s mgasps=36.978 number=11003960 hash="57e63b…9ad207" age=20m52s dirty=0.00B
INFO [10-06|12:27:53.158] Imported new chain segment blocks=39 txs=7742 mgas=483.612 elapsed=8.006s mgasps=60.406 number=11003999 hash="710994…399ecb" age=12m22s dirty=0.00B
INFO [10-06|12:27:59.388] Imported new chain segment blocks=40 txs=6121 mgas=472.947 elapsed=6.230s mgasps=75.913 number=11004039 hash="2d6e9b…4bbd95" dirty=0.00B
INFO [10-06|12:27:59.388] Imported new block headers count=1 elapsed=20.143s number=11004040 hash="e74b91…cd9d46"
INFO [10-06|12:27:59.389] Upgrading chain index type=bloombits percentage=43
INFO [10-06|12:27:59.392] Imported new block headers count=1 elapsed=3.861ms number=11004041 hash="2f7438…47fa95"
INFO [10-06|12:27:59.396] Imported new block headers count=2 elapsed=3.927ms number=11004043 hash="bf32fa…b0ddc3"
INFO [10-06|12:27:59.542] Imported new block headers count=1 elapsed=7.050ms number=11004044 hash="e80916…cce2a8"
INFO [10-06|12:27:59.782] Imported new chain segment blocks=1 txs=155 mgas=12.477 elapsed=169.439ms mgasps=73.638 number=11004040 hash="e74b91…cd9d46" dirty=0.00B
INFO [10-06|12:28:00.253] Imported new chain segment blocks=3 txs=641 mgas=37.294 elapsed=469.657ms mgasps=79.406 number=11004043 hash="bf32fa…b0ddc3" dirty=0.00B
INFO [10-06|12:28:00.363] Imported new chain segment blocks=1 txs=157 mgas=12.281 elapsed=109.632ms mgasps=112.022 number=11004044 hash="e80916…cce2a8" dirty=0.00B
INFO [10-06|12:28:00.363] Fast sync complete, auto disabling
Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
970 EVO NVMe M.2 SSD 1TB
32GB of RAM and --cache set to 8GB
WyseNynja
on 7 Oct 2020
Why do some of the nodes have 785M entries, but others have 603M?
https://github.com/ethereum/go-ethereum/issues/14647#issuecomment-682098181:
Every time you restart, geth goes into a mode where, while importing state, also reads the entire db to create a bloom filter of existing states. Which slows down sync. Also, every time you restart, you then choose a new pivot point to sync to, so more state to download.
Also, if you have slow progress, this will get worse. Every ~128 blocks, the peer(s) we're syncing against "forget" about the state (due to pruning). And so we need to move the pivot point. Which means that some of the stuff you already downloaded is discarded, and some new stuff needs to be fetched. So if the progress is too slow, it will have to download a lot _more_ data than if the progress is fast: essentially as you are syncing, the data becomes stale.
holiman
on 8 Oct 2020
Thanks. I think @Incrediblez7 must have much slower hardware than me if they ended up with 180M extra states.
WyseNynja
on 9 Oct 2020
Thanks. I think @Incrediblez7 must have much slower hardware than me if they ended up with 180M extra states.
Hmm... I'm running geth on Vultr with 64GB RAM and 16 CPU Cores so the hardware shouldn't be an issue. Probably because of the internet issue and the peers I'm connected to.
Incrediblez7
on 28 Oct 2020
Your storage matters more than your CPU or memory. What storage do you have? HDD, SSD, or NVME? How many IOPS?
On Oct 28, 2020, at 1:48 AM, Incrediblez7 notifications@github.com wrote:
Thanks. I think @Incrediblez7 must have much slower hardware than me if they ended up with 180M extra states.Hmm... I'm running geth on Vultr with 64GB RAM and 16 CPU Cores so the hardware shouldn't be an issue. Probably because of the internet issue and the peers I'm connected to.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
WyseNynja
on 28 Oct 2020
1280GB SSD with 10k+ IOPS on Vultr.
Corrections: I recently upgraded to 1600GB SSD with 24 CPU Cores and 96GB RAM.
Incrediblez7
on 30 Oct 2020
no point 24 core and 1 million ram. really. all should get the "samsung 980 pro 1tb". that's the only way
AndreiD
on 30 Oct 2020
Since I'm located in China and I couldn't get a decent amount of peers, I'm running Geth on a VPS, so I can't change the NVME myself. But isn't ~15k IOPS enough? Or more disk performance is required?
Incrediblez7
on 30 Oct 2020
I have a NanoPC T4 with 1TB NVMe SSD which has been syncing for about 6 weeks. I'm running the EthArmbian build available here: https://github.com/diglos/userpatches
I noticed a reply above in which @SirLancelot-OG was able to sync a Raspberry Pi in only about 9 days. Given that this is similar hardware, I'm wondering if anyone can help me troubleshoot my setup. My node has managed to process 773755887 states in 6 weeks:
Nov 01 21:34:40 ethnode-bc89073e geth[16112]: INFO [11-01|21:34:40.721] Downloader queue stats receiptTasks=0 blockTasks=0 itemSize=187.24KiB throttle=351
Nov 01 21:34:42 ethnode-bc89073e geth[16112]: INFO [11-01|21:34:42.779] Imported new block headers count=1 elapsed=34.322ms number=11173619 hash="67cad5…f81212"
Nov 01 21:34:45 ethnode-bc89073e geth[16112]: INFO [11-01|21:34:45.971] Imported new state entries count=384 elapsed="15.75µs" processed=773754623 pending=193816 trieretry=0 coderetry=0 duplicate=0 unexpected=393
Nov 01 21:35:00 ethnode-bc89073e geth[16112]: INFO [11-01|21:35:00.800] Imported new state entries count=384 elapsed="15.167µs" processed=773755007 pending=194380 trieretry=0 coderetry=0 duplicate=0 unexpected=393
Nov 01 21:35:05 ethnode-bc89073e geth[16112]: INFO [11-01|21:35:05.286] Imported new block headers count=1 elapsed=446.894ms number=11173620 hash="885043…71fd63"
Nov 01 21:35:17 ethnode-bc89073e geth[16112]: INFO [11-01|21:35:17.935] Imported new block headers count=1 elapsed=419.383ms number=11173621 hash="4da9b0…851a0f"
Nov 01 21:35:20 ethnode-bc89073e geth[16112]: INFO [11-01|21:35:20.856] Imported new state entries count=496 elapsed="694.163µs" processed=773755503 pending=195065 trieretry=1 coderetry=0 duplicate=0 unexpected=393
Nov 01 21:35:30 ethnode-bc89073e geth[16112]: INFO [11-01|21:35:30.835] Imported new block headers count=1 elapsed=554.866ms number=11173622 hash="a85c36…87c3b3"
Nov 01 21:35:33 ethnode-bc89073e geth[16112]: INFO [11-01|21:35:33.590] Imported new block headers count=2 elapsed=39.649ms number=11173624 hash="0e4464…0ca9ed"
Nov 01 21:35:36 ethnode-bc89073e geth[16112]: INFO [11-01|21:35:36.124] Imported new state entries count=384 elapsed=3.030ms processed=773755887 pending=195608 trieretry=0 coderetry=0 duplicate=0 unexpected=393
Is there a way to determine whether my sync is slow because of disk or network IO? I have run iostats on the device during sync and get the following for SSD throughput (nvme0n1):
avg-cpu: %user %nice %system %iowait %steal %idle
14.34 0.00 11.51 7.27 0.00 66.87
Device tps kB_read/s kB_wrtn/s kB_read kB_wrtn
nvme0n1 2927.80 32683.20 342.40 163416 1712
mmcblk1 0.00 0.00 0.00 0 0
mmcblk1rpmb 0.00 0.00 0.00 0 0
mmcblk1boot1 0.00 0.00 0.00 0 0
mmcblk1boot0 0.00 0.00 0.00 0 0
mmcblk0 65.80 1172.00 2.40 5860 12
zram0 0.00 0.00 0.00 0 0
zram1 318.60 1273.60 0.80 6368 4
I have run most of the sync without port forwarding enabled, but one week ago I enabled port forwarding to the device for UDP & TCP port 30303.
Can anyone suggest monitoring or other tests I can perform to understand where the bottleneck is?
gomes7997
on 1 Nov 2020
Can anyone suggest monitoring or other tests I can perform to understand where the bottleneck is?
I'll second that. I am also finding it hard to tell if there is an issue syncing without knowing whether I am catching up or falling behind. And a generic what to do about it.
Duncan-Brain
on 2 Nov 2020
It took me approximately 36 hours to sync.
Here is the current state :
INFO [11-02|05:35:50.478] Imported new block receipts count=1 elapsed=4.983ms number=11175392 hash="d37205…f8de8a" age=24m44s size=87.40KiB
INFO [11-02|05:35:50.492] Committed new head block number=11175392 hash="d37205…f8de8a"
INFO [11-02|05:35:50.562] Deallocated fast sync bloom items=619579733 errorrate=0.001
970 EVO NVMe M.2 SSD 1TB
AMD Ryzen 7 1700 (3.0 GHz)
DDR4 Corsair Vengeance LPX, Noir, 4 x 8 Go, 3200 MHz,
CAS 16
Good luck !
Xavier59
on 2 Nov 2020
Sharing data of my fully synced node also, maybe it is useful to others.
D:\Program Files\Geth>geth --datadir s:\Ethereum inspect
INFO [11-11|19:19:15.793] Maximum peer count ETH=50 LES=0 total=50
INFO [11-11|19:19:16.156] Set global gas cap cap=25000000
INFO [11-11|19:19:16.160] Allocated cache and file handles database=s:\Ethereum\geth\chaindata cache=512.00MiB handles=8192
INFO [11-11|19:19:27.022] Opened ancient database database=s:\Ethereum\geth\chaindata\ancient
INFO [11-11|19:19:27.046] Disk storage enabled for ethash caches dir=s:\Ethereum\geth\ethash count=3
INFO [11-11|19:19:27.051] Disk storage enabled for ethash DAGs dir=C:\Users\giuse\AppData\Local\Ethash count=2
INFO [11-11|19:19:27.059] Loaded most recent local header number=11237906 hash="d8c4b8…c3f1e9" td=18641055599883751407245 age=53s
INFO [11-11|19:19:27.069] Loaded most recent local full block number=11237906 hash="d8c4b8…c3f1e9" td=18641055599883751407245 age=53s
INFO [11-11|19:19:27.079] Loaded most recent local fast block number=11237906 hash="d8c4b8…c3f1e9" td=18641055599883751407245 age=53s
INFO [11-11|19:19:27.087] Loaded last fast-sync pivot marker number=10760885
INFO [11-11|19:19:35.100] Inspecting database count=4996000 elapsed=8.002s
INFO [11-11|19:19:43.108] Inspecting database count=10062000 elapsed=16.010s
INFO [11-11|19:19:51.115] Inspecting database count=14730000 elapsed=24.017s
INFO [11-11|19:19:59.123] Inspecting database count=19415000 elapsed=32.025s
...
INFO [11-11|19:50:45.078] Inspecting database count=1815272000 elapsed=31m17.980s
INFO [11-11|19:50:53.083] Inspecting database count=1821919000 elapsed=31m25.985s
INFO [11-11|19:51:01.090] Inspecting database count=1828550000 elapsed=31m33.992s
INFO [11-11|19:51:09.097] Inspecting database count=1835247000 elapsed=31m41.999s
+-----------------+--------------------+------------+-----------+
| DATABASE | CATEGORY | SIZE | ITEMS |
+-----------------+--------------------+------------+-----------+
| Key-Value store | Headers | 54.97 MiB | 100438 |
| Key-Value store | Bodies | 3.52 GiB | 98390 |
| Key-Value store | Receipt lists | 4.48 GiB | 98390 |
| Key-Value store | Difficulties | 7.11 MiB | 113813 |
| Key-Value store | Block number->hash | 6.04 MiB | 113813 |
| Key-Value store | Block hash->number | 439.41 MiB | 11237928 |
| Key-Value store | Transaction index | 30.13 GiB | 898783141 |
| Key-Value store | Bloombit index | 1.76 GiB | 5617664 |
| Key-Value store | Contract codes | 264.01 MiB | 44710 |
| Key-Value store | Trie nodes | 130.27 GiB | 880287080 |
| Key-Value store | Trie preimages | 2.98 GiB | 45170757 |
| Key-Value store | Account snapshot | 0.00 B | 0 |
| Key-Value store | Storage snapshot | 0.00 B | 0 |
| Key-Value store | Clique snapshots | 0.00 B | 0 |
| Key-Value store | Singleton metadata | 151.00 B | 5 |
| Ancient store | Headers | 4.75 GiB | 11147907 |
| Ancient store | Bodies | 108.13 GiB | 11147907 |
| Ancient store | Receipt lists | 50.63 GiB | 11147907 |
| Ancient store | Difficulties | 173.74 MiB | 11147907 |
| Ancient store | Block number->hash | 404.00 MiB | 11147907 |
| Light client | CHT trie nodes | 0.00 B | 0 |
| Light client | Bloom trie nodes | 0.00 B | 0 |
+-----------------+--------------------+------------+-----------+
| TOTAL | 337.97 GIB | |
+-----------------+--------------------+------------+-----------+
ERROR[11-11|19:51:16.769] Database contains unaccounted data size=126.40KiB count=2748
Neurone
on 12 Nov 2020
I'm trying to do a fastsync on a local server but it seems the disk speed drops off after about 30 minutes of "importing new state entry". I'm running the sync process on machine with SSD, 16GB RAM and i5. Cache is setup to 12GB (fills up only to 50%).
The process starts with about 100-150MB/s I/O and after 30 minutes drops to 10-20MB/s I/O.
Is this normal?
What are the average disk speeds on the import states task?
peterkvasnica
on 20 Nov 2020
I just finished syncing a new node the final count was processed=644366474
gregnuj
on 28 Nov 2020
just hit processed=648537261 using around 300GB
LucasBruder
on 2 Dec 2020
Finished syncing last night, final was: knownStates: 640595237.
Hardware:
Samsung 970 EVO Plus 1TB NVMe SSD
32 GB DDR4 3200MHz RAM - cachesize of 16GB for geth
i7-9700K CPU @ 3.6GHz
MikeJerred
on 3 Dec 2020
it would help if you write your hardware too not just the number...
AndreiD
on 3 Dec 2020
I just synced another eth1 full node in 64 hours, processing 650.634.632 state entries, using 282.75 GIB. Full stats below.
Please note that during synchronization the same machine was running a Teku beacon node, so performance could have been better without it. I wanted to test if those two clients can both run smoothly on this mini PC and, spoiler alert, yes they can 😉.
KEY | VALUE
--- | ---
Fast sync complete | 64 hours
Processed state entries | 650.634.632
Storage used | 282.75 GIB
OS | Ubuntu 20.04.1
Client | Geth/v1.9.24-stable-cc05b050/linux-amd64/go1.15.5
Machine | MINISFORUM DeskMini UM300 Mini PC
Processor | AMD Ryzen™ 3 3300U , 4 Cores/4 Threads
Memory | DDR4 8GB Dual channel (min. 1 GB reserved for graphic card, so actual RAM 7GB!)
Storage n.1 | Kingston M.2 2280 256GB SATA SSD _(used by Teku)_
Storage n.2 | Crucial BX500 1TB SATA SSD _(used by Geth)_
Connectivity | Ethernet 1Gb
ISP | Fastweb, Italy
./geth
INFO [12-02|03:57:17.926] Starting Geth on Ethereum mainnet...
INFO [12-02|03:57:17.927] Bumping default cache on mainnet provided=1024 updated=4096
WARN [12-02|03:57:17.927] Sanitizing cache to Go's GC limits provided=4096 updated=2323
INFO [12-02|03:57:17.934] Maximum peer count ETH=50 LES=0 total=50
INFO [12-02|03:57:17.934] Smartcard socket not found, disabling err="stat /run/pcscd/pcscd.comm: no such file or directory"
INFO [12-02|03:57:17.936] Set global gas cap cap=25000000
INFO [12-02|03:57:17.936] Allocated trie memory caches clean=580.00MiB dirty=580.00MiB
...
INFO [12-02|03:57:18.015] Writing default main-net genesis block
INFO [12-02|03:57:18.279] Persisted trie from memory database nodes=12356 size=1.78MiB time=84.346878ms gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=0.00B
INFO [12-02|03:57:18.279] Initialised chain configuration config="{ChainID: 1 Homestead: 1150000 DAO: 1920000 DAOSupport: true EIP150: 2463000 EIP155: 2675000 EIP158: 2675000 Byzantium: 4370000 Constantinople: 7280000 Petersburg: 7280000 Istanbul: 9069000, Muir Glacier: 9200000, YOLO v2: <nil>, Engine: ethash}"
...
INFO [12-02|03:57:18.280] Initialising Ethereum protocol versions="[65 64 63]" network=1 dbversion=<nil>
WARN [12-02|03:57:18.280] Upgrade blockchain database version from=<nil> to=8
INFO [12-02|03:57:18.281] Loaded most recent local header number=0 hash="d4e567…cb8fa3" td=17179869184 age=51y7mo4w
INFO [12-02|03:57:18.281] Loaded most recent local full block number=0 hash="d4e567…cb8fa3" td=17179869184 age=51y7mo4w
INFO [12-02|03:57:18.281] Loaded most recent local fast block number=0 hash="d4e567…cb8fa3" td=17179869184 age=51y7mo4w
INFO [12-02|03:57:18.281] Regenerated local transaction journal transactions=0 accounts=0
INFO [12-02|03:57:18.298] Allocated fast sync bloom size=1.13GiB
INFO [12-02|03:57:18.304] Starting peer-to-peer node instance=Geth/v1.9.24-stable-cc05b050/linux-amd64/go1.15.5
...
INFO [12-02|03:57:18.667] Initialized fast sync bloom items=12356 errorrate=0.000 elapsed=367.872ms
...
INFO [12-02|03:57:28.334] Block synchronisation started
...
INFO [12-02|03:57:35.319] Imported new state entries count=1152 elapsed=5.229ms processed=45201 pending=20345 trieretry=0 coderetry=0 duplicate=0 unexpected=0
INFO [12-02|03:57:35.425] Imported new state entries count=1152 elapsed=4.800ms processed=46353 pending=20673 trieretry=0 coderetry=0 duplicate=0 unexpected=0
INFO [12-02|03:57:35.751] Imported new state entries count=1536 elapsed=8.549ms processed=47889 pending=19409 trieretry=0 coderetry=0 duplicate=0 unexpected=0
...
INFO [12-04|20:52:06.008] Imported new state entries count=1996 elapsed=62.015ms processed=650631754 pending=9510 trieretry=0 coderetry=0 duplicate=126032 unexpected=765459
INFO [12-04|20:52:06.239] Imported new state entries count=2191 elapsed=141.737ms processed=650633945 pending=2499 trieretry=0 coderetry=0 duplicate=126032 unexpected=765459
INFO [12-04|20:52:10.981] Imported new state entries count=687 elapsed=45.228ms processed=650634632 pending=0 trieretry=0 coderetry=0 duplicate=126032 unexpected=765459
INFO [12-04|20:52:10.998] Committed new head block number=11388325 hash="4dfe95…2574b4"
INFO [12-04|20:52:11.049] Deallocated fast sync bloom items=646967270 errorrate=0.006
...
INFO [12-04|20:52:11.049] Deallocated fast sync bloom items=646967270 errorrate=0.006
...
INFO [12-04|20:58:18.180] Fast sync complete, auto disabling
./geth inspect
INFO [12-04|22:37:47.671] Maximum peer count ETH=50 LES=0 total=50
INFO [12-04|22:37:47.671] Smartcard socket not found, disabling err="stat /run/pcscd/pcscd.comm: no such file or directory"
INFO [12-04|22:37:47.672] Set global gas cap cap=25000000
...
INFO [12-04|22:37:52.507] Loaded most recent local header number=11388879 hash="0ec041…573550" td=19169271088158317937607 age=3m28s
INFO [12-04|22:37:52.507] Loaded most recent local full block number=11388879 hash="0ec041…573550" td=19169271088158317937607 age=3m28s
INFO [12-04|22:37:52.507] Loaded most recent local fast block number=11388879 hash="0ec041…573550" td=19169271088158317937607 age=3m28s
INFO [12-04|22:37:52.510] Loaded last fast-sync pivot marker number=11388325
INFO [12-04|22:38:00.533] Inspecting database count=10557000 elapsed=8.000s
INFO [12-04|22:38:08.533] Inspecting database count=21006000 elapsed=16.000s
INFO [12-04|22:38:16.533] Inspecting database count=31660000 elapsed=24.000s
...
INFO [12-04|22:52:30.881] Inspecting database count=1561224000 elapsed=14m38.348s
INFO [12-04|22:52:38.881] Inspecting database count=1573276000 elapsed=14m46.348s
INFO [12-04|22:52:46.881] Inspecting database count=1581359000 elapsed=14m54.348s
+-----------------+--------------------+------------+-----------+
| DATABASE | CATEGORY | SIZE | ITEMS |
+-----------------+--------------------+------------+-----------+
| Key-Value store | Headers | 49.31 MiB | 90033 |
| Key-Value store | Bodies | 3.28 GiB | 90033 |
| Key-Value store | Receipt lists | 4.08 GiB | 90033 |
| Key-Value store | Difficulties | 5.66 MiB | 99853 |
| Key-Value store | Block number->hash | 4.68 MiB | 99771 |
| Key-Value store | Block hash->number | 445.31 MiB | 11388911 |
| Key-Value store | Transaction index | 31.01 GiB | 924916880 |
| Key-Value store | Bloombit index | 1.80 GiB | 5693440 |
| Key-Value store | Contract codes | 1.59 GiB | 333905 |
| Key-Value store | Trie nodes | 71.08 GiB | 647171117 |
| Key-Value store | Trie preimages | 8.82 MiB | 132275 |
| Key-Value store | Account snapshot | 0.00 B | 0 |
| Key-Value store | Storage snapshot | 0.00 B | 0 |
| Key-Value store | Clique snapshots | 0.00 B | 0 |
| Key-Value store | Singleton metadata | 151.00 B | 5 |
| Ancient store | Headers | 4.83 GiB | 11298879 |
| Ancient store | Bodies | 111.38 GiB | 11298879 |
| Ancient store | Receipt lists | 52.63 GiB | 11298879 |
| Ancient store | Difficulties | 176.19 MiB | 11298879 |
| Ancient store | Block number->hash | 409.47 MiB | 11298879 |
| Light client | CHT trie nodes | 0.00 B | 0 |
| Light client | Bloom trie nodes | 0.00 B | 0 |
+-----------------+--------------------+------------+-----------+
| TOTAL | 282.75 GIB | |
+-----------------+--------------------+------------+-----------+
ERROR[12-04|22:52:52.717] Database contains unaccounted data size=128.10KiB count=2785
What I am looking for from this issue is an approximate formula for states, and some link to a blog or something that has accumulated a bunch of failures of when your system is not ever going to catch up to the header block.
The current method I am using uses the estimate of 3-6k state changes per block (Reference) . Of those 3-6k changes, 2-4k are additions to the state tree and 1-2k are deletions. So delta ~ 3-6k, net ~1-2k per block.
If we use these estimates on @Neurone local state above to make a lower bound ( 650,634,632 in 64 hours) "Total states" = 650,634,632 - (3000x4x60x64) = 604554632 at the start of sync, and "Total states" = 604554632+(1000x4x60x64) = 619914632 at the end of sync.
I think this is a good formula for estimating in lieu of a better way -- so far for me it has correctly predicted my _inability_ to catch up to the header block, I just can't quite figure out my system's issue (my data here). The 3-6k seems outdated so there is also that.
Any formulas or indicators for "never catching up"?
Duncan-Brain
on 11 Dec 2020
Running a 4gb Raspberry Pi and approaching 900 million state entries and 325gb used.
processed=898488115
0xEther
on 13 Dec 2020
FYI, I finished syncing last night:
State entry: 664 401 205
Hardware NUC i5-8259U -16 GB RAM - 1TB NMVE Sabrent Q
estebaneu
on 6 Jan 2021
Related issues
phpsamsb
·
3Comments
JMaxU
·
3Comments
RoodyChan
·
3Comments
AdrianScott
·
3Comments
aomiba
·
3Comments
Most helpful comment
I just finished fast syncing with Geth 1.7.3 64-bit for Windows using an SSD and wanted to leave this data point for others. It took a little more than 2 days.
Yep, finished syncing just short of block 5,000,000. That's about 81 million known states for 5 million blocks. The database size on disk is 66,323,816,699. I kept getting crashes using the 1.7.2 that came with the wallet even going to an SSD. Parity in warp mode was also crashing.