parity-ethereum 🚀 - Memory usage spike induced crash

Can you please run with --l shutdown=trace and try to catch any additional logs that prints?

joshua-mir on 17 Feb 2019

It should actually be -l shutdown=trace ... I will run it and report as soon as I have another crash.

SimonHausdorf on 17 Feb 2019

😄1

2019-02-17 16:50:23 UTC IO Worker #1 INFO import    26/50 peers   5 MiB chain 115 MiB db 0 bytes queue 53 KiB sync  RPC:  0 conn,   12 req/s,  779 µs
2019-02-17 16:50:28 UTC Verifier #1 INFO import  Imported #7232632 0xa7ab…a4b7 (106 txs, 7.99 Mgas, 1846 ms, 23.46 KiB)
2019-02-17 16:50:32 UTC Verifier #1 INFO import  Imported #7232633 0xe47c…a4b9 (60 txs, 7.98 Mgas, 1556 ms, 24.82 KiB)
2019-02-17 16:50:53 UTC IO Worker #0 INFO import    27/50 peers   5 MiB chain 116 MiB db 0 bytes queue 53 KiB sync  RPC:  0 conn,    7 req/s,  754 µs
2019-02-17 16:51:17 UTC Verifier #1 INFO import  Imported #7232634 0x924f…a471 (96 txs, 7.97 Mgas, 1768 ms, 21.29 KiB)
2019-02-17 16:51:40 UTC IO Worker #0 INFO import    28/50 peers   6 MiB chain 115 MiB db 0 bytes queue 53 KiB sync  RPC:  0 conn,   12 req/s,  951 µs
2019-02-17 16:51:44 UTC Verifier #1 INFO import  Imported #7232635 0x4773…f70f (91 txs, 7.99 Mgas, 951 ms, 24.36 KiB)
2019-02-17 16:51:54 UTC main INFO parity_ethereum::run  Finishing work, please wait...
2019-02-17 16:51:54 UTC main TRACE shutdown  [IoService] Closing...
2019-02-17 16:51:54 UTC  TRACE shutdown  [IoWorker] Closing...
2019-02-17 16:51:54 UTC  TRACE shutdown  [IoWorker] Closed
2019-02-17 16:51:54 UTC  TRACE shutdown  [IoWorker] Closing...
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closed
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closing...
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closed
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closing...
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closed
2019-02-17 16:51:55 UTC main TRACE shutdown  [IoService] Closed.
2019-02-17 16:51:55 UTC main TRACE shutdown  [IoService] Closing...
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closing...
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closed
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closing...
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closed
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closing...
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closed
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closing...
2019-02-17 16:51:55 UTC  TRACE shutdown  [IoWorker] Closed
2019-02-17 16:51:55 UTC main TRACE shutdown  [IoService] Closed.
2019-02-17 16:51:55 UTC main TRACE shutdown  [VerificationQueue] Closing...
2019-02-17 16:51:55 UTC main TRACE shutdown  [VerificationQueue] Closed.

SimonHausdorf on 18 Feb 2019

@SimonHausdorf Are you sure there isn't some other process sending e.g. SIGTERM to the parity process?

2019-02-16 20:44:28 UTC Finishing work, please wait...
2019-02-16 20:45:35 UTC Shutdown is taking longer than expected.
2019-02-16 20:49:35 UTC Shutdown timeout reached, exiting uncleanly.

I'm glad I'm not the only one: https://github.com/paritytech/parity-ethereum/issues/10364

c0deright on 18 Feb 2019

I'm seeing this behavior as well. I put in some logging to trace it. Will report back if I find anything new. As a result of this, my nodes are pretty unusable because they shut down or crash after a short period of time. I'm seeing this behavior on both 2.2.10-stable and 2.3.3-beta.

lampshade9909 on 18 Feb 2019

@SimonHausdorf Are you sure there isn't some other process sending e.g. SIGTERM to the parity process?

yes, I am running parity in screen if this makes any difference. On this machine is nothing else running than Parity. Ubuntu 18.10

Maybe we can collect some information how you guys are running parity and on which system @c0deright @lampshade9909

SimonHausdorf on 18 Feb 2019

@SimonHausdorf I'm running on AWS with Ubuntu Server 18.04 LTS (HVM), SSD Volume Type - ami-0bdf93799014acdc4. Nothing else is running on this instance, only parity. c5.xlarge instance with 200 GB of disk space.

lampshade9909 on 18 Feb 2019

So also Ubuntu 18.x

We have a kovan server running on 16.04 without any problems. But it is 2.3.0-beta. I will setup two new nodes with Ubuntu 16.04 and 18.10 and let them run during the night to see if one of them crashes.

SimonHausdorf on 18 Feb 2019

I don't run parity 2.2.10 and 2.3.3 anymore because of https://github.com/paritytech/parity-ethereum/issues/10361 (have downgraded to 2.2.9 and 2.3.2).

My setup is AWS ec2 t3.medium instance, EBS gp2 volume, Ubuntu 16.04 LTS, parity being started and stopped via self-written systemd service file:

% cat /etc/systemd/system/parity-ethereum.service
[Unit]
Description=Ethereum Parity daemon
After=network.target

[Service]
User=foobar
Group=foobar

Type=forking
PIDFile=/home/foobar/.local/share/io.parity.ethereum/parity.pid
ExecStart=/home/foobar/parity-binary daemon /home/foobar/.local/share/io.parity.ethereum/parity.pid

Restart=always
PrivateTmp=true
TimeoutStopSec=900s
TimeoutStartSec=3s
StartLimitInterval=120s
StartLimitBurst=5

[Install]
WantedBy=multi-user.target

/home/foobar/parity-binary is a symlink to the actual binary so I can switch versions easily.

c0deright on 18 Feb 2019

@SimonHausdorf It may be important to note that I'm making a very significant number of jsonrpc calls to my nodes. Calls like: "eth_getBalance", "eth_call", "eth_blockNumber", "eth_getTransactionReceipt", "eth_getTransactionByHash", etc.

lampshade9909 on 18 Feb 2019

same for us, we have a lot of the rpc calls you mentioned

SimonHausdorf on 18 Feb 2019

The two new nodes are still running, our main node crashed 2 times during that night.

SimonHausdorf on 19 Feb 2019

@SimonHausdorf Mine still crashes consistently. Approximately 50-100 minutes after running. Here's the configuration I'm using:

parity --warp --pruning=fast --no-serve-light --max-peers=250 --snapshot-peers=50 --min-peers=50 --mode=active --tracing=off --db-compaction=ssd --cache-size=7168 --rpccorsdomain=* --rpcaddr=0.0.0.0 --jsonrpc-hosts=all --jsonrpc-server-threads=3 --log-file /home/ubuntu/parityLogfile &

lampshade9909 on 19 Feb 2019

On a site note unrelated to that issue:

why do you @lampshade9909 and @SimonHausdorf run parity with all the parameters from the command line respectively from screen instead of letting the OS/systemd/whatever handle starting and stopping parity like every other daemon (sshd, cron, syslog, ...), too? And why don't you put all the config in config.toml and just use 'parity' command?

I'm just curious.

c0deright on 19 Feb 2019

😕1

I am running it as a service now to restart after crash automatically. As to the config, just laziness :)

SimonHausdorf on 19 Feb 2019

Just appearing here with +1 message since my issue was closed as dupe of this one.

I just tested that when I'm trying to rerun my application that sends transactions I get

Unhandled Exception: Nethereum.JsonRpc.Client.RpcClientUnknownException: Error occurred when trying to send rpc requests(s) ---> System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: The write operation failed, see inner exception. ---> System.Net.Http.WinHttpException: The server returned an invalid or unrecognized response

Pzixel on 19 Feb 2019

@Pzixel that doesn't appear related to the issue at hand, but just to make sure, can you find out which rpc method is returning the invalid response?

joshua-mir on 19 Feb 2019

@joshua-mir can't say it now since I restarted the app, but it should be sendrawtransaction, getreceipt or getcode.

However it may be unrelated because it works when I restart client. Maybe some keepalive options or something else that prevents reconnect when server is restarted.

Pzixel on 19 Feb 2019

On a site note unrelated to that issue:

why do you @lampshade9909 and @SimonHausdorf run parity with all the parameters from the command line respectively from screen instead of letting the OS/systemd/whatever handle starting and stopping parity like every other daemon (sshd, cron, syslog, ...), too? And why don't you put all the config in config.toml and just use 'parity' command?

I'm just curious.

This is just the habit I've gotten myself into. Certainly not the ideal way to do it. So you have your parity node being restarted once it shuts down? I need to figure out how to do that, just haven't had the need since my parity nodes were pretty reliable and didn't shutdown/crash much up until this last update.

Edit: I installed immortal and will use that to persist the parity node. Maybe i'll move to the config file next ;-)

lampshade9909 on 19 Feb 2019

@lampshade9909 that's fine but it seems that clients may be not ok with reconnecting to restarted server, see my comment above.

Pzixel on 19 Feb 2019

I am now seeing this shutdown issue:

2019-02-20 17:03:30 UTC 22/75 peers 303 MiB chain 564 MiB db 17 KiB queue 72 KiB sync RPC: 0 conn, 0 req/s, 18610 µs2019-02-20 17:03:30 UTC 21/75 peers 303 MiB chain 564 MiB db 26 KiB queue 72 KiB sync RPC: 0 conn, 0 req/s, 18610 µs

2019-02-20 17:05:25 UTC Syncing #7527099 0xac8b…a5e3 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 4 Qed #7527104 20/75 peers 303 MiB chain 564 MiB db 36 KiB queue 72 KiB sync RPC: 0 conn, 0 req/s, 18610 µs2019-02-20 17:05:25 UTC 20/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 0 req/s, 18610 µs

2019-02-20 17:05:27 UTC 19/75 peers 303 MiB chain 564 MiB db 20 KiB queue 72 KiB sync RPC: 0 conn, 2 req/s, 18610 µs
2019-02-20 17:05:28 UTC Syncing #7527103 0xbf50…773d 2.74 blk/s 54.8 tx/s 1.5 Mgas/s 0+ 7 Qed #7527111 14/75 peers 303 MiB chain 564 MiB db 42 KiB queue 72 KiB sync RPC: 0 conn, 7 req/s, 395 µs
2019-02-20 17:05:28 UTC Imported #7527111 0x4003…add1 (0 txs, 0.00 Mgas, 688 ms, 0.51 KiB) + another 6 block(s) containing 49 tx(s)
2019-02-20 17:05:30 UTC Imported #7527115 0xbc61…b484 (30 txs, 0.77 Mgas, 30 ms, 4.45 KiB) + another 2 block(s) containing 25 tx(s)
2019-02-20 17:06:00 UTC 0/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 26 req/s, 340 µs
2019-02-20 17:06:03 UTC Imported #7527122 0x6b68…f0f8 (8 txs, 0.19 Mgas, 52 ms, 1.66 KiB) + another 6 block(s) containing 54 tx(s)
2019-02-20 17:06:30 UTC 2/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 26 req/s, 82 µs
2019-02-20 17:06:43 UTC Imported #7527123 0x7a10…3003 (8 txs, 0.41 Mgas, 132 ms, 2.31 KiB)
2019-02-20 17:07:02 UTC Imported #7527125 0xab5a…f13c (3 txs, 0.06 Mgas, 481 ms, 0.85 KiB)
2019-02-20 17:07:06 UTC 3/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 2 req/s, 140 µs
2019-02-20 17:08:07 UTC 2/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 2 req/s, 198442 µs2019-02-20 17:08:07 UTC 2/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 2 req/s, 198442 µs

2019-02-20 17:08:07 UTC 2/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 2 req/s, 198442 µs
2019-02-20 17:08:07 UTC 2/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 2 req/s, 198442 µs
2019-02-20 17:08:30 UTC Imported #7527126 0xfb9f…baac (1 txs, 0.02 Mgas, 15691 ms, 0.70 KiB)
2019-02-20 17:08:43 UTC 1/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 2 req/s, 1023942 µs
2019-02-20 17:08:43 UTC 1/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 2 req/s, 1023942 µs
2019-02-20 17:11:18 UTC 1/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 0 req/s, 1023942 µs
2019-02-20 17:11:59 UTC 1/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 0 req/s, 1023942 µs2019-02-20 17:12:08 UTC 1/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 0 req/s, 1023942 µs
2019-02-20 17:12:04 UTC 1/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 0 req/s, 1023942 µs

2019-02-20 17:12:11 UTC Finishing work, please wait...
2019-02-20 17:12:11 UTC 0/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 18 req/s, 1023942 µs2019-02-20 17:12:11 UTC 0/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 18 req/s, 1023942 µs2019-02-20 17:12:11 UTC 0/75 peers 303 MiB chain 564 MiB db 0 bytes queue 72 KiB sync RPC: 0 conn, 18 req/s, 1023942 µs

2019-02-20 17:13:12 UTC Shutdown is taking longer than expected.
2019-02-20 17:17:12 UTC Shutdown timeout reached, exiting uncleanly.

As well as a memory leak:

[1082346.387016] Out of memory: Kill process 64976 (parity) score 335 or sacrifice child
[1082346.387442] Killed process 64976 (parity) total-vm:32278588kB, anon-rss:22075784kB, file-rss:0kB, shmem-rss:0kB
[1082349.951772] oom_reaper: reaped process 64976 (parity), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

Parity 2.2.10 upgraded from 2.2.9.

Serpent6877 on 21 Feb 2019

Just a +1 here, having the same issue running in AWS EC2 with the standard Ubuntu 18.04 AMI using 2.2.10. At some points it seems to be basically causing the entire networking of the machine to break until restarting. Have never had this problem before, and it seems to be all our parity nodes exhibiting the same behavior.

cheeseandcereal on 21 Feb 2019

I also have to add that this affected all of my Parity daemons running at the same time. This is on Ubuntu 18.04.1.

Serpent6877 on 21 Feb 2019

@Serpent6877 did your nodes happen to be doing the same activity or were they on the same machine? ie, was this simultaneously at the same blockheight, or some spike in resource usage affected them all at the same time?

joshua-mir on 21 Feb 2019

👍1

All on same dedicated server. It appears one of them, I assume ETH, used up all the memory and was killed by OOM. At the same time it appears there was a major networking issue as the others stopped responding to RPC calls from the pools. I run 5 Parity instances. ETH, ETC, EXP, ELLA, MUSIC. I wasn't able to grab too much additional information as it caused the server to go wonky.

Serpent6877 on 22 Feb 2019

I will note I run 5 (private) parity instances, each on dedicated VMs, and I observed this behavior on ALL of my Ropsten, and Mainnet configured nodes running 2.2.10, however they did NOT crash at the same time/processing the same block height. They crash at seemingly random times. It does appear to be OOM or something similar, as the entire machine will lock up and require a forced restart in order to recover, so it was hard to introspect on exactly why they crashed.

Right now I reverted to 2.2.8 which does not have this issue.

cheeseandcereal on 25 Feb 2019

❤1 👍1

@cheeseandcereal 2.2.9 should be fine, too.

c0deright on 28 Feb 2019

@joshua-mir do we already know the reason for the shutdown?

We downgraded to 2.2.9 but it was still crashing, so we are now on back 2.2.8 which means that we are missing 3 releases so far.

SimonHausdorf on 1 Mar 2019

👍1

It looks like this is an OOM issue, most likely caused by a spike in memory usage (needs investigation) or memory leak (unlikely, but we should certainly be looking into it from this perspective as well)

joshua-mir on 1 Mar 2019

👍1

It looks like this is an OOM issue, most likely caused by a spike in memory usage (needs investigation) or memory leak (unlikely, but we should certainly be looking into it from this perspective as well)

I had OOM's two times while syncing ropsten network from scratch on node with 15GB memory and cache size set to 10GB on v2.2.11 and on v2.3.4.

Right now it's running on v2.3.5 so far stable.

Also noticed recent parity versions are consuming CPU much more comparing to the previous ones.

gituser on 2 Mar 2019

👍1

After investigation of our logs we see that the 'Shutdown is taking longer than expected' problem also occurred at every restart in version 2.2.7

We do not have earlier logs. Since 2.2.7 every version that we use have this problem(we do not tried 2.4.0 branch), which means that as of now every Parity version that support post-Constantinople ETH cannot be closed/restarted gracefully.

zet-tech on 4 Mar 2019

👍1

@zet-tech that looks more like #10364 than this issue

joshua-mir on 4 Mar 2019

hi

Any chance of some help, thank you :)

2 full trace nodes mainnet. (only showing 1 node in logs)

2.2.7 mem in use for months 4.8GB synced
2.2.11 mem in use > 15GB and crashes @ 30GB if we have sent requests to it

2.2.7 to 2.2.11 memory profile

mem-profile2-2-7-to-2-2-11

parity mem

today's tests

v-2-2-11

Logs as:

https://www.dropbox.com/s/rr4yfw063k551q7/4096-nohup.zip?dl=0

https://www.dropbox.com/s/2asevqc3mzssiwl/8192-nohup.zip?dl=0

debug logs

https://www.dropbox.com/s/szom8ulmc0dz42w/nohup.zip?dl=0

Anything I can provide you to help in finding a solution.

Thanks ...

TJ

t-j-turner on 4 Mar 2019

👍1

@t-j-turner I wonder what --cache-size you have in your settings?
I think the issue somehow correlates to the --cache-size parameter.

gituser on 4 Mar 2019

@t-j-turner I wonder what --cache-size you have in your settings?
I think the issue somehow correlates to the --cache-size parameter.

I was running it with 8192 for months and tried 4096 today ..
same result high memory and increases ...

t-j-turner on 4 Mar 2019

@t-j-turner alright thanks, how about v2.3.5?
For me it seems to be stable with --cache-size 4096

gituser on 4 Mar 2019

@t-j-turner alright thanks, how about v2.3.5?
For me it seems to be stable with --cache-size 4096

Thanks I'll try v2.3.5

parity --l discovery --pruning archive --tracing=on --db-compaction hdd --db-path /paritydata --cache-size 8192 --jsonrpc-interface=all --ws-interface=all --ws-origins=all --no-persistent-txqueue --max-peers=25

t-j-turner on 4 Mar 2019

We think that all those problems (shutdown, increased memory consumption and this #10447) are all caused by the same or very similar memory leaks. All those problems started around 2.2.7 after Constantinople enforced upgrade.

zet-tech on 4 Mar 2019

@t-j-turner
@zet-tech

Could you please check if you're affected by https://github.com/paritytech/parity-ethereum/issues/10361, too? Might be related. For me the last release without issues is 2.2.9/2.3.2 which we're currently running.

c0deright on 4 Mar 2019

Both disk reads and general RAM usage increased after update do 2.2.11. RAM consumption increased by 7.5GB from 21GB to 28.5GB for 3 nodes (ETH, ETC and EXP) on the same VM.

Disk reads are very random. On average there were 20M reads with max of 60M. After update we have 50M on average with 80-100M max.
Disk writes are identical before and after the update.
Moreover we noted other changes: network consumption is lowered by 20%, CPU usage is higher by 2% (8 core) which is 20% of one core.

zet-tech on 4 Mar 2019

👍1

Hey @joshua-mir I think we're seeing the same thing - having CPU usage spike to 100%

nikilster on 8 Mar 2019

Parity Ethereum version: 2.2.11
Operating system: Linux
Installation: one-line installer
Fully synchronized: yes
Network: ethereum
Restarted: yes

We're seeing CPU spike up to 100% under a similar query profile as before (when CPU never went above ~33%)

dtran320 on 8 Mar 2019

Is there any other information we can help provide for this issue? We're seeing it across both full-archive trade as well as warp-synced nodes, and we see spikes in memory and CPU on nodes even without any JSON-RPC requests.

dtran320 on 19 Mar 2019

Just saw this again but on 2.2.9 with Ethereum Classic:

2019-03-20 01:35:49 UTC Imported #7694081 0x9503…c762 (1 txs, 0.02 Mgas, 0 ms, 0.63 KiB)
2019-03-20 01:35:57 UTC 71/75 peers 284 MiB chain 516 MiB db 0 bytes queue 1 MiB sync RPC: 0 conn, 28 req/s, 93 µs
2019-03-20 01:36:07 UTC Imported #7694082 0xf9fb…ae14 (0 txs, 0.00 Mgas, 1 ms, 0.52 KiB)
2019-03-20 01:36:14 UTC Imported #7694083 0x5773…04b7 (1 txs, 0.02 Mgas, 2 ms, 0.63 KiB)
2019-03-20 01:36:27 UTC 71/75 peers 284 MiB chain 516 MiB db 0 bytes queue 1 MiB sync RPC: 0 conn, 27 req/s, 96 µs
2019-03-20 01:36:57 UTC 71/75 peers 284 MiB chain 516 MiB db 0 bytes queue 1 MiB sync RPC: 0 conn, 28 req/s, 84 µs
2019-03-20 01:37:27 UTC 71/75 peers 284 MiB chain 516 MiB db 0 bytes queue 1 MiB sync RPC: 0 conn, 28 req/s, 105 µs
2019-03-20 01:37:41 UTC Imported #7694084 0x0d53…e5e8 (0 txs, 0.00 Mgas, 2 ms, 0.52 KiB)
2019-03-20 01:37:44 UTC Imported #7694085 0x8734…78fe (9 txs, 0.22 Mgas, 7 ms, 1.65 KiB)
2019-03-20 01:37:57 UTC 71/75 peers 284 MiB chain 517 MiB db 0 bytes queue 1 MiB sync RPC: 0 conn, 28 req/s, 97 µs
2019-03-20 01:38:10 UTC Imported #7694086 0x5dcb…7e0f (3 txs, 0.06 Mgas, 2 ms, 0.86 KiB)
2019-03-20 01:38:19 UTC Imported #7694088 0x308b…d7d3 (0 txs, 0.00 Mgas, 3 ms, 0.52 KiB) + another 1 block(s) containing 4 tx(s)
2019-03-20 01:38:27 UTC 71/75 peers 284 MiB chain 517 MiB db 0 bytes queue 1 MiB sync RPC: 0 conn, 22 req/s, 100 µs
2019-03-20 01:38:57 UTC 71/75 peers 284 MiB chain 517 MiB db 0 bytes queue 1 MiB sync RPC: 0 conn, 28 req/s, 85 µs
2019-03-20 01:39:05 UTC Finishing work, please wait...
2019-03-20 01:40:05 UTC Shutdown is taking longer than expected.
2019-03-20 01:44:05 UTC Shutdown timeout reached, exiting uncleanly.
2019-03-20 05:11:16 UTC Starting Parity-Ethereum/v2.2.9-stable-5d5b372-20190203/x86_64-linux-gnu/rustc1.31.1

This is running for a pool that had about 8Gh on it. It's been a few weeks and don't see much progress on this. This seems like a pretty substantial error.

Serpent6877 on 20 Mar 2019

I am confused why this is in a stable release. Our nodes still keep crashing in our production environment. It's hard to keep track of transaction etc while this is happening and we are missing a lot of transaction due to this.

SimonHausdorf on 20 Mar 2019

@SimonHausdorf welcome to parity and ethereum world!

gituser on 20 Mar 2019

The more correct answer is that stable here means: no new features, but all bugfixes are backported.

joshua-mir on 20 Mar 2019

👍1

@SimonHausdorf we can help you out here - wanna dm me on twitter? twitter.com/Nikilster

nikilster on 20 Mar 2019

Anyone testing 2.3.7 to see if this is fixed?

Serpent6877 on 20 Mar 2019

@Serpent6877 the past two patch releases have no related fixes. I'll let people know here immediately when we have more information.

joshua-mir on 20 Mar 2019

2.3.8 got released with the note:

Parity-Ethereum 2.3.8-stable is a bugfix release that improves performance and stability. This patch release contains a critical bug fix where serving light clients previously led to client crashes. Upgrading is highly recommended.

We are testing right now

SimonHausdorf on 24 Mar 2019

still crashing with 2.3.8

SimonHausdorf on 25 Mar 2019

Is this bug really not going to be addressed until 2.6???

This bug is making upgrading ANY Parity nodes completely impossible for us. If there are any further serious security/consensus updates, we will be UNABLE to upgrade and FORCED to migrate all of our running nodes to geth.

This issue is labeled a Priority 0, but how can it not be targeted to be fixed until 2.6? If that's really the case, I know we're going to have to start migrating to geth sooner rather than later, so we aren't accidentally caught off guard with any surprise necessary updates.

Is there no more news regarding this? Is more info needed to help debug?

cheeseandcereal on 3 Apr 2019

we still don't really know what the cause of this issue is. Is it related to the RPC calls? Are we the only one having problems? Is there any workaround and then can we expect the fix? We are also considering to move to geth again.

SimonHausdorf on 3 Apr 2019

What happens if you just add more RAM to the (V)M?

c0deright on 3 Apr 2019

The cause of the issue is that your OS is killing parity (hence shutdown prompts) due to memory spikes in parity - this is what we're trying to identify the source of - the likelihood of crashing can be reduced if you reduce the size of parity's cache, so there is some "headspace" before parity is killed, or as suggested above, adding more RAM.

@cheeseandcereal no, bugfixes get backported - the only scenario where fixes are introduced in the milestone they are labelled with, is if they are not labelled as backports during the PR process, which only happens if they are a breaking change. This will likely be part of the 2.4.x stable and 2.5.x beta tracks, if found before 2.5 gets marked stable.

joshua-mir on 3 Apr 2019

We upgraded now to 2.4.5 with a 16GB machine and it crashed twice within 30 minutes. Memory usage is 1.3% at the moment.

SimonHausdorf on 9 Apr 2019

Just a heads up for people using my systemd file from above (https://github.com/paritytech/parity-ethereum/issues#issuecomment-464786173):

TimeoutStartSec=3s

3 seconds isn't enough anymore for TimeoutStartSec since parity v2.4.x.

With v2.3.x it took a second until the systemctrl start parity call did return (I guess until parity forked). Now with v2.4.x it seems that it takes 4 to 5 seconds for parity to fork which did result in systemd restarting parity because TimeoutStartSec was reached.

Setting TimeoutStartSec=10s did work for me.

c0deright on 9 Apr 2019

Ok, can confirm this issue with v2.3.9 after upgrading production server from v2.2.9:

AWS ec2 t3.medium (4GB RAM)

2019-04-09 10:11:52  main INFO parity_ethereum::run  Starting Parity-Ethereum/v2.3.9-stable-0b42826-20190401/x86_64-linux-gnu/rustc1.33.0
[..]
2019-04-11 07:14:57  IO Worker #2 INFO import     9/10 peers    146 MiB chain  312 MiB db  0 bytes queue  132 KiB sync  RPC:  0 conn,    0 req/s,   25 µs
2019-04-11 07:15:37  IO Worker #0 INFO import     9/10 peers    146 MiB chain  312 MiB db  0 bytes queue  132 KiB sync  RPC:  0 conn,    0 req/s,   23 µs
2019-04-11 07:27:39  IO Worker #0 INFO import     0/10 peers    146 MiB chain  312 MiB db  234 KiB queue  132 KiB sync  RPC:  0 conn,    0 req/s,   24 µs
2019-04-11 07:47:04  IO Worker #0 INFO import     0/10 peers    146 MiB chain  312 MiB db  234 KiB queue  132 KiB sync  RPC:  0 conn,    0 req/s,   24 µs
2019-04-11 08:08:48  main INFO parity_ethereum::run  Starting Parity-Ethereum/v2.3.9-stable-0b42826-20190401/x86_64-linux-gnu/rustc1.33.0

v2.3.9 did run for less than 48 hours when it was killed by the kernel this morning:

[Thu Apr 11 08:08:46 2019] cron invoked oom-killer: gfp_mask=0x24201ca, order=0, oom_score_adj=0
[..]
[Thu Apr 11 08:08:46 2019] 10045 total pagecache pages
[Thu Apr 11 08:08:46 2019] 0 pages in swap cache
[Thu Apr 11 08:08:46 2019] Swap cache stats: add 0, delete 0, find 0/0
[Thu Apr 11 08:08:46 2019] Free swap  = 0kB
[Thu Apr 11 08:08:46 2019] Total swap = 0kB
[Thu Apr 11 08:08:46 2019] 1033112 pages RAM
[Thu Apr 11 08:08:46 2019] 0 pages HighMem/MovableOnly
[Thu Apr 11 08:08:46 2019] 36786 pages reserved
[Thu Apr 11 08:08:46 2019] 0 pages cma reserved
[Thu Apr 11 08:08:46 2019] 0 pages hwpoisoned
[Thu Apr 11 08:08:46 2019] [ pid ]   uid  tgid total_vm      rss nr_ptes nr_pmds swapents oom_score_adj name
[Thu Apr 11 08:08:46 2019] [  345]     0   345     8016      440      22       3        0             0 systemd-journal
[Thu Apr 11 08:08:46 2019] [  381]     0   381    23693       44      16       3        0             0 lvmetad
[Thu Apr 11 08:08:46 2019] [  414]     0   414    10598      202      23       3        0         -1000 systemd-udevd
[Thu Apr 11 08:08:46 2019] [  825]   100   825    24023       53      16       3        0             0 systemd-timesyn
[Thu Apr 11 08:08:46 2019] [  828]     0   828     4031      214      12       3        0             0 dhclient
[Thu Apr 11 08:08:46 2019] [  968]   104   968    64098      233      27       4        0             0 rsyslogd
[Thu Apr 11 08:08:46 2019] [  970]     0   970     7154       78      17       3        0             0 systemd-logind
[Thu Apr 11 08:08:46 2019] [  972]     0   972     6511       51      18       3        0             0 atd
[Thu Apr 11 08:08:46 2019] [  989]     0   989     1099       22       8       3        0             0 acpid
[Thu Apr 11 08:08:46 2019] [  991]   107   991    10722      108      26       3        0          -900 dbus-daemon
[Thu Apr 11 08:08:46 2019] [ 1003]     0  1003     6932       70      18       3        0             0 cron
[Thu Apr 11 08:08:46 2019] [ 1048]     0  1048    16378      180      39       4        0         -1000 sshd
[Thu Apr 11 08:08:46 2019] [ 1073]     0  1073     4868       58      14       3        0             0 irqbalance
[Thu Apr 11 08:08:46 2019] [ 1077]   112  1077     5993      126      16       3        0             0 nrpe
[Thu Apr 11 08:08:46 2019] [ 1093]   109  1093    14620     1138      29       3        0             0 snmpd
[Thu Apr 11 08:08:46 2019] [ 1097]     0  1097     3618       35      12       3        0             0 agetty
[Thu Apr 11 08:08:46 2019] [ 1107]  1000  1107  1141648   937692    1920       8        0             0 parity-binary  <<---
[Thu Apr 11 08:08:46 2019] [ 1109]     0  1109     3664       34      12       3        0             0 agetty
[Thu Apr 11 08:08:46 2019] [ 1212]     0  1212    16352      116      23       3        0             0 master
[Thu Apr 11 08:08:46 2019] [ 1215]   113  1215    16881      114      25       3        0             0 qmgr
[..]
[Thu Apr 11 08:08:46 2019] Out of memory: Kill process 1107 (parity-binary) score 943 or sacrifice child
[Thu Apr 11 08:08:46 2019] Killed process 1107 (parity-binary) total-vm:4566592kB, anon-rss:3750768kB, file-rss:0kB

There wasn't even an attempt to shutdown the application, it was SIGKILLED right away.

Running v2.4.5 now

c0deright on 11 Apr 2019

Same problem with v2.4.5.
I'm back at v2.2.9 for the time being.

Some debugging info while v2.4.5 was running:

log_parity_mem.sh:

#!/bin/sh

F="/proc/$(pidof parity parity-binary)/status"
L="$HOME/.local/share/io.parity.ethereum/memory.log"

grep -E '^VmRSS:|VmData:' "$F" | tr -d '\n' | awk '{print strftime("%Y-%m-%d %H:%M")"\t"$2"\t"$4}' >> "$L" 2>/dev/null

memory.log:

2019-04-12 00:00        2848744 3565616
2019-04-12 01:00        2848744 3565616
2019-04-12 02:00        2873272 3565616
2019-04-12 03:00        2829524 3565616
2019-04-12 04:00        2880056 3565616
2019-04-12 05:00        2880412 3565616
2019-04-12 06:00        2912112 3631152
2019-04-12 07:00        2924428 3631152
2019-04-12 08:00        2943600 3631152
2019-04-12 09:00        2890648 3575624
2019-04-12 10:00        2883072 3575624
2019-04-12 11:00        2854908 3575624
2019-04-12 12:00        2831324 3575624
2019-04-12 13:00        2879380 3575624
2019-04-12 14:00        3124508 3968840
2019-04-12 15:00        3163796 3968840
2019-04-12 16:00        3145720 3837768
2019-04-12 17:00        3193028 3968840
2019-04-12 18:00        3240096 3968840
2019-04-12 19:00        3195692 3968840

2019-04-12 19:01        3204136 3968840
2019-04-12 19:02        3206432 3968840
2019-04-12 19:03        3216936 3968840
2019-04-12 19:04        3204816 3903304
2019-04-12 19:05        3210468 3903304
2019-04-12 19:06        3210732 3903304
2019-04-12 19:07        3236340 3968840 <<--
2019-04-12 19:08        3422196 4165448
2019-04-12 19:09        3587260 4427592
2019-04-12 19:10        3612416 4362056
2019-04-12 19:11        3616208 4362056
2019-04-12 19:12        3626244 4427592
2019-04-12 19:13        3674216 4427592 <<--
2019-04-12 19:14        3679892 4427592
2019-04-12 19:15        3679980 4427592
2019-04-12 19:16        3679980 4427592
2019-04-12 19:17        3680096 4427592
2019-04-12 19:18        3680096 4427592
2019-04-12 19:19        3680096 4427592
2019-04-12 19:20        3680096 4427592
2019-04-12 19:21        3680096 4427592

19:07

parity RSS size: 3160 MByte
parity Data size: 3875 MByte

19:13

parity RSS size: 3588 MByte
parity Data size: 4323 MByte

Within 6 minutes parity RSS size grew by 428 MByte, Data grew by 448 MByte

That's an instance that is not heavily queried via RPC at all.

ReadOPS vs. NetworkOut

parity

At the same time the memory spike happened disk read OPS went through the roof and outgoing network traffic spiked, too.

c0deright on 12 Apr 2019

👍1

Hi Joshua.
I also get this error. I have just installed the latest 2.5.0 beta.
I have a i3 8th gen Intel based Linux ubuntu 19.04. 16GB RAM, SSD disk 500GB and a swap file created for additional memory of 2GB.
The error in the sysog is: OOM-Kill. See attachments.

mexican1973 on 25 Apr 2019

Needs @5chdn back!!

I'm sure he'll quickly find the root cause of this issue :)

Also pinging @tomusdrw.

gituser on 25 Apr 2019

👀1

Ok I have some more detail on this issue from the kernel.log

Apr 22 16:25:32 auraserver kernel: [ 15.355051] rfkill: input handler disabled
Apr 22 17:21:48 auraserver kernel: [ 3391.541636] rfkill: input handler enabled
Apr 22 17:22:06 auraserver kernel: [ 3409.778462] rfkill: input handler disabled
Apr 22 17:22:19 auraserver kernel: [ 3421.992384] rfkill: input handler enabled
Apr 22 17:22:45 auraserver kernel: [ 5.929259] rfkill: input handler disabled
Apr 22 18:51:28 auraserver kernel: [ 5329.029022] rfkill: input handler enabled
Apr 22 21:29:47 auraserver kernel: [ 6.104559] rfkill: input handler disabled
Apr 22 22:42:18 auraserver kernel: [ 4356.929697] rfkill: input handler enabled
Apr 22 22:42:40 auraserver kernel: [ 5.984493] rfkill: input handler disabled
Apr 23 11:09:15 auraserver kernel: [44800.224303] Verifier #1 invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 23 11:09:15 auraserver kernel: [44800.224313] oom_kill_process.cold.29+0xb/0x1d6
Apr 23 11:09:15 auraserver kernel: [44800.224437] [ 1513] 1000 1513 98368 106 114688 105 0 gsd-rfkill
Apr 23 11:09:15 auraserver kernel: [44800.224464] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/[email protected],task=parity,pid=25317,uid=1000
Apr 23 11:09:15 auraserver kernel: [44800.224468] Out of memory: Kill process 25317 (parity) score 880 or sacrifice child
Apr 23 11:09:15 auraserver kernel: [44800.224533] Killed process 25317 (parity) total-vm:17468556kB, anon-rss:15216040kB, file-rss:0kB, shmem-rss:0kB
Apr 23 21:44:23 auraserver kernel: [ 5.992519] rfkill: input handler disabled
Apr 24 00:30:04 auraserver kernel: [ 9946.503612] Xorg invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 24 00:30:04 auraserver kernel: [ 9946.503619] oom_kill_process.cold.29+0xb/0x1d6
Apr 24 00:30:04 auraserver kernel: [ 9946.503739] [ 2109] 1000 2109 98368 0 114688 214 0 gsd-rfkill
Apr 24 00:30:04 auraserver kernel: [ 9946.503764] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/[email protected],task=parity,pid=5426,uid=1000
Apr 24 00:30:04 auraserver kernel: [ 9946.503768] Out of memory: Kill process 5426 (parity) score 887 or sacrifice child
Apr 24 00:30:04 auraserver kernel: [ 9946.503831] Killed process 5426 (parity) total-vm:18648964kB, anon-rss:15500416kB, file-rss:0kB, shmem-rss:0kB
Apr 24 21:09:14 auraserver kernel: [84293.816484] parity invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Apr 24 21:09:14 auraserver kernel: [84293.816491] oom_kill_process.cold.29+0xb/0x1d6
Apr 24 21:09:14 auraserver kernel: [84293.816611] [ 2109] 1000 2109 98368 0 114688 229 0 gsd-rfkill
Apr 24 21:09:14 auraserver kernel: [84293.816645] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/[email protected],task=parity,pid=7627,uid=1000
Apr 24 21:09:14 auraserver kernel: [84293.816648] Out of memory: Kill process 7627 (parity) score 839 or sacrifice child
Apr 24 21:09:14 auraserver kernel: [84293.816712] Killed process 7627 (parity) total-vm:17939056kB, anon-rss:15204340kB, file-rss:0kB, shmem-rss:0kB
Apr 25 00:23:10 auraserver kernel: [95929.267001] rfkill: input handler enabled
Apr 25 22:10:35 auraserver kernel: [ 5.909278] rfkill: input handler disabled

Maybe that can help.

mexican1973 on 25 Apr 2019

Looks like I could solve it by a workaround. I killed the DB and then I created a new swap file of 32GB size and restarted the sync from scratch. Now it has been syncing nicely and I am already a round 4.8M blocks downloading. Lets see how long this goes well.

mexican1973 on 26 Apr 2019

👍1

Looks like I could solve it by a workaround. I killed the DB and then I created a new swap file of 32GB size and restarted the sync from scratch. Now it has been syncing nicely and I am already a round 4.8M blocks downloading. Lets see how long this goes well.

If there is a memory leak, which this looks more like to me, then your swap will just be a fix for an extended amount of time or until you restart the daemon. I have a 64GB system with 32GB free and it runs my system out of memory. Especially when I have five chains running it. This is on Ubuntu 18.04. I see most everyone posting on this issue is running similar. What test OS does Paritytech use? How many others here are running pools or apps utilizing the RPC port?

Serpent6877 on 1 May 2019

I am seeing this on Parity 2.4.5 on two seperate archive nodes.

mewwts on 5 May 2019

I noticed when you start to sync with --no-warp it doesn't come. Only when syncing with just "parity" which is in warp mode the memory issues come. Me node got very slow because 16GB memory were not enough.

mexican1973 on 6 May 2019

@mexican1973 I have warp sync turned off, and I have memory leaks that results in crashes.

mewwts on 6 May 2019

I used parity 2.2.11 without memory issues, but now all tags got removed: https://hub.docker.com/r/parity/parity/tags

Are you kidding? I have to use bugged version because you depricated working one?

Pzixel on 6 May 2019

@Pzixel You don't have to use parity-ethereum.

After all it's an open source project and you're free to help in debugging/fixing this issue or simply don't use parity-ethereum at all.

Or do you pay the developers and are somehow entitled to a working copy?

You see, we all use the software for free (as in free beer) and have the same issue but demanding anything or cracking jokes isn't helping.

c0deright on 6 May 2019

I don't ask anything to do, I ask not to do. Actually, not to remove previous images, especially when new images are flawed.

You don't pay github to store your repos but you probably would be suprised and dissapointed if they remove all of them one day.

I don't get why removing old versions when new ones are buggied? You force people to upgrade to make them suffer. That's not how it should work, no mater OSS or not.

Pzixel on 6 May 2019

I agree with @Pzixel the parity version cycle is not optimal.

They are deprecating old versions too early which results in buggy new version promoted as stable.

gituser on 6 May 2019

We removed old images from dockerhub due to their recent security breach. The correct issue to talk about this would be https://github.com/paritytech/parity-ethereum/issues/10627, please keep conversation in this issue on topic, thanks 🙏

joshua-mir on 6 May 2019

👍2

Parity Ethereum version: 2.4.5
Operating system: Linux
Installation: Arch Package Repo
Fully synchronized: yes
Network: ethereum
Restarted: yes

Just to add to the dataset of cases this issue shows up, seems to be a memory leak and not necessarily a memory "spike". I have my cache set to 8gb and Parity regularly, multiple times per day now, exceeds this threshold until the server ultimately runs out of memory (~16gb).

Today, I was maintaining under the 8gb threshold for a few hours until I started my script that does the hard work, ram usage immediately began to climb at a much quicker pace... The script utilizes the following web3 methods through web sockets:

web3.eth.getBlock(blockNumber)
web3.currentProvider.send('parity_getBlockReceipts', [ block_hex ])
web3.currentProvider.send('trace_transaction', [ tx_hash ])

dafky2000 on 10 May 2019

👍1

@dafky2000 Interesting is that I run a pool which uses these same methods and have same issue.

Serpent6877 on 11 May 2019

stack backtrace:
   0:     0x5644798d919d - <no info>
   1:     0x5644798d8d82 - <no info>
   2:     0x5644798d7f76 - <no info>
   3:     0x564478475b58 - <no info>
   4:     0x564479946528 - <no info>
   5:     0x564479945fc1 - <no info>
   6:     0x564479945ea5 - <no info>
   7:     0x56447996615c - <no info>
   8:     0x56447905633d - <no info>
   9:     0x5644790c6daf - <no info>
  10:     0x5644790cf8d5 - <no info>
  11:     0x5644790e79b5 - <no info>
  12:     0x564478315b1a - <no info>
  13:     0x5644784f3ed5 - <no info>
  14:     0x5644785ce98a - <no info>
  15:     0x56447828f065 - <no info>
  16:     0x5644784d0c48 - <no info>
  17:     0x56447822d466 - <no info>
  18:     0x56447821b10c - <no info>
  19:     0x5644781fa7d2 - <no info>
  20:     0x56447830145f - <no info>
  21:     0x56447830289d - <no info>
  22:     0x56447830289d - <no info>
  23:     0x564478378725 - <no info>
  24:     0x5644784625cc - <no info>
  25:     0x56447846d943 - <no info>
  26:     0x5644782f3c76 - <no info>
  27:     0x564479792c21 - <no info>
  28:     0x564479794a25 - <no info>
  29:     0x564479795ba0 - <no info>
  30:     0x564479951ed9 - <no info>
  31:     0x564479795e9f - <no info>
  32:     0x56447978ee35 - <no info>
  33:     0x56447978df4d - <no info>
  34:     0x56447976fcad - <no info>
  35:     0x56447976fe47 - <no info>
  36:     0x564479770730 - <no info>
  37:     0x564479768209 - <no info>
  38:     0x564479795512 - <no info>
  39:     0x5644797952ed - <no info>
  40:     0x564479796bcd - <no info>
  41:     0x564479795beb - <no info>
  42:     0x564479951ed9 - <no info>
  43:     0x56447979253f - <no info>
  44:     0x56447995121d - <no info>
  45:     0x7f542e56d163 - <no info>
  46:     0x7f542e479dee - <no info>
  47:                0x0 - <no info>

Thread 'jsonrpc-eventloop-0' panicked at 'Transaction replayed; qed: Execution(Internal("InvalidStateRoot(0x8d7446cb286cdb80ec06a7cc912c1c4fe44a29ece0840772f4f69a8b1204fa12)"))', src/libcore/result.rs:997

SimonHausdorf on 18 May 2019

Seems like 128 GB ram are not sufficient for our operations. Still crashing with newest version. Do we at least already know the reason for the memory leak?

SimonHausdorf on 26 May 2019

I think i will switch over to geth as soon as 1.9 comes. Seems parity is not useable anymore and support is poor since afri left.

mexican1973 on 26 May 2019

I think i will switch over to geth as soon as 1.9 comes.

Do you have any idea how to move accounts over from parity to geth?

SimonHausdorf on 26 May 2019

Use the best tools for your use-case 👍

@SimonHausdorf Just move your key files from parity's key file folder to geth's - it's all the same standard - either way you shouldn't be managing your keys on your node unless you are running a PoA/PoS authority/signer, both geth and parity are moving away from supporting that.

joshua-mir on 26 May 2019

support is poor since afri left.

@mexican1973 Why do you think he's left? @soc1c is here :)

Seems parity is not useable anymore

@mexican1973 Well it's been working for me quite stable - I didn't have much of OOMs lately.
Did you try tuning your --cache-size parameter? I've been running parity v2.4.6 with 20GB RAM + --cache-size 8192 with --pruning fast and --tracing on. Works for me.

@joshua-mir

either way you shouldn't be managing your keys on your node

Many users are still relying on personal RPC and I don't think it's a good idea to remove this anytime soon unless you want to make a chaos for many services using parity and make them to switch off Ethereum/Ethereum Classic support for quite some time to rework what works already.

gituser on 27 May 2019

👍1

Is there any progress in finding the root cause of this?

We're still running v2.2.9 because of this single bug and I curse the day the next ethereum consensus upgrade/change is coming and we have no other option as to upgrade (or change to geth).

@SimonHausdorf Just move your key files from parity's key file folder to geth's - it's all the same standard - either way you shouldn't be managing your keys on your node unless you are running a PoA/PoS authority/signer, both geth and parity are moving away from supporting that.

@joshua-mir Could you elaborate what that means exactly? How am I supposed to handle/manage my keys if not in the parity keys directory? What's the best practice way of handling keys with parity other than dumping them in the keys directory?

c0deright on 1 Aug 2019

@c0deright we've put complete removal of key management mostly on ice while we do other refactoring, but I meant using tools like geth's clef or doing key management entirely on the client side or in external backends, instead of signing transactions using parity's rpc endpoints - in practice this wouldn't mean changing logic, but changing the endpoint you call when making a transaction.

have you tried 2.6 beta to see if you are still seeing these issues? It's hard to find the root cause because while it is relatively easy to reproduce this issue, it takes a long time to get a single reproduction, (and in that time if we're running with the logging necessary to identify the cause, the log files end up eating all the space)

joshua-mir on 1 Aug 2019

@joshua-mir

we've put complete removal of key management mostly on ice while we do other refactoring, but I meant using tools like geth's clef or doing key management entirely on the client side or in external backends, instead of signing transactions using parity's rpc endpoints - in practice this wouldn't mean changing logic, but changing the endpoint you call when making a transaction.

I tend to disagree with this. If you're relying e.g. on parity_localtransactions
https://wiki.parity.io/JSONRPC-parity-module#parity_localtransactions it won't be possible anymore to track those locally via this RPC call to track e.g. balance changes or pending transactions via this specific RPC call.

Not sure about clef tool, how mature is it?
And also geth didn't remove their accounting system it's still there.. so..
Also there is no point in using 99999 tools to make an interaction with Ethereum blockchain, it's not convenient at all. For beginners it's important to have a tool where you can do most important tasks: create new keys, sign and send transactions.

gituser on 1 Aug 2019

This is starting to go off topic from this issue - we won't make any changes unless the upgrade path is clear and straightforward, the same likely applies to geth, but obviously I can't speak for them. We're trying to avoid the problem we had back in 1.11 -> 2.0 with no clear update path for many users.

For beginners it's important to have a tool where you can do most important tasks: create new keys, sign and send transactions.

Totally agree, but parity-ethereum wasn't designed to be that tool (or rather, it originally was, but since 2.0 we've been moving away from it,) do you primarily sign transactions using parity-ethereum? There are much better and safer tools for that usecase. I would like to point out this issue thread is for an issue experienced almost exclusively by operators of archive/archive+tracing nodes, who especially shouldn't be using their nodes for signing transactions and key management.

it won't be possible anymore to track those locally via this RPC call to track e.g. balance changes or pending transactions via this specific RPC call.

Hypothetically, you still submit transactions to your node over rpc, and they are recognized as local transactions - you just submit them presigned (by an external tool,) via eth_sendrawtransaction.

joshua-mir on 1 Aug 2019

@joshua-mir

have you tried 2.6 beta to see if you are still seeing these issues?

Testing now.

c0deright on 8 Aug 2019

Still having this issue on 2.6-beta.

mewwts on 12 Aug 2019

Can't say for sure because of https://github.com/paritytech/parity-ethereum/issues/10960 but top reports the v2.6.0 binary using 89% of my 4GB RAM (5437612 VIRT, 3.38g RES).

So I would assume this issue is still a thing in v2.6.0

c0deright on 13 Aug 2019

Hi all,

I did some testing with 2.6.1 on my dedicated test node and it looks good so far.

Although 4GB of RAM isn't enough for v2.6.x anymore I'm not seeing any kinds of memory leaks so far.

7 days runtime: memory usage

parity

same graph over 90 days for the bigger picture

parity2

c0deright on 22 Aug 2019

Hi all,

I did some testing with 2.6.1 on my dedicated test node and it looks good so far.

Although 4GB of RAM isn't enough for v2.6.x anymore I'm not seeing any kinds of memory leaks so far.

7 days runtime: memory usage

same graph over 90 days for the bigger picture

from you analysis it's obviously a bug, as parity-v2.6.1 now consumes double memory comparing to parity-v2.2.9 this should be investigated and fixed as there is no reason why it should be like that.

gituser on 22 Aug 2019

@c0deright thank you for sharing that!

@gituser I'm not sure I agree. I think that Parity should take advantage of the larger headroom on the 8Gb server and put the available memory to good use. I'd expect it to use even more memory on a 16Gb server. The important datum is that it does not leak. That said it's worrying that recent versions misbehave when memory starved.

dvdplm on 22 Aug 2019

@dvdplm

I'm not sure I agree. I think that Parity should take advantage of the larger headroom on the 8Gb server and put the available memory to good use. I'd expect it to use even more memory on a 16Gb server. The important datum is that it does not leak. That said it's worrying that recent versions misbehave when memory starved.

Yes, I agree with you of course parity should use all available resources if possible and auto tune if the user decides to run it without specifying any manual options on restricting it's memory usage.

However, if the user only wants to give parity e.g. 4GB of memory parity should honor this requirement.

And third: if parity v2.2.9 worked just fine on 4GB and didn't require that high I/O on the same hardware the newest parity should also work on the same hardware and shouldn't require more memory.

gituser on 26 Aug 2019

Still no sign of a memory leak here
parity

c0deright on 26 Aug 2019

17 days later and still no issues. Parity v2.6.1 now running over a month.
Did anyone else here affected by the memory leak test with v2.6.x?

combined

c0deright on 16 Sep 2019

I've been seeing some issues on Digital Ocean, but the nodes we migrated to GCP have been running for a week+ without issues at all.

mewwts on 18 Sep 2019

OS: Ubuntu 18.04
Parity: 2.6.3-beta
Function: Pool node

[677998.988070] Out of memory: Kill process 20573 (parity) score 425 or sacrifice child
[677998.988812] Killed process 20573 (parity) total-vm:46384256kB, anon-rss:27971796kB, file-rss:0kB, shmem-rss:0kB
[678001.469503] oom_reaper: reaped process 20573 (parity), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

Another out of memory issue. You can see the memory increase in my weekly graph.

memory-week

First start of the new Parity was the beginning of the week. I run three Parity on this server. One for ETC, ETH, ELLA. However ELLA has no miners on it. ETC/ETH get upwards of 1TH on the pool but only moderate RPC traffic during that time.

ETC Config:
[network]
port = 38000
allow_ips = "public"
nat = "extip:"
min_peers = 100
max_peers = 125

[rpc]
port = 18000
interface = "local"
apis = ["eth", "net", "parity", "rpc"]

[websockets]
disable = true

[footprint]
pruning = "fast"
pruning_memory = 256
cache_size_db = 512
cache_size = 4096

ETH Config:
[network]
port = 38001
allow_ips = "public"
nat = "extip:"
min_peers = 100
max_peers = 150

[rpc]
port = 18001
interface = "local"
apis = ["eth", "rpc", "net"]
server_threads = 12
processing_threads = 12

[websockets]
disable = true

[footprint]
pruning = "fast"
pruning_memory = 256
cache_size_db = 2048
cache_size = 8192

This is running on a dedicated dual Intel(R) Xeon(R) CPU E5-2690 server with 64GB. Without these parity it runs about 20GB steady. Previous Parity 2.2.9 seemed to work without any memory climbing.

Serpent6877 on 2 Oct 2019

Can you try running with the same config but without specifying the cache size options? i.e. remove cache_size_db and cache_size.
We have some anecdotal indication that specifying the cache size manually can result in this kind of odd memory consumption.

dvdplm on 2 Oct 2019

👍1

@dvdplm I also have this issue with parity-v2.5.9-stable on ethereumclassic network with --cache-size 1024 in VM with 8GB memory.

parity consumes nearly all available memory and part of the swap and OOMs at some point.

gituser on 2 Oct 2019

@gituser can you try the same thing: run without specifying the cache-size option and report back if that changes anything? Thanks!

dvdplm on 2 Oct 2019

@dvdplm, I've restarted the parity now without the cache-size option and will see how it goes and report back. Right after the restart - there is a minimum memory consumption.

gituser on 2 Oct 2019

@dvdplm I commented out the cache entries and restarted. I am still seeing a steady climb in memory usage by Parity. You can see on my graph the restart of two of the Parity and the memory usage drop. Roughly 20GB for each Parity tho I was only setting 10GB of cache size.

memory-week

You can see by my monthly the memory increased usage. The left flatter side is Parity 2.2.9. The switch to Parity 2.6.3-beta on the right.

memory-month

I run ELLA, ETH, ETC pools. I noticed that the ELLA Parity is not having the same crashing issues and I wonder if this is not RPC related as I have no miners on ELLA. Any next steps I can take?

Serpent6877 on 7 Oct 2019

…I commented out the cache entries and restarted… I was only setting 10GB of cache size.

Not sure I understand: was this last run with a config without any explicit --cache-* settings or did you set a cache size?

if this is not RPC related as I have no miners on ELLA. Any next steps I can take?

You mean you suspect it might be RPC related? That would be a good thing as it might be a bit easier to debug. It would be very interesting to see more info on the kinds of RPC calls that is hitting the nodes. I noticed your ETH and ETC configs differs slightly in the APIs that are active. Do you know if that results in any change in the mem usage pattern? I guess not, or if it does it's probably drowned out in the noise.
One thing that would be super useful but perhaps hard for you to do is to try running with a single API at a time to help us figure out which one is leaking (or at least help disprove the RPC-hypothesis). If that is not possible, running with -l rpc to log all RPC calls would also be useful. Finally, there's a PR pending to improve the thread handling in jsonrpc. It is possible that might help somewhat (or at least make it easier to debug); it'd require running a nightly build until it ends up in a release though: would it be an option for you to run a non-release build?

dvdplm on 7 Oct 2019

@dvdplm This last run is without any cache settings on either of the three Parity instances. But was stating that I had only set 10GB cache previously and it appeared to be using well over 20GB for each Parity.

I am wondering if this isn't RPC related. I am doing average over a minute of 36 RPC calls a second. Here is the RPC calls I am performing:

eth_getWork
eth_submitWork
eth_getBlockByNumber
eth_submitHashrate
eth_submitLogin
eth_getBlockByHash
eth_getUncleByBlockNumberAndIndex
eth_getTransactionReceipt
eth_getBalance
eth_sign
eth_sendTransaction

I could possibly run the nightly build on a single block. ETC might be best. It seems to die the quickest.

Serpent6877 on 7 Oct 2019

stating that I had only set 10GB cache previously and it appeared to be using well over 20GB

I see, that makes sense, thank you.

Here is the RPC calls I am performing:

Excellent, that is useful. I'll try to repeat it on my end.

dvdplm on 7 Oct 2019

@Serpent6877 Your config above says you have apis = ["eth", "net", "parity", "rpc"] (ETC) and apis = ["eth", "rpc", "net"] activated. If you only use the list of eth_* RPC calls above, then do you think you could try running with apis = ["eth"] only to see if we can isolate the problem to that API only?

Also, may I ask how you are submitting the http requests to the parity node? Is it a single thread/process or several? From localhost or from the network?

Finally it would be useful to know if you have a rough idea of the mix of the RPC calls: are some more common than others?

dvdplm on 7 Oct 2019

@dvdplm It seems to be consuming 1GB a day between the two Parity running. The ELLA Parity seems to be flatlined right now.

I can set it to just eth. I did not realize that was for the RPC calls. I just looked and it does appear I am also doing these additional calls:

net_peerCount
net_listening
net_version

This is a modified form of Sammy's Open Ethereum Pool found here: https://github.com/sammy007/open-ethereum-pool

It is wrote in GO and does these calls:

rpcResp, err := r.doPost(r.Url, "eth_getWork", []string{})

I changed my config to use "eth", "net". Will see how that goes. I will also try and get you more info on the RPC calls. A majority of it will be the eth_getWork/eth_submitWork commands.
e open

Serpent6877 on 8 Oct 2019

@Serpent6877 any updates on your end? On my side I've tried to hit my node (compiled from current master) with a mix of your RPC calls at ~300req/s. Memory consumption is very stable here but I'm running macOS so it's not really indicative. FWIW we have a new version of jsonrpc coming up that should provide better performance and be less resource consuming but I'm not so sure it will fix the leaks you're seeing. As soon as it lands I'll point you to a build and ask you to try it.

dvdplm on 9 Oct 2019

@dvdplm I have not made the changes to the apis as of yet. There was a small memory usage increase over the last several days as seen here:

memory-week

But it has tapered off slightly today.

memory-day

I think I will need a few more days to see if it continues to increase more or has finally leveled out. I will try and capture more info on the types of API call's being performed.

Serpent6877 on 10 Oct 2019

Ok, thanks. No rush ofc. These things are really tricky to pin down.

We merged https://github.com/paritytech/parity-ethereum/pull/11151 today which contains some good changes to threads usage. When I load tested that build today I saw better performance and much diminished variance in request times. But I could not reproduce the memory leak on my system with the current master so not sure if it fixes your issue. If you have the means to check out master and build parity and try it under your workload that'd be ideal of course.

dvdplm on 10 Oct 2019

@dvdplm Sorry for the lack of updates. I have had a server fan issue that caused a reboot and thus waiting to collect more data. I do notice that it does seem to be more stable without the cache settings. I'll let it run for a few days then restart with the cache to see if that makes a difference.

Serpent6877 on 15 Oct 2019

Excellent, thank you for the update.

dvdplm on 15 Oct 2019

This matches exactly what I'm seeing using Parity 2.5.9 on my NanoPC T4.

Stable connection, good number of peers. After 8 hours memory usage through the roof, low peer count and continually losing sync. Restart and the problem goes away.

I've deleted the cache setting as mentioned early - will monitor and let you know.

sverzijl on 24 Oct 2019

@Serpent6877 @sverzijl any news on you guys' nodes?

dvdplm on 1 Nov 2019

@dvdplm Since removing the cache and the restarts things seem pretty stable to me. I have not had a chance to re-enable the cache and test that. Mainly due to not wanting to lose any further miners. I am waiting for the next beta to test anything further.

Serpent6877 on 4 Nov 2019

@Serpent6877 ok, thank you for the report!

dvdplm on 4 Nov 2019

I'm syncing a new fresh node under the same conditions I was having issues with before. I have to non-warp fully sync my node to be able to test under the conditions that I was having before, but when that's finished, I'll confirm I'm running parity 2.5.10 (or later, with #11151), and test again and see if I run into the same issues. My guess is it will probably be a couple weeks before I'm able to properly attempt to reproduce this, but I'll update when I do.

cheeseandcereal on 5 Nov 2019

@cheeseandcereal thank you for keeping us in the loop here and for your patience.

dvdplm on 6 Nov 2019

Yesterday, because of the the upcoming Istanbul Ethereum Hardfork (block 9069000, targeted for the 4th of December) I upgraded my production boxes from v2.2.9 to v2.6.5-beta.

After several months of running the v2.6.x branch on my test node, I finally made the switch on prod after increasing RAM to at least 8GB.

So far no issues. IO dropped on machines previously running with 4GB of RAM, so that's good. I even kept cache=2000 in the config of my boxes although others suggested to get rid of it completely.

Didn't see any signs of a memory leak for months.

c0deright on 26 Nov 2019

So I upgraded to 2.6.6-beta from my 2.6.3-beta and the memory seems okay. However a whole new issue has come about that is even worse. Parity simple freezes up completely. Stops logging. Unresponsive to RPC. Nada. I have to do a kill -9 just to stop it. Then when restarted it has to sync to catch up. So it was definitely in a completely locked state. I am at a loss on how to proceed since I need this for Istanbul.

[parity]
chain = "mainnet"
base_path = "/pools/ethpool/parityeth"
identity = "mydomain.org"

[network]
port = 38001
allow_ips = "public"
nat = "extip:x.x.x.x"
min_peers = 75
max_peers = 100

[rpc]
port = 18001
interface = "local"
apis = ["eth","net","parity"]
#server_threads = 12
#processing_threads = 12

[websockets]
disable = true

[footprint]
pruning = "fast"
pruning_memory = 256
#cache_size_db = 2048
#cache_size = 8192

[misc]
log_file = "/pools/ethpool/logs/parity.log"
color = false
logging = "jsonrpc=trace"

[account]
unlock = ["0x0000"]
password = ["/pools/ethpool/passeth"]

[ui]
disable = true

[mining]
author = "0x0000"

I commented out the server_threads and processing_threads to test that out. Any help appreciated.

Serpent6877 on 6 Dec 2019

Nothing obvious come to mind. A few things to try:

start parity with an empty db: parity --base-path=/path/to/somewhere/temporary: does it still lock up?
start parity without rpc enabled: parity --no-jsonrpc --no-ipc: still locks up?
compile master from source and check if master works (make sure you have a backup of your db for this)
enable full logging with RUST_LOG=trace parity --your --settings --here: when it locks up, is there anything logged at all right before? (logs are very verbose at that level)

dvdplm on 6 Dec 2019

Any news from you @Serpent6877? I just had the exact same thing happen to me with v2.6.6:

At 05:05:09 UTC+1 parity-v2.6.6-beta stopped working while or after importing block #9123828.

CPU Usage dropped to zero
network activity dropped to idle
disk IO dropped to zero
logging to $log_file stopped

Had to kill parity. After restart of v2.6.6 parity did take some time to sync block #9123828 (see below) but then went on.

Just now, I upgraded to 2.6.7 and commented out my cache_size = 2000. But without cache_size Disk-Reads and CPU usage went through the roof, so I reverted back to cache_size = 2000 with v2.6.7.

log

[..]
2019-12-18 05:04:03  IO Worker #0 INFO import    12/12 peers    146 MiB chain  357 MiB db  0 bytes queue    2 MiB sync  RPC:  0 conn,    1 req/s,  900 µs
2019-12-18 05:04:09  Verifier #0 INFO import  Imported #9123821 0xe1f9…d331 (195 txs, 9.95 Mgas, 732 ms, 30.85 KiB)
2019-12-18 05:04:20  Verifier #0 INFO import  Imported #9123822 0x5c60…9f64 (154 txs, 9.94 Mgas, 1008 ms, 26.69 KiB)
2019-12-18 05:04:21  Verifier #0 INFO import  Imported #9123823 0x37d2…0c7b (147 txs, 9.96 Mgas, 880 ms, 30.62 KiB)
2019-12-18 05:04:26  Verifier #1 INFO import  Imported #9123824 0x65b4…0b82 (134 txs, 9.95 Mgas, 1264 ms, 31.80 KiB)
2019-12-18 05:04:33  IO Worker #2 INFO import    12/12 peers    146 MiB chain  358 MiB db 736 bytes queue    3 MiB sync  RPC:  0 conn,    0 req/s,  900 µs
2019-12-18 05:04:34  Verifier #1 INFO reorg  Reorg to #9123825 0x28a5…ee8d (0x6922…e1a3 #9123823 0x37d2…0c7b 0x65b4…0b82)
2019-12-18 05:04:34  Verifier #1 INFO import  Imported #9123825 0x28a5…ee8d (231 txs, 9.95 Mgas, 714 ms, 39.11 KiB)
2019-12-18 05:04:37  Verifier #1 INFO import  Imported #9123826 0x69c5…5da7 (151 txs, 9.96 Mgas, 1478 ms, 27.72 KiB)
2019-12-18 05:05:03  IO Worker #3 INFO import    12/12 peers    146 MiB chain  358 MiB db  0 bytes queue    2 MiB sync  RPC:  0 conn,   12 req/s,  900 µs
2019-12-18 05:05:06  Verifier #1 INFO import  Imported #9123827 0x7b70…a4ea (91 txs, 9.95 Mgas, 1056 ms, 24.34 KiB)
2019-12-18 05:05:09  Verifier #1 INFO import  Imported #9123828 0xe177…bcbd (253 txs, 9.95 Mgas, 1424 ms, 40.17 KiB)
2019-12-18 08:19:34  main INFO parity_ethereum::run  Finishing work, please wait...
2019-12-18 08:19:34  main TRACE shutdown  Shutting down Client Service
2019-12-18 08:19:34  main TRACE shutdown  ClientService shut down
2019-12-18 08:19:34  main TRACE shutdown  [IoService] Closing...
2019-12-18 08:19:34   TRACE shutdown  [IoWorker] Closing...
2019-12-18 08:45:44  main INFO parity  starting in daemon mode
2019-12-18 08:45:44  main INFO parity_ethereum::run  Starting Parity-Ethereum/v2.6.6-beta-5162bc2-20191205/x86_64-linux-gnu/rustc1.39.0
2019-12-18 08:45:44  main INFO parity_ethereum::run  Keys path /home/foo/.local/share/io.parity.ethereum/keys/ethereum
2019-12-18 08:45:44  main INFO parity_ethereum::run  DB path /home/foo/.local/share/io.parity.ethereum/chains/ethereum/db/906a34e69aec8c0d
2019-12-18 08:45:44  main INFO parity_ethereum::run  State DB configuration: fast
2019-12-18 08:45:44  main INFO parity_ethereum::run  Operating mode: active
2019-12-18 08:45:44  main INFO ethcore_service::service  Configured for Ethereum using Ethash engine
2019-12-18 08:45:47  tokio-runtime-worker-0 INFO miner  Updated conversion rate to Ξ1 = US$121.90 (39064020 wei/gas)
2019-12-18 08:45:51  IO Worker #2 INFO network  Public node URL: enode://censored@myip:30303
2019-12-18 08:46:01  IO Worker #3 INFO import  Syncing #9123828 0xe177…bcbd     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #9123828    3/12 peers      1 MiB chain   56 MiB db  0 bytes queue    1 MiB sync  RPC:  0 conn,    0 req/s,   25 µs
2019-12-18 08:46:06  IO Worker #1 INFO import  Syncing #9123828 0xe177…bcbd     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #9123828    2/12 peers      2 MiB chain   56 MiB db  0 bytes queue    1 MiB sync  RPC:  0 conn,    0 req/s,   25 µs
2019-12-18 08:46:11  IO Worker #0 INFO import  Syncing #9123828 0xe177…bcbd     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #9123828    4/12 peers      7 MiB chain   56 MiB db  0 bytes queue    1 MiB sync  RPC:  0 conn,    0 req/s,   32 µs
2019-12-18 08:46:16  IO Worker #3 INFO import  Syncing #9123828 0xe177…bcbd     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #9123828    5/12 peers     10 MiB chain   56 MiB db  0 bytes queue    1 MiB sync  RPC:  0 conn,    0 req/s,   32 µs
2019-12-18 08:46:21  IO Worker #1 INFO import  Syncing #9123829 0x77a2…bb1d     0.20 blk/s   36.4 tx/s    2.0 Mgas/s    126+    1 Qed  #9123957    4/12 peers     10 MiB chain   58 MiB db   14 MiB queue    1 MiB sync  RPC:  0 conn,    0 req/s,   32 µs
2019-12-18 08:46:26  IO Worker #2 INFO import  Syncing #9123833 0x6728…90c3     0.60 blk/s  111.4 tx/s    6.0 Mgas/s    123+    1 Qed  #9123957    4/12 peers     10 MiB chain   60 MiB db   14 MiB queue    1 MiB sync  RPC:  0 conn,    0 req/s,   32 µs
[..]

c0deright on 18 Dec 2019

My archive node got stuck syncing the same block on v2.6.6 which is a problem I haven't had for many versions.

mewwts on 18 Dec 2019

~~Exactly after #9123828? Strange coincidence.~~

c0deright on 18 Dec 2019

Sorry for being unclear. Here's the events:

2019-12-16 03:11:40 UTC Imported #9113540 0x6e58…6f11 (81 txs, 9.94 Mgas, 1814 ms, 12.80 KiB)
2019-12-16 03:11:56 UTC Imported #9113541 0x03b4…b2cc (131 txs, 9.94 Mgas, 1388 ms, 23.50 KiB)
2019-12-16 03:12:01 UTC Imported #9113542 0x7a20…007e (87 txs, 9.93 Mgas, 1105 ms, 15.91 KiB)
2019-12-17 15:35:12 UTC Finishing work, please wait...
Loading config file from /root/.local/share/io.parity.ethereum/config.toml
2019-12-17 15:35:25 UTC Starting Parity-Ethereum/v2.6.6-beta-5162bc2-20191205/x86_64-linux-gnu/rustc1.39.0

Note how it did nothing between 3AM and 3PM. It then continued to stall on the same block

2019-12-17 15:35:39 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed        #0    1/25 peers     90 KiB chain    7 KiB db  0 bytes queue    6 KiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:35:44 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed      #131    2/25 peers     90 KiB chain    7 KiB db  0 bytes queue    1 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:35:49 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed      #763    2/25 peers     90 KiB chain    7 KiB db  0 bytes queue    1 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:35:54 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed     #2288    3/25 peers     90 KiB chain    7 KiB db  0 bytes queue  708 KiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:35:59 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed     #3048    4/25 peers     90 KiB chain    7 KiB db  0 bytes queue  652 KiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:36:04 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed     #3175    5/25 peers     90 KiB chain    7 KiB db  0 bytes queue    3 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:36:09 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed     #3175    5/25 peers     90 KiB chain    7 KiB db  0 bytes queue    6 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:36:14 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed     #3177    5/25 peers     90 KiB chain    7 KiB db  0 bytes queue   10 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:36:19 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed     #6360    6/25 peers    277 KiB chain    7 KiB db  0 bytes queue   10 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:36:24 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed    #11938    5/25 peers    356 KiB chain    7 KiB db  0 bytes queue    5 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:36:29 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed    #13081    5/25 peers    623 KiB chain    7 KiB db  0 bytes queue    9 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:36:34 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed    #13081    5/25 peers    623 KiB chain    7 KiB db  0 bytes queue   12 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-17 15:36:39 UTC Syncing #9113542 0x7a20…007e     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed    #13081    5/25 peers    633 KiB chain    7 KiB db  0 bytes queue   20 MiB sync  RPC:  0 conn,    0 req/s,    0 µs

untill I manually restarted it and then it would sync.

mewwts on 18 Dec 2019

Ok, not the same block number but likely the same issue.

c0deright on 18 Dec 2019

👍1

@c0deright @dvdplm I actually just had a lockup last night. I have just been testing different config changes to see what is causing it. So far no luck. Last block before lock was 9123621.

2019-12-18 03:07:35 UTC Verifier #14 INFO import  Imported #9123617 0x1f1c…e668 (265 txs, 9.99 Mgas, 487 ms, 43.54 KiB)
2019-12-18 03:07:44 UTC IO Worker #2 INFO import    74/75 peers      5 MiB chain  416 MiB db  0 bytes queue   20 MiB sync  RPC:  0 conn,   12 req/s, 7956 µs
2019-12-18 03:07:56 UTC Verifier #26 INFO import  Imported #9123618 0x3e05…dd81 (156 txs, 9.22 Mgas, 305 ms, 32.65 KiB)
2019-12-18 03:08:14 UTC Verifier #2 INFO import  Imported #9123619 0x5e58…3870 (175 txs, 9.99 Mgas, 874 ms, 30.30 KiB)
2019-12-18 03:08:19 UTC IO Worker #0 INFO import    74/75 peers      6 MiB chain  416 MiB db  0 bytes queue   20 MiB sync  RPC:  0 conn,   15 req/s, 7623 µs
2019-12-18 03:08:25 UTC Verifier #31 INFO import  Imported #9123621 0x837a…4394 (142 txs, 9.99 Mgas, 421 ms, 45.20 KiB) + another 1 block(s) containing 224 tx(s)
2019-12-18 05:16:06 UTC Starting Parity-Ethereum/v2.6.6-beta-5162bc2-20191205/x86_64-linux-gnu/rustc1.39.0
2019-12-18 05:16:06 UTC Keys path /pools/ethpool/parityeth/keys/ethereum
2019-12-18 05:16:06 UTC DB path /pools/ethpool/parityeth/chains/ethereum/db/906a34e69aec8c0d
2019-12-18 05:16:06 UTC State DB configuration: fast
2019-12-18 05:16:06 UTC Operating mode: active
2019-12-18 05:16:07 UTC Configured for Ethereum using Ethash engine
2019-12-18 05:16:09 UTC Public node URL: enode://93505aed3f62a6d7c7963e7887b330c54a7184ee829cca10e74125b052db857a878e00ef9ef91cdeb9fa169f262d25beba487ee373ab87e8f3b0ac1af0772683@45.43.30.3:38001
2019-12-18 05:16:09 UTC Removed existing file '/pools/ethpool/parityeth/jsonrpc.ipc'.
2019-12-18 05:16:09 UTC Updated conversion rate to Ξ1 = US$123.72 (38489370 wei/gas)
2019-12-18 05:16:24 UTC Syncing #9123621 0x837a…4394     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #9123621   15/75 peers    389 KiB chain  241 MiB db  0 bytes queue    1 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
2019-12-18 05:16:29 UTC Syncing #9123629 0xc0c3…d54f     1.60 blk/s  287.8 tx/s   13.9 Mgas/s      0+  455 Qed  #9124090   17/75 peers    684 KiB chain  242 MiB db   58 MiB queue  949 KiB sync  RPC:  0 conn,    0 req/s,    0 µs

Here is my current config during the lockup:

[parity]
chain = "mainnet"
base_path = "/pools/ethpool/parityeth"
identity = "mydomain.org"

[network]
port = 38001
allow_ips = "public"
nat = "extip:xx.xx.xx.x"
min_peers = 75
max_peers = 100

[rpc]
port = 18001
interface = "local"
apis = ["eth","net","parity"]
#server_threads = 12
#processing_threads = 12

[websockets]
disable = true

[footprint]
pruning = "fast"
pruning_memory = 256
#cache_size_db = 2048
#cache_size = 8192

[misc]
log_file = "/pools/ethpool/logs/parity.log"
color = false
#logging = "jsonrpc=trace"

[account]
unlock = ["0xXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"]
password = ["/pools/ethpool/passeth"]

[ui]
disable = true

[mining]
author = "0xXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

Serpent6877 on 18 Dec 2019

Parity 2.2.6-beta:

2019-12-31 05:18:55 UTC Imported #9189355 0xfa1d…03ea (97 txs, 9.91 Mgas, 750 ms, 30.95 KiB)
2019-12-31 05:19:07 UTC   65/75 peers      3 MiB chain  366 MiB db  0 bytes queue   24 MiB sync  RPC:  0 conn,    3 req/s,  962 µs
2019-12-31 05:19:12 UTC Stage 1 block verification failed for 0xa5f0…42e0: Block(InvalidTransactionsRoot(Mismatch { expected: 0xa5758f07fad869b84c6ac62e4a6a502c7fec25667447635c3ebc26919acc8c06, found: 0x7da15537f4171cf88f24b0afdd385b348e21054ad4464f05c6551c9be13ec8f8 }))
2019-12-31 05:19:12 UTC 
Bad block detected: InvalidTransactionsRoot(Mismatch { expected: 0xa5758f07fad869b84c6ac62e4a6a502c7fec25667447635c3ebc26919acc8c06, found: 0x7da15537f4171cf88f24b0afdd385b348e21054ad4464f05c6551c9be13ec8f8 })
RLP: f94949f90215a09fd8152ef85b9899d3cce0d8f340bbe0ed2732b2dd1a281a6d5fdcdae9b7bd57a01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d4934794829bd824b016326a401d083b33d092293333a830a019dced45482c827958e490854cb491799548a5629a09786e85bb23537a087f75a07da15537f4171cf88f24b0afdd385b348e21054ad4464f05c6551c9be13ec8f8a0cabf2f15c98fdc794e01f1b4dc03d635f5f726e42a18a02d702294cf429f7512b90100484088821203380888d00a095470033440810390544a92d080c048100a20d35604526021250154c0081010000a22415b32c00600810006a90a8a408468a0066050900200088b8513bc8804c88680820c3452c502420508100062026004061018004048b009884c0d0401600c610100120020121000080814008007383181150454042102020420648208000040000701404028405180800a00102022a07141125f040500240531807881408040b0d02400404400c0800140000000021a00384392a00042c5a201c8043278000400a860044052000041802c80303550510980000212402031101080006041800000208108ce08818103a01310e41054285000608708e7c19838e221838c37ed8397abd7834fa3e2845e0ada3a947070796520e4b883e5bda9e7a59ee4bb99e9b1bca0f8996c571db7fd53f823f07ec5dcc06fbbfcd08d496811b6e036080080e77169886f6b53e4000bd51ff9472df870830421ac850f224d4a008301637894d6ac90c8cca2ae8b002926c875334a0011849bcb88013ce6e419c770008025a0c5a7ae6ce0f4802f3c17c74e073d2b57402e47484b19f49a7bd561c076d91adaa029a901f355dc44e78af1d62807ca359067ddac5449bfefde14c670644a15b883f8ad830421ad850f224d4a008301637894dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb0000000000000000000000009815ccd7b770052b51631b4dc8427b8c0f10435a0000000000000000000000000000000000000000000000000000000008fb118b25a055115f539add2b5da7905948086bbffc682c055641b83e5b65f1bead4cdd2699a050eaa394be34f40ea9ccedbc93aca5eee2a6ebf07f91e0c5d13819502ba54ba8f86c0285098bca5a008252089488d34944cf554e9cccf4a24292d891f620e9c94f88528c63dafa1ff0008026a04b753c04432cc055e44b100af04441b7e1ab5a79541e157954757636e58453c2a00452226ae9690084e4f4b5a468a30993cc0d83036bb1ec53b1b72ca399da3f6bf86c0985098bca5a008252089488d34944cf554e9cccf4a24292d891f620e9c94f885e5b64e5a98130008025a09d0b40366e4526f62b7775246ccfcbf841fb44cd0531c07605c90169d1cd5c87a07e3dffebf4f17f99e472c0a1019ec0bf1272c4f52c3e1919792e4f2ebcf7a6bcf8aa4e8509502f9000830249f094794a8d6966ee087da2dee792bbcf70039c568f1a80b844a9059cbb000000000000000000000000c94bc6a1d7ba62466aa82aaac1ecb468fa4604850000000000000000000000000000000000000000000000000000000070401d501ba07f5e15aa333b5b6e00b7c4d6f0006b397355fbecc857c8b680a82083c4de417da059afd7af5c40cd071d11cc48d877aafec2190a6b1185484daff0f8d2c9a36304f8ac8301ea048509502f900082ea60944ecdb6385f3db3847f9c4a9bf3f9917bb27a545280b844a9059cbb0000000000000000000000005db32ba298f365cc45c498858c95d4582f1fa4bb0000000000000000000000000000000000000000000000000000003b5f2f360026a07798876fca2b94d84948fb34dfc086b1d99f257fd0110fd6af173976d3a43002a01504d79fc9c3629e8a2fd91d29e6e498a81975075c52303e68a8ec6a9bdc20c4f86f829e0c8509502f900083015f909497976ceaabc35134bc411dcc64bfe1792379d755882914085137280000801ba0ab6fe63e82f747351021d92d22f42409f3130e49bc2b31299757435395aa4684a01be5ac51c7760c6b6cddda38377c6df8caccd4a623b1ada8f13966963dfc49d6f8a908850861c4680082f618945b8174e20996ec743f01d3b55a35dd376429c59680b844a9059cbb00000000000000000000000078a7a105cb4170b083601686abaf8ae1481b6004000000000000000000000000000000000000000000000010a47b1971bcec00001ba0f781bf9da80df57ea5ab6009d463fb73ab7df5687753fc2641fd5882c83f4716a04fa054b08ffc1eb789dd9969a831c4e3c85e3e8e3e8e000aa86d1111f0750b72f8ad83033d2e850826299e00830222e094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000c1a49a7ec34374a750b3aa04e625badac8d201d3000000000000000000000000000000000000000000000000000000000090f56026a0b374711536105d7f3c486b1e9ac63db59ea51db65bc017f9aadad4835c7269bda0123231e6811b1d173b6973bc761dbf4d43143192c09f93f4f8485e24604cda6ef8a909850826299e0082fde89468e54af74b22acaccffa04ccaad13be16ed14eac80b844a9059cbb0000000000000000000000008babf0ba311aab914c00e8fda7e8558a8b66de5d0000000000000000000000000000000000000000000000000000000a65f1850025a015cb09494cc16dd669b62dbaae9f86378e264539094854eadc8122fbcddeffa4a03da0cee2239e65aea5d520f996f047b629516b12a3bd4a0274e6adce2182a08bf8a910850826299e0082fde89468e54af74b22acaccffa04ccaad13be16ed14eac80b844a9059cbb0000000000000000000000008babf0ba311aab914c00e8fda7e8558a8b66de5d0000000000000000000000000000000000000000000000000000005d21dba00025a0cd9f635657d9c4d4fa64868b0e53a8529bffc0154754719fb47e2754142800d7a04b7580c91ecacf43b1e1d42d0916d97a9ffeb322d84d4fee8045da2bcb342029f8ad8306f57d850826299e00830222e094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb0000000000000000000000000af96d163186d4cf5cdb80c876c562e4af1c3fef000000000000000000000000000000000000000000000000000000006712676526a007568fa55b66fb51c150239daea8341c647e5a8e72a4af126c63fb09dbd4bbffa00ee539e90df8b5b511e26c19a70c21179cd17be537b99de9782afd9654a0edf7f8ad8306f57e850826299e00830222e094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb00000000000000000000000033aabbada1175ba56302b92ce7de5a8ce12c11bc0000000000000000000000000000000000000000000000000000000177e3680b26a08197587048dfe4fd95d953cbacb17dafb8515e1ff87ce2d2b6cb81633923baf5a04cca7e0492c6d7e3e758ca734f53520c8bb8a268f312e07289b01c25b116efc0f86e8309e7c385012a05f200825208943c4e3bace461a1081a51e3ea0f2685738cc8aea4870221b262dd80008025a090ea6808b0d4854f2e8f512fdd05af6d9c71878f44d890868b0a6305d433b3b2a046ec8d411cf4fee42249ebd9a4393792f8e22946937924e5417fd4a6cf66f69af8ac8303bdc8850826299e0082fde89468e54af74b22acaccffa04ccaad13be16ed14eac80b844a9059cbb000000000000000000000000f669a34c182a47cfe15b79560750fb9be9eb3d750000000000000000000000000000000000000000000000000000000147d357001ba07058c7360c136c76e869281051c513e5f02bd459729706752e3f7c0566426300a0055f406438f3d7d33cc76775602ad42e70a83a9e7bd719c38e2819d0022825f3f8ac8303bdc9850826299e0082fde89468e54af74b22acaccffa04ccaad13be16ed14eac80b844a9059cbb00000000000000000000000083779c0c31cacdd5ad191096631e2c28258d6ce800000000000000000000000000000000000000000000000000000001004ccb001ca036145e4dbea34e48cf12ac3c8b868103eb939da80e9bb027abcd8d6751d82c3aa0362ab1fe949d7de60a78ede1e844d0a6ada041756c58b9ac2e93e815aa384056f8ac8303bdca850826299e0082fde89468e54af74b22acaccffa04ccaad13be16ed14eac80b844a9059cbb000000000000000000000000e81c5eda4b6920c84e9cdfacf41a0e590e83631b00000000000000000000000000000000000000000000000000000000f5cf3f001ba040e004e7588df9fc5f300ef6db20fc345c34a933b4a434810cf2729f86cf663da035a666f39d232d7eab823c9d67a55bb6b95452d9629957bea2fa597e63c203b1f8ad8306c3c3850826299e00830222e094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000e92c07073990860417a56a0b009925c3d76635730000000000000000000000000000000000000000000000000000000055f3322325a0571430055706ea89bdf22ab13aa835db20c92d1aee9eed8efcd4cfa0905571a8a04b54ea6c6deca8ef95342204c0effdbdac56b97dfc9c44824920d952ebd57b7af8a906850826299e0082fde89468e54af74b22acaccffa04ccaad13be16ed14eac80b844a9059cbb0000000000000000000000007b4dc72249eaaafe09dafb30b492e693af20f38400000000000000000000000000000000000000000000000000000001dcd6500025a0c657d63b4d90ddcfd76488dbbb9b2c8c177acfba1faf7d47bf52809b3db5259fa05c0f4d9e3115e07c243d0cd8695fbf3f8e3d71ec157b38d82229cead868ffe5bf8aa80850826299e0083015f9094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000f17fa7b7439226354bb4a469d585f087fd11b2820000000000000000000000000000000000000000000000000000000011e1a30025a071468bb618fc7c1c919eddad3e26bf5cf50e0b98193f0b418c4afb5a6dcd1f6aa032f785471304aa5761cce289ca355155a54f4f1ec3d3d5001cddf64cea03b56cf8ad8304fee5850826299e00830222e094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000077c15634402077b6193b49f14ad6359d553d7e400000000000000000000000000000000000000000000000000000004ddad6ec026a079f76ad0949d6fd71b6f50a7c85a525d225714d0c2c08d77cb41edcf62e4ca54a00befe9a4a72ececf76524b9247cafe22defd3157b31321699519c4ba3324ac88f8aa018507ea8ed400830186a094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000efd52349ee3192c9c94de236b938039114e6fa5500000000000000000000000000000000000000000000000000000000042c1d8025a04cb3705eddb5ca8d59f30d8046a8a87d8db9aa5b27222a9b97a81056de567e29a0740d4f66db80b6356234ce02b31321778695dbba0d649c63afd04a8c9ca7da4df8ab81b98506fc23ac0083011940944fe327c5a809fa721d47b80c5038a0b393e6130580b844a9059cbb00000000000000000000000064756d7fcc0371797fe7b4cfef10a3a6669c64f3000000000000000000000000000000000000000000000000000000036044710025a009956d6b480cb70255b3332ebecf3ce0ecefa57a6e5df04eb95a9a0ba48d371ca00b391ac1dcc4243efa8eaaac7cbbb8431677bb98a979a269ff7d3b9505c69d18f86b088506fc23ac008252089468da78fcb1192ff3553dcb782a5f4216038752cc8705c280063fcc00801ba088524c7f769b435496e20a9e76a44b38819610b5d75aeadae20b328f2c0704aba0248048fd0200e180e88e00ee93ca2872a3e2750971f29d8f11544e525e443843f8ad8301062e8506fc23ac00830186a094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb0000000000000000000000008a9c3060ad2c63428e04faf367438b3b4aededad000000000000000000000000000000000000000000000000000000001a831c0026a05eaf4f45e472b251fdbfd24cbf1d9112f043f2878868753e32352f23d068fcfaa0134aa3ea94a68ac35eb66e22116a17c6698911aed9e322ce37d1a9b8bfdec735f8ab8227738506fc23ac0082ea6094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000fc7feafbb5ef449264ae36d139534898efa95910000000000000000000000000000000000000000000000000000000000afbea6826a09204fa1c46f5aabdb5d40b24de2498aa3c75863cd76ade59118b417a391ee709a02947f04f8ca3f5ad248640725089cd57f78a8610775db2ac495530c38af00fc7f8ac8302c3be8506fc23ac0082ef0d94dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb0000000000000000000000000528711e2c1b00ad67609c4b193aebad02c3045a00000000000000000000000000000000000000000000000000000000121a821425a0af54843c4a44b8a046fc49e5c0d89dd1f1c777d8e5b318fdb03b6b2a3fcdf2b8a00c171108a347f059cce6f8f221e36358ca468d6f96bb34132f86518cb2e325d2f8ad8302c3bf8506fc23ac0083011e43943c7b464376db7c9927930cf50eefdea2eff3a66a80b844a9059cbb000000000000000000000000df88f6b5590967d5d0ad30bfcc31aa343fce14b4000000000000000000000000000000000000000000000000000000049db5975326a00e14e593707f034e18777e5c46ab5819b9ddfbb0c7e9645dbd96fcbb66993a07a0149a3978cd6b9ccf30c22359d05ecfcffcb45e7b4b363d4afc757ece3d4eec86f8ac8302c3c08506fc23ac0082ef1994dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb0000000000000000000000002458167092dac3dbe44af56c7e38443fcfaa7d85000000000000000000000000000000000000000000000000000000000f13ef8025a0af69a8cc2718efcc906a3e1e16daf82425dc56640606139875645be5cfee1ecda0730bbf6c5670e960e3a5c79e26738b8f012d91bd10629e45a2b3faa232f7a8e4f8ad8302c3c18506fc23ac0083011e43943c7b464376db7c9927930cf50eefdea2eff3a66a80b844a9059cbb000000000000000000000000987e31ad1d826dd46e6e428e49bdd4d81e6d8c4d000000000000000000000000000000000000000000000000000000054771ca4c26a0eb814a5c0d361d665e344d7251cc43ae7f92e2fda84f49d44d2812c5cec3de12a07ea5bbb7bc36ef2700bc58830c16cab42ce277054b2e5f6895b299fdace701e4f8ad8308664a8506fc23ac008301388094d7cc16500d0b0ac3d0ba156a584865a43b0b005080b844a9059cbb0000000000000000000000004bf218f218dc427cf7c0d6f06bf2cf4653a1d8a80000000000000000000000000000000000000000000000000000001343bf774026a0c2c4efe7d5da9c7ab22f1fce626e778a5c037aaf9e8fb2bc65fdb52080a958a4a058cb81ed4e72d28616f12db6c3785fe79d546fb024e8af3c2412b942799b1308f8ac8308664b8506fc23ac008301388094d7cc16500d0b0ac3d0ba156a584865a43b0b005080b844a9059cbb0000000000000000000000004bf218f218dc427cf7c0d6f06bf2cf4653a1d8a800000000000000000000000000000000000000000000000000000012739bc640269f5ee36a782186685f030dd9f2b852b168cc06ab20e5e465b66703ef074b1753a052e9417917aff05216b1bcc6e4f9dbe26a300f76cb3c75f468bdab5e31db7ae8f86d82f1238505d21dba0082c35094f772fb803c421d586cd23d72b45fe45d66aa75fe876f9d8a5ca13800801ba0b2fc9d9fb072a79749603223dd9a366e63fd0afc60508528a4103b2c46ea02fba047cf1d6d96d4e1595729f91e65ce5f27f6272087da4a098e1fc80e4bfbb31033f8708306114c85059682f0008301388094c581426ecc3e5cdf7ef74fbd65f5bf63fe81846d886982ac03098e04008026a0add9a617efddd469680f0299adbf2b22e7c1d15a43f79b65f1b25a88dade01cba053960ce78d25205d8357765919fb98f11b74f3cfefde793c25262d2ec0bc454ef8ab820dfb85051f4d5c0082ea6094e183c454ab04205415dc42e1bb48313a912d148e80b844a9059cbb00000000000000000000000099199f6182efe1254fe6514b79fad90adda30aa10000000000000000000000000000000000000000000001833909cec3973c00001ca0ca6f7862bb970ab9b3e1e59ceca8c4a884c3204324c6a78b6192f8df1ef09b89a0018cccd3862dd53d14eb3364630a96ed17684efc8eaf94dfa6671dcc7b8897d4f8ad8382f8be85051f4d5c00830249f094c65e61626bb0cb58d31f0212cd4a0dab4d64743d80b8446ea056a900000000000000000000000093065b5c7eb63333b8e57a73012d25f68789578500000000000000000000000000000000000000000005760ee3c5062c3add000026a0de0bc147924299d95c1a5a18c92c4163cf71dc21abc611b3e1631f9e34d4db7ba00a04d2effbb693ed9bdeb53a88c3f731c6fb19504b138f076bcc85f35d3cb1ccf88b822fc88504a817c8008323dbb094b1e867f3a3bfe5c3417c792f3fe12ff683ca1a3380a4c01a8c8400000000000000000000000000000000000000000000000000000000000030081ba0a5adb0919624c55cd7e3344eb1007770059c823ab931f6584ac660b8bf8a3f2ea0265fafe470caddfdfc78f9a563f0a43c01133296780291acb8de86d4b6882a74f86d8201468504a817c800825208944c65d791917f9da00b0780019ab269ce52c9f1e987016fd15e70a8008026a052351a3ec9ef67dcdd54e55fdf79cf212cb63b90143dc01cc1019358cf3ba491a00fea29fd5b957c5aa4441b0963ed99f389d8aaa6ffd4e376b5144cd0eac029ccf88b823c258504a817c8008323dbb09414884a04608413a0837ed4d9d1fa0a33eb45640080a4c01a8c840000000000000000000000000000000000000000000000000000000000003cbc1ba09d1cbdb51b7fc509bca57bbdaa0840dd253a27d31636b41f54ba6dbe625543d2a06ebd86721bf875af0335f6cda1a954461054885fba5be2fbd1c420279d93bd47f8aa808504a817c800830186a0941e701cbb418017b22acfc1dd64aad8b0ee0f075f80b844a9059cbb000000000000000000000000bfd8af6d5b215aeb89b235d05890dbc9e9cabf94000000000000000000000000000000000000000000000000000000cf2ccf188026a011f7543200e14921f81162569c5c94437c1b2dad0377012b6bc136611a930302a02f86fb1124ce2a3a7c540a33dcd10a71e2a60a8f11b232b9a967387c7a5d0c03f8aa088504a817c800830186a094b3dd5dce850dca7519e74a943568b69f958df52c80b844a9059cbb0000000000000000000000001e5ae30fd2ee4678e42c798f4ec8d79d452f356300000000000000000000000000000000000000000000003ceea22ccaf310000026a05e0ec8b967ec9458046692f008895cff46483d20d9b5fefd310eee1b0a4cd9d5a07a0754a5e2ea707e3510433828f7aa5f98ec0c620ce6c21335130b4954e51a43f8ab81938504a817c80083015f9094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000e7be476dd358dd4079f712081580ed6fbe1008f2000000000000000000000000000000000000000000000000000000000243d58025a045f5d9c0e1a73c1a6eb004d15ebe2a2a24461615665cceaae2aeedb06e61becba068fca1bde7c5f185dd7b6ef5ad9cfd7b923e7f78d996c88948deb6719d17c056f8ac8211628504a817c80083015f9094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb0000000000000000000000001ae124ee9340cb3f64f4976c8e6b690489220776000000000000000000000000000000000000000000000000000000002e344ee025a0093ed6c4ca613085b28827e9a06565096efa5f1cf81ee7216e1f16a6c9d8bbf4a072448128b0868631e4707b3f25f5e57181f4640307ac5ebfb0e4485ce73e28d2f86e82dc7e8504a817c80083015f9094ec95ac0707626db2d80c207e7d64645632f5df11870aa87bee5380008025a0142b1d17d7133d09a9ba8f29b53280ad9d1453608a2a55d2aea0a233571e7485a0561bef2b2468896ed3ff02ead1b35665d12a619f3d0cdb117e7113f278fb5e6ff86e82dc7f8504a817c80083015f909489fe3170fe95694902b304dd7613679d43122763870aa87bee5380008025a094425ba586e8adad38903401d850bc336041c88c0ddacc96f38131a2e4266ed3a06045e01ad04ae883151f693a0f54ba30d2b29ca96770b2d62b02b446243e5140f86e82dc808504a817c80083015f90944e7b82037ea30e8e30c28411a3dfb4f8f88d5779870aa87bee5380008025a0a487604d6beab5f566f5a1bb927860225a0282a1926d01e98ba9a1124c1a490ea0353c752dd471b1454ea11a786a5bb3934496e03e37ce08ec3fd4dfa0a13ec2d9f86e82dc818504a817c80083015f9094f259cf3493e42aa070cd60174c0816a55c25d1a7870aa87bee5380008026a0a31c839cfda37ae25ae48676fbfb81e3ef3db65c4bc1df84234794839641957ca063a41d3dbb90a263ae728fbf73e2ccae47fd94cdc46ca850e8319e9032afd2aaf86e82dc828504a817c80083015f909460d19b069eba263f4aadde1df02544a52f019a3b870aa87bee5380008026a0c7e9aba4a1ebbd8be815af0b99f2eb33f48016bd5121ac49a53c4ca4c3312ef9a034fa5233cd3c9e4d6913f44a6bde5e7ef85609d1a134d9a59b212cf8324e1905f86e82dc838504a817c80083015f90947ca8b22c514a9b355058ec216f209ca2e087789c870aa87bee5380008026a087cdd1341aeb56850f09f4e8474a8d5d40cc99a1a3ae736e2f5573847b265eaca07c32fd2b2a14c0902c0cc0d4dcf33d02a6b27e05e5c9a546299dd0acb53fd0f8f8aa018504a817c80083014dd2940000000000085d4780b73119b644ae5ecd22b37680b844a9059cbb0000000000000000000000003f5ce5fbfe3e9af3971dd833d26ba9b5c936f0be00000000000000000000000000000000000000000000011478b7c30abc30000025a08b63be77e4986e3227ce6c4e115a5c01dbe86dde8b73d6de4ce76b1058103289a05d5ecdc9ee097d25cb456cfb822074670db79f8e6c1c6c22c49271bd3dea5414f8ac823c148504a817c800830105b894dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb0000000000000000000000000b0bcc694a541da9cbe474ccf90528c25f1b769a000000000000000000000000000000000000000000000000000000047783de401ca0fc1d4af6d1f78fbc4be9c41682d115cd0de6e0319bff005976f0e9468bba5d65a050b0fb41832b2d5a10bc5f46607bc12e70fb5405ff519109453ede5cd37fac7df9016d823ce98504a817c8008323dbb09414884a04608413a0837ed4d9d1fa0a33eb45640080b901040d59b564000000000000000000000000dac17f958d2ee523a2206206994597c13d831ec7000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000800000000000000000000000000000000000000000000000000000016f5a64bdc20000000000000000000000000000000000000000000000000000000000000044a9059cbb0000000000000000000000005c143d1a3062bb62a72e14566be36616a08d4829000000000000000000000000000000000000000000000000000000008e83c3c0000000000000000000000000000000000000000000000000000000001ba073d633b682f756b166620f5dfad2d620118d1c174c8c67d0bb2675bcfd83f706a06a4d29fbbae4664fbb44edcadb5ffcb1e0fb39c7f9fc77d9bc90d5924eec9a6cf8ac82054b8504a817c800830186a094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000bfd8af6d5b215aeb89b235d05890dbc9e9cabf9400000000000000000000000000000000000000000000000000000000b4f5d21525a02f7c218db11d6ad4d57e17ac9bcd04e4c694fc623d177d8e662fcff264919906a06f27ebd4c4e2d917e92bf6be103df2c59435bfb141583135b16a6f0c253cbc49f8aa821dae8504a817c80082ea6094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb0000000000000000000000003f6c1b3224ce07237f609cf3ccd246808c2dd7a80000000000000000000000000000000000000000000000000000000006e4eeee1ba0f6158ed4be1ab10f028d47cdc41baabc791d93b29d7c28e8876730554645f8409ffd4471f58fe5452d8b31b346181781149db22c9817e05a3cbbb44de3c3daf9f890808501dcd650008303f14594bcf935d206ca32929e1b887a07ed240f0d8ccd22876a94d74f430000a48853b53e00000000000000000000000000000000000000000000000000000000000089a425a06f793e58a01cc1b2469bcf75453098a53969eb5b8959b0c4a87fac13d41ea9dca055b05174f8451e6316d3a4a9051307d0d633b5f624538999b8cea8627acebe2cf86d82b1cb8504a817c80082520894b49a897c934b0bef60ef6d0080bfd7a1ffacc0478702ed965c2188008026a027685d9d736543635d136c0699eda0afc142dbd638fe1446265d3edbf9178f44a02741b6d3f07732ec737466c9e35cf4618357cc627e827fe7abc1eb6f5ee2cfbaf9014c8205f8850141dd760083035b6094069c66319b4faa64f48612d59e5dfc4986814fa980b8e4c26ea0dd000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000821ab0d44149800000000000000000000000000005c8ff149b68949a7aa05fe44b48f81fe819b05e1000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000403234323065626661303532383063613366333065393562366430333938306265333135653064633335613366306561613665643834343137663331353835333626a062ba9189ff3b94b2b51db4674e2c7efd6ad236d2d08b8072fa67a115cc7889d7a03f3262db9ff7ec665ddfb85b4f927547e8e446451a84ebc31bcec13ca0603a26f8aa808504a817c8008301521894a0b86991c6218b36c1d19d4a2e9eb0ce3606eb4880b844a9059cbb0000000000000000000000003f5ce5fbfe3e9af3971dd833d26ba9b5c936f0be000000000000000000000000000000000000000000000000000000033ab7678026a03ed3bdc81a8ff2573978769fa24d83225d9ee1c3b1bd2c0d51ca95499ec4890da044f9c20ec6a8c04afbb7f6e000b03a20453248c4a81bbabb96ac54f032d05f30f8aa81da850430e2340082fde89472955ecff76e48f2c8abcce11d54e5734d6f365780b844a9059cbb0000000000000000000000003e9afaa4a062a49d64b8ab057b3cb51892e17ecb0000000000000000000000000000000000000000000000a10107dc90aa1ad8001ba0de2d12abe8902c12000450b58f15149317ac17c05ff40d6f9b5d78de55db81faa0763990ee48a9006b979828c595083b2f63dde6b8bb3b5c6f4d3ca76ab4f935abf8aa08850430e234008302166094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000b8001c3ec9aa1985f6c747e25c28324e4a361ec10000000000000000000000000000000000000000000000000000000148ea8f601ba0189b8481280ae32cdc18b9ab61c9b9c2d4a281fff6cf67de20e85c5b276a434aa02ee727fa2d65ba533d3e855391e2f810a65e431aa3e8a45e9d1d43b2b67901e2f8aa80850430e234008302166094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000b8001c3ec9aa1985f6c747e25c28324e4a361ec1000000000000000000000000000000000000000000000000000000004c6be7601ca0267404ea0953c037c6268377b8bbd8e5ee4e40ded88714f662d0dc9b0e3ec383a041299add2efaae7db7edee1fcbec78b362ee955a1cac1e415ea00289c1a7763bf8aa0c850430e234008302166094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000b8001c3ec9aa1985f6c747e25c28324e4a361ec100000000000000000000000000000000000000000000000000000000c95830801ba029a55301865cc88a69a7165668a3342872175631dfc047e5aafb2a7a7909d0b6a01c8c84637811f9d06fd450ad3a6aa84c93cbfc713fdac094942c33513652ec79f8aa21850421fb8180830186a094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb0000000000000000000000000e58a7652143a0a275750e5ac57a4fb14eb4a18d0000000000000000000000000000000000000000000000000000000023b738101ca025a268f14e7d9ac04f16cb7dc87c695ec155edff69cbe3daae672de279233df0a073f3d1b47f1e884cb4360dfa43bf8df81fa947d8e9093c43b36b13ac40f8e525f8ab8230b58503b9aca00082ea60945b8174e20996ec743f01d3b55a35dd376429c59680b844a9059cbb0000000000000000000000009fa700f5aec333c89ae43413a5eeef426475d0ae000000000000000000000000000000000000000000000012dfccf97b325bc00026a0268859f55eaab33410733784a800adc04a91e69379085b7bc0de2084ee6e8c0ba02621c9674d500602132d4ea8b0628af81cb2491ce03ab8a28e203ff79377940ff8ab8230c68503b9aca00082ea60945b8174e20996ec743f01d3b55a35dd376429c59680b844a9059cbb00000000000000000000000059f7a6b9740a6bdc13da3b83ca35c7f368c243e900000000000000000000000000000000000000000000000358d08ae3309e000026a0feb95626710ecc7db63e864fdf117b60a2858d52f3f901421b3a2e0433782d3da015ffe04a9c30037c782d04bd59994992771fd2a1366e61a8b6494b656e7654acf8ab822e3e8503b9aca00082ea60945b8174e20996ec743f01d3b55a35dd376429c59680b844a9059cbb0000000000000000000000003b2a0a8dda7ecd429798eb96a8f1bb81524523ed000000000000000000000000000000000000000000000012dfd9db4f1c50600026a0080e60fc33879f07fb0fcadc00e92a32cda92426f54e8b9f2ccb9718dc1ffd1ca039f589e3ee94a3d680b218f2e36de46cda326b2ba7ea60d1b1684f8682b87bf2f86d81bc8503b9710536825208946ba533dcc1bac9595bd764f93fcbc7ec81410441880d20611c577cde058026a066ea742882a3fb66001be3d08a6f02999141daa97f1b0010ffb1fb08a9fff14ca03547dd5a2918f3361d9a5ab94bd6a0e585854a8a1bfe029bde2b2e9a1794075df86f82093585037e11d60083015f9094876eabf441b2ee5b5b0554fd502a8e0600950cfa880c1e0e9974463c008026a01193e3ab7bc5712b1bf1718d61afc5496425845dbe80fddfc7f7fcc77c6694afa03b67ae94eb5caa020a630198f63f12aeda2098e4136f5bfbd2bc1a8ecec53155f86e8264778502cb41780082520894bdd8d9f16057abd5324f267bf4cf5987c87a0f6d88016368c231c1a0008025a0a44405b46a7531bf0ff0df058372a161441aa056a0af9570acc3559cdf4e3ddfa07b4ea86f2e378e9e98ac4b180c164dbf6cab5cb48c6ce825a655af6b9a33c4d1f901ed82cedb8502cb41780083017d5094121effb8160f7206444f5a57d13c7a4424a237a480b9018439125215000000000000000000000000f281e85a0b992efa5fda4f52b35685dc5ee67bea00000000000000000000000000000000000000000000000026db992a3b18000000000000000000000000000000000000000000000000000000000000000000c0000000000000000000000000000000000000000000000000000000005e1414b20000000000000000000000000000000000000000000000000000000000002c5b00000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000417c725edd8ae0d0ebb54ee129ce4b186ed2ac82107f5b2ed72804f9b4efbf0a6534e843aab4eb95300f4ab763c26d927283c6fa5abab70c45517d18a6a8b0952c1b000000000000000000000000000000000000000000000000000000000000001ca0348b0d364e90ee37a29ca3c4d70639866ce1c1f81ad6e0cd7bda53bcce04518fa023f176f7b72802a08c811d485af85c9ee18788246a9544881de5d77a0c010be2f8aa228502cb41780083015f9094cadb96858fe496bb6309622f9023ba2defb5d54080b844a9059cbb0000000000000000000000008f20a07e0541ca2db9152d7e521aee5d639b211d0000000000000000000000000000000000000000000000d7f7801d10d0c4c40025a08273e07f106541f0c6d927f8caca8afe762266d932a1b51295244cdbf2956905a0302806618a1dd3010c48a9449a5be63a63d56e4ca0979d2c02753dd7d0be7021f86f83019b888502cb41780083015f909465976a99eecea78f38ab18907e77cbcff81d81518705543df729c0008026a0515a24e1ea21b4be58d7a00cf539ecf46e7d0340f155df3438350a87adfb69f1a05feb74cae3c2f8d1d9b4e7636d25e138cd8bb68a10d4ca21df5dc5a9c9145cc6f9010c8209ab85028fa6ae008316e360940ba45a8b5d5575935b8158a88c631e9f9c95a2e580b8a468c180d50000000000000000000000000000000000000000000000000000000000000060000000000000000000000000000000000000000000000000000000000000003200000000000000000000000000000000000000000000000000000000003f04cf0000000000000000000000000000000000000000000000000000000000000013363239303634383636323737323531373835310000000000000000000000000025a0717b34b14b38631f2fda570773ff1a017e01120557000a64b6dd43a1d7ef48eca0105b65253a18d6d9cf23021d7dfda9e4e87a8158e4b01e3b564ac3e85ff4af1ff8a9808502540be40082ea6094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb00000000000000000000000057a00f0b6607754af27eff5b77632cf3c9f100c700000000000000000000000000000000000000000000000000000000224c178726a06c5fba4a1da836fbc36fe9a587d940d128504335fc726ae96a76acd67b5733aca07b6059a99fb915d9a71000975dc5f475786f86575243058b21a84b9585dd9279f8ac8317326e8502540be40082fde894d1ceeeeee83f8bcf3bedad437202b6154e9f540580b844ca722cdc9e7da25a9c389d64a38907aa432062207d49de3d3cb63497feff608c80d323bcfe4129c5c81c50d7f2293896eb5cde2217d055a0a7f8a9f2675ac6dd3c153b4b1ba0f3edd2aa5578e71e068b508e8fd7b4de4c4b8212deea413fcf2c175c69ece321a05aa88be287c00999d01e450f8ac94039e822d0baeba262a3904db9e3466840a6f8ac8317326f8502540be40082fde894d1ceeeeee83f8bcf3bedad437202b6154e9f540580b844ca722cdcbfb3299e3f1c807fc4a02337448e4706bb8f4c99a84394c3d91d5242f0d84a4ba9221b73147a8f12da3bb2783051798f967144a4a93b838511dcb19f8e90a1a31ba0ff72d63bd59ccc8ab6a95f50806d647e12a2c6f1cd96cc9176066fcb35f4184da03e36a7b739909f487a256413e901b2af6a0410b20199cab7c7ceb6acdd038a4bf86e8201c58502540be40083015f909457e0c81d89a1526bc01d8ee1e695cfb586db0a5587071afd498d00008025a0b491d6dfcf701b96034697d5390c3ef958f90f68910fd3abead1b179f877c4aba0099dc3a39d115a6fc23afab22dcbd68e138351e9fa6673e2674ab8933d92840bf8aa048502540be400830186a094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000108fb8c18dd3ce3a0ec957533a63ece39750e7d20000000000000000000000000000000000000000000000000000000002faf0801ba0257d3de84cddda15daa952c73c54c755d00c63c39ecbebd0dfa3c0d643e1c34fa02caa306412f30ff0b6c61364b3c6b36899682c0ed7976060f74575998a5da1fbf8ab82045d8502540be40082916c947d1afa7b718fb893db30a3abc0cfc608aacfebb080b844a9059cbb0000000000000000000000003f5ce5fbfe3e9af3971dd833d26ba9b5c936f0be000000000000000000000000000000000000000000108b2a2c2802909400000025a0c96a7e3a01789ef5c80ae233b392ed76d7279ec41af410b2afbb046cc9d1f350a057201812061ba1d87161017b77ba24085591a47a54dc758255d7838cfad15165f8aa098502540be40083015f9094c8422a8a386eb49ecbf060198938df163c18840680b844a9059cbb000000000000000000000000499bc6b4b9e7da1fd8a775106a389f4c1d1d4b9800000000000000000000000000000000000000000000001b1ae4d6e2ef50000026a07ddae53dcb862c6d7a0cd0831770bd91f41b585021169ec7857332975d984ad8a03231d9d576999c3dc83ca493d4d4c74e0ebd322d473d61f3ae1ab18c3085ea01f87683054ab88502540be4008252889423b1dcf95efa3f1d8ab7eca2903d8839e25ea389871ff973cafa800088e8d6c1800cbe8f9326a0a8e3a154d9eb114985917688365844cf4dbfe88a8cd0817730444c0b7747c728a0633794fa42ac1e1a6c4afb825f9087e5e5bd89dc197d3a15c7d797c691ceffe4f86b2a8502540be400825208943e5abd204b7df2afabe7d25ee5f58eeded353e1c87038d7ea4c680008025a08bc5c19bff1855d7abcb83c386026f025eadda0c84f3fa5263f6940c05ce39bda01b8ff51859da1dad9384552180b20735cf39dd42f14e9659672978c9c0e6cf07f8ab81f68502540be400830186a09407a88907b8b46a2124bb49cc67ce65dbe06e8a1b80b844a9059cbb0000000000000000000000007d344d40963994e1c8246dc5a64daca6cf08cf210000000000000000000000000000000000000000000000000000128dbec0e0001ba07300945c0fbaebd8a3151436a2926e715484f124c7959dff7bff1fd166bea04ca07afad5e320ff9249fdb2767911ffdd96b7984df14335d0e44607ee777ab3429ef8ad830108048502540be40083419ce09433fbc07d9d9456d9e6d8316aa87792a20d3ad44880b844a9059cbb000000000000000000000000c18653ae95dce4dd94b81852ef2e65f9ba85ce14000000000000000000000000000000000000000000000020f7fc1d4dc17080001ca059bfa28182676b3c011dbfd11b4d1e25bc0baa7ce3a25588494bf16b7137a0a2a0678dae4fd78d8e064bc7e900785cae40f0a7d20bd3d31f078a37e2c43be2217cf86b19850224155574825208946b151d0c4fdb7980b40152ee08380ff7299fcbfe876a94d74f4300008026a016e2c17e10942e7c5cff8fe0064c68abd6b1f91b36a0e0ad3f25272c0a79ecefa070a4fe10e35a0784dde49695bc59da76a8a55c8cde3fac6902470cf5ab86749ef902ae834538db850218711a0083061a80942a0c0dbecc7e4d658f48e01e3fa353f44050c20880b90244ef34358800000000000000000000000000000000000000000000000007389d3b231e130c000000000000000000000000000000000000000000000156073b9df9bbf3ecbe000000000000000000000000000000000000000000000000000000000000271000000000000000000000000000000000000000000000000000000000e2e4014300000000000000000000000000000000000000000000000007389d3b231e130c00000000000000000000000000000000000000000000000000000000e2e404d700000000000000000000000000000000000000000000000000038d7ea4c6800000000000000000000000000000000000000000000000000000118d42d8e5842b000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080fb784b7ed66730e8b1dbd9820afd29931aab03000000000000000000000000e8207eac5da52d9625e43f5fb6ddbb7310f4ae5b000000000000000000000000e8207eac5da52d9625e43f5fb6ddbb7310f4ae5b000000000000000000000000000000000000000000000000000000000000001b000000000000000000000000000000000000000000000000000000000000001b50a1bd885a654a8849955d11f72962ce588930168eff6094f1c6a8f63d2c66774a52a0af30c65cf71b80d0ca1e38526a4b2c1499880e8eaa222a79e90dd0a7636881132f8de78df6f79d62953461b9ddd7636b32c2d262fee44066424c016b0d07c8d754fd6e1f9d2e559f0eb2e97cef5234620201561b5c4a70821971f7c2bf25a00dae3280dbcb9fa75a1f879f0e37b8d2cedf6716f3350e5e2c098d7b28812aeba06701f0d63060ed71392ab4fd68eec43930050b285f2f91ed4ac96add55a8ac04f8a908850218711a0082a10594dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb00000000000000000000000075e89d5979e4f6fba9f97c104c2f0afb3f1dcb88000000000000000000000000000000000000000000000000000000002818bfdd25a0b73d41f3a276c81ce380c8f79ea5a62e75e659310d7983dc34d3bf42168027bea04409e097e5832cc4b7b0c354e848e8cc605d2817008658676434c6cbda6272f6f8aa038501dcd65000830927c0947c5cb1220bd293ff9cf903915732e51a7129203880b844a9059cbb000000000000000000000000ae63fdc2cacf58b010b1b8ae79b546a1c9b8026e0000000000000000000000000000000000000000000001b813e1a4fe1d18000026a020bef9aed534e1a87997cac7d6cf93965cd3e4adcc19f038829d0d9075765feea0174bd8aa30b3174d5063989cf67f799ba3204df4f73cf76672e53b800e83cd0df86e825e5b8501dcd65000830186a0943df2b998d1aa8acc1bcb5812828de7847a1703498702d79883d200008025a06606a70c7ea74b806adc98f5f0271b0d855ffff065bd1c47ae914f6f69569182a06e821eee273567c165eba12d5103feb5f3dc84aa4d5d100ca2839268befe65a9f8ab82032f8501dcd6500082ea6094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb0000000000000000000000000a106d66be726a445d7e6e4f7c9a48579fb69224000000000000000000000000000000000000000000000000000000001d6a366025a0ae1273cf92b271f826df46326f3a8f42522a514699b2ced08441e1da9bf33648a016753fbceed51454eec1a48ecf82aef341796f0a37e30b39d1b081811cec1311f86c808501dcd650008252089487b6e87c763bca2f45b2b1090dce00eb897f2a38880278e47f242c58008025a0020a42d1e7cae57d455ca296d4b1d046194f716e86df4f20bbba76e313c8e29da00dafb12d813e6c5f4bd384d3775019f1b493c1c1eb5c366bb160cf66a06e70b3f86f827ffe8501dcd650008301388094c5081471c33dac824060a4ed3b95c5f08429ae6288010f85c50951e000801ba0b96c45b6f31dc5048cdb1f5bd0776ddceec4fd663c37d9e5f46608d2f5aa58fba01595dbde0bcec646245a8e54bcf5be00a4c9fcca438902d332d2ee9683771037f890808501dcd650008303a51994bcf935d206ca32929e1b887a07ed240f0d8ccd22876a94d74f430000a48853b53e00000000000000000000000000000000000000000000000000000000000086cd25a012f97a513c19a363149d8f713edad1c941f6e5ab1b89640e0c0e14568a264d8fa01329aa91f82e5db6ffc863eda6bf0144ba7d69b85d2ef71a60e9aef1f937274af8aa819c8501dcd6500082eb1b94dac17f958d2ee523a2206206994597c13d831ec780b844095ea7b30000000000000000000000000eee3e3828a45f7601d5f54bf49bb01d1a9df5eaffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff26a0500169deab316b4db49ef96f20b065c391387db394c54ca78fc9c8db368dd81aa02cf4fa8ee2df203bdf5a37a70643af0227bfc024c90fe4bb321ec0df6380037df8a9308501dcd6500082ea6094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb000000000000000000000000ac5f70950463ac95f49a75df2437c620eb123a730000000000000000000000000000000000000000000000000000000001ba814025a0236930dc2e66566587641d86a9dbe198d5a309f8d2fc75070ccd06e81396cc88a03563e5b14bbc71520b277877349268dbddb994644a1fdf01c1a32f97e7de2d3cf86c808501dcd6500082520894c76535cb84ade872535f0a3baa4d9a70a47fa38a8805d102b36e97a4008026a0349cebafb6f18688f0f6c046d9968a1451f25e5cc9634072c7a61401841f44fea0378dcdc7c74af5ee0990f6ee8d49fd09fc13bb7521b3645874a72f8b8829cfadf8ab8201ae8501dcd6500082ea6094dac17f958d2ee523a2206206994597c13d831ec780b844a9059cbb0000000000000000000000008616811ccbdc417a1b851c4529b6a0e5d6016c9b00000000000000000000000000000000000000000000000000000000111b45c025a083a354f44062d62417e1a41d0158b24c14ef085ef817f66ad1ae65f3fcab8ee3a07e04cdd02e631702fa7044278855a8eb5f1d84d3988486c79864f4fce311e218f86e8202468501dcd65000825230941f973b233f5ebb1e5d7cfe51b9ae4a32415a3a088801b0d1eabb020564801ba0aa3a6caecd96eb66f17c9b829fe6473d44414e28d23a36bd46fe4f2d70dc6236a01be450e8895c700404237a354b82ec2bac1cdf9ae16a36b12cb4104d02a4b393f86c808501dcd650008252089448c0aa2d5e71a7e83204ff681b6304513cdfa4ff88050d2d34df9c74008026a08f7cb4ab3e97d431f4920be086d1ae960edffdd29319551d8bcf328b9d4c9961a0760e9d77ddfda19dfdd0532cd02f31de0e388f305da94a79664bb2ed37e7fa13f8aa138501dcd6500083017c079434848f3c3934733d3ebc6f82489346592566c77780b844715d0e33000000000000000000000000000000000000000000000000000000000000000200000000000000000000000027c1248026d2ffc163759882753bad600e7ab7b126a027b3825f9ac78127eae2596039624ca0a4f36a19839f04ea6cea78c50a4367e8a07ebfadddd55a4a00c57a54e253a82b7d65477c0c5041c3965363a8a8177554d2f86c808501dcd65000825208949045c39c4ebdf00761c718a2867cb172b883f24988018a59e9721180008025a0bf2050150da7240367cf3c710373c5b37fdcde2768120c4862f7f50959ada37ca063428d1b1f6047eccd2bfe96a21e77b233f10cb394dc87f63a961d80b2b74655f8a9188501dcd6500082ea60944b401c84a19a41fff2d8dda140661af93176052780b844a9059cbb0000000000000000000000003fd3b0a4da408cc82042f56b4d3e65a683ca0b4100000000000000000000000000000000000000000000d3ed7a09be58b980000026a09215ff71496ce6448ad54811d5bd271cb3accab0e950ca1e2292c84a477f8c75a0604792c985057637fdc629660adebbafb93cc024c83162b198bc51b0f6627d5ff890808501dcd650008303f14594bcf935d206ca32929e1b887a07ed240f0d8ccd22876a94d74f430000a48853b53e00000000000000000000000000000000000000000000000000000000000089a425a06f793e58a01cc1b2469bcf75453098a53969eb5b8959b0c4a87fac13d41ea9dca055b05174f8451e6316d3a4a9051307d0d633b5f624538999b8cea8627acebe2cf86d808501dcd6500083015f9094f996eadfa33ab0692a314fae20d873d680d7918f880243b81de1f594008025a0074f587f0cc484b2a5fb7b276d8403109932e33efc761c1b3ca77c911d0dc4efa00193a96db056b30a9a563a56a1505a2c678629a1cd4fb0914dfb5637efd1514cf86e8301c47d8501dcd650008259d89457820e0ec07ccc3a8e9e0803b47ec0bb76d1d0dc870133e947b2b000801ba00cf25da4640e5f2ae5b51ee290b198010b6684db3a7f97e89cb152055bf0b9d5a074c143600e73d2f02c7dcc52a946e183fea91a59cc9e5d09c9fabdba378f95c1f86c2f8501dcd650008252089492114693b5bda8c2c50aa0e7d0f333cc30f2e5278832a767a9562d00008025a06d4acbb0bcca48716e3610e8577bb039767647bd0324fec5307a270840e482b8a01374a840dfd5da94d9d57b5a4b12c71fb2878cc9bb797094845de2ebdb78be99f86d8201e68501dcd65000825230941f973b233f5ebb1e5d7cfe51b9ae4a32415a3a0887f1a5750adad3af801ca02513a03698f86c936746cdcea6947ca365615548f741643b5ea5a9bec72d8d14a00c4670c087ddb526e7a8fe6af63180093b3a9e5504ebb08a996fa100262470def86e8203e58501dcd650008252089493b0c4f94b1c119fd9972adc22b7b1d2472d8abb880de2d8660a418000801ba0ada35c72b717e482dad3d582c74f0e10db41ca3e71358f2d0273f94e2b04b694a0217f78e660dfe52a958e9dde8c914d9bea91ce95ca96f3ddd4ea6b36ca142503f86a81f48501dcd6500083030d40942e63b6d7ef8fc8a2ea4c96bb9a633423682f0f798084a3e76c0f26a04de807f582bbf467b4f8f0ca48d0bf0d67882292de2430adc9e2ef4c1cff9fb7a0757d21decea7a07af195f91f64d9cc92df3b8ae52efff253aab6865cb7514dc4f89b048501dcd6500083030d4094bb99a5b15fc7e5e8273cc3e380741f1899485aea80b63059544137574d425271417762586e59586e354d3861514879317273656d4d387a6b70676567366d683739447a416b6536506374364625a0d7ed0d3c4358c3a8d2dc1fe6d3b3ba870d1316563386578d72ea4dfb760050cda048c16945aee4156137886e7d87700e9b5ad8992a91624d1b4c5133d9ec131c14f86b808501dcd6500082520894989eca3429b083931a1c91278995253a7c91f8e78726cdba7f65e4008025a06b878d5bdb6e722aad51703023c05c4b3d975e18aafb9bfc8e2d7daab60707cba0578d8506d35959d22931a3b65695c4b69fa79bb9806fed43b59510a49add13ddf86c808501dcd6500082520894bca1460e801bb38ec421ccf711d73e1a62bdb349880feaab924f1ba8008025a0145147645ddee99644dc72fbdc23ba6aef604c47f0cd4876e50d75c40b1da9bda0592d93d4439dfec9394082cdf8610fd13ae76dbcaadfb4e6a497d04d75a2c9d9f86e8261418501dcd6500082520894373570ca9f74edb28a92a40cb5d5843768f636d1881d9cc7cb6dba80008026a0d792e2c29f5beb7f4505a38ac8a4fcefb5bb356c9428624c87cdd967eceb0e28a0774fb03fd1f17af608a4a77d466034252b9dc8740d869acb062c6fa65cb8c35ec0
Header: Header { parent_hash: 0x9fd8152ef85b9899d3cce0d8f340bbe0ed2732b2dd1a281a6d5fdcdae9b7bd57, timestamp: 1577769530, number: 9189357, author: 0x829bd824b016326a401d083b33d092293333a830, transactions_root: 0x7da15537f4171cf88f24b0afdd385b348e21054ad4464f05c6551c9be13ec8f8, uncles_hash: 0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347, extra_data: [112, 112, 121, 101, 32, 228, 184, 131, 229, 189, 169, 231, 165, 158, 228, 187, 153, 233, 177, 188], state_root: 0x19dced45482c827958e490854cb491799548a5629a09786e85bb23537a087f75, receipts_root: 0xcabf2f15c98fdc794e01f1b4dc03d635f5f726e42a18a02d702294cf429f7512, log_bloom: 0x484088821203380888d00a095470033440810390544a92d080c048100a20d35604526021250154c0081010000a22415b32c00600810006a90a8a408468a0066050900200088b8513bc8804c88680820c3452c502420508100062026004061018004048b009884c0d0401600c610100120020121000080814008007383181150454042102020420648208000040000701404028405180800a00102022a07141125f040500240531807881408040b0d02400404400c0800140000000021a00384392a00042c5a201c8043278000400a860044052000041802c80303550510980000212402031101080006041800000208108ce08818103a01310e4105428500060, gas_used: 5219298, gas_limit: 9939927, difficulty: 2506618482254369, seal: [[160, 248, 153, 108, 87, 29, 183, 253, 83, 248, 35, 240, 126, 197, 220, 192, 111, 187, 252, 208, 141, 73, 104, 17, 182, 224, 54, 8, 0, 128, 231, 113, 105], [136, 111, 107, 83, 228, 0, 11, 213, 31]], hash: Some(0xa5f03d8bfbab59c5de8e25bf6c5dc3a6c2abfb8f7d7364ef1abbb4aa42ea42e0) }
Uncles: 
Transactions:[Tx 0] UnverifiedTransaction { unsigned: Transaction { nonce: 270764, gas_price: 65000000000, gas: 91000, action: Call(0xd6ac90c8cca2ae8b002926c875334a0011849bcb), value: 89199960000000000, data: [] }, v: 37, r: 89401898469950585399273705064832071148569439697573449180197918307469554686682, s: 18843437407917014050859576565293715783359686085759650980645068077179084322947, hash: 0x31c27e981124c58a8a65453c0d9927003d83917245b05b04050b190a9c1c8b09 }
[Tx 1] UnverifiedTransaction { unsigned: Transaction { nonce: 270765, gas_price: 65000000000, gas: 91000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 152, 21, 204, 215, 183, 112, 5, 43, 81, 99, 27, 77, 200, 66, 123, 140, 15, 16, 67, 90, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 251, 17, 139] }, v: 37, r: 38477286449566470752363200794890248405557270857544266604717036611837458458265, s: 36599599094577205764377226754985924303624348675561191803430673990051114077096, hash: 0xe8e67ccb2c0676cd4acf64a21dc1ce6d251c1ac3e58f3e1d31a2d0a28294606a }
[Tx 2] UnverifiedTransaction { unsigned: Transaction { nonce: 2, gas_price: 41000000000, gas: 21000, action: Call(0x88d34944cf554e9cccf4a24292d891f620e9c94f), value: 5948239000000000000, data: [] }, v: 38, r: 34130598970018977360170106403319709688966689640870638988298551488992837391298, s: 1954370395355989953250635532988387882416241989788657127276649876321042317163, hash: 0x6e94a950c9e493c833462d2541304cc4d56c9fabb40083b4227290214248a718 }
[Tx 3] UnverifiedTransaction { unsigned: Transaction { nonce: 9, gas_price: 41000000000, gas: 21000, action: Call(0x88d34944cf554e9cccf4a24292d891f620e9c94f), value: 6799139000000000000, data: [] }, v: 37, r: 71032995724501499470487842942346875170096057463834411571257387032380101713031, s: 57100962899144490617145850400108233122538758599318319765944274690232594966204, hash: 0x6854a039267695156e3c8a230e7f774525b027a3efbe0089a464dad3e5f3ecc1 }
[Tx 4] UnverifiedTransaction { unsigned: Transaction { nonce: 78, gas_price: 40000000000, gas: 150000, action: Call(0x794a8d6966ee087da2dee792bbcf70039c568f1a), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 201, 75, 198, 161, 215, 186, 98, 70, 106, 168, 42, 170, 193, 236, 180, 104, 250, 70, 4, 133, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 112, 64, 29, 80] }, v: 27, r: 57609964919423560457019023868041163119317049150832170823354292783166814830973, s: 40566530363417540918855158211790264481733094418177153470858206241799444587268, hash: 0xad3515157f27cbda03851cba57f07d404339f399ebc88113915933b7a028321d }
[Tx 5] UnverifiedTransaction { unsigned: Transaction { nonce: 125444, gas_price: 40000000000, gas: 60000, action: Call(0x4ecdb6385f3db3847f9c4a9bf3f9917bb27a5452), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 93, 179, 43, 162, 152, 243, 101, 204, 69, 196, 152, 133, 140, 149, 212, 88, 47, 31, 164, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 59, 95, 47, 54, 0] }, v: 38, r: 54094724484856922689875782044031654628590855848630360496470391128149624893442, s: 9507125391851935188972314363386730465391372216708213168867655684800026255556, hash: 0x57bbabb362ce574e3175c6dbd33f85c4d50eff8bdde42edf5bd36f3e1036a073 }
[Tx 6] UnverifiedTransaction { unsigned: Transaction { nonce: 40460, gas_price: 40000000000, gas: 90000, action: Call(0x97976ceaabc35134bc411dcc64bfe1792379d755), value: 2960000000000000000, data: [] }, v: 27, r: 77543206218897730514443068671764565260881452058336068752061918995376270624388, s: 12618244194715478897787263659147825407761013899450549372931295955382307670486, hash: 0x271f5effe2b45aebfe5f3da61d8c7b4a0278bbc9e110617270eb29d0332a6c34 }
[Tx 7] UnverifiedTransaction { unsigned: Transaction { nonce: 8, gas_price: 36000000000, gas: 63000, action: Call(0x5b8174e20996ec743f01d3b55a35dd376429c596), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 120, 167, 161, 5, 203, 65, 112, 176, 131, 96, 22, 134, 171, 175, 138, 225, 72, 27, 96, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 164, 123, 25, 113, 188, 236, 0, 0] }, v: 27, r: 111950519355385280631987926848098880255232162644033525130407997944922013386518, s: 36015995075249703856042998878709886371758141658148382035485167206450565811058, hash: 0x97de4325c79f61514a2e433a9d8b62a2eec28bac037bea6f955db1e01e8f3ce6 }
[Tx 8] UnverifiedTransaction { unsigned: Transaction { nonce: 212270, gas_price: 35000000000, gas: 140000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 193, 164, 154, 126, 195, 67, 116, 167, 80, 179, 170, 4, 230, 37, 186, 218, 200, 210, 1, 211, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 144, 245, 96] }, v: 38, r: 81169734625108639457167644412956360723485313211160024482793028757725927074237, s: 8230318027692879917510931578948022549006633128006920730699140491453133544046, hash: 0x62da75632d1ccf7c37102f472fe30441905e046f1cddb033414d079b2b0a92ce }
[Tx 9] UnverifiedTransaction { unsigned: Transaction { nonce: 9, gas_price: 35000000000, gas: 65000, action: Call(0x68e54af74b22acaccffa04ccaad13be16ed14eac), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 139, 171, 240, 186, 49, 26, 171, 145, 76, 0, 232, 253, 167, 232, 85, 138, 139, 102, 222, 93, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 101, 241, 133, 0] }, v: 37, r: 9857303866275140255655740418172200415124108932316069194011845440660408565668, s: 27875207150390267851128969726736607619686590211341641216635392663752240439435, hash: 0x993d2ec213e84cfdf24466a870d5d3f568b4e5b77647e8058502f511f76952d8 }
[Tx 10] UnverifiedTransaction { unsigned: Transaction { nonce: 16, gas_price: 35000000000, gas: 65000, action: Call(0x68e54af74b22acaccffa04ccaad13be16ed14eac), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 139, 171, 240, 186, 49, 26, 171, 145, 76, 0, 232, 253, 167, 232, 85, 138, 139, 102, 222, 93, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 93, 33, 219, 160, 0] }, v: 37, r: 93005748243564872659846179457552907630946376061043053536197460464180797112535, s: 34131073596048519175308020061344325247843709743908801850164840607193813491753, hash: 0xc46163468e89ddd77c9402c937983859660c9fe7703fa0e7b4914562aec8acc0 }
[Tx 11] UnverifiedTransaction { unsigned: Transaction { nonce: 456061, gas_price: 35000000000, gas: 140000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 249, 109, 22, 49, 134, 212, 207, 92, 219, 128, 200, 118, 197, 98, 228, 175, 28, 63, 239, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 103, 18, 103, 101] }, v: 38, r: 3319130195398379664119368346412528325717446538454373290154532986661478120447, s: 6737387540680693023561178224465781423582381991216915568374539200295608577527, hash: 0x49e7f980f688927f8ebbca4abf755227907ce1f0726ffdfd8f539fe4c6566092 }
[Tx 12] UnverifiedTransaction { unsigned: Transaction { nonce: 456062, gas_price: 35000000000, gas: 140000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 51, 170, 187, 173, 161, 23, 91, 165, 99, 2, 185, 44, 231, 222, 90, 140, 225, 44, 17, 188, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 119, 227, 104, 11] }, v: 38, r: 58615761754890034277272735821935985864457129915558667129233404039091655523061, s: 34733549342750355412039204790392791304490815168713869851623565035420322820032, hash: 0x561181f0a6c5d86771d7e935e0a62e6dcf080b59e8152a502cb7feecbdb40fee }
[Tx 13] UnverifiedTransaction { unsigned: Transaction { nonce: 649155, gas_price: 5000000000, gas: 21000, action: Call(0x3c4e3bace461a1081a51e3ea0f2685738cc8aea4), value: 600000000000000, data: [] }, v: 37, r: 65547210425070530709716838347452927745564775836123419102995953973373351211954, s: 32079850209797306261199354452385501753306694365774459001583476121315645519514, hash: 0x5868ad91c6d432cb6e34bba98d827de49cf906205f67d1018d6ff1fb4c783105 }
[Tx 14] UnverifiedTransaction { unsigned: Transaction { nonce: 245192, gas_price: 35000000000, gas: 65000, action: Call(0x68e54af74b22acaccffa04ccaad13be16ed14eac), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 246, 105, 163, 76, 24, 42, 71, 207, 225, 91, 121, 86, 7, 80, 251, 155, 233, 235, 61, 117, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 71, 211, 87, 0] }, v: 27, r: 50815896487658202609519326704763812978891516879436711253662328536301295526656, s: 2429859127828939317258151767182069744750553526882955652024879046270231520755, hash: 0xc768347d6d04baf08af94182fbb78e4b9f4e7508ce3b73500de1fd62b57fe832 }
[Tx 15] UnverifiedTransaction { unsigned: Transaction { nonce: 245193, gas_price: 35000000000, gas: 65000, action: Call(0x68e54af74b22acaccffa04ccaad13be16ed14eac), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 131, 119, 156, 12, 49, 202, 205, 213, 173, 25, 16, 150, 99, 30, 44, 40, 37, 141, 108, 232, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 76, 203, 0] }, v: 28, r: 24460881624940959868279349154758967671166366668656504473501437425195968638010, s: 24500329872797920032121921429164497751394675720079470304196622334773832466518, hash: 0x70e94344bc7a4c8a1330592c5b1fe51c58ca144ce9e1348960e9a78ddf896c97 }
[Tx 16] UnverifiedTransaction { unsigned: Transaction { nonce: 245194, gas_price: 35000000000, gas: 65000, action: Call(0x68e54af74b22acaccffa04ccaad13be16ed14eac), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 232, 28, 94, 218, 75, 105, 32, 200, 78, 156, 223, 172, 244, 26, 14, 89, 14, 131, 99, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 245, 207, 63, 0] }, v: 27, r: 29343829895898361633987606255408260736935433589663898747558466294141953205821, s: 24266588133609244201781027506903250645839874322700161563486229252095176999857, hash: 0xf9ec7047c52f414aef4fcf176425ca2982d28ff00a645a2b91d89aa4a535c8c0 }
[Tx 17] UnverifiedTransaction { unsigned: Transaction { nonce: 443331, gas_price: 35000000000, gas: 140000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 233, 44, 7, 7, 57, 144, 134, 4, 23, 165, 106, 11, 0, 153, 37, 195, 215, 102, 53, 115, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 85, 243, 50, 35] }, v: 37, r: 39386886195829137278009076615379758161546077394975833742361832634236787847592, s: 34073496729082148486111564043368873951639319308431341883750736273335398005626, hash: 0xbd37cf02d8ad0b423adfd738af9d35943d211a3dcd0eb2a9577cad6650407403 }
[Tx 18] UnverifiedTransaction { unsigned: Transaction { nonce: 6, gas_price: 35000000000, gas: 65000, action: Call(0x68e54af74b22acaccffa04ccaad13be16ed14eac), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 123, 77, 199, 34, 73, 234, 170, 254, 9, 218, 251, 48, 180, 146, 230, 147, 175, 32, 243, 132, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 220, 214, 80, 0] }, v: 37, r: 89713138286646171937621910740971160105153380283167991571660231334625648387487, s: 41639820474941761702558624792094838791922177536669093189703226562349775191643, hash: 0xe15324a9edbfb7c04d7d2b7b20bb073d7761e62b86a628e26dd990ee5ed1a67e }
[Tx 19] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 35000000000, gas: 90000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 241, 127, 167, 183, 67, 146, 38, 53, 75, 180, 164, 105, 213, 133, 240, 135, 253, 17, 178, 130, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 17, 225, 163, 0] }, v: 37, r: 51235995436527447325842908141417929001151114603339385593465002383189700583274, s: 23052973502586769634985776651960036463831802289853588570520854646927107798380, hash: 0x1da0ffb066812742e2dce3e5870193e263d1d1a6408764635311411c90733846 }
[Tx 20] UnverifiedTransaction { unsigned: Transaction { nonce: 327397, gas_price: 35000000000, gas: 140000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 124, 21, 99, 68, 2, 7, 123, 97, 147, 180, 159, 20, 173, 99, 89, 213, 83, 215, 228, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 221, 173, 110, 192] }, v: 38, r: 55167003112008145430867650003297023762598452687937448072637832067177934801492, s: 5399330328834428083501027008775050293247346987109271545204847238841941404808, hash: 0x2f983001aaf0e23db4f3e51c62284ac8c16fb69c5e51186d1ce244987e96d00d }
[Tx 21] UnverifiedTransaction { unsigned: Transaction { nonce: 1, gas_price: 34000000000, gas: 100000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 239, 213, 35, 73, 238, 49, 146, 201, 201, 77, 226, 54, 185, 56, 3, 145, 20, 230, 250, 85, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 44, 29, 128] }, v: 37, r: 34692817670098204595404141135221217509437279765052714896073086715971822321193, s: 52491807458493317370561412139518474548062921277426005020791859130544039647821, hash: 0x6b2cb623fd50e1d6cf58cf4bc3f95dc8935e6bc26398187cf696179ad8d4d196 }
[Tx 22] UnverifiedTransaction { unsigned: Transaction { nonce: 185, gas_price: 30000000000, gas: 72000, action: Call(0x4fe327c5a809fa721d47b80c5038a0b393e61305), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 100, 117, 109, 127, 204, 3, 113, 121, 127, 231, 180, 207, 239, 16, 163, 166, 102, 156, 100, 243, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 96, 68, 113, 0] }, v: 37, r: 4334831032555185915164858992577158326154281542456262541260474802315146901276, s: 5076336289032413629109635399282801947515215435280591101964667374612884397336, hash: 0x5495bc8885e1b40daec09cc2c7cd39025c7e4c6c5e62f9095ad21cea5757fd3a }
[Tx 23] UnverifiedTransaction { unsigned: Transaction { nonce: 8, gas_price: 30000000000, gas: 21000, action: Call(0x68da78fcb1192ff3553dcb782a5f4216038752cc), value: 1621230000000000, data: [] }, v: 27, r: 61659956835762361059088110805031936258336476927491552626458632991324183659691, s: 16509922720103685838899214670122186149451993957905799956252108813620612446275, hash: 0x724d7e497a922ca701cb3f57311bb03e074ceb415254735c378daa609e51157d }
[Tx 24] UnverifiedTransaction { unsigned: Transaction { nonce: 67118, gas_price: 30000000000, gas: 100000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 138, 156, 48, 96, 173, 44, 99, 66, 142, 4, 250, 243, 103, 67, 139, 59, 74, 237, 237, 173, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 26, 131, 28, 0] }, v: 38, r: 42827153125419360182656627701599106657973532275893260455305743965727352225018, s: 8725822114812439384511973334973914235310948482349081991808686067264991971125, hash: 0xd55ca31feff8994cd0ee7f17c65c38308ae7205286930c79359e551f9d312b44 }
[Tx 25] UnverifiedTransaction { unsigned: Transaction { nonce: 10099, gas_price: 30000000000, gas: 60000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 252, 127, 234, 251, 181, 239, 68, 146, 100, 174, 54, 209, 57, 83, 72, 152, 239, 169, 89, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 251, 234, 104] }, v: 38, r: 66046469480354132502221529294513015019212597856656804592283755552046141335305, s: 18671931497283373581880097962898485950165530108997077943647895567976790757319, hash: 0xf254a1da0f463eada36f5c0f06ab5bd865c044dfaac67c9d192703d590f3ee34 }
[Tx 26] UnverifiedTransaction { unsigned: Transaction { nonce: 181182, gas_price: 30000000000, gas: 61197, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 40, 113, 30, 44, 27, 0, 173, 103, 96, 156, 75, 25, 58, 235, 173, 2, 195, 4, 90, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 26, 130, 20] }, v: 37, r: 79304076311448958476099727184447934508986078345434976940261512428355180163768, s: 5468509228052065753533948169553742583182695832347472125884076559594485392850, hash: 0x13ce91b116b166dd0b58ca1b679c1ee2aace1f549ac340a550eaafb2b332fee4 }
[Tx 27] UnverifiedTransaction { unsigned: Transaction { nonce: 181183, gas_price: 30000000000, gas: 73283, action: Call(0x3c7b464376db7c9927930cf50eefdea2eff3a66a), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 223, 136, 246, 181, 89, 9, 103, 213, 208, 173, 48, 191, 204, 49, 170, 52, 63, 206, 20, 180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 157, 181, 151, 83] }, v: 38, r: 6369301296334098981670921407294429345014850595022417877496695389932075170311, s: 9318748076009855096288050632406758452526156809533464162724424092367373397126, hash: 0x831f6b9b4bb9b0fa867601b69ee6615eb07969d53452aad20f0803409513a0c8 }
[Tx 28] UnverifiedTransaction { unsigned: Transaction { nonce: 181184, gas_price: 30000000000, gas: 61209, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 36, 88, 22, 112, 146, 218, 195, 219, 228, 74, 245, 108, 126, 56, 68, 63, 207, 170, 125, 133, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 15, 19, 239, 128] }, v: 37, r: 79341432441206166978064816606989921746634703232220368946912185365986729139917, s: 52036734059117980176359085674410138731396410920159351968568679168431950899428, hash: 0x198dc21412494c9a402e1208149e32e2ced0b650292d72f3273da92976beb2c2 }
[Tx 29] UnverifiedTransaction { unsigned: Transaction { nonce: 181185, gas_price: 30000000000, gas: 73283, action: Call(0x3c7b464376db7c9927930cf50eefdea2eff3a66a), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 152, 126, 49, 173, 29, 130, 109, 212, 110, 110, 66, 142, 73, 189, 212, 216, 30, 109, 140, 77, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 71, 113, 202, 76] }, v: 38, r: 106521955899360090072819177412559022183017088946694557743863431069128489950738, s: 57284244267238397721645593893370044175676733673443219790853287003190043541988, hash: 0x97eafa443ae9cadd9220b119582c33096bf66378ea94903d5131bdb6814a6414 }
[Tx 30] UnverifiedTransaction { unsigned: Transaction { nonce: 550474, gas_price: 30000000000, gas: 80000, action: Call(0xd7cc16500d0b0ac3d0ba156a584865a43b0b0050), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 75, 242, 24, 242, 24, 220, 66, 124, 247, 192, 214, 240, 107, 242, 207, 70, 83, 161, 216, 168, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 19, 67, 191, 119, 64] }, v: 38, r: 88096650417496255305853266383531130717193495252058861036014557579912822741156, s: 40163097352525144221902726521465382109787824315893318219210388958688036197128, hash: 0x81271743cf82d4d044eb37b724f22fcfefcf811721c77daa85bf078127800633 }
[Tx 31] UnverifiedTransaction { unsigned: Transaction { nonce: 550475, gas_price: 30000000000, gas: 80000, action: Call(0xd7cc16500d0b0ac3d0ba156a584865a43b0b0050), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 75, 242, 24, 242, 24, 220, 66, 124, 247, 192, 214, 240, 107, 242, 207, 70, 83, 161, 216, 168, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 115, 155, 198, 64] }, v: 38, r: 167653190915502700599355175302888393134462970540880196347937293081147807571, s: 37501780828081855220448132677019635641156268859925380987808030660889033800424, hash: 0xf4e2f26cdd2661fc59292413f9051d1c1a551ba454bcb6b19ab5ecbc15bd3be4 }
[Tx 32] UnverifiedTransaction { unsigned: Transaction { nonce: 61731, gas_price: 25000000000, gas: 50000, action: Call(0xf772fb803c421d586cd23d72b45fe45d66aa75fe), value: 31416940000000000, data: [] }, v: 27, r: 80958020387535665698817687143181297743826833470137728365876758841382731907835, s: 32480152696983700390180199849025639620083182858182264895169315676573062598707, hash: 0x501b53f6827bb34394a3bbe1990fffcc66dfb71df5702df7fe208273ff67eeda }
[Tx 33] UnverifiedTransaction { unsigned: Transaction { nonce: 397644, gas_price: 24000000000, gas: 80000, action: Call(0xc581426ecc3e5cdf7ef74fbd65f5bf63fe81846d), value: 7602828250000000000, data: [] }, v: 38, r: 78634674953195228334247824361010347556588614074262607929292690619026055758283, s: 37807082555696015534416791742416109686947740508830428955438292000569323963726, hash: 0x30b046c16a8581e4b590a8f9d35b4068977af39fe70ff01b0c7b46eb2f2bcf84 }
[Tx 34] UnverifiedTransaction { unsigned: Transaction { nonce: 3579, gas_price: 22000000000, gas: 60000, action: Call(0xe183c454ab04205415dc42e1bb48313a912d148e), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 153, 25, 159, 97, 130, 239, 225, 37, 79, 230, 81, 75, 121, 250, 217, 10, 221, 163, 10, 161, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 131, 57, 9, 206, 195, 151, 60, 0, 0] }, v: 28, r: 91564146309402132934526455362112596050777908608785665796542948241745181186953, s: 701085096586433909388705113656278895351558540616411813802734908260592687060, hash: 0x1a347f9b89c8c667135bdd1077a934d74d0a691a767ceaae9cc2c557fee79944 }
[Tx 35] UnverifiedTransaction { unsigned: Transaction { nonce: 8583358, gas_price: 22000000000, gas: 150000, action: Call(0xc65e61626bb0cb58d31f0212cd4a0dab4d64743d), value: 0, data: [110, 160, 86, 169, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 147, 6, 91, 92, 126, 182, 51, 51, 184, 229, 122, 115, 1, 45, 37, 246, 135, 137, 87, 133, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 118, 14, 227, 197, 6, 44, 58, 221, 0, 0] }, v: 38, r: 100434221669801837027725134976613433488212113436722069701696732931061471042427, s: 4531651710760350266003353233026522621902837382792108095068606357582055780812, hash: 0x9b550895fbd4166b77ece89ad56873016b9341433ba5aa363665536d4d0f23da }
[Tx 36] UnverifiedTransaction { unsigned: Transaction { nonce: 12232, gas_price: 20000000000, gas: 2350000, action: Call(0xb1e867f3a3bfe5c3417c792f3fe12ff683ca1a33), value: 0, data: [192, 26, 140, 132, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 48, 8] }, v: 27, r: 74938503190806886887566849690066480403373229656510833252772896017965994622766, s: 17356952681675029103133660862136254765276326430576301336209116546106785278580, hash: 0x10fcb788be958545e12267a37906f40f78b79ec643d5d0c56314cb90898639fa }
[Tx 37] UnverifiedTransaction { unsigned: Transaction { nonce: 326, gas_price: 20000000000, gas: 21000, action: Call(0x4c65d791917f9da00b0780019ab269ce52c9f1e9), value: 404420000000000, data: [] }, v: 38, r: 37183477616449064557766549177792299331057633990340878296589640190265019573393, s: 7198424744018776376832861628852162610410675648004017063711656565113224178124, hash: 0x7f63ff2d358cf276bce76c7c58c797c50bf38307730d43e29f0027de31f86f42 }
[Tx 38] UnverifiedTransaction { unsigned: Transaction { nonce: 15397, gas_price: 20000000000, gas: 2350000, action: Call(0x14884a04608413a0837ed4d9d1fa0a33eb456400), value: 0, data: [192, 26, 140, 132, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 60, 188] }, v: 27, r: 71063898258092501179322077199862350858249956421899183875837923512973794821074, s: 50089275349792449618005023025373581771583944683790536918324862203988748254535, hash: 0x6cf0841f7dc6a11707177ba23cc81afaa350c0a7a44528f161f14663f7331fab }
[Tx 39] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 20000000000, gas: 100000, action: Call(0x1e701cbb418017b22acfc1dd64aad8b0ee0f075f), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 191, 216, 175, 109, 91, 33, 90, 235, 137, 178, 53, 208, 88, 144, 219, 201, 233, 202, 191, 148, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 207, 44, 207, 24, 128] }, v: 38, r: 8126310745698930447697232848084566971159756117618121710359252639500517704450, s: 21497194190622015370070300961323005868357593050234386532473330882795433430019, hash: 0x2f70fddf8f9fd7f9c02552ca87300a8fb3b965a3414016d9e9a43da0ce370439 }
[Tx 40] UnverifiedTransaction { unsigned: Transaction { nonce: 8, gas_price: 20000000000, gas: 100000, action: Call(0xb3dd5dce850dca7519e74a943568b69f958df52c), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 30, 90, 227, 15, 210, 238, 70, 120, 228, 44, 121, 143, 78, 200, 215, 157, 69, 47, 53, 99, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 60, 238, 162, 44, 202, 243, 16, 0, 0] }, v: 38, r: 42543528973537918613399234968002387986050055621716349256589631607941999942101, s: 55195119675593299262040445542035326628504114006019957491790420429156869544515, hash: 0x6d511bf1d05bcfe42f01190ed77a6ba5b736e5727af71c035fdd03c5a5c9f32b }
[Tx 41] UnverifiedTransaction { unsigned: Transaction { nonce: 147, gas_price: 20000000000, gas: 90000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 231, 190, 71, 109, 211, 88, 221, 64, 121, 247, 18, 8, 21, 128, 237, 111, 190, 16, 8, 242, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 67, 213, 128] }, v: 37, r: 31643966962147160626666868283101591036157539591662068869674721379402326851275, s: 47486898013984179201360105372523854218492661176429155603459995684409068470358, hash: 0xb79199c42d4cbc3261e1bcea9d0ace95e320415078b87975805a24060ec565d6 }
[Tx 42] UnverifiedTransaction { unsigned: Transaction { nonce: 4450, gas_price: 20000000000, gas: 90000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 26, 225, 36, 238, 147, 64, 203, 63, 100, 244, 151, 108, 142, 107, 105, 4, 137, 34, 7, 118, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 46, 52, 78, 224] }, v: 37, r: 4181842434446481640791827322272036580744667190661075403976477870776397642740, s: 51684701761164204456938347675728433565919852762249767846939412484891074242770, hash: 0x6d636ab22705be579a5a4444fd8c2447660a57bdae3c8fb762f85163536b0319 }
[Tx 43] UnverifiedTransaction { unsigned: Transaction { nonce: 56446, gas_price: 20000000000, gas: 90000, action: Call(0xec95ac0707626db2d80c207e7d64645632f5df11), value: 3000000000000000, data: [] }, v: 37, r: 9122432188823685503716538934008477869692673871123957121431603760072167945349, s: 38948260529398761873977129523755788030561896328454058313253306059425071193711, hash: 0xc30cc86ff5c01c543a6ecd4845c84f18b17c3fa8f89da02fb7c999cd725b4cd6 }
[Tx 44] UnverifiedTransaction { unsigned: Transaction { nonce: 56447, gas_price: 20000000000, gas: 90000, action: Call(0x89fe3170fe95694902b304dd7613679d43122763), value: 3000000000000000, data: [] }, v: 37, r: 67059546018115122301683173943907000406336056749816435838695738462483633303251, s: 43545492625539348368600246600670688704146806537676658858076353788944263827776, hash: 0xfcb8f50e32c4a5dc429e828ac72ee68eae787255b93a9e2984e0655bb3319010 }
[Tx 45] UnverifiedTransaction { unsigned: Transaction { nonce: 56448, gas_price: 20000000000, gas: 90000, action: Call(0x4e7b82037ea30e8e30c28411a3dfb4f8f88d5779), value: 3000000000000000, data: [] }, v: 37, r: 74418496176330924463781170508885007225123009775663178016089794149308476049678, s: 24079400538692972361863385275308187130662334630704061527756796728932392944345, hash: 0x9503b55d35831215a0f2b9b9970afc5d0499c5040382453d3c950d9ba97a19cf }
[Tx 46] UnverifiedTransaction { unsigned: Transaction { nonce: 56449, gas_price: 20000000000, gas: 90000, action: Call(0xf259cf3493e42aa070cd60174c0816a55c25d1a7), value: 3000000000000000, data: [] }, v: 38, r: 73777374398120587275587013486640306718887060343537345000480051924459602417020, s: 45068936743320728757307055220252223245636290613151232655841529047975617090218, hash: 0xbd4b6cb5c28e48a315e0fc57022c0342d5ada81e73481e12bd991c942b46af57 }
[Tx 47] UnverifiedTransaction { unsigned: Transaction { nonce: 56450, gas_price: 20000000000, gas: 90000, action: Call(0x60d19b069eba263f4aadde1df02544a52f019a3b), value: 3000000000000000, data: [] }, v: 38, r: 90423116871272186375075528285735704241415439619028354969270904771564788592377, s: 23962547232296121871260251539132099658738574218802859817853331908129459149061, hash: 0xf6c96581a95bc63dba73ac4d259dcdc9e45fef0bdf263839ed83f5fe6225d4c9 }
[Tx 48] UnverifiedTransaction { unsigned: Transaction { nonce: 56451, gas_price: 20000000000, gas: 90000, action: Call(0x7ca8b22c514a9b355058ec216f209ca2e087789c), value: 3000000000000000, data: [] }, v: 38, r: 61425882076759169145648378177965479385449528060272680692194798359244186214060, s: 56176882883099049284840346775126884920240735583518606055542634681951041081592, hash: 0x2d294af59befd328f8833149e429d015dac36dfe9c15986f1a006bf5d2c79c65 }
[Tx 49] UnverifiedTransaction { unsigned: Transaction { nonce: 1, gas_price: 20000000000, gas: 85458, action: Call(0x0000000000085d4780b73119b644ae5ecd22b376), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63, 92, 229, 251, 254, 62, 154, 243, 151, 29, 216, 51, 210, 107, 169, 181, 201, 54, 240, 190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 20, 120, 183, 195, 10, 188, 48, 0, 0] }, v: 37, r: 63047718376600540382191801162272163867293841609407524556398742922259866858121, s: 42232598844351543315418074625499471539688140082003933788723114062817597740052, hash: 0x5ac46708e955ec4e4963a99f9fe2cc373416f541dd417cfb54f27bb3438c84e6 }
[Tx 50] UnverifiedTransaction { unsigned: Transaction { nonce: 15380, gas_price: 20000000000, gas: 67000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 11, 204, 105, 74, 84, 29, 169, 203, 228, 116, 204, 249, 5, 40, 194, 95, 27, 118, 154, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 119, 131, 222, 64] }, v: 28, r: 114034593791350355475653140417300495499634426795967808410663511159434878016869, s: 36497727074605556768308529541809831908066070461033195969119011169206559288445, hash: 0xe8c94db94b5671cc38e297c45858ef9cb30f0bdf57726e7ad9505db862519f9d }
[Tx 51] UnverifiedTransaction { unsigned: Transaction { nonce: 15593, gas_price: 20000000000, gas: 2350000, action: Call(0x14884a04608413a0837ed4d9d1fa0a33eb456400), value: 0, data: [13, 89, 181, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 218, 193, 127, 149, 141, 46, 229, 35, 162, 32, 98, 6, 153, 69, 151, 193, 61, 131, 30, 199, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 111, 90, 100, 189, 194, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 68, 169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 92, 20, 61, 26, 48, 98, 187, 98, 167, 46, 20, 86, 107, 227, 102, 22, 160, 141, 72, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 142, 131, 195, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] }, v: 27, r: 52394439768504535783188963076813655698472457457909962281996211283062054713094, s: 48081498932043049047158473889922153722583228020468755752153475408101867559532, hash: 0xe3875bdd711c6f30d6b560546bf73c840c20e4538d0e1b55be08e2153abfbfa0 }
[Tx 52] UnverifiedTransaction { unsigned: Transaction { nonce: 1355, gas_price: 20000000000, gas: 100000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 191, 216, 175, 109, 91, 33, 90, 235, 137, 178, 53, 208, 88, 144, 219, 201, 233, 202, 191, 148, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 180, 245, 210, 21] }, v: 37, r: 21478024497080341272048043021550799675616484291483394094519757313097900071174, s: 50277260874903644346890569145050979006101778333080257552479460768204065651785, hash: 0x86823dede3245cc85cff96cc7608cdc481ce38afaed3ac65467b23748359b320 }
[Tx 53] UnverifiedTransaction { unsigned: Transaction { nonce: 7598, gas_price: 20000000000, gas: 60000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63, 108, 27, 50, 36, 206, 7, 35, 127, 96, 156, 243, 204, 210, 70, 128, 140, 45, 215, 168, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 228, 238, 238] }, v: 27, r: 111307050323354131025653829882241458524553950526483928973633773571028090746944, s: 447484698475193051514044475430537568119348682656027038252789364536402762489, hash: 0xe7eb60769ec86c81b07607fb669e69ff5e4dee976faac2284d61da87cfcb659b }
[Tx 54] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 8000000000, gas: 258373, action: Call(0xbcf935d206ca32929e1b887a07ed240f0d8ccd22), value: 30000000000000000, data: [136, 83, 181, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 137, 164] }, v: 37, r: 50420944985191357794498051537783174217440489577744802141381095747874308663772, s: 38758119407932424196430294873233210604960596392430317807338888616392514256428, hash: 0x3e9e37c5ed8b42b26955c095ce132e5ea7b408afcf3b6be0b094ff6cc5100f15 }
[Tx 55] UnverifiedTransaction { unsigned: Transaction { nonce: 45515, gas_price: 20000000000, gas: 21000, action: Call(0xb49a897c934b0bef60ef6d0080bfd7a1ffacc047), value: 824180000000000, data: [] }, v: 38, r: 17824599296758790202439672081474239975167735407714419368985335646048721801028, s: 17756307985665832371540060418952690429703671100484120712833575831657395703738, hash: 0x58cd5fe4643afd0f60255825dfe1a723169d388ab12afef9b0841473ed85fa83 }
[Tx 56] UnverifiedTransaction { unsigned: Transaction { nonce: 1528, gas_price: 5400000000, gas: 220000, action: Call(0x069c66319b4faa64f48612d59e5dfc4986814fa9), value: 0, data: [194, 110, 160, 221, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 33, 171, 13, 68, 20, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 92, 143, 241, 73, 182, 137, 73, 167, 170, 5, 254, 68, 180, 143, 129, 254, 129, 155, 5, 225, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 64, 50, 52, 50, 48, 101, 98, 102, 97, 48, 53, 50, 56, 48, 99, 97, 51, 102, 51, 48, 101, 57, 53, 98, 54, 100, 48, 51, 57, 56, 48, 98, 101, 51, 49, 53, 101, 48, 100, 99, 51, 53, 97, 51, 102, 48, 101, 97, 97, 54, 101, 100, 56, 52, 52, 49, 55, 102, 51, 49, 53, 56, 53, 51, 54] }, v: 38, r: 44656297188821008093573272800730302673088170313354308018097254639123201296855, s: 28584734106201651426899651391821555693346272518485126410187207018267763816998, hash: 0x7e46c01648dbd48204e9b482a922f2a5cdd9ff64ed960fed0aeb75ff1f5e4f7d }
[Tx 57] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 20000000000, gas: 86552, action: Call(0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63, 92, 229, 251, 254, 62, 154, 243, 151, 29, 216, 51, 210, 107, 169, 181, 201, 54, 240, 190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 58, 183, 103, 128] }, v: 38, r: 28417511167676967817202413463323153446828621146868614422583902663781103470861, s: 31198557959943716761499965580072236529480353977874601654340392697998963924784, hash: 0x4e6971503c57d52d633b4c498c9d189d75e16bba35bb46c379fef1f8bc7d0068 }
[Tx 58] UnverifiedTransaction { unsigned: Transaction { nonce: 218, gas_price: 18000000000, gas: 65000, action: Call(0x72955ecff76e48f2c8abcce11d54e5734d6f3657), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 62, 154, 250, 164, 160, 98, 164, 157, 100, 184, 171, 5, 123, 60, 181, 24, 146, 225, 126, 203, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 161, 1, 7, 220, 144, 170, 26, 216, 0] }, v: 27, r: 100493089369477048476047813348174417354270121620986282367403707441302283715066, s: 53474626691111054600926067868397800090619996201514030562628115360392404350379, hash: 0x4eb11a49bf5c885596e180e0c902cd136d359aa7bfb77e06b3962b2fb2a48b68 }
[Tx 59] UnverifiedTransaction { unsigned: Transaction { nonce: 8, gas_price: 18000000000, gas: 136800, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 184, 0, 28, 62, 201, 170, 25, 133, 246, 199, 71, 226, 92, 40, 50, 78, 74, 54, 30, 193, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 72, 234, 143, 96] }, v: 27, r: 11130284173606909793558532002178843015042966212833078290729618051078120162122, s: 21214808619669153793089125728360313425213634675956064796111105864791450255842, hash: 0xfdd67216730af2bf9cab41617951323169f9f6f4b3250587fab1f6fb177ed909 }
[Tx 60] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 18000000000, gas: 136800, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 184, 0, 28, 62, 201, 170, 25, 133, 246, 199, 71, 226, 92, 40, 50, 78, 74, 54, 30, 193, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 76, 107, 231, 96] }, v: 28, r: 17392876422273586342707999904739533464416839773051971829716774257206855910275, s: 29473844719601338036847205263784610511371849954130059978692848630005324281403, hash: 0x28b0234fc8655547809a050a2104a7a02130634e641bc5c9ec738bf1e9f5aa37 }
[Tx 61] UnverifiedTransaction { unsigned: Transaction { nonce: 12, gas_price: 18000000000, gas: 136800, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 184, 0, 28, 62, 201, 170, 25, 133, 246, 199, 71, 226, 92, 40, 50, 78, 74, 54, 30, 193, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 201, 88, 48, 128] }, v: 27, r: 18836929443659101566477069861057690040275936202224659747364324949739067527350, s: 12913032061597798913475726004896200903018347260410133803629528585981972900985, hash: 0x8a4a082fa8980ec902d13b14e6c2d548e812880dfcfc252e510cfc8ead1a5a22 }
[Tx 62] UnverifiedTransaction { unsigned: Transaction { nonce: 33, gas_price: 17750000000, gas: 100000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 14, 88, 167, 101, 33, 67, 160, 162, 117, 117, 14, 90, 197, 122, 79, 177, 78, 180, 161, 141, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 35, 183, 56, 16] }, v: 28, r: 17022528909308197076308926861690219047248877400582962825611325553002303143408, s: 52446768754980882721106944046390760258402352829803750668822004835468367684901, hash: 0xac25e0f12d6c013459b693067af1ffb836aa36f047965f0d32eaffaa38fbd248 }
[Tx 63] UnverifiedTransaction { unsigned: Transaction { nonce: 12469, gas_price: 16000000000, gas: 60000, action: Call(0x5b8174e20996ec743f01d3b55a35dd376429c596), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 159, 167, 0, 245, 174, 195, 51, 200, 154, 228, 52, 19, 165, 238, 239, 66, 100, 117, 208, 174, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 223, 204, 249, 123, 50, 91, 192, 0] }, v: 38, r: 17428800317555358168346571781763226194455038541684187871500346780623685192715, s: 17247584235333987265177754880616166347902188295909231348093377412732051493903, hash: 0x8a2bf5b95e2068095085043a6dd5d995694dd2ee0d186233d258aef0211863e0 }
[Tx 64] UnverifiedTransaction { unsigned: Transaction { nonce: 12486, gas_price: 16000000000, gas: 60000, action: Call(0x5b8174e20996ec743f01d3b55a35dd376429c596), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 89, 247, 166, 185, 116, 10, 107, 220, 19, 218, 59, 131, 202, 53, 199, 243, 104, 194, 67, 233, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 88, 208, 138, 227, 48, 158, 0, 0] }, v: 38, r: 115214924833703837916972226074965903015220577079573782372258489140996981927229, s: 9950663824433285772423505335140150550648559650855132398874738866717791245484, hash: 0x11ab230f0deade819f55b73fbfc853c86536c0b9c308019f6128bd82d98bb65e }
[Tx 65] UnverifiedTransaction { unsigned: Transaction { nonce: 11838, gas_price: 16000000000, gas: 60000, action: Call(0x5b8174e20996ec743f01d3b55a35dd376429c596), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 59, 42, 10, 141, 218, 126, 205, 66, 151, 152, 235, 150, 168, 241, 187, 129, 82, 69, 35, 237, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 223, 217, 219, 79, 28, 80, 96, 0] }, v: 38, r: 3643908014555598710836352697254805687685283544355454974660305538961404591388, s: 26215661584399764411555586830350380949882240273364554823431016696403824507890, hash: 0x0dee948f699dc684437aa6c84bd643c645174f8149c3004800082db1f2a57d5a }
[Tx 66] UnverifiedTransaction { unsigned: Transaction { nonce: 188, gas_price: 15996093750, gas: 21000, action: Call(0x6ba533dcc1bac9595bd764f93fcbc7ec81410441), value: 945862696102583813, data: [] }, v: 38, r: 46550154463383464147965920927672333031808919153864752688404090004097741091148, s: 24099554833178292440730081894445967889922116202023899610724493605492313294685, hash: 0xb4d69b7c12e54850791ec9877bf4f6952b575addbb2d6d4bbea1820be7003d09 }
[Tx 67] UnverifiedTransaction { unsigned: Transaction { nonce: 2357, gas_price: 15000000000, gas: 90000, action: Call(0x876eabf441b2ee5b5b0554fd502a8e0600950cfa), value: 873151430000000000, data: [] }, v: 38, r: 7950616264044187468814551055118309822860666097534796520393350661318587618479, s: 26869648232807847135572644664532471649358471261060618235332389309476831048021, hash: 0x693b6af802a68f81a3c3ea1d1b510d7448f8aa63362a722168eef5b5b5271921 }
[Tx 68] UnverifiedTransaction { unsigned: Transaction { nonce: 25719, gas_price: 12000000000, gas: 21000, action: Call(0xbdd8d9f16057abd5324f267bf4cf5987c87a0f6d), value: 100038800000000000, data: [] }, v: 37, r: 74299492140794065732021103736359323634303502543175305245699913007567484108255, s: 55773456937602040901208362872799682150907418727060266902542658452920886084817, hash: 0x4b076ffb557b8ca52ddd52fd2e2f6929491dd0eac8aa4f0fffa31d5055c10d26 }
[Tx 69] UnverifiedTransaction { unsigned: Transaction { nonce: 52955, gas_price: 12000000000, gas: 97616, action: Call(0x121effb8160f7206444f5a57d13c7a4424a237a4), value: 0, data: [57, 18, 82, 21, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 242, 129, 232, 90, 11, 153, 46, 250, 95, 218, 79, 82, 179, 86, 133, 220, 94, 230, 123, 234, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 38, 219, 153, 42, 59, 24, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 192, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 94, 20, 20, 178, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 44, 91, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 65, 124, 114, 94, 221, 138, 224, 208, 235, 181, 78, 225, 41, 206, 75, 24, 110, 210, 172, 130, 16, 127, 91, 46, 215, 40, 4, 249, 180, 239, 191, 10, 101, 52, 232, 67, 170, 180, 235, 149, 48, 15, 74, 183, 99, 194, 109, 146, 114, 131, 198, 250, 90, 186, 183, 12, 69, 81, 125, 24, 166, 168, 176, 149, 44, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] }, v: 28, r: 23765951055147655635382104858916268933980130882730554006658936530490326536591, s: 16257580927490270931807567007043534029238167205481484533547206382126368885730, hash: 0x509f1f86368db0dd3e83212b4be809365800610714cacb28ca1fde0e1c59b1dc }
[Tx 70] UnverifiedTransaction { unsigned: Transaction { nonce: 34, gas_price: 12000000000, gas: 90000, action: Call(0xcadb96858fe496bb6309622f9023ba2defb5d540), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 143, 32, 160, 126, 5, 65, 202, 45, 185, 21, 45, 126, 82, 26, 238, 93, 99, 155, 33, 29, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 215, 247, 128, 29, 16, 208, 196, 196, 0] }, v: 37, r: 59005407145095704119976173411714840589217146701133619335551347877586018855173, s: 21781734654726194700904785875297798016502893110389741888250628970958326493217, hash: 0xeffe3037f0c4c2cbd69d817a418e6106978ec2f0a60b3e27f4188178a005b669 }
[Tx 71] UnverifiedTransaction { unsigned: Transaction { nonce: 105352, gas_price: 12000000000, gas: 90000, action: Call(0x65976a99eecea78f38ab18907e77cbcff81d8151), value: 1500000000000000, data: [] }, v: 38, r: 36796611524588058723808039693055633357528466647124260225488956999065017215473, s: 43385735748104778700426401549429677609984437768887891206833688620781410737350, hash: 0x5295d437285524beeb87fffa13f55f7c470b04bd6012e2c953735543eacd8120 }
[Tx 72] UnverifiedTransaction { unsigned: Transaction { nonce: 2475, gas_price: 11000000000, gas: 1500000, action: Call(0x0ba45a8b5d5575935b8158a88c631e9f9c95a2e5), value: 0, data: [104, 193, 128, 213, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 96, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 50, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63, 4, 207, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 19, 54, 50, 57, 48, 54, 52, 56, 54, 54, 50, 55, 55, 50, 53, 49, 55, 56, 53, 49, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] }, v: 37, r: 51329037749519054575278736741340085735554386439800715899221853479134041164012, s: 7398486740244478915553240372928249739605912863774462102059766493806620618527, hash: 0xed04b960986370ec6b7b63aa5be82b9e7c555df0895c312f0b29d833010d22d6 }
[Tx 73] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 10000000000, gas: 60000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 87, 160, 15, 11, 102, 7, 117, 74, 242, 126, 255, 91, 119, 99, 44, 243, 201, 241, 0, 199, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 34, 76, 23, 135] }, v: 38, r: 49018923841126528874435910061491493525476044353914422156664977119639919604652, s: 55804716522437135847197140539542395464665044161631932910266366425117665104505, hash: 0x706c1f86ee2e63e88f0f214ad0b3dbf33a7e9d1c2fed2e2a69d5461b28db6c6d }
[Tx 74] UnverifiedTransaction { unsigned: Transaction { nonce: 1520238, gas_price: 10000000000, gas: 65000, action: Call(0xd1ceeeeee83f8bcf3bedad437202b6154e9f5405), value: 0, data: [202, 114, 44, 220, 158, 125, 162, 90, 156, 56, 157, 100, 163, 137, 7, 170, 67, 32, 98, 32, 125, 73, 222, 61, 60, 182, 52, 151, 254, 255, 96, 140, 128, 211, 35, 188, 254, 65, 41, 197, 200, 28, 80, 215, 242, 41, 56, 150, 235, 92, 222, 34, 23, 208, 85, 160, 167, 248, 169, 242, 103, 90, 198, 221, 60, 21, 59, 75] }, v: 27, r: 110332218919011250214541875031799303100475610306822785040268601524799591473953, s: 41005952129363053207328861790189504897199785686744574984873046480251997667494, hash: 0x9273eb73104f525eaa61c28b5ad84821f0c2ae65d57fa7a479e76fcb22b7c30c }
[Tx 75] UnverifiedTransaction { unsigned: Transaction { nonce: 1520239, gas_price: 10000000000, gas: 65000, action: Call(0xd1ceeeeee83f8bcf3bedad437202b6154e9f5405), value: 0, data: [202, 114, 44, 220, 191, 179, 41, 158, 63, 28, 128, 127, 196, 160, 35, 55, 68, 142, 71, 6, 187, 143, 76, 153, 168, 67, 148, 195, 217, 29, 82, 66, 240, 216, 74, 75, 169, 34, 27, 115, 20, 122, 143, 18, 218, 59, 178, 120, 48, 81, 121, 143, 150, 113, 68, 164, 169, 59, 131, 133, 17, 220, 177, 159, 142, 144, 161, 163] }, v: 27, r: 115542675540968752880526541466894576040632987737266654688502684968187793250381, s: 28139963885032997692929877846967361023528972142918384680191836039825489889867, hash: 0xfb9f076453afb5dfd46e5eda0d1eb620c2572d1aa1e3f3eaacf0cb8d3e23325d }
[Tx 76] UnverifiedTransaction { unsigned: Transaction { nonce: 453, gas_price: 10000000000, gas: 90000, action: Call(0x57e0c81d89a1526bc01d8ee1e695cfb586db0a55), value: 2000000000000000, data: [] }, v: 37, r: 81673988577012898422200220198114903071437889583618101495895412799545844155563, s: 4349560877969703252874216083440467011818468768821575539104550566441100936203, hash: 0xdfce78526f8305f08d79f17529df96e6f65684a7f47734faf3aa11443c6b8d9a }
[Tx 77] UnverifiedTransaction { unsigned: Transaction { nonce: 4, gas_price: 10000000000, gas: 100000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 143, 184, 193, 141, 211, 206, 58, 14, 201, 87, 83, 58, 99, 236, 227, 151, 80, 231, 210, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 250, 240, 128] }, v: 27, r: 16956858550007934157627062125072327464661849924194117447413827923210936632143, s: 20202463320490970353529171866998535600306523515543744879665519015580024021499, hash: 0xa7dc9bb795e6734967bd584d69e503d3ec346ab1cb5dc79f7863af8e854e61e2 }
[Tx 78] UnverifiedTransaction { unsigned: Transaction { nonce: 1117, gas_price: 10000000000, gas: 37228, action: Call(0x7d1afa7b718fb893db30a3abc0cfc608aacfebb0), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63, 92, 229, 251, 254, 62, 154, 243, 151, 29, 216, 51, 210, 107, 169, 181, 201, 54, 240, 190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 139, 42, 44, 40, 2, 144, 148, 0, 0, 0] }, v: 37, r: 91103039537974587975142450234079160729087698621468558327122604681167107126096, s: 39407923060651688050700687243341150279233230417709038517586322481411192344933, hash: 0x2ffeff466724b4fcb07b3e402f46ef52ff16d30a8805350e74341f9625b449b9 }
[Tx 79] UnverifiedTransaction { unsigned: Transaction { nonce: 9, gas_price: 10000000000, gas: 90000, action: Call(0xc8422a8a386eb49ecbf060198938df163c188406), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 73, 155, 198, 180, 185, 231, 218, 31, 216, 167, 117, 16, 106, 56, 159, 76, 29, 29, 75, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 27, 26, 228, 214, 226, 239, 80, 0, 0] }, v: 38, r: 56925860898933739069669093448581193221294612339821502668010841737695464016600, s: 22703721369253395942599127780468978619087005866591658880474376034939790813697, hash: 0x8d4d0ae840656a88863c7a0fa86e11a7a39f85e398b5be74c86765a28aa4a0b6 }
[Tx 80] UnverifiedTransaction { unsigned: Transaction { nonce: 346808, gas_price: 10000000000, gas: 21128, action: Call(0x23b1dcf95efa3f1d8ab7eca2903d8839e25ea389), value: 9000000000000000, data: [232, 214, 193, 128, 12, 190, 143, 147] }, v: 38, r: 76390746314440268275227179746748018101410796653443462767317093823268138764072, s: 44877176803773591698277555490910754898396320321896456938126499115518537957348, hash: 0x51b95346d5baf981b66eb5f532f85950b70a6f882f8c87b86c687d885be932f7 }
[Tx 81] UnverifiedTransaction { unsigned: Transaction { nonce: 42, gas_price: 10000000000, gas: 21000, action: Call(0x3e5abd204b7df2afabe7d25ee5f58eeded353e1c), value: 1000000000000000, data: [] }, v: 37, r: 63220891067536679030329156557543533259778295885636749926719485917859276536253, s: 12466797626367706792588023984530440743372970275644378958288263351555752120071, hash: 0x808d05f07bfe40e630f3e22d24fff1a756c44045172e29c08135e56d7f6d362d }
[Tx 82] UnverifiedTransaction { unsigned: Transaction { nonce: 246, gas_price: 10000000000, gas: 100000, action: Call(0x07a88907b8b46a2124bb49cc67ce65dbe06e8a1b), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 125, 52, 77, 64, 150, 57, 148, 225, 200, 36, 109, 197, 166, 77, 172, 166, 207, 8, 207, 33, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 141, 190, 192, 224, 0] }, v: 27, r: 52017001527506632371468616796484518149805315550646085345569239163423934029900, s: 55625355488707984623549909554431500090271625323396589125412414403584581649054, hash: 0x286f07dbfc19d272d81b06ae8e3c8c7984772eb14609ffc261927887abee9904 }
[Tx 83] UnverifiedTransaction { unsigned: Transaction { nonce: 67588, gas_price: 10000000000, gas: 4300000, action: Call(0x33fbc07d9d9456d9e6d8316aa87792a20d3ad448), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 193, 134, 83, 174, 149, 220, 228, 221, 148, 184, 24, 82, 239, 46, 101, 249, 186, 133, 206, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 32, 247, 252, 29, 77, 193, 112, 128, 0] }, v: 28, r: 40594432887757822084230022527553788193890168322141113649843065596234471809186, s: 46838551896610693425031595954051200547452407520160263413098271974219552268668, hash: 0x12d8a71feb7dca2659033e03bed3ba6703696076e22ac0bbefcf4cd5e40f8907 }
[Tx 84] UnverifiedTransaction { unsigned: Transaction { nonce: 25, gas_price: 9195312500, gas: 21000, action: Call(0x6b151d0c4fdb7980b40152ee08380ff7299fcbfe), value: 30000000000000000, data: [] }, v: 38, r: 10351525541215941077599307921406251639101961816224733246985825607965711396079, s: 50950555458823648149362957908894760240496605537302943098687535917258500371614, hash: 0x2f7d94bcadf5032ae58ab1559db32800c6c273e254fbd12e92d0f63f00661423 }
[Tx 85] UnverifiedTransaction { unsigned: Transaction { nonce: 4536539, gas_price: 9000000000, gas: 400000, action: Call(0x2a0c0dbecc7e4d658f48e01e3fa353f44050c208), value: 0, data: [239, 52, 53, 136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 56, 157, 59, 35, 30, 19, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 86, 7, 59, 157, 249, 187, 243, 236, 190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 39, 16, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 226, 228, 1, 67, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 56, 157, 59, 35, 30, 19, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 226, 228, 4, 215, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 141, 126, 164, 198, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 17, 141, 66, 216, 229, 132, 43, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 251, 120, 75, 126, 214, 103, 48, 232, 177, 219, 217, 130, 10, 253, 41, 147, 26, 171, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 232, 32, 126, 172, 93, 165, 45, 150, 37, 228, 63, 95, 182, 221, 187, 115, 16, 244, 174, 91, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 232, 32, 126, 172, 93, 165, 45, 150, 37, 228, 63, 95, 182, 221, 187, 115, 16, 244, 174, 91, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 27, 80, 161, 189, 136, 90, 101, 74, 136, 73, 149, 93, 17, 247, 41, 98, 206, 88, 137, 48, 22, 142, 255, 96, 148, 241, 198, 168, 246, 61, 44, 102, 119, 74, 82, 160, 175, 48, 198, 92, 247, 27, 128, 208, 202, 30, 56, 82, 106, 75, 44, 20, 153, 136, 14, 142, 170, 34, 42, 121, 233, 13, 208, 167, 99, 104, 129, 19, 47, 141, 231, 141, 246, 247, 157, 98, 149, 52, 97, 185, 221, 215, 99, 107, 50, 194, 210, 98, 254, 228, 64, 102, 66, 76, 1, 107, 13, 7, 200, 215, 84, 253, 110, 31, 157, 46, 85, 159, 14, 178, 233, 124, 239, 82, 52, 98, 2, 1, 86, 27, 92, 74, 112, 130, 25, 113, 247, 194, 191] }, v: 37, r: 6187846982191572631004888916199479732991172029688892949238719350077363661547, s: 46591652444787896556825038033559093560613930693598609406730043827004851661828, hash: 0x93e406590f3abacbee97b3528baa4a6910febd991de4337929aaf2f55a373922 }
[Tx 86] UnverifiedTransaction { unsigned: Transaction { nonce: 8, gas_price: 9000000000, gas: 41221, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 117, 232, 157, 89, 121, 228, 246, 251, 169, 249, 124, 16, 76, 47, 10, 251, 63, 29, 203, 136, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 40, 24, 191, 221] }, v: 37, r: 82881484143578267224367854997286026515420638557157975745001697431211361707966, s: 30774725413549224700829117210396029363535097338333591653645341506779463447286, hash: 0x74e351ca421fd64437817c0c9072fe99d6d95c5204eebbedbb4f9e7953e84887 }
[Tx 87] UnverifiedTransaction { unsigned: Transaction { nonce: 3, gas_price: 8000000000, gas: 600000, action: Call(0x7c5cb1220bd293ff9cf903915732e51a71292038), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 174, 99, 253, 194, 202, 207, 88, 176, 16, 177, 184, 174, 121, 181, 70, 161, 201, 184, 2, 110, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 184, 19, 225, 164, 254, 29, 24, 0, 0] }, v: 38, r: 14811435345296760688403951603497526158565892372168283512896188836531228008430, s: 10537204412804009708605908218846763408468821329400679941476505301814530526477, hash: 0xe06929422d275f9f4e143eac2aa073080a04cf91b37b383da8aefab28a39fcca }
[Tx 88] UnverifiedTransaction { unsigned: Transaction { nonce: 24155, gas_price: 8000000000, gas: 100000, action: Call(0x3df2b998d1aa8acc1bcb5812828de7847a170349), value: 800000000000000, data: [] }, v: 37, r: 46147664566379285841965692174986268285957460272658022009027496376052431098242, s: 49984316935567352221733891962125067889137838967232688351318592054558304658857, hash: 0x667363762278746f067b396eecacc5ddd3407bffa0491c1330fad3ec481de030 }
[Tx 89] UnverifiedTransaction { unsigned: Transaction { nonce: 815, gas_price: 8000000000, gas: 60000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 16, 109, 102, 190, 114, 106, 68, 93, 126, 110, 79, 124, 154, 72, 87, 159, 182, 146, 36, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 29, 106, 54, 96] }, v: 37, r: 78735038197642206187992412998938819103305201872732973584453984244219944056392, s: 10158043679052733947298953042406785244267782844998934922253032310661226173201, hash: 0xfa3130021a05d8bc9fefcd0da4bd4733e325eff5dfbf1918c92b200e109890dd }
[Tx 90] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 8000000000, gas: 21000, action: Call(0x87b6e87c763bca2f45b2b1090dce00eb897f2a38), value: 178143420000000000, data: [] }, v: 37, r: 922755342112664362468200097448292429412572920081302839598298198231208223389, s: 6190488103830627351474512597380570123601924128801052239232061485504028111027, hash: 0xf02ed0d0f31413771e34a67a0cd831a9e87deb019fb9a16c103538b2c6d482df }
[Tx 91] UnverifiedTransaction { unsigned: Transaction { nonce: 32766, gas_price: 8000000000, gas: 80000, action: Call(0xc5081471c33dac824060a4ed3b95c5f08429ae62), value: 76426800000000000, data: [] }, v: 27, r: 83869177623711702705834620243765777328266663871014510406594495046212574140667, s: 9763347501702177968936648212514653676363858648763902703530320907126737145911, hash: 0x4d916ab5c6275724e5db75b4789f8f622824115fda48197aa612f5e5e8764c46 }
[Tx 92] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 8000000000, gas: 238873, action: Call(0xbcf935d206ca32929e1b887a07ed240f0d8ccd22), value: 30000000000000000, data: [136, 83, 181, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 134, 205] }, v: 37, r: 8582420396767885633206430981418926917424713458613018274853628358200033430927, s: 8667562084945718499446491862016754199493249335981012433491387690643420030794, hash: 0xa9cb990d628f39d1325ba9e71e8edfa4849cb0ae883dea3fcc2027775d941630 }
[Tx 93] UnverifiedTransaction { unsigned: Transaction { nonce: 156, gas_price: 8000000000, gas: 60187, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [9, 94, 167, 179, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 14, 238, 62, 56, 40, 164, 95, 118, 1, 213, 245, 75, 244, 155, 176, 29, 26, 157, 245, 234, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255] }, v: 38, r: 36187525420229394307122078089786369270451298468142825344734668681433603495962, s: 20334605310261137835189882475590266555590380441392284444494016422727234093949, hash: 0x0d17853c59694682ae6f4d7559cacb087e8113e5dfdcb52c0b52a175bc599e25 }
[Tx 94] UnverifiedTransaction { unsigned: Transaction { nonce: 48, gas_price: 8000000000, gas: 60000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 172, 95, 112, 149, 4, 99, 172, 149, 244, 154, 117, 223, 36, 55, 198, 32, 235, 18, 58, 115, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 186, 129, 64] }, v: 37, r: 16016805862115431284819325941056729173397291431945840643038792129557030161544, s: 24149084114126096445516222156364076955478643496195651218633748763949429304636, hash: 0xb52e8d15127b27bc543f36cd9a00b1403524c924cea2fb47976eb56b833db63b }
[Tx 95] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 8000000000, gas: 21000, action: Call(0xc76535cb84ade872535f0a3baa4d9a70a47fa38a), value: 419119210000000000, data: [] }, v: 38, r: 23797522916083637607228056269848953433301417942758309973153534664843089364222, s: 25127752352231865449916760166989498260182640832256639317653677736252991459245, hash: 0x4811904545d43298a1ef66b1dafa5c0468c7836e190ef53dc5259c721a00da70 }
[Tx 96] UnverifiedTransaction { unsigned: Transaction { nonce: 430, gas_price: 8000000000, gas: 60000, action: Call(0xdac17f958d2ee523a2206206994597c13d831ec7), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 134, 22, 129, 28, 203, 220, 65, 122, 27, 133, 28, 69, 41, 182, 160, 229, 214, 1, 108, 155, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 17, 27, 69, 192] }, v: 37, r: 59541565567667536354486530769976797395897327516278478566817839288883731992291, s: 56999906780305805938083965786701736155248250520031645727188525565965375758872, hash: 0x45089c538e15f4e84e79b92cb5dd3fb128c5ff1e07c3d7be9d04bb2d86b20589 }
[Tx 97] UnverifiedTransaction { unsigned: Transaction { nonce: 582, gas_price: 8000000000, gas: 21040, action: Call(0x1f973b233f5ebb1e5d7cfe51b9ae4a32415a3a08), value: 121827996029027684, data: [] }, v: 27, r: 76996411490199710269261089914192476564239397468328656866631305562372811022902, s: 12615846451398844702957735335683717286716300437891814430090774683867821683603, hash: 0x2ccf8142165b7fa37be4a4c6bd3b81d30026438e77bdfaacfb92b48efafc0a16 }
[Tx 98] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 8000000000, gas: 21000, action: Call(0x48c0aa2d5e71a7e83204ff681b6304513cdfa4ff), value: 363996850000000000, data: [] }, v: 38, r: 64901073314524736443424467041737202447704934573584370004194902307199743990113, s: 53398738797520785048862478095984843593471497858927393845210593443402238917139, hash: 0x64d1340e1f3b08ff39e7562e0acb580433e3e91f87b4c6498463da4290eacbf3 }
[Tx 99] UnverifiedTransaction { unsigned: Transaction { nonce: 19, gas_price: 8000000000, gas: 97287, action: Call(0x34848f3c3934733d3ebc6f82489346592566c777), value: 0, data: [113, 93, 14, 51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 39, 193, 36, 128, 38, 210, 255, 193, 99, 117, 152, 130, 117, 59, 173, 96, 14, 122, 183, 177] }, v: 38, r: 17957366523862900408416756597583176158392938831647963399307900045446342731752, s: 57330086693599105446015700890881961880501049378287517555727173552633416078546, hash: 0xdb0ed63b926d1f97ca3fb28b64c178fada212b057067d26bb4d124a20041b069 }
[Tx 100] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 8000000000, gas: 21000, action: Call(0x9045c39c4ebdf00761c718a2867cb172b883f249), value: 111000000000000000, data: [] }, v: 37, r: 86448845892781229211223714820210162734209303820097480646617555365377692509052, s: 44896557793476492027401825131316389650644964183341785231481957917894009898581, hash: 0x05f53dd37daf36683a73785ae62359ffdff5afca3c10ad22efa6e45d98663555 }
[Tx 101] UnverifiedTransaction { unsigned: Transaction { nonce: 24, gas_price: 8000000000, gas: 60000, action: Call(0x4b401c84a19a41fff2d8dda140661af931760527), value: 0, data: [169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63, 211, 176, 164, 218, 64, 140, 200, 32, 66, 245, 107, 77, 62, 101, 166, 131, 202, 11, 65, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 211, 237, 122, 9, 190, 88, 185, 128, 0, 0] }, v: 38, r: 66076542681042239696064292527127976136744916100444395797542199513896735181941, s: 43548492693517531733598195119532127881264816280518309036377469854464432307551, hash: 0x9bda3f4f97e360503acdb3fd2ddc3647cac62e397bc137c8b47ecdfc2ed6a020 }
[Tx 102] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 8000000000, gas: 258373, action: Call(0xbcf935d206ca32929e1b887a07ed240f0d8ccd22), value: 30000000000000000, data: [136, 83, 181, 62, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 137, 164] }, v: 37, r: 50420944985191357794498051537783174217440489577744802141381095747874308663772, s: 38758119407932424196430294873233210604960596392430317807338888616392514256428, hash: 0x3e9e37c5ed8b42b26955c095ce132e5ea7b408afcf3b6be0b094ff6cc5100f15 }
[Tx 103] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 8000000000, gas: 90000, action: Call(0xf996eadfa33ab0692a314fae20d873d680d7918f), value: 163176450000000000, data: [] }, v: 37, r: 3306381637136691838942149556173578605041046599504312428855752296026786874607, s: 713208719443112599635621490322895451890195454408989357232685770025961869644, hash: 0x1d00943ec8dce5803eb593750f215e416586e3ad7eb489f902747ebd4da1f015 }
[Tx 104] UnverifiedTransaction { unsigned: Transaction { nonce: 115837, gas_price: 8000000000, gas: 23000, action: Call(0x57820e0ec07ccc3a8e9e0803b47ec0bb76d1d0dc), value: 338552000000000, data: [] }, v: 27, r: 5855977467054211964567437629400473751222016457398132319353495523876788025813, s: 52809756925843263545117779513079010199167481315323405639343598445895628068289, hash: 0x6e213de2ba40a673aaa36c807df9197e6072b73e2936d359c6c0c31d942d823a }
[Tx 105] UnverifiedTransaction { unsigned: Transaction { nonce: 47, gas_price: 8000000000, gas: 21000, action: Call(0x92114693b5bda8c2c50aa0e7d0f333cc30f2e527), value: 3650000000000000000, data: [] }, v: 37, r: 49434252997710573901430475993205165944199686898331260903913361824348759163576, s: 8800059624991816880293641620364295760471644153453577344366134395748933484185, hash: 0x7577f25c1faf18bfa4f93cb680eea36d54959154554fc0643c53c3bd9cc9ba82 }
[Tx 106] UnverifiedTransaction { unsigned: Transaction { nonce: 486, gas_price: 8000000000, gas: 21040, action: Call(0x1f973b233f5ebb1e5d7cfe51b9ae4a32415a3a08), value: 68017391499137967, data: [] }, v: 28, r: 16770251243173916279165291598961339123905978518069977067174077610446732234004, s: 5552211663742648801661137587266031651743769846840141846023323445680501190878, hash: 0x14cb2c0f4efd81f23756d99fa8abb2f16b0e86ae65bfe6f69b82c507eff0036f }
[Tx 107] UnverifiedTransaction { unsigned: Transaction { nonce: 997, gas_price: 8000000000, gas: 21000, action: Call(0x93b0c4f94b1c119fd9972adc22b7b1d2472d8abb), value: 1000600000000000000, data: [] }, v: 27, r: 78538756929843764851162050660994967985885891419931583569551414047658989631124, s: 15151548001026079073848105701130564880497729621998966766494682182731355661571, hash: 0xa75ae7e1ec75dae8a5addd16d04569e840203e0726c2fbf0f0cbf03ea38ec384 }
[Tx 108] UnverifiedTransaction { unsigned: Transaction { nonce: 244, gas_price: 8000000000, gas: 200000, action: Call(0x2e63b6d7ef8fc8a2ea4c96bb9a633423682f0f79), value: 0, data: [163, 231, 108, 15] }, v: 38, r: 35238052791119376053891456612058750535052364827147803398958333352484859256759, s: 53141692931840359457731960038661646194681740695276765229962226775448160914884, hash: 0xe998bd26232aa10c8aa7191bd6d7609a8dbb1bdd2b7d685deedee2d69de01c30 }
[Tx 109] UnverifiedTransaction { unsigned: Transaction { nonce: 4, gas_price: 8000000000, gas: 200000, action: Call(0xbb99a5b15fc7e5e8273cc3e380741f1899485aea), value: 0, data: [48, 89, 84, 65, 55, 87, 77, 66, 82, 113, 65, 119, 98, 88, 110, 89, 88, 110, 53, 77, 56, 97, 81, 72, 121, 49, 114, 115, 101, 109, 77, 56, 122, 107, 112, 103, 101, 103, 54, 109, 104, 55, 57, 68, 122, 65, 107, 101, 54, 80, 99, 116, 54, 70] }, v: 37, r: 97666096547146497947439949136994705920662112319964826161647786874041450647757, s: 32908253143518308047116463061277194848154260414667620717262077616717070277652, hash: 0xc3a026d78de40cc9f9082281ce9bc63f10b001f9518c4f286cc730453b4dc71f }
[Tx 110] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 8000000000, gas: 21000, action: Call(0x989eca3429b083931a1c91278995253a7c91f8e7), value: 10922250000000000, data: [] }, v: 37, r: 48636974774853456330207103272684128070387629490503334586799803743202919778251, s: 39601261379159381481991352778148800174782898319033212798630412089526589395933, hash: 0x0defac7ff739e97ea48225fdf1c1ccdb8fc8aa64fed84b1fecf97a24ba686020 }
[Tx 111] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 8000000000, gas: 21000, action: Call(0xbca1460e801bb38ec421ccf711d73e1a62bdb349), value: 1146917700000000000, data: [] }, v: 37, r: 9189864313783409731180805569420405408098346400268067530868301628327214295485, s: 40336371921168326529902182320380426034185898831246530710514070243708772731353, hash: 0xcd39284d8f8d8938a569d8069c90a05183a0e7a4ac9f2b32516185b0b5fd7008 }
[Tx 112] UnverifiedTransaction { unsigned: Transaction { nonce: 24897, gas_price: 8000000000, gas: 21000, action: Call(0x373570ca9f74edb28a92a40cb5d5843768f636d1), value: 2133800000000000000, data: [] }, v: 38, r: 97506787158546413740328322845821191918579721704426780869561229393442158349864, s: 53966026327469475947306962505818647909527354040214635148890222406426437403486, hash: 0x4f8ad2e095eb3d96f36fc3e3c6ba262bf0bc5f676dc4758146ad9df930ed556c }

2019-12-31 05:19:38 UTC   14/75 peers      8 MiB chain  366 MiB db  0 bytes queue   23 MiB sync  RPC:  0 conn,   16 req/s, 1390 µs
2019-12-31 05:20:08 UTC   28/75 peers      2 MiB chain  366 MiB db  0 bytes queue   22 MiB sync  RPC:  0 conn,   16 req/s, 1723 µs
2019-12-31 05:20:38 UTC   15/75 peers      4 MiB chain  366 MiB db  0 bytes queue   23 MiB sync  RPC:  0 conn,   15 req/s, 1696 µs
2019-12-31 05:21:03 UTC Syncing #9189356 0x9fd8…bd57     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #9189101   16/75 peers      4 MiB chain  366 MiB db  0 bytes queue   22 MiB sync  RPC:  0 conn,   15 req/s, 1693 µs
2019-12-31 05:21:08 UTC Syncing #9189356 0x9fd8…bd57     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #9189167   18/75 peers      4 MiB chain  366 MiB db  0 bytes queue   23 MiB sync  RPC:  0 conn,   14 req/s, 1631 µs
2019-12-31 05:21:13 UTC Syncing #9189356 0x9fd8…bd57     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #9189351   17/75 peers      4 MiB chain  366 MiB db  0 bytes queue   23 MiB sync  RPC:  0 conn,   15 req/s, 1534 µs
2019-12-31 05:21:18 UTC Syncing #9189356 0x9fd8…bd57     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed  #9189351   19/75 peers      4 MiB chain  366 MiB db  0 bytes queue   24 MiB sync  RPC:  0 conn,   14 req/s, 1565 µs

Just had this issue that then put Parity into a loop trying to sync block #9189356. Just upgraded to 2.6.8-beta for hard fork.

Serpent6877 on 31 Dec 2019

I've been testing different toml configuration, same problem. BUFF CACHE is slowly eating all the mem space after each parity restart. since version 2.6.8 beta

macdasi on 9 Jan 2020

Hi, I think the same problem remains in v2.7.2-stable...I'm running a PoA in a AWS server 4GB RAM and some validators crash repeatedly. I'm running it with docker-compose and after crashing I can see it exit with code 137 (Out Of Memory). I also check kernel logs and here is what I see:

14 23:58:50 ip-172-31-3-238 kernel: parity invoked oom-killer: gfp_mask=0x14200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 May 14 23:58:50 ip-172-31-3-238 kernel: oom_kill_process+0x21f/0x420 May 14 23:58:50 ip-172-31-3-238 kernel: out_of_memory+0x116/0x4e0 May 14 23:58:50 ip-172-31-3-238 kernel: [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name May 14 23:58:50 ip-172-31-3-238 kernel: Out of memory: Kill process 21241 (parity) score 913 or sacrifice child May 14 23:58:50 ip-172-31-3-238 kernel: oom_reaper: reaped process 21241 (parity), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

Some comments above thinks it is because of RPC calls and I support it...I'm going to explain why:

I've been experiencing this crashes since november (versions 2.5.x, 2.6.x and 2.7.2). In november I was testing a blockexplorer which builds a database by its own (with constant rpc-calls) and I forgot I didn't stop it! After some months breaking my head trying to know why my validators where crashing I read this issue and remembered this test-blockexplorer...I'll comment if they stop crashing but I'm almost sure this was the reason because my validators were crashing even when no real users where using the blockchain...

I don't know if this issue is still under review or not...

gperezalba on 15 May 2020

Closing the issue due to its stale state

rakita on 27 Jul 2020

Parity-ethereum: Memory usage spike induced crash

Most helpful comment

All 130 comments

Some debugging info while v2.4.5 was running:

19:07

19:13

ReadOPS vs. NetworkOut

7 days runtime: memory usage

same graph over 90 days for the bigger picture

7 days runtime: memory usage

same graph over 90 days for the bigger picture

log

Related issues