Blockscout: JSON RPC Endpoint Variant

[Eika4]

No Endpoint Variant available for aleth (formerly cpp-ethereum).

Need to add RPC endpoint variant configuration for aleth.

[igorbarinov]

@Eika4 which chains are using aleth at the moment? I'm trying to figure out the use case
What is the difference in API between aleth vs geth/partiy

[Eika4]

We have forked cpp-ethereum as a starting point for the Petrichor blockchain. We were able to get blockscout working by making a couple adjustments to apps/ethereum_jsonrpc/lib/ethereum_jsonrpc/receipt.ex
Changes can be seen here: https://github.com/poanetwork/blockscout/compare/master...petrichor-dev:petrichor#diff-82f44fdd597b763a7f1d8ad48dd3e0ad

The only issue we have now is that cpp-ethereum does not have websocket support yet, so blocks are only imported after restarting blockscout. Is there any way to automatically import new blocks without the websocket?
The Petrichor/blockscout explorer can be seen here: https://explorer.petrichor-dev.com/

[goodsoft]

Is there any way to automatically import new blocks without the websocket?

We are working on making websocket connection optional right now.
The update is on the way.

[goodsoft]

@Eika4 I have created a PR with a fix that makes websockets optional, please give it a try.
Note, that you have to set subscribe_named_arguments.transport to nil in config to disable websocket.

[Eika4]

@goodsoft The fix does enable new blocks to be imported without restarting blockscout, but the browser window must be refreshed to see the new blocks. Still a big improvement.
There is a message at the top of the screen "Indexing Tokens - We're indexing this chain right now. Some of the counts may be inaccurate." that never goes away, and I wonder if this is related to the websocket being unavailable.
Also, all transactions have "(Awaiting internal transactions for status)" next to them, which may be related to the websocket too.
These messages can be see on the explorer site: https://explorer.petrichor-dev.com

[goodsoft]

but the browser window must be refreshed to see the new blocks

The browser fails to connect to the websocket of Blockscout app server:

Considering you have nginx there, you might want to check, whether the following lines are present in the virtual host config:

proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";

[goodsoft]

There is a message <...> that never goes away, and I wonder if this is related to the websocket being unavailable.

Also, all transactions have "(Awaiting internal transactions for status)" next to them, which may be related to the websocket too.

None of these are related to websockets, but rather to internal transactions fetcher.
Internal transactions are fetched using different APIs for different nodes, and the default is ganache for dev environment, and parity for prod and test.
cpp-ethereum seems to be similar to geth variant, so you should run the Blockscout app server with the following environment variable:

ETHEREUM_JSONRPC_VARIANT=geth

Also, please check, whether internal transactions fetcher logs any errors:

grep internal_transaction logs/*/*.log

[Eika4]

I have switched to the Geth variant, and added the two lines above to the nginx config. I am still required to refresh the browser in order to see new blocks.
I am receiving errors from the internal transactions fetcher:

2019-04-23T20:32:49.267 fetcher=internal_transaction count=10 [error] Task #PID<0.559.0> started from Indexer.Fetcher.InternalTransaction terminating
** (BadMapError) expected a map, got: {"gas", "0x5208"}
    (ethereum_jsonrpc) lib/ethereum_jsonrpc/geth.ex:81: anonymous fn/4 in EthereumJSONRPC.Geth.debug_trace_transaction_response_to_internal_transactions_params/2
    (elixir) lib/enum.ex:1331: anonymous fn/3 in Enum.map/2
    (elixir) lib/enum.ex:3015: anonymous fn/3 in Enum.map/2
    (elixir) lib/stream.ex:1556: anonymous fn/3 in Enumerable.Stream.reduce/3
    (elixir) lib/stream.ex:1047: anonymous fn/3 in Stream.with_index/2
    (elixir) lib/enum.ex:3350: Enumerable.Map.reduce_list/3
    (elixir) lib/stream.ex:1568: Enumerable.Stream.do_each/4
    (elixir) lib/enum.ex:3015: Enum.map/2
    (ethereum_jsonrpc) lib/ethereum_jsonrpc/geth.ex:80: EthereumJSONRPC.Geth.debug_trace_transaction_response_to_internal_transactions_params/2
    (elixir) lib/enum.ex:1327: Enum."-map/2-lists^map/1-0-"/2
    (ethereum_jsonrpc) lib/ethereum_jsonrpc/geth.ex:68: EthereumJSONRPC.Geth.debug_trace_transaction_responses_to_internal_transactions_params/2
    (indexer) lib/indexer/fetcher/internal_transaction.ex:175: Indexer.Fetcher.InternalTransaction.run/2
    (elixir) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
    (elixir) lib/task/supervised.ex:35: Task.Supervised.reply/5
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Function: &Indexer.BufferedTask.log_run/1
    Args: [%{batch: [{1942, <<116, 144, 90, 168, 154, ...

error.log:2019-04-23T20:33:35.425 fetcher=internal_transaction count=10 [error] Task #PID<0.1541.0> started from Indexer.Fetcher.InternalTransaction terminating
error.log:    (ethereum_jsonrpc) lib/ethereum_jsonrpc/geth.ex:81: anonymous fn/4 in EthereumJSONRPC.Geth.debug_trace_transaction_response_to_internal_transactions_params/2
error.log:    (ethereum_jsonrpc) lib/ethereum_jsonrpc/geth.ex:80: EthereumJSONRPC.Geth.debug_trace_transaction_response_to_internal_transactions_params/2
error.log:    (ethereum_jsonrpc) lib/ethereum_jsonrpc/geth.ex:68: EthereumJSONRPC.Geth.debug_trace_transaction_responses_to_internal_transactions_params/2
error.log:    (indexer) lib/indexer/fetcher/internal_transaction.ex:175: Indexer.Fetcher.InternalTransaction.run/2
error.log:    Args: [%{batch: [{938, <<4, 48, 94, 201, 201, 30, 13, 243, 6, 118, 243, 124, 115, 149, 254, 0, 90, 255, 191, 119, 21, 143, 21, 227, 216, 231, 201, 191, 169, 93, 113, 239>>, 6}, {938, <<68, 80, 250, 248, 12, 215, 22, 0, 126, 231, 16, 230, 151, 170, 60, 202, 152, 140, 183, 220, 216, 167, 17, 105, 74, 246, 98, 67, 4, 131, 167, 244>>, 5}, {1778, <<249, 237, 2, 145, 77, 32, 74, 4, 241, 23, 50, 214, 90, 149, 101, 190, 193, 165, 205, 231, 139, 130, 44, 58, 142, 242, 223, 120, 20, 129, 39, 18>>, 6}, {1778, <<12, 161, 141, 123, 71, 229, 69, 159, 1, 195, 153, 48, 94, 22, 126, 71, 135, 128, 135, 169, 82, 177, 108, 204, 36, 120, 112, 11, 86, 107, 38, 255>>, 4}, {1778, <<19, 105, 204, 80, 9, 112, 116, 9, 16, 203, 26, 89, 51, 55, 12, 242, 96, 110, 48, 160, 2, 79, 20, 4, 75, 102, 46, 88, 105, 118, 226, 236>>, 1}, {1778, <<28, 69, 188, 198, 10, 162, 126, 144, 200, 75, 26, 131, 81, 99, 252, 229, 88, 29, 183, 138, 66, 19, 54, 208, 155, 202, 234, 10, 126, 117, 152, 37>>, 5}, {1778, <<152, 240, 63, 185, 85, 140, 53, 152, 131, 212, 42, 106, 81, 81, 194, 222, 26, 84, 35, 81, 22, 153, 48, 144, 79, 154, 141, 202, 103, 169, 98, 27>>, 3}, {1778, <<154, 126, 227, 202, 87, 68, 166, 64, 80, 101, 124, 238, 185, 15, 136, 148, 195, 152, 245, 241, 180, 170, 55, 217, 93, 12, 63, 188, 253, 112, 36, 17>>, 2}, {1942, <<116, 144, 90, 168, 154, 29, 57, 222, 156, 199, 22, 165, 82, 34, 242, 41, 158, 164, 135, 77, 97, 128, 147, 176, 36, 245, 20, 199, 5, 107, 185, 48>>, 3}, {1942, <<207, 58, 179, 187, 163, 34, 187, 139, 76, 182, 187, 240, 233, 247, 126, 54, 228, 124, 206, 158, 86, 7, 126, 56, 240, 164, 80, 48, 194, 22, 173, 13>>, 5}], callback_module: Indexer.Fetcher.InternalTransaction, callback_module_state: [transport: EthereumJSONRPC.HTTP, transport_options: [http: EthereumJSONRPC.HTTP.HTTPoison, url: "http://localhost:8545", http_options: [recv_timeout: 60000, timeout: 60000, hackney: [pool: :ethereum_jsonrpc]]], variant: EthereumJSONRPC.Geth], metadata: [fetcher: :internal_transaction]}]
indexer.log:2019-04-23T20:32:48.353 application=indexer fetcher=internal_transaction count=10 [debug] fetching internal transactions for transactions

[goodsoft]

The lines in nginx config helped browser to successfully connect to the app server.
I don't see errors anymore, and the websocket seems to be alive.
I can't confirm, whether new blocks arrive, though, as there seems to be too large of an interval between blocks.

Regarding internal transactions, there seems to be some kind of incompatibility in internal transaction format. I'll need a bit of your help on this.

Please download this file, and run the following queries against your node:

curl --data @trace.txt -H "Content-Type: application/json" -X POST localhost:8545 > result1.txt

curl --data '{"method":"debug_traceTransaction","params":["0x74905aa89a1d39de9cc716a55222f2299ea4874d618093b024f514c7056bb930"],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST localhost:8545 > result2.txt

And attach the resulting result1.txt and result2.txt files.

[Eika4]

result1.txt
result2.txt

[goodsoft]

Sorry, I got an error in the second query, it should go like this:

curl --data '{"method":"debug_traceTransaction","params":["0x74905aa89a1d39de9cc716a55222f2299ea4874d618093b024f514c7056bb930", {}],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST localhost:8545 > result2.txt

Please try again.

[Eika4]

result2.txt

[goodsoft]

Apparently cpp-ethereum doesn't support debug_traceTransaction queries with custom JS tracer that we use for go-ethereum to match Parity's API.
I'm going to implement tracer natively in Elixir, so that it works for both variants.

[goodsoft]

@Eika4 I added a PR #1857 with native Elixir tracer, may I ask you to try it out on your branch?

[Eika4]

I have applied the changes to our branch and recompiled. I don't immediately see any changes on the explorer. If you would like to share your ssh public key, I can provide you with access to the blockscout server.

[goodsoft]

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAweNYeJzQk2F2GM4PLuLskETJUahVlk6f13VO890erXkFYYJ87P9JN+Ev57RWVtfko+GRZ8Ndn0c69x85xLVGo8cqwl2y/AzuzWwawECef4DQoqAH99D+JEz3KXL9H6N3FaRv4d4wxdOydb1KGTGohleUJop1nTbbFJazaBhNmV3zSN+PsWP9Izw61wLjpERX9JOoNxFsfZcMw9ILv4VlphVziQo1XgTnF3bPor5yc5JY1oReM5T+yPTPfZAfSDU7nAnQ1lVVl5aztnwTsuQjRjAr919BMj+BOOb6C+toKEy9cmQILQDETlrs4da+MWrzYiUyFk8E+g7oUJA5r5TpGw== goodsoft@goodsoft

[Eika4]

ssh [email protected]
screen -ls
blockscout: screen -r 885
petrichor node: screen -r 31095

[goodsoft]

I found another incompatibility and fixed it. Please apply this patch and restart the app server.

[Eika4]

The patch has been applied and I do notice an immediate difference. I no longer see error messages either, so the changes appear to have been a success.

[goodsoft]

Good. From what I see, all transactions have been successfully indexed as well.

[Eika4]

I was worried that blockscout would not be compatible with the C++ ethereum client, but it looks like you made all the necessary changes for us. Great job. Thank you.

[vbaranov]

@Eika4 did you move https://explorer.petrichor-dev.com/ somewhere? It doesn't open right now

[Eika4]

Yes, sorry for not updating you. We moved domains, and changed the spelling to match the currency. Thank you.
https://explorer.petrachor.com

[Eika4]

@goodsoft @vbaranov We have an open issue on Gitcoin. If either of you are interested, I would like to give you the job.
https://gitcoin.co/issue/petrachor/blockscout/1/3038