Blockscout: Feature request: support transaction indexing more frequently

[rjl493456442]

After https://github.com/poanetwork/blockscout/issues/1530 be fixed, the transactions can be indexed with a given time interval TXS_COUNT_CACHE_PERIOD. Appreciate for the awesome work!

However the TXS_COUNT_CACHE_PERIODcan only support hour level configuration. For some private network, it is definitely too long to update the transaction counter every 2 hours.

Could we support indexing transaction in minute or second level? For some private network, is it possible to index transaction in real time?

[ayrat555]

@rjl493456442 yep, It's easy to resolve. I'll work on it

[rjl493456442]

@ayrat555 Appreciate!

[rjl493456442]

@ayrat555 I tested your PR, it works well! Could we also do the same thing for wallet address counter?

[ayrat555]

@rjl493456442 what's wrong with it? doesn't it update?

[rjl493456442]

@ayrat555 The transaction counter can be updated by setting interval, so what about wallet address counter, do we have something similar to config the interval?

[ayrat555]

@rjl493456442 can you check #1706?

[rjl493456442]

Wow, so kind of you. So the ADDRESS_WITH_BALANCES_UPDATE_INTERVAL is also second level?

[ayrat555]

what do you mean by second level?

[rjl493456442]

ADDRESS_WITH_BALANCES_UPDATE_INTERVAL=2 means updating the wallet address counter every 2 seconds.

[ayrat555]

yes

[rjl493456442]

@ayrat555

Does your PR break something?

2019-04-08T02:02:43.872 [info] Application explorer exited: exited in: Explorer.Application.start(:normal, [])
    ** (EXIT) an exception was raised:
        ** (RuntimeError) could not lookup Explorer.Repo because it was not started or it does not exist
            (ecto) lib/ecto/repo/registry.ex:18: Ecto.Repo.Registry.lookup/1
            (ecto) lib/ecto/repo/queryable.ex:131: Ecto.Repo.Queryable.execute/4
            (ecto) lib/ecto/repo/queryable.ex:18: Ecto.Repo.Queryable.all/3
            (ecto) lib/ecto/repo/queryable.ex:74: Ecto.Repo.Queryable.one!/3
            (explorer) lib/explorer/chain.ex:1025: Explorer.Chain.fetch_min_and_max_block_numbers/0
            (explorer) lib/explorer/chain/block_number_cache.ex:64: Explorer.Chain.BlockNumberCache.update_cache/0
            (explorer) lib/explorer/chain/block_number_cache.ex:26: Explorer.Chain.BlockNumberCache.setup/1
            (explorer) lib/explorer/application.ex:40: Explorer.Application.start/2

[ayrat555]

@rjl493456442 no it shouldn't. are you still experiencing this?

[rjl493456442]

Yes. I can run the master branch without this error.

[ayrat555]

@rjl493456442 I think you set update period too small. so Repo can not be initialized. Can you set it to a bigger number and try again?

[rjl493456442]

@ayrat555 I set ADDRESS_WITH_BALANCES_UPDATE_INTERVAL to 3600, still doesn't work.

[rjl493456442]

@ayrat555 Do you need the detailed docker output?

[ayrat555]

I think the error is not related to #1706. https://github.com/poanetwork/blockscout/pull/1711 should fix it. Can you try it?

[rjl493456442]

@ayrat555

2019-04-08T11:33:16.785 [info] Application explorer exited: Explorer.Application.start(:normal, []) returned an error: shutdown: failed to start child: Explorer.Counters.AddressesWithBalanceCounter
    ** (EXIT) an exception was raised:
        ** (ArithmeticError) bad argument in arithmetic expression
            :erlang.*(1000, "20")
            (stdlib) timer.erl:220: :timer.seconds/1
            (explorer) lib/explorer/counters/addresses_with_balance_counter.ex:67: Explorer.Counters.AddressesWithBalanceCounter.schedule_next_consolidation/0
            (explorer) lib/explorer/counters/addresses_with_balance_counter.ex:48: Explorer.Counters.AddressesWithBalanceCounter.init/1
** (Mix) Could not start application explorer: Explorer.Application.start(:normal, []) returned an error: shutdown: failed to start child: Explorer.Counters.AddressesWithBalanceCounter
    ** (EXIT) an exception was raised:
        ** (ArithmeticError) bad argument in arithmetic expression
            :erlang.*(1000, "20")
            (stdlib) timer.erl:220: :timer.seconds/1
            (explorer) lib/explorer/counters/addresses_with_balance_counter.ex:67: Explorer.Counters.AddressesWithBalanceCounter.schedule_next_consolidation/0
            (explorer) lib/explorer/counters/addresses_with_balance_counter.ex:48: Explorer.Counters.AddressesWithBalanceCounter.init/1
            (stdlib) gen_server.erl:374: :gen_server.init_it/2
            (stdlib) gen_server.erl:342: :gen_server.init_it/6
            (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
            (stdlib) gen_server.erl:374: :gen_server.init_it/2
            (stdlib) gen_server.erl:342: :gen_server.init_it/6
            (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

[rjl493456442]

Set ADDRESS_WITH_BALANCES_UPDATE_INTERVAL to 20.

[ayrat555]

@rjl493456442 my bad. there is an error in types. I updated the PR. can you please try it?

[rjl493456442]

@ayrat555 After testing your latest #1706 and #1711 , it works(services can be started and wallet address counter can be updated in the specified interval). But there still some error logs. Please take a look.

2019-04-09T01:49:41.277 [error] GenServer Tzdata.EtsHolder terminating
** (MatchError) no match of right hand side value: {:error, :cannot_create_table}
    (tzdata) lib/tzdata/ets_holder.ex:59: Tzdata.EtsHolder.load_ets_table/1
    (tzdata) lib/tzdata/ets_holder.ex:52: Tzdata.EtsHolder.load_release/0
    (tzdata) lib/tzdata/ets_holder.ex:23: Tzdata.EtsHolder.handle_cast/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: {:"$gen_cast", :new_release_has_been_downloaded}
State: "2018e"
warning: deprecated time unit: :microseconds. A time unit should be :second, :millisecond, :microsecond, :nanosecond, or a positive integer
  (explorer) lib/explorer/repo/prometheus_logger.ex:8: Explorer.Repo.PrometheusLogger.log/1
  (telemetry) /opt/app/deps/telemetry/src/telemetry.erl:113: anonymous fn/4 in :telemetry.execute/3
  (stdlib) lists.erl:1338: :lists.foreach/2
  (ecto_sql) lib/ecto/adapters/sql.ex:786: Ecto.Adapters.SQL.log/4

warning: deprecated time unit: :seconds. A time unit should be :second, :millisecond, :microsecond, :nanosecond, or a positive integer
  (elixir) lib/calendar/iso.ex:703: Calendar.ISO.from_unix/2
  (elixir) lib/calendar/datetime.ex:118: DateTime.from_unix/3
  (elixir) lib/calendar/datetime.ex:166: DateTime.from_unix!/3
  (ethereum_jsonrpc) lib/ethereum_jsonrpc/block.ex:447: EthereumJSONRPC.Block.entry_to_elixir/1

warning: deprecated time unit: :seconds. A time unit should be :second, :millisecond, :microsecond, :nanosecond, or a positive integer
  (elixir) lib/calendar/iso.ex:717: Calendar.ISO.precision_for_unit/1
  (elixir) lib/calendar/iso.ex:708: Calendar.ISO.from_unix/2
  (elixir) lib/calendar/datetime.ex:118: DateTime.from_unix/3
  (elixir) lib/calendar/datetime.ex:166: DateTime.from_unix!/3

[ayrat555]

@rjl493456442 it's warnings. It's related to new elixir versions. It's ok