Eos: MONGODB eosio.token transfer info(act.data) is binary format in action_traces collection
aiguo186
All 18 comments
According developer i have modified abi-serializer-max-time-ms to a large value,but it doesn't work.
The following is my config.ini
get-transactions-time-limit = 30
max-transaction-time=3000
http-server-address = 0.0.0.0:9876
p2p-listen-endpoint = 0.0.0.0:14422
p2p-server-address = 172.26.247.121:14422
access-control-allow-origin = *
chain-state-db-size-mb = 2048
http-validate-host = 0
allowed-connection = any
log-level-net-plugin = info
max-clients = 120
connection-cleanup-period = 30
network-version-match = 0
sync-fetch-span = 2000
enable-stale-production = false
required-participation = 33
plugin = eosio::chain_api_plugin
plugin = eosio::history_plugin
plugin = eosio::history_api_plugin
plugin = eosio::chain_plugin
#
plugin = eosio::mongo_db_plugin
mongodb-uri = mongodb://47.92.86.161:27017/eosmain
mongodb-store-blocks = false
mongodb-store-block-states = false
mongodb-store-transaction-traces = false
mongodb-filter-out = spammer::
mongodb-filter-out = eosio:onblock:
mongodb-queue-size = 2048
mongodb-abi-cache-size = 10240
abi-serializer-max-time-ms = 999900000
#
p2p-peer-address = bp.superone.io:57576
p2p-peer-address = onedapp.oneeos.org:9000
p2p-peer-address = mainnet.eos.ren:9376
p2p-peer-address = eos.nodepacific.com:9876
p2p-peer-address = fullnode.eoslaomao.com:443
p2p-peer-address = mainnet.eoseco.com:10010
p2p-peer-address = eosbp.eosvillage.io:8181
p2p-peer-address = testnode.eosgalaxy.io:9876
p2p-peer-address = node1.eoscannon.io:59876
p2p-peer-address = bp.eosmedi.com:9877
p2p-peer-address = bp.eosbeijing.one:8080
p2p-peer-address = mainnet.eosbattles.com:9875
p2p-peer-address = 807534da.eosnodeone.io:19872
act.data is not stored as json.
The data in the database is as follows：
{
"_id" : ObjectId("5b9765ba603fe26f6e6ee23d"),
"receipt" : {
"receiver" : "eosiochengyu",
"act_digest" : "4f4d437d52552e4c3197419532e8befe5ef1d48973f6c53524de463aaed683cc",
"global_sequence" : 412608830,
"recv_sequence" : 4001,
"auth_sequence" : [
[
"eosbetdice11",
15853661
]
],
"code_sequence" : 55,
"abi_sequence" : 55
},
"act" : {
"account" : "eosbetdice11",
"name" : "betreceipt",
"authorization" : [
{
"actor" : "eosbetdice11",
"permission" : "active"
}
],
"data" : "677a6099e7c95c72a03d9baa21ea305500a6823403ea3055803801000000000004454f5300000000004401000000000004454f53000000009d723a4221f56d5ce1f25bc831d68ff19adf06c0a1e1f4fba9b9fb79eb949f95001f408fe653285c49848d64e5ad86a3ed3e44e12bb3e70248d90f693a3d62d56f834f4f2f429e1233162d8396c950e8118a63f2d8c666e87628513a09490991a26060000000000000003d00000000000000"
},
"elapsed" : 14,
"cpu_usage" : 0,
"console" : "",
"total_cpu_usage" : 0,
"trx_id" : "23ba07a521a18c49860624346f04ef76fd5d52e2563e53013f80af40739a4d76",
"createdAt" : ISODate("2018-09-11T06:50:34.211Z")
}
aiguo186
on 11 Sep 2018
The root cause is more likely that the abi file of eosbetdice11 cannot decode the action data.
taokayan
on 11 Sep 2018
@taokayan But the eosio.token transfer also cannot decode,why?
{
"_id" : ObjectId("5b9765ba603fe26f6e6ee26f"),
"receipt" : {
"receiver" : "oooooooooooh",
"act_digest" : "20fffafe8047cac4bb40138accf442e891d03e977f0ce7e165486576663a0685",
"global_sequence" : 412608882,
"recv_sequence" : 6778,
"auth_sequence" : [
[
"eosknightsio",
262302
]
],
"code_sequence" : 2,
"abi_sequence" : 2
},
"act" : {
"account" : "eosio.token",
"name" : "transfer",
"authorization" : [
{
"actor" : "eosknightsio",
"permission" : "active"
}
],
"data" : "401dce8db9093155d028a594524a29a58c0000000000000004454f53000000002f656f736b6e69676874733a6d6174657269616c2d73616c653a38373a3131343431313a68657974736e7a7867346765"
},
"elapsed" : 41,
"cpu_usage" : 0,
"console" : "",
"total_cpu_usage" : 0,
"trx_id" : "e720c58abda47262cf8961145c59e87e446c56420ff2aace06b155855983c893",
"createdAt" : ISODate("2018-09-11T06:50:34.248Z")
}
aiguo186
on 11 Sep 2018
We meet the same problem. Use same node, same config.ini. When use local mongodb sync data is json format , but modify mongodb-uri use remote mongodb sync data is binary format. Why?
yaanhyy
on 12 Sep 2018
@yaanhyy
We looked at the source code, which might be caused by the following code, but the exact reason is that we still need to debug the source code
"eos/libraries/chain/include/eosio/chain/abi_serializer.hpp"
`
try {
auto abi = resolver(act.account);
if (abi.valid()) {
auto type = abi->get_action_type(act.name);
if (!type.empty()) {
try {
mvo( "data", abi->_binary_to_variant( type, act.data, recursion_depth, deadline, max_serialization_time ));
mvo("hex_data", act.data);
} catch(...) {
// any failure to serialize data, then leave as not serailzed
mvo("data", act.data);
}
} else {
mvo("data", act.data);
}
} else {
mvo("data", act.data);
}
} catch(...) {
mvo("data", act.data);
}
out(name, std::move(mvo));
}`
@yaanhyy
Use same node, same config.ini. When use local mongodb sync data is json format , but modify mongodb-uri use remote mongodb sync data is binary format. Why?
Our situation is like this:
1.Mainnet node mongodb plugin store transfer info as binary format
2.It is json format in my test node
3.It doesn't matter if the mongodb server is local or remote.
aiguo186
on 12 Sep 2018
@aiguo186
I add log as you show above,get log:
2018-09-12T08:49:10.261 thread-2 abi_serializer.hpp:306 add ] abi is invalid.
2018-09-12T08:49:10.261 thread-2 abi_serializer.hpp:306 add ] abi is invalid.
2018-09-12T08:49:10.261 thread-2 abi_serializer.hpp:306 add ] abi is invalid.
2018-09-12T08:49:10.262 thread-2 abi_serializer.hpp:306 add ] abi is invalid.
2018-09-12T08:49:10.263 thread-2 abi_serializer.hpp:306 add ] abi is invalid.
2018-09-12T08:49:10.265 thread-2 abi_serializer.hpp:306 add ] abi is invalid.
I wonder why abi is invalid when I config mongodb-uri as remote. when I change config to local mongodb, abi is valid.
yaanhyy
on 12 Sep 2018
@taokayan
how to make store transfer info as json format? :)
aiguo186
on 12 Sep 2018
@yaanhyy Your log does not correspond to any code I can find. What version are you running?
heifner
on 12 Sep 2018
Note that the abi-serializer-max-time-ms = 999900000 needs to be on the nodeos pushing into mongodb at the time the data is stored. The binary data is converted to json and stored in mongodb as json. If that conversion fails then it will be stored as hex.
Note it can fail because of improper abi on the account OR if abi-serializer-max-time-ms is set too low. For eosio transfer the only way I know it can fail is too small of a value for abi-serializer-max-time-ms.
heifner
on 12 Sep 2018
@heifner the log code is added by me for test. I re-sync data and set abi-serializer-max-time-ms = 999900000 ,then solve the problem.
yaanhyy
on 13 Sep 2018
@heifner @yaanhyy
You need to synchronize the Mongo data from scratch, and if you synchronize from halfway through, you will make an error (due to incomplete "accounts" collection) .Of course abi-serializer-max-time-ms must big enough
aiguo186
on 13 Sep 2018
Yes, It's much likely because the original setabi action(which happened at the very beginning) was not captured by the mongodb_plugin.
taokayan
on 13 Sep 2018
Hi everyone, I'm having the exact same problem (raw hex data in act.data) but setting up abi-serializer-max-time-ms = 999900000didn't solve the issue for me.
I need to be able to process the information contained inside act.dataso how can I fix this?
Here's my config.ini for information:
agent-name = myAgent
blocks-dir = "blocks"
chain-state-db-size-mb = 65536
reversible-blocks-db-size-mb = 2048
contracts-console = false
http-server-address = 0.0.0.0:8888
p2p-server-address = MY_IP:9876
p2p-listen-endpoint = 0.0.0.0:9876
validation-mode = light
http-validate-host = false
access-control-allow-origin = *
wasm-runtime = wabt
allowed-connection = any
sync-fetch-span = 1000
plugin = eosio::chain_api_plugin
plugin = eosio::chain_plugin
plugin = eosio::mongo_db_plugin
plugin = eosio::net_api_plugin
mongodb-filter-on = *
mongodb-queue-size = 10240
mongodb-abi-cache-size = 10240
mongodb-uri = mongodb://USER:[email protected]:27017/
mongodb-block-start = 1
mongodb-store-blocks = 0
mongodb-store-block-states = 0
mongodb-filter-out = eosio:onblock:
mongodb-filter-out = blocktwitter::
mongodb-filter-out = spammer::
abi-serializer-max-time-ms = 999900000
The server should be powerful enough: Core i7 4.2Ghz, 64go RAM, 1To NVMe > 1500 IOPS, 1Gb internet connection.
fabricus
on 22 Nov 2018
Hi everyone, I'm having the exact same problem (raw hex data in
act.data) but setting upabi-serializer-max-time-ms = 999900000didn't solve the issue for me.I need to be able to process the information contained inside
act.dataso how can I fix this?Here's my config.ini for information:
agent-name = myAgent blocks-dir = "blocks" chain-state-db-size-mb = 65536 reversible-blocks-db-size-mb = 2048 contracts-console = false http-server-address = 0.0.0.0:8888 p2p-server-address = MY_IP:9876 p2p-listen-endpoint = 0.0.0.0:9876 validation-mode = light http-validate-host = false access-control-allow-origin = * wasm-runtime = wabt allowed-connection = any sync-fetch-span = 1000 plugin = eosio::chain_api_plugin plugin = eosio::chain_plugin plugin = eosio::mongo_db_plugin plugin = eosio::net_api_plugin mongodb-filter-on = * mongodb-queue-size = 10240 mongodb-abi-cache-size = 10240 mongodb-uri = mongodb://USER:[email protected]:27017/ mongodb-block-start = 1 mongodb-store-blocks = 0 mongodb-store-block-states = 0 mongodb-filter-out = eosio:onblock: mongodb-filter-out = blocktwitter:: mongodb-filter-out = spammer:: abi-serializer-max-time-ms = 999900000The server should be powerful enough: Core i7 4.2Ghz, 64go RAM, 1To NVMe > 1500 IOPS, 1Gb internet connection.
Synchronize nodes from scratch
aiguo186
on 22 Nov 2018
Yes, It's much likely because the original setabi action(which happened at the very beginning) was not captured by the mongodb_plugin.
Synchronize nodes from scratch
I just did and ended up with the raw hex in the act.data :-(
fabricus
on 22 Nov 2018
Yes, It's much likely because the original setabi action(which happened at the very beginning) was not captured by the mongodb_plugin.
Synchronize nodes from scratch
I just did and ended up with the raw hex in the act.data :-(
you should restart node by "--delete-all-blocks "
aiguo186
on 22 Nov 2018
Yes, It's much likely because the original setabi action(which happened at the very beginning) was not captured by the mongodb_plugin.
Synchronize nodes from scratch
I just did and ended up with the raw hex in the act.data :-(
you should restart node by "--delete-all-blocks "
Indeed it worked. For future reference I had to delete the mongo database + state & blocks folder of the node. Then from a fresh install, starting everything over finally worked.
Thanks for the feedback.
fabricus
on 22 Nov 2018
Related issues
congnghebitcoin
·
3Comments
dimakomar
·
3Comments
yashbhavsar007
·
3Comments
jcalfee
·
3Comments
chanwitkepha
·
3Comments
Most helpful comment
@heifner @yaanhyy
You need to synchronize the Mongo data from scratch, and if you synchronize from halfway through, you will make an error (due to incomplete "accounts" collection) .Of course abi-serializer-max-time-ms must big enough