Solidity: Documentation not clear how event data is stored

[miohtama]

The documentation explains in-depth how indexed event logs are stored:

http://solidity.readthedocs.io/en/latest/contracts.html#events

However it fails to provide explanation how other kind of data is stored. I am trying to figure out how to read back a string stored in the event in Solidity contract through geth eth_getFilterChanges.

[chriseth]

This is more of a question about web3.js. Does the example code given in the documentation work for you? Does it provide the value for the non-indexed argument as part of the result variable in the callback?

[mancze]

I have stumbled on this ambiguity too. The current docs read (followed how I interpret them):

Up to three parameters can receive the attribute indexed which will cause the respective arguments to be searched for: It is possible to filter for specific values of indexed arguments in the user interface.

That's nice. Seems that those arguments are stored as-is and filterable in addition.

If arrays (including string and bytes) are used as indexed arguments, the Keccak-256 hash of it is stored as topic instead.

Hmm. Never heard of "topics", but fine. I can imagine there is some search tree using indexed argument as a key - topic. And only string and bytes are hashed, otherwise not. Yet that's a technical detail, I don't care how indexes are implemented.

...
All non-indexed arguments will be stored in the data part of the log.

(started being nervous) Uhm, and indexed arguments won't?

NOTE: Indexed arguments will not be stored themselves. You can only search for the values, but it is impossible to retrieve the values themselves.

Bang! Ok, now I'm not sure about anything. Questions raised:

1) Are the string and bytes the only types being hashed as topics? Can it change in the future?
2) When reading log data from web3 are the actual values of topics included with the each event or not?
3) If included, are values of hashed indexed arguments (string and bytes) replaced with hashes? Or missing (undefined)?
4) If not included, is it possible to read the actual argument value from somewhere?
5) Is it possible to watch for any record of the "indexed event" without using filters?

[chriseth]

@mancze indeed, the documentation at that point is a little thin. Would you be willing to fix it?

The low-level effect of calling an event is that a so-called log is created. Indexed arguments are stored in the "topic" field of a log. This field is put inside a bloom filter (a special kind of database index) which in turn means that it is rather fast to search for the value if you know it, but it is impossible to retrieve the value. Non-indexed argumenst are stored in the data part of the log which can be retrieved later. In addition to that, strings and bytes are stored by hash inside the topic field.

so:

1. 256 bit types are stored without being hashed, but they cannot be retrieved either
2. no, since they are not part of the database anymore
3. -
4. no, unfortunately not.
5. not sure what you mean, for me using a filter is the definition of watching :)

[mancze]

Yes, I'm willing to prepare PR. Yet I have more questions to ask because it is still not clear to me. Take for example event Transfer(address indexed from, address indexed to, uint tokens) from ERC20.

token.Transfer({ _from: sender, _to: recipient }).watch(...)

Easy. In this case we watch for the events with filtered (fixed) _from and _to so basically we are able to reconstruct all arguments of the event even if web3 would not provide them out-of-the-box. No questions here.

token.Transfer().watch(...)

This is what I meant by question no. 5: Is it possible to watch for all Transfer events like that? Aren't _from and _to filters mandatory?

I assume it is possible to watch like this. But you are telling, that it is not possible to determine senders and recipients of the transfers from the logs and we will be only able to tell the amounts (tokens) transferred?

And let me emphasize, that by "determining" I mean from anywhere from logs. I understand that those arguments are not in data fields. Yet the actual values have to be in topics and topics are part of the database, aren't they? So it should be possible to read those values of indexed arguments directly from topics (not from data).

[ChrisChinchilla]

@mancze Hey, there, I'm working on the Solidity docs for the team right now, before I start on this myself, did you ever make a start to the PR you mention above?

[mancze]

Hi. No, I did not make the PR. I was unable to find out how precisely are indexed arguments handled.

[tzapu]

I hope I got this right (please anyone correct if it's wrong), @mancze, here is my understanding of it:
For simple types, indexed arguments will be in the topics array as is (address, uint, ...) but for complex types (string, ...), the topic will be a hash of the value.
This pretty much means that the simple ones you can retrieve straight from the topic but the others are unretrievable.

@ChrisChinchilla it's so good to see someone starting on the docs, this phrase "Indexed arguments will not be stored themselves. You can only search for the values, but it is impossible to retrieve the values themselves." is so confusing and borderline misleading (:P) if the above is correct.

thanks

[ChrisChinchilla]

Thanks @tzapu once I have cleared 0.5.0 tasks, I will follow up with this.

[ChrisChinchilla]

@miohtama @tzapu @mancze I have a PR here https://github.com/ethereum/solidity/pull/4952 but I also have a remaining work, please take a look at changes, and follow there to let us know if the changes help, or need more work.