Web3.py: Support for named tuples, dictionaries, and arbitrary objects with attributes

[kclowes]

What was wrong?

After #1235 gets merged, we'll have tuple support, but not support for rich object representation (see comment here). It would be neat to have support for named tuples, dictionaries, and arbitrary objects with attributes. These should probably be introduced in 3 separate PRs.

[davesque]

To clarify, as I understand it, the issue is that we don't support returning rich objects like dictionaries or other mappings. Right now, a result with a tuple ABI encoding is just returned as a python tuple and it's difficult to parse its structure.

[banteg]

It can be parsed somewhat like this:

def decode(value, abi):
    """Decode tuple as dict."""

    abi = abi.get("outputs", abi)

    # list of values
    if isinstance(value, list):
        return [decode(x, y) for x, y in zip(value, abi)]

    # complex value
    if 'components' in abi:
        inner = {}
        for x, y in zip(value, abi["components"]):
            inner.update(decode(x, y))
        result = {abi["name"]: inner}

        return result.get("", result)

    # basic value
    return {abi["name"]: value}

For example, this

[('0x89d24A6b4CcB1B6fAA2625fE562bDD9a23260359',
  (1054885587528333985413443, 1717372972689245809897586),
  (1012922935596035117, 1007394154240642891, 1557792587),
  '0x787F552BDC17332c98aA360748884513e3cB401a',
  '0xad91a0ddf799176a0A87a32Dafe8F3dd28479918',
  (0,),
  (0,),
  False),
 (1012923172873366382, 1007394278124839761, 1557792630),
 (996316811659981092,),
 (5447677486,)]

becomes this

[{'token': '0x89d24A6b4CcB1B6fAA2625fE562bDD9a23260359',
  'totalPar': {'borrow': 1054885587528333985413443,
   'supply': 1717372972689245809897586},
  'index': {'borrow': 1012922935596035117,
   'supply': 1007394154240642891,
   'lastUpdate': 1557792587},
  'priceOracle': '0x787F552BDC17332c98aA360748884513e3cB401a',
  'interestSetter': '0xad91a0ddf799176a0A87a32Dafe8F3dd28479918',
  'marginPremium': {'value': 0},
  'spreadPremium': {'value': 0},
  'isClosing': False},
 {'borrow': 1012923172873366382,
  'supply': 1007394278124839761,
  'lastUpdate': 1557792630},
 {'value': 996316811659981092},
 {'value': 5447677486}]

[pipermerriam]

Returning rich objects probably won't be a default API since name isn't a required ABI field last time I checked. However, I can see this as something that could very easily be supported either under the existing API or with some minor modifications to the contract API. This really needs someone to take the torch and run with it to explore ways that this can be implemented in a flexible maintainable way.

[banteg]

A few options on how to handle the API side of things:

1. Subclass ContractCaller, add an option or shortcut to override caller class when creating a contract. This requires the least changes but will only work with the caller calls.
2. Add an option to call_contract_function, which will also propagate to ContractFunction.call, ContractFunctions and Contract.

As for the decoder, I think it belongs to web3._utils.abi as it's very similar to abi_data_tree and abi_sub_tree.

Overall it's coming together nicely, I'll make a pull request soon.

[pipermerriam]

Other options to look into.

1. contract level middlewares that only operate on calls that originate from the contract.
2. contract level abi decoder that does the additonal level of logic to convert things to rich logic.

A first step towards this that I would be supportive of would be to let the native layer return a NamedTuple when it detects it is possible since this would be a backwards compatible change and would still provide users with named attribute access to struct fields.

[fubuloubu]

A first step towards this that I would be supportive of would be to let the native layer return a NamedTuple when it detects it is possible since this would be a backwards compatible change and would still provide users with named attribute access to struct fields.

Add my support that this sounds like a pretty good way to handle this additional functionality.

Also really support adding this feature. Having to convert smart contract struct objects to/from tuples is really a hassle, and I was hoping that there'd be a better way eventually. It might help to think about how developers might interact with this API, for example I was developing a Transaction object that had some specialized features for txn verification, but ultimately wanted to be able to submit via a smart contract call directly without having to create obtuse syntax.