Web3.py: eth_abi.exceptions.InsufficientDataBytes: Tried to read 32 bytes. Only got 0 bytes

[brenzi]

• Version: 3.8.1
• Python: 3.5
• OS: linux
• Parity v1.7.0-unstable-b0e4c91-20170502/x86_64-linux-gnu/rustc1.17.0

What was wrong?

While trying to fix #181, I updated my toolchain and now I'm having problems interacting with my contract.

Setting up web3 works fine

# load contract ABI
contractAbi = json.loads(open('../contracts/MeasPub.abi').read())
measpub = web3.eth.contract(abi = contractAbi)
f=open('MeasPub.address')
measpub.address = f.read().strip()
f.close()
f=open('publisher.address') 
publisher_addr = f.read().strip()
f.close()

web3.personal.unlockAccount(account = publisher_addr, passphrase='*******')

but then I want to call a function

      N = measpub.call({'from':publisher_addr}).getSubscriberCount() 

which fails with:

Traceback (most recent call last):
  File "/home/ethadm/.virtualenvs/dapp/lib/python3.5/site-packages/web3/contract.py", line 767, in call_contract_function
    output_data = decode_abi(output_types, return_data)
  File "/home/ethadm/.virtualenvs/dapp/lib/python3.5/site-packages/eth_abi/abi.py", line 108, in decode_abi
    return decoder(stream)
  File "/home/ethadm/.virtualenvs/dapp/lib/python3.5/site-packages/eth_abi/decoding.py", line 102, in __call__
    return self.decode(stream)
  File "/home/ethadm/.virtualenvs/dapp/lib/python3.5/site-packages/eth_utils/functional.py", line 22, in inner
    return callback(fn(*args, **kwargs))
  File "/home/ethadm/.virtualenvs/dapp/lib/python3.5/site-packages/eth_abi/decoding.py", line 140, in decode
    yield decoder(stream)
  File "/home/ethadm/.virtualenvs/dapp/lib/python3.5/site-packages/eth_abi/decoding.py", line 102, in __call__
    return self.decode(stream)
  File "/home/ethadm/.virtualenvs/dapp/lib/python3.5/site-packages/eth_abi/decoding.py", line 165, in decode
    raw_data = cls.read_data_from_stream(stream)
  File "/home/ethadm/.virtualenvs/dapp/lib/python3.5/site-packages/eth_abi/decoding.py", line 247, in read_data_from_stream
    len(data),
eth_abi.exceptions.InsufficientDataBytes: Tried to read 32 bytes.  Only got 0 bytes

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "publisher.py", line 96, in <module>
    main()
  File "publisher.py", line 62, in main
    N = measpub.call({'from':publisher_addr}).getSubscriberCount() 
  File "/home/ethadm/.virtualenvs/dapp/lib/python3.5/site-packages/eth_utils/string.py", line 85, in inner
    return force_obj_to_text(fn(*args, **kwargs))
  File "/home/ethadm/.virtualenvs/dapp/lib/python3.5/site-packages/web3/contract.py", line 779, in call_contract_function
    raise_from(BadFunctionCallOutput(msg), e)
  File "/home/ethadm/.virtualenvs/dapp/lib/python3.5/site-packages/web3/utils/exception_py3.py", line 2, in raise_from
    raise my_exception from other_exception
web3.exceptions.BadFunctionCallOutput: Could not decode contract function call getSubscriberCount return data 0x for output_types ['uint256']

And here's my contract, which I did successfully test with truffle/mocca//testRPC:

pragma solidity ^0.4.6;
contract MeasPub         
{
    address public publisher;
    string public description;
    uint public price_per_second; // [wei per second]


    // dictionary that maps addresses to balances
    // always be careful about overflow attacks with numbers
    mapping (address => uint) private balances;

    // dictionary that remebers the timestamp when the last measurement was published to that subscriber
    mapping (address => uint) private last_publish_block;

    // dictionary that provides an iterable list of subscribers
    mapping (uint => address) private subscriber_index;
    uint subscriber_count;

    // dictionary for RSA public keys
    mapping (address => string) private subscriber_pubkey;

    event LogPublished(
    address _subscriber,    // the subscriber
    bytes _pwdenc,          // the encoded password needed for decryption of data stored in ipfs
    bytes _ipfsaddr,        // the ipfs address where the encrypted measurement can be fetched by the subscriber
    uint _cost       // cost of single publishing item
    );

    event LogDebug(
    string _msg
    );

    // The constructor. It accepts a string input that describes the nature of this publishing.
    function MeasPub() public
    {
        publisher = msg.sender;
        price_per_second = 0; // Wei
    }

    // the publisher may change pricing anytime
    function setPricePerSecond(uint _price) public
    {
        if (msg.sender == publisher)
        price_per_second = _price;
    }

    function publish(address _subscriber, bytes _pwdenc, bytes _ipfsaddr) public returns (bool covered)
    {
        if (msg.sender == publisher)
        {
            uint cost;
            cost = (now - last_publish_block[_subscriber]) * price_per_second;
            if (balances[_subscriber] >= cost)
            {
                balances[_subscriber] -= cost;
                // send money to publisher
                if (!publisher.send(cost)) 
                {
                    balances[_subscriber] += cost;  
                    return false;
                }
                last_publish_block[_subscriber] = now;
                LogPublished(_subscriber, _pwdenc, _ipfsaddr, cost);
                return true;
            }
            LogDebug("subscriber has insufficient funds");
            return false;

        }
        LogDebug("only publisher can publish");
        return false;
    }

    function getSubscriberCount() public returns (uint count)
    {
        //if (msg.sender == publisher)
        return subscriber_count;
    }

    function getSubscriber(uint _index) public returns (address _subscriber, string _pubkey)
    {
        if (msg.sender == publisher)
        return (subscriber_index[_index],subscriber_pubkey[subscriber_index[_index]]);
    }

    function subscribe(string _pubkey) payable public returns (bool success) {
        if (last_publish_block[msg.sender] == uint(0x0))
        {
            // subscriber is new to us
            last_publish_block[msg.sender] = now;
            subscriber_index[subscriber_count] = msg.sender;
            subscriber_count += 1;
        }
        subscriber_pubkey[msg.sender] = _pubkey;
        balances[msg.sender] += msg.value;
        LogDebug("new subscription successful");
        return true;
    }



    /**********
    Standard kill() function to recover funds
    **********/
    function kill()
    {
        if (msg.sender == publisher)
        suicide(publisher);  // kills this contract and sends remaining funds back to creator
    }

}

[mikeshultz]

Seeing a similar issue with a function that returns boolean.

>>> contract.call().myFunction(int("0xdeadbeef", 16))
Traceback (most recent call last):
  File "/var/venvs/web3/lib/python3.6/site-packages/web3/contract.py", line 767, in call_contract_function
    output_data = decode_abi(output_types, return_data)
  File "/var/venvs/web3/lib/python3.6/site-packages/eth_abi/abi.py", line 108, in decode_abi
return decoder(stream)
  File "/var/venvs/web3/lib/python3.6/site-packages/eth_abi/decoding.py", line 102, in __call__
return self.decode(stream)
  File "/var/venvs/web3/lib/python3.6/site-packages/eth_utils/functional.py", line 22, in inner
return callback(fn(*args, **kwargs))
  File "/var/venvs/web3/lib/python3.6/site-packages/eth_abi/decoding.py", line 140, in decode
yield decoder(stream)
  File "/var/venvs/web3/lib/python3.6/site-packages/eth_abi/decoding.py", line 102, in __call__
return self.decode(stream)
  File "/var/venvs/web3/lib/python3.6/site-packages/eth_abi/decoding.py", line 165, in decode
raw_data = cls.read_data_from_stream(stream)
  File "/var/venvs/web3/lib/python3.6/site-packages/eth_abi/decoding.py", line 247, in read_data_from_stream
    len(data),
eth_abi.exceptions.InsufficientDataBytes: Tried to read 32 bytes.  Only got 0 bytes

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/var/venvs/web3/lib/python3.6/site-packages/eth_utils/string.py", line 85, in inner
    return force_obj_to_text(fn(*args, **kwargs))
  File "/var/venvs/web3/lib/python3.6/site-packages/web3/contract.py", line 779, in call_contract_function
    raise_from(BadFunctionCallOutput(msg), e)
  File "/var/venvs/web3/lib/python3.6/site-packages/web3/utils/exception_py3.py", line 2, in raise_from
    raise my_exception from other_exception
web3.exceptions.BadFunctionCallOutput: Could not decode contract function call myFunction return data 0x for output_types ['bool']

[mikeshultz]

I'm assuming our issues here are the same, and it looks like the problem is likely with ethereum-abi-utils. Perhaps only with solidity contracts? Anyway, it seems that functions that return 0 and false have a 0 byte return, and when being parsed by read_data_from_stream, it looks for a 32 byte return.

I was considering submitting a patch, but I'm not entirely sure that a 0 byte return is acceptable in any case, or in some cases. Maybe someone with more knowledge on EVM contracts could shed some light here?

EDIT: Just did a test with JSON-RPC using a test contract of mine, and the return does appear to be 0x:

$ curl --data '{"jsonrpc":"2.0", "method":"eth_call", "params":[{"from": "0xaccount", "to": "0x648f26c9df8c99efd282a240604e3784266c42bc", "data": "0x72abc8b70000000000000000000000000000000000000000000000000000000000000001"}, "latest"], "id":1}' "http://test.rpc.server"
{"jsonrpc":"2.0","id":1,"result":"0x"}

EDIT: And yet, the JSON-RPC docs say this:

WRONG: 0x (should always have at least one digit - zero is "0x0")

Which suggests an invalid function call(or perhaps the ABI not matching your compiled code), which I don't think is the case here, but others finding this issue might have that problem. I have triple-checked mine and tested the same address and function in geth-1.6.1-stable.

[mikeshultz]

I just put together a test contract that returned both false and 0 from some of its functions and they all returned as they should 0x0000000000000000000000000000000000000000000000000000000000000000. I can only suspect that both my and your contracts are not deployed properly, the address for the contract is wrong, or the ABI is a mismatch for the deployed code.

One potential bug that sticks out on your contract(@brenzi), is that its definition says it returns a variable that is not set. Instead you use a return statement within the function. I can't guarantee its your problem, but perhaps getSubscriberCount should probably be something like this:

function getSubscriberCount() public constant returns (uint)
{
    return subscriber_count;
}

I'm going to dig into mine further.

[mikeshultz]

@brenzi It could also be that subscriber_count was not initialized with a value(0). At least, it appears to have been my problem that for me it was an uninitialized struct in a mapping.

Importantly, in the geth console, 0x apparently will still translate to false(and perhaps 0 in your case), but using JSON-RPC(that web3.py uses), does not and errors out. I can't really say what's correct behavior here, since it does seem like an error condition that geth just copes with for whatever reason.

But once that struct was initialized in my contract, the proper return value came back.

[brenzi]

@mikeshultz Thanks for the suggestion. I've changed the constructor

        function MeasPub() public
        {
                publisher = msg.sender;
                price_per_second = 0x0; // Wei
                subscriber_count = 0x0;
        }

and added the "constant" keyword as you suggested

        function getSubscriberCount() public constant returns (uint count)
        {
                //if (msg.sender == publisher)
                return subscriber_count;
        }

Unfortunately, this doesn't change the result. Still get

InsufficientDataBytes: Tried to read 32 bytes.  Only got 0 bytes

EDIT: I should read more carefully. Will try your suggestion with the (uint) return syntax now...

[brenzi]

When I do as you wrote it works:

  function getSubscriberCount() public constant returns (uint)
        {
                //if (msg.sender == publisher)
                return subscriber_count;
        }

Thanks!

[pipermerriam]

This looks like the same error as #161 . This is what happens when you try to read data from an empty address (an address that has no code associated with it).

If you call web3.eth.getCode(address) on the address of the contract you are interacting with, what is the result?

[mikeshultz]

@pipermerriam My problem was not with an invalid address. The other contract methods worked without issue. The problem was that my contract was returning a value from an uninitialized struct(uninitialized memory, I guess) which caused the error condition. The 0x return seems to be a catch-all for every kind of error condition, since @brenzi's problem was different than mine, which also wasn't an invalid contract address.

[pipermerriam]

@mikeshultz thanks for the clarification. Sorry for missing that. Definitely a different issue.

What would you expect the behavior to be in this situation? I assume to have it interpret the 0x to mean that all return values should be returned in their Zero representation.

• bytesXX > "" (empty string)
• uint/int > 0
• address > 0x000000000.... (the zero address)

[mikeshultz]

That's a good question. The geth console just treats it as false/0 but I'm not so sure that's helpful since it is pointing out an error condition. I don't think I have any good ideas here. A more informative exception could help, but really you're trying to do a catch-all on every kind of EVM error there is. There may be no good way to handle this in the library.

[carver]

I'll add that the spec is somewhat ambiguous about when 0x is an okay value to return, and when it must be 0x0.

When encoding QUANTITIES (integers, numbers): encode as hex, prefix with "0x", the most compact representation (slight exception: zero should be represented as "0x0"). Examples:

• WRONG: 0x (should always have at least one digit - zero is "0x0")

When encoding UNFORMATTED DATA (byte arrays, account addresses, hashes, bytecode arrays): encode as hex, prefix with "0x", two hex digits per byte. Examples:

• 0x (size 0, "")

Open questions:

• Is a boolean a QUANTITY?
• Is a contract call result always UNFORMATTED DATA, or does the RPC spec peek at the contract ABI?

[webjunkie01]

Hi guys,

I'm using version 3.16.2 and experiencing the same issue when calling the method approveAndCall of a standard ERC20 token contract.

What surprises me is that in JavaScript when I call the method it works as expected. But in Python is a different story.

Even if I copy paste the encoded data generated from the JavaScript file and try to call the contract method (approveAndCall) in Python still returns

web3.exceptions.BadFunctionCallOutput: Could not decode contract function call myFunction return data 0x for output_types ['bool']

[pipermerriam]

I'm inclined to update eth-abi to accept 0x as False, however, I don't think this is an ok solution, or at least I'm worried about it.

If we start accepting 0x as a valid return of False then that means that if someone calls a contract function which returns a bool on the wrong contract, or a non contract, and no data is returned, then web3.py would tell the use that the contract returned False.

This seems like a very bad thing as it hides the fact that they are not interacting with the contract they expect.

[pipermerriam]

https://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI#types

bool: equivalent to uint8 restricted to the values 0 and 1

@carver I think this indicates that booleans are QUANTITY

Now what I'm curious about is whether solidity is returning 0x (empty return value) for false, or if javascript is incorrectly interpreting no return value as false.

[pipermerriam]

@webjunkie01 one thing to add to this, approveAndCall is a method which requires a transaction so you probably shouldn't be getting a return value (or really interacting with it via the call() api).

[giacomolicari]

I get the same error when using Ganache-cli and trying to execute a contract method call() by passing a block_identifier different than 'latest'.

is_executed_latest = contract.functions.isExecuted(parameter).call( block_identifier='latest')

is_executed_prev = contract.functions.isExecuted(parameter).call( block_identifier=block_identifier) # a block number - like 95

The first contract call goes through and returns a number, whereas the second one raises an error:
web3.exceptions.BadFunctionCallOutput: Could not decode contract function call isExecuted return data b'\x00' for output_types ['uint256']

It seems that the decoder isn't able to decode the x00 value, which would be 0.
I don't know if this is related with Ganache-cli, but actually I can't test my code due to the aforementioned error.

[giacomolicari]

I used a workaround to make the code working:

def read_data_from_stream(self, stream):
    data = stream.read(self.data_byte_size)
    return data

decoding.Fixed32ByteSizeDecoder.read_data_from_stream = read_data_from_stream

[pipermerriam]

I'm reciting from memory, but I believe that is an invalid response for a uint256. I think that the response should be 32 bytes in length, even if the return value is 0.

[jfdelgad]

Having the same issue, after upgrading to 4.5. This happens when I try to read events

[carver]

@jfdelgad can you give more background? Code for the contract, and the python code.

[jfdelgad]

I solved this by changing the function that generates the event from external to public in solidity. For web3js this was irrelevant, but was not the case for web3py.

[jasonrhaas]

Any other insights as to what is causing this problem? My contract code works great on the testrpc but when I try to interact with it on Ropsten, I can a similar error.

web3.exceptions.BadFunctionCallOutput: Could not decode contract function call tokenURI return data b'' for output_types ['string']

I verified that the contract deployed successfully and that the ABI is correct. Even the basic .call() functions in the contract appear to be returning b'' and then throwing the error message.

[dylanjw]

My contract code works great on the testrpc but when I try to interact with it on Ropsten, I can a similar error.

Is your contract returning an empty byte string when run in testrpc? What is the expected return value?

[jasonrhaas]

@dylanjw So on the local testrpc the response is this:

DEBUG:web3.providers.HTTPProvider:Getting response HTTP. URI: http://testrpc:8545, Method: eth_call, 
Response: {'id': 1, 'jsonrpc': '2.0', 'result': '0x0000000000000000000000000000000000000000000000000000000000000000'}

On Ropsten, the response is this:

DEBUG:web3.providers.HTTPProvider:Getting response HTTP. URI: https://ropsten.infura.io, Method: eth_call,
Response: {'jsonrpc': '2.0', 'id': 1, 'result': '0x'}

The converted response should be 0, which works on the testrpc. But on Ropsten its just giving me 0x

[dylanjw]

@jasonrhaas Could you try calling your contract function through a local node, rather than through Infura?

[jasonrhaas]

@dylanjw I will try to set that up, do you think its an infura problem? The local nodes are a pain to test with since they take forever to sync.

[pipermerriam]

@jasonrhaas not trying to be dismissive, but from what I've seen this is almost always an issue with either the contract code not being present (I see you did check that) or a user error in making the request (maybe to the wrong address, maybe with the wrong abi, etc).

This one may be a 3rd category which could be something wrong with your testrpc server or potentially your testrpc behaving incorrectly?

Can you provide an MCVE for this and then we can try to debug ourselves. It's fine to use infura for the provider.

[jasonrhaas]

@pipermerriam I had the wrong contract address 😞. Super embarrassing, I was relying what I saw on http://ropsten.etherscan.io and it was pointing to the wrong address. Is returning 0x a standard error code? Or maybe its just a catch all? It might be helpful to add some suggestions or a more descriptive error code if its always an issue with the contract code not being present. Maybe something like ContractError or NoContractCodeFound? Just an idea. Thanks for your help.

[carver]

Is returning 0x a standard error code?

Right, nodes return 0x on a failure. So web3.py doesn't really have any more information that "something went wrong." It could be a revert in the code, or talking to the wrong contract...

It might be helpful to add some suggestions or a more descriptive error code if its always an issue with the contract code not being present. Maybe something like ContractError or NoContractCodeFound?

There actually is code to do this. It's likely that the address you used was to a different live contract.

[pipermerriam]

I think this can be safely closed.

[carver]

I think this can be safely closed.

Although it's maybe a reasonable idea to raise a new ContractError from the InsufficientDataBytes (after we checked that there is code at the address). It would send people in the right direction faster (to debug their contract).