Ganache-cli: How come I must set the gas-limit to at least twice the gas used in a transaction?

[barakman]

When testing a transaction via Truffle & Ganache, it appears that I must set the gas-limit of both sides to at least twice the gas used in the transaction.

Expected Behavior

One should be able to set the gas-limit (of both sides) to the gas used in the transaction.

Current Behavior

One must set the gas-limit (of both sides) to at least twice the gas used in the transaction.

Steps to Reproduce (for bugs)

On-Chain Code:

pragma solidity 0.4.25;

contract MyContract {
    bool public x;

    function func(uint i) external {
        for (uint n = 0; n < i; n++)
            x = !x;
    }
}

Off-Chain Code:

contract("Test", function(accounts) {
    let myContract;
    before(async function() {
        myContract = await artifacts.require("MyContract").new();
    });
    it("test", async function() {
        let response = await myContract.func(1000);
        console.log("gasUsed =", response.receipt.gasUsed);
    });
});

The results depending on the gas-limit configuration of Truffle and Ganache:

Truffle  | Ganache  | Test Result
---------|----------|------------
12803719 | 12803719 | Failure: VM Exception while processing transaction: out of gas
12803719 | 12803720 | Failure: VM Exception while processing transaction: out of gas
12803720 | 12803719 | Failure: Exceeds block gas limit
12803720 | 12803720 | Success: gasUsed = 6401860

The Exceeds block gas limit error occurs at the before part, so I'm pretty sure that it is due to a violation of the gas-limit configuration between Truffle and Ganache (i.e., Truffle gas-limit must be less than or equal to Ganache gas-limit).

But the other cases imply that in order for the test to pass, I must set the gas-limit in both Truffle and Ganache to at least twice the gas used in a transaction (12803720 = 6401860 * 2).

Can you please explain this behavior?

Is it possible that the response.receipt.gasUsed returned from Ganache is only half of the actual gas used in the transaction?

My Environment

• NodeJS Version: 8.11.2
• Operating System: Windows 10
• Truffle: 4.1.15 ("truffle": "4.1.15")
• Ganache: 6.2.5 ("ganache-cli": "6.2.5")

Thanks

[davidmurdoch]

When you set x = true; then x = false; you incur 2 SSTORE gas costs (20000 for x = true; and 5000 for x = false;) and 1 SSTORE gas refund of 15000 (given whenever x is set _back_ to false). Gas refunds are refunded _after_ the transaction is completed. What appears to be happening is that this contract function actually uses 12803720 gas, but after the transaction is complete some of that gas is refunded, resulting in a total gas used of 6401860.

Additionally, the accumulated refund can not exceed half the gas used for the context (in this case, there is only the single calling context, which is the top-level call to func), which is exactly was appears to be happening here.

Closing as everything seems to be working as intended. Let me know if you believe this is not the case.

[barakman]

@davidmurdoch:
Thank you very much for the detailed explanation.
My primary concern was for receipt.gasUsed not reflecting the correct amount of gas-used in a transaction, since I have relied on it in order to produce an estimation of our expected overall gas-cost.

I take it (according to your description), that gasUsed here is accurate despite the fact that I needed to configure the gas-limit to a much higher value, because the latter stems from the "intermediate" gas consumption (i.e., before the refund).

If my understanding is correct and I can continue relying on receipt.gasUsed for estimations, then it's all good (please inform me if otherwise).

Thanks again for your help :)

[davidmurdoch]

You should be using eth_estimateGas for calculating gas estimations. This will return the gasUsed + gasRefund. Note that in Ganache we currently respect the gas limit when running estimateGas, so an OOG error is still possible when using the default gasLimit for transactions with very high gas usage (like in your example).

[barakman]

@davidmurdoch :

I have done so in the past (via web3.js), but I found it to be:

1. Not always accurate (in particularly, when the estimated function makes an external function call).
2. Much more difficult to take a given sequence (test, deployment script, etc) and estimate it; for each transaction, one needs to call estimateGas and then send.
3. Inconsistent between different versions of web3.js (namely v0.x and v1.x).
4. Buggy; not sure if the problem was on Ganache side or on Web3 side, but I found myself debugging web3.js source code as a result of transaction.estimateGas throwing an exception (I never posted it, because it was hard to describe and I couldn't be bothered at some point).

Should I expect any problem using gasUsed for this matter?

Thanks :)

[davidmurdoch]

1. Yes, this is a known issue in ganache and we are working on a solution right now.
2. Gas costs can fluctuate given the current state of the chain and storage; you really should be using estimateGas whenever it makes sense to do so. Using JavaScript's async/await and [email protected] should make transitioning your code to calling web3.eth.estimateGas relatively straight forward.
3. Upgrading from web3 v0.x to v1.x is a breaking change and you'll need to upgrade your calling code.
4. Not sure what the problem could be. If you do run into this issue again please file an issue here so we can fix it if it is a Ganache problem.

But yes, you will run into problems using only gasUsed in transactions that contain gas refunds.

Thanks!

[barakman]

@davidmurdoch :

1. Thanks.
2. It is because the gas-estimation depends on the current state of the chain, that I need to succeed each transaction.estimateGas() with a transaction.send() (so that the transaction will take effect, the state of the chain will change, and the gas-estimation of the next transaction will be corresponding to the new state of the chain). I find the alternative - printing receipt.gasUsed after every transaction - to be much more simple, given that I already have a sequence at hand (whether it's a test, a deployment script or a runtime script).
3. I am well aware of those breaking changes. I currently use v0.x in my Truffle tests (due to the fact that Truffle 4.x relies on this version), and v1.x for my deployment and runtime scripts (which I execute directly via node).
4. I kinda recall that it happened with web3.js v1.x, and that I ended up replacing let gas = await transaction.estimateGas() with let gas = (await web3.eth.getBlock("latest")).gasLimit, which proved much more robust for my purpose (since my entire code is "in-house", I don't need to worry about using the maximum gas-limit). I will see if I can reproduce it and report accordingly.

Thanks