Truffle: Support resetting accounts after each test

[Nick-Lucas]

Issue

Good tests should always start from a clean slate, and not depend on the results of previous tests. Currently the test network is reset before each Contract suite, and it seems there is no way to reset programmatically in tests or with some configuration.

Steps to Reproduce

• Create a test file with a Contract block
• Add multiple It(...) tests which assume a blank slate and make changes to accounts ether balances (etc)
• Observe that all but the first test will fail, as the state is not blank anymore

Expected Behavior

A setting or globally available function would allow for resetting of accounts, when the developer needs to start from a blank slate more regularly

Potential solution

I've had a read of how truffle works, and played with a solution like the following:

https://github.com/trufflesuite/truffle-core/blob/develop/lib/test.js#L218

      global.resetState = function() {
        runner.resetState(function() { console.log("State Reset By Test")  })
      }
          // OR
      global.resetState = function() {
        runner.initialize(function(err) { 
            err 
            ? console.log('Failed Resetting State') 
            : console.log("State Reset By Test")  }
        )
      }

Unfortunately this doesn't seem to actually work, but providing a working reset method or helpers class in the global scope would open up a lot of doors for developers.

[cgewecke]

@Nick-Lucas Thanks, this makes sense and is nicely expressed. In practice what people often do is manage their own contract instantiation in a beforeEach block. Example:

contract('SimpleContract', function(accounts){
   let simpleContract;

   beforeEach(async function(){
     simpleContract = await SimpleContract.new();
   });

Are there use cases where the strategy above isn't adequate?

[Nick-Lucas]

Yes, to clarify, a different solution is needed to reset Ether values on the accounts. Redeploying contracts is a decent workaround for resetting application state, but if I want my contracts to receive/send Ethereum then testing this functionality is difficult without a proper blank slate for each test.

[cgewecke]

@Nick-Lucas Ah yes, there it is! Right in the title of your issue, sorry. Agree, this option would be nice.

An interim workaround is to fetch balances at the beginning of a test and set expectations relative to that baseline. But it would be convenient not to do this for every test.

As a note, the value of reverting at the contract suite level instead of per test is that it speeds things up (deploying contracts can take a while). Your point about starting from a clean slate make sense but there's also some execution overhead associated with it in this context.

[Nick-Lucas]

Yes, I believe this is what I'll be doing for the time-being.

As far as performance, that definitely makes this an optional function call, for developers to use or ignore. The current trade-off was a good call for most purposes.

I'd be happy to submit a PR for this, but my concept code wasn't actually working, so any additional guidance would be really welcome. I'm guessing it's something to do with misusing the TestRunner code

[cgewecke]

@Nick-Lucas Ok thanks! Will have a look over there and see how this might be done . . .

[eduardonunesp]

@Nick-Lucas I assume that you are using the ganache-cli on tests? I wonder if the evm_snapshot and evm_revert makes sense. ganache-cli methods

• evm_snapshot : Snapshot the state of the blockchain at the current block. Takes no parameters. Returns the integer id of the snapshot created.
• evm_revert : Revert the state of the blockchain to a previous snapshot. Takes a single parameter, which is the snapshot id to revert to. If no snapshot id is passed it will revert to the latest snapshot. Returns true.

[Nick-Lucas]

Hey @eduardonunesp apologies for the slow reply, crazy end to the week!

I'm using truffle's testing framework with its default provider, which I understand is ganache behind the scenes, but it abstracts away the API and doesn't seem to inject that into the global scope either. I've seen in the source code that snapshots are already used for the reset operation at the start of Contract blocks, so it does sound about right.

That said you now have me wondering if I could connect to ganache from the test code, without truffle knowing. I expect it's communicated with over the the local network so could probably call the endpoints myself?

[justinkchen]

I'm also having trouble resetting in between tests. (it works at the contract level of each file but I need to reset account balances more often). Was there ever a solution? I'm trying evm_snapshot and evm_revert but the revert keeps having issues it seems

[eduardonunesp]

const saveState = async () => {
  return await web3.currentProvider.send({
    jsonrpc: "2.0",
    method: "evm_snapshot",
    id: 0
  })
}

const revertState = async (id) => {
  await web3.currentProvider.send({
    jsonrpc: "2.0",
    method: "evm_revert",
    params: [id],
    id: 0
  })
}

contract('SomeContract', () => {
  let id
  beforeEach(async () => {
    id = await saveState()
  })

  afterEach(async() => {
    await revertState(id)
  })
})

Note: I didn't test

[stale[bot]]

Thank you for raising this issue! It has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If you would like to keep this issue open, please respond with information about the current state of this problem.

[sparrowDom]

Has anybody tested solution presented by @eduardonunesp? And if yes how long does it take to save/revert state?

[stale[bot]]

Thanks for your response! This issue is no longer considered stale and someone from the Truffle team will try to respond as soon as they can.

[AlexanderPoschenrieder]

const saveState = async () => {
  return await web3.currentProvider.send({
    jsonrpc: "2.0",
    method: "evm_snapshot",
    id: 0
  })
}

const revertState = async (id) => {
  await web3.currentProvider.send({
    jsonrpc: "2.0",
    method: "evm_revert",
    params: [id],
    id: 0
  })
}

contract('SomeContract', () => {
  let id
  beforeEach(async () => {
    id = await saveState()
  })

  afterEach(async() => {
    await revertState(id)
  })
})

Note: I didn't test

A little fix:

let lastSnapshot;

const saveState = async () => {
  const { result } = await web3.currentProvider.send({
    jsonrpc: '2.0',
    method: 'evm_snapshot',
    id: 0
  });

  return parseInt(result, 0);
};

const revertState = async () => {
  await web3.currentProvider.send({
    jsonrpc: '2.0',
    method: 'evm_revert',
    params: lastSnapshot,
    id: 0
  });

  lastSnapshot = await saveState();
};

Puting this on a little helper and calling saveState on before function and revertState on beforeEach works pretty well.
lastSnapshot is a little workaround because ganache-cli hangs after the second consecutive revert on the same snapshot.

[gnidan]

We will look into specing out requirements for this. We'll need to preserve backwards compatibility while providing a nice interface for doing this. Thank you!

[stale[bot]]

Thank you for raising this issue! It has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If you would like to keep this issue open, please respond with information about the current state of this problem.

[stale[bot]]

There has been no new activity on this issue since it was marked as stale 7 days ago, so it is being automatically closed. If you'd like help with this or a different problem, please open a new issue. Thanks!

[ejwessel]

@gnidan Has there been any progress on speccing the requirements and identifying issues with backwards compatibility?

Also has @tcoulter 's position changed on not evm_reverting for every test per https://github.com/trufflesuite/truffle/pull/47#issuecomment-161432778 ?

[gnidan]

Hey @ejwessel, nope, no progress yet on this specifically. Cursèd Stalebot is now disabled so we won't risk this issue getting lost to history again.

As for position on this: I'd say no, Tim probably feels the same way as he used to, and I agree with him. But! Definitely we could support this as a configuration option.

Not sure about team bandwidth to work on this in the near-term, so I'm going to slap the "help wanted" label on this issue. If anyone wants to work on a PR, I'd love to see the addition of configuration options to customize this revert behavior. Might be worth syncing up before starting work, though. If anyone's interested, let me know!

[ejwessel]

Definitely we could support this as a configuration option.

I'd be down for this. IMO think its better than not having it.

Not sure about team bandwidth to work on this in the near-term, so I'm going to slap the "help wanted" label on this issue. If anyone wants to work on a PR, I'd love to see the addition of configuration options to customize this revert behavior. Might be worth syncing up before starting work, though. If anyone's interested, let me know!

I just started looking at:
https://github.com/trufflesuite/truffle/blob/master/packages/truffle-core/lib/testing/testrunner.js
and it's' usage in:
https://github.com/trufflesuite/truffle/blob/master/packages/truffle-core/lib/test.js

[ejwessel]

@gnidan
Been poking around. Are either of the two files I noted earlier areas I should start with?