Truffle: Truffle Tests using TestRPC appear unstable/inconsistent, passing/failing on same test without cause.

[pmumby]

• [x] I've asked for help in the Truffle Gitter before filing this issue.

---

Issue

Given the sample project in the repo at: https://github.com/BattleDrome/truffle_bug
truffle test is experiencing some sort of race condition where running subsequent tests sometimes passes all tests, and sometimes fails some tests. Sometimes it takes 4-5 subsequent tests to encounter the issue. But as test complexity increases, number of failures increases.

Seems to be a race condition between the migration scripts, and the tests, where the migrations sometimes fail during the test, resulting in state not being properly applied.

Steps to Reproduce

1. Check out the repo from https://github.com/BattleDrome/truffle_bug
2. truffle compile
3. truffle migrate --reset
4. truffle test

Repeat step 4 as many times as needed to see inconsistent tests.

Expected Behavior

Tests should behave consistently. If failures occur randomly we can't "trust" the tests, defeating the purpose.

Actual Results

Tests pass sometimes, and fail sometimes.

Screenshot of passed test:

Screenshot of failed test:

These tests were run immediately subsequent to one another with no other changes either to code or environment.

Environment

• Operating System: Windows, using bash on ubuntu environment in windows (truffle installed within ubuntu vm)
• Truffle version: 3.4.9
• Ethereum client: TestRPC 4,1,1
• node version: 8.2.1
• npm version: 5.4.0

[SCBuergel]

We ran into the same issue. It is hard to reproduce and it is hard to provide a minimum working example. [I cannot post the code that we got from the customer to audit.] When minimizing the contract that failed to a hello-world-style example with getter and setter functions all tests seemed to always be working well. The issue seems to surface beyond a certain complexity.

[pmumby]

Yeah I found the same thing. The example code repo I linked is a minimal working example. I slowly added complexity until the issue presented itself regularly enough to verify. Still intermittent, but doing 5-6 subsequent tests should exhibit the instability.

The more the complexity ramps up, the more frequently the test failures are encountered (until you hit a point where you have to run it 8-9 times just to get a passed test). Which is where I'm at in my main project currently.

The issue appears to be a race condition between the migration scripts, and the test framework. ie: somehow the migrations are not fully completing on the chain in TestRPC before the tests run, and as a result the tests fail for various reasons.

So far I've been unable to nail down exactly where this race condition is though.

[mcdee]

As a workaround you can change the migration code to not use async. For example, with the repo you linked you can change migrations/4_setup.js to be

module.exports = function(deployer, network, accounts) {
    var primary = accounts[0];
    return ContractA.deployed().then(aInstance => {
        return ContractB.deployed().then(bInstance => {
            return aInstance.setBAddress(bInstance.address).then(() => {
                return bInstance.setAAddress(aInstance.address);
            });
        });
    });
};

[SCBuergel]

Some observations, that might help:

• When contract is "too simple" (too fast?) this issue happens less frequently.
• When javascript is "too slow" this issue happens less frequently, e.g. a simple console.log before proceeding usually prevents the issue from happening at all.
• This issue usually surfaces more frequently with promises than with async.

[pmumby]

Thanks for the suggestion, but I should note, that before creating this
test case, the problem originally showed in code that was NOT using async
at all...

In the case of my example yes I'm using async, but it doesn't matter. This
problem will exhibit without async as well. Async does seem to make it
worse however (why I added it to make this example). For example turning
async on in the example code appears to alleviate it, until you push the
complexity up a little more, then it shows up again anyway.

In my main project, with async on in the migrations, it errors out 100% of
the time, with async removed using nested promises with .then() it only
fails 60% to 75% of the time, but still a major issue.

On Sat, Sep 9, 2017 at 5:20 PM, Jim McDonald [e-mail]
wrote:

As a workaround you can change the migration code to not use async. For
example, with the repo you linked you can change migrations/4_setup.js to
be

module.exports = function(deployer, network, accounts) {
var primary = accounts[0];
return ContractA.deployed().then(aInstance => {
return ContractB.deployed().then(bInstance => {
return aInstance.setBAddress(bInstance.address).then(() => {
return bInstance.setAAddress(aInstance.address);
});
});
});
};

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/trufflesuite/truffle/issues/557#issuecomment-328304175,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABM8ckQtoLyZkAfUiUZDrDKPmBVJyA9Gks5sgwEXgaJpZM4PLRD3
.

[pmumby]

Yes thanks for this, you are correct, adding some additional "time
consuming" tasks in the right spot can sometimes bounce you out of the
apparent race condition.

Also your comment about promises vs async, I should further clarify my
other note.

Async in the migrations seems to make it much worse.
Async in the tests seems to alleviate the problem somewhat.

On Sat, Sep 9, 2017 at 6:18 PM, Sebastian C. Bürgel <
[email protected]> wrote:

Some observations, that might help:

• When contract is "too simple" (too fast?) this issue happens less
  frequently.
• When javascript is "too slow" this issue happens less frequently,
  e.g. a simple console.log before proceeding usually prevents the issue
  from happening at all.
• This issue usually surfaces more frequently with promises than with
  async.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/trufflesuite/truffle/issues/557#issuecomment-328306714,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABM8cr20acGl8Zz_xZh4maMODOUNllTmks5sgw6fgaJpZM4PLRD3
.

[tcoulter]

Hi there. Creator of Truffle here.

When your migrations are run, your contracts aren't being deployed. Instead, the migrations are staging deployment tasks to be later performed by the migration system. So the code:

deployer.deploy(A);

Doesn't actually deploy A. Instead it says, "Please deploy A for me." Making the request for A's deployment async doesn't make much sense in this context. You can think of the deployer as a deployment stager; it'll stage deployment tasks for later execution.

As of yet, Truffle's migration system doesn't yet support async. Your best bet at this moment is to not use async code in your migrations. And you don't really need to. For most tasks, say, two deployments, you can simply say:

deployer.deploy(A);
deployer.deploy(B); 

The deployer knows that the deployment of A is staged before B, and it'll run the deployments sequentially.

If you need more specialty deployments, like deploying B by passing A's address into B's constructor, you'd do something like this:

deployer.deploy(A).then(function() {
  return deployer.deploy(B, A.address);
});

This will ensure deployment of B happens after the deployment of A, and the code that evaluates A.address is also run after A is deployed.

So, with that in mind, this is all a long way of saying that async is the cause of your race condition. We'll likely upgrade the migration system some time in the future to support async/instantaneous deployments (i.e., the removal of staged deploys), but that will be after async is included in node's LTS support.

(As an aside, the reason your tests work well with async is because you're not using the deployer.)

I'm going to close this ticket as this isn't a bug, per se. Please see our documentation for more information on migrations.

Cheers!

CC @gnidan

[pmumby]

Hi Tim, and thanks for the detailed response.

Perhaps I shouldn't have used Async in my example, but as I commented in my
earlier note. This issue DOES present itself when no use of async exists.
(in either the tests or the migrations).

In my original project I had used promises with .then() throughout the
migrations and the tests. And that's where I first experienced it. Again as
complexity increases, probability of a randomly failed test increases.

Use of async in the tests seems to reduce this probability, and use of
async in the migrations seems to increase it. But the problem is still
present. And at the current size of my project even with pure promise based
code, I'm seeing failed tests very frequently. (with the exact same
behavior outlined in this issue).

My problem is that it seems the steps of the migration are not always
completed before the tests are run. And using for example A.deployed() to
get a promise for the eventual deployed contract which is queued, should
wait until the migration is successful, and then return. But it in fact
does not appear to (in some cases).

The example code was meant to demonstrate that, and the use of async seemed
to help it along, but I could modify that repo to exhibit the problem using
nothing but promises if that would be helpful.

On Sun, Sep 10, 2017 at 2:50 AM, Tim Coulter [e-mail]
wrote:

Hi there. Creator of Truffle here.

When your migrations are run, your contracts aren't being deployed.
Instead, they are staging deployment tasks to be later performed by the
migration system. So the code:

deployer.deploy(A);

Doesn't actually deploy A. Instead it says, "Please deploy A for me."
Making the request for A's deployment async doesn't make much sense in
this context. You can think of the deployer as a deployment stager; it'll
stage deployments tasks for later execution.

As of yet, Truffle's migration system doesn't yet support async. Your
best bet at this moment is to not use async code in your migrations. And
you don't really need to. For most tasks, say, two deployments, you can
simply say:

deployer.deploy(A);
deployer.deploy(B);

The deployer knows that the deployment of A is staged before B, and it'll
run the deployments sequentially.

If you need more specialty deployments, like deploying B by passing A's
address into B's constructor, you'd do something like this:

deployer.deploy(A).then(function() {
return deployer.deploy(B, A.address);
});

This will ensure deployment of B happens after the deployment of A, and
the code that evaluates A.address is also run after A is deployed.

So, with that in mind, this is all a long way of saying that async is the
cause of your race condition. We'll likely upgrade the migration system
some time in the future to support async/instantaneous deployments (i.e.,
the remove of staged deploys), but that will be after async is included
in node's LTS support.

(As an aside, the reason your tests work well with async is because
you're not using the deployer.)

I'm going to close this ticket as this isn't a bug, per se. Please see our
documentation
http://truffleframework.com/docs/getting_started/migrations for more
information on migrations.

Cheers!

CC @gnidan https://github.com/gnidan

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/trufflesuite/truffle/issues/557#issuecomment-328323842,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABM8clJGuwVsXxZQd5xWaqUHI_Pia5DBks5sg4bCgaJpZM4PLRD3
.

[tcoulter]

Before I look into this further, can you remove all async and await expressions from your migrations? The migrations themselves shouldn't be async, and you shouldn't be calling await with the deployer.

If migrations still aren't being set up correctly, then I'll look into this further. As it stands now, the most likely culprit is async/await.

[pmumby]

Hi Tim, thanks!

I've updated the example repo. All migrations have been updated to use
promises. All references to async/await have been removed from the 4
migration files:
1_initial_migration.js
2_deploy_a.js
3_deploy_b.js
4_setup.js

It should also be noted, I've previously verified this issue continues to
exist even if I condense these down to a single migration script which does
the deployment/setup of all contracts all with proper chained promises as
well. The same problem occurs.

I've verified this is still failing at least 50% of the time (in my case it
failed 3 times, before I got a successful test).

Thanks for looking into it further!

On Sun, Sep 10, 2017 at 12:52 PM, Tim Coulter [e-mail]
wrote:

Before I look into this further, can you remove all async and await
expressions from your migrations? The migrations themselves shouldn't be
async, and you shouldn't be calling await with the deployer.

If migrations still aren't being set up correctly, then I'll look into
this further. As it stands now, the most likely culprit is async/await.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/trufflesuite/truffle/issues/557#issuecomment-328355636,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABM8cka5JrMxL2pOwH09Z_rmPL0dCPp9ks5shBPFgaJpZM4PLRD3
.

[SCBuergel]

Same thing here, failing for a vanilla migrations script without async if
1) the contract is sufficiently complex AND
2) the JS test code is sufficiently efficient (no console.logs, statements ideally inlined)

[tcoulter]

If using deployer.then(), which you are in the fourth migration, you should return further promises so they're included in the chain. e.g., this line will run asynchronously because the enclosing .then() doesn't know they're executing.

TL;DR: stick a return on the start of that line, it should fix your issue.

[tcoulter]

Better yet, make it a single chain, without nested promises:

    var aInstance;
    var bInstance;
    deployer.then(function(){
        return ContractA.deployed();
    }).then(function(instance){
        aInstance = instance;
        return ContractB.deployed();
    }).then(function(instance){
        bInstance = instance;
    }).then(function(){            
        aInstance.setBAddress(bInstance.address);
        bInstance.setAAddress(aInstance.address);
        console.log("Setup Migration Done!");
    });

[tcoulter]

And I just noticed: aInstance.setBAddress() is a transaction, hence, a promise. You should do these individually (or use Promise.all() to put them in one .then()):

    var aInstance;
    var bInstance;
    deployer.then(function(){
        return ContractA.deployed();
    }).then(function(instance){
        aInstance = instance;
        return ContractB.deployed();
    }).then(function(instance){
        bInstance = instance;
    }).then(function(){            
        return aInstance.setBAddress(bInstance.address);
    }).then(function() {
        return bInstance.setAAddress(aInstance.address);
    }).then(function() {
        console.log("Setup Migration Done!");
    });

[JonathanScialpi]

This article helped me resolve this issue and made my code much cleaner: https://medium.com/coinmonks/testing-solidity-with-truffle-and-async-await-396e81c54f93

[Shirikatsu]

Is this still the same on latest versions?

Now we can simply run truffle test without prior compilation or deployment and it gets compiled and deployed on the fly. This means we don't have control over how this happens yet I am still having issues where there seem to be race conditions and tests fail mid-execution throwing errors like:

{ AssertionError: expected promise to be rejected but it was fulfilled with { Object (tx, receipt, ...) }
  message: 'expected promise to be rejected but it was fulfilled with { Object (tx, receipt, ...) }',
  showDiff: true,
  actual: 
   { tx: '0xeb51cef6708fe4a17c6a9f4eca4de0178795c3ae1738e3d1fc74931fe7ea72a7',
     receipt: 
      { transactionHash: '0xeb51cef6708fe4a17c6a9f4eca4de0178795c3ae1738e3d1fc74931fe7ea72a7',
        transactionIndex: 0,
        blockHash: '0xd0f18a5f18a03001e7b63554834712634578bc4ab9c10dcf3fe3d5b0bc553fe2',
        blockNumber: 172,
        from: '0xf8dee77490590ec7c9856caa79bf680474cac8af',
        to: '0x0e60af7ee2e62e24f39de38ee052907357cfa07e',
        gasUsed: 50648,
        cumulativeGasUsed: 50648,
        contractAddress: null,
        logs: [],
        status: true,
        logsBloom: '0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
        v: '0x1c',
        r: '0x3de661523731aa289b925600eebe1dd7e2a33ffe201dd1b9102b1d0d6351abf7',
        s: '0x32346a8b7b0345a5660258059b734066cf58c5b8d83058646becc41b7d2a5949',
        rawLogs: [] },
     logs: [] },
  expected: undefined }
TruffleConfig {
  _deepCopy: [ 'compilers' ],
  _values: 
   { truffle_directory: '/home/cc/Repositories/ion/node_modules/truffle',
     working_directory: '/home/cc/Repositories/ion',
     network: 'development',
     networks: { development: [Object], clique: [Object], coverage: [Object] },
     verboseRpc: false,
     gas: null,
     gasPrice: null,
     from: null,
     confirmations: 0,
     timeoutBlocks: 0,
     production: false,
     skipDryRun: false,
     build: null,
     resolver: 
      TestResolver {
        resolver: [Object],
        source: [Object],
        search_path: '/tmp/test-1191019-11489-7ctub6.kmz2',
        seen: [],
        require_cache: {},
        cache_on: true },
     artifactor: Artifactor { destination: '/tmp/test-1191019-11489-7ctub6.kmz2' },
     ethpm: 
      { ipfs_host: 'ipfs.infura.io',
        ipfs_protocol: 'https',
        registry: '0x8011df4830b4f696cd81393997e5371b93338878',
        install_provider_uri: 'https://ropsten.infura.io/truffle' },
     ens: { enabled: false, registryAddress: null },
     compilers: { solc: [Object], vyper: {} },
     logger: 
      Console {
        log: [Function: bound consoleCall],
        debug: [Function: bound consoleCall],
        info: [Function: bound consoleCall],
        warn: [Function],
        error: [Function: bound consoleCall],
        dir: [Function: bound consoleCall],
        time: [Function: bound consoleCall],
        timeEnd: [Function: bound consoleCall],
        trace: [Function: bound consoleCall],
        assert: [Function: bound consoleCall],
        clear: [Function: bound consoleCall],
        count: [Function: bound consoleCall],
        countReset: [Function: bound countReset],
        group: [Function: bound consoleCall],
        groupCollapsed: [Function: bound consoleCall],
        groupEnd: [Function: bound consoleCall],
        Console: [Function: Console],
        dirxml: [Function: dirxml],
        table: [Function: table],
        markTimeline: [Function: markTimeline],
        profile: [Function: profile],
        profileEnd: [Function: profileEnd],
        timeline: [Function: timeline],
        timelineEnd: [Function: timelineEnd],
        timeStamp: [Function: timeStamp],
        context: [Function: context],
        [Symbol(counts)]: Map {} },
     build_directory: '/home/cc/Repositories/ion/build',
     contracts_directory: '/home/cc/Repositories/ion/contracts',
     contracts_build_directory: '/tmp/test-1191019-11489-7ctub6.kmz2',
     migrations_directory: '/home/cc/Repositories/ion/migrations',
     migrations_file_extension_regexp: /^\.(js|es6?)$/,
     test_directory: '/home/cc/Repositories/ion/test',
     test_file_extension_regexp: /.*\.(js|ts|es|es6|jsx|sol)$/,
     example_project_directory: '/home/cc/Repositories/ion/node_modules/truffle/example' },
  truffle_directory: '/home/cc/Repositories/ion/node_modules/truffle',
  working_directory: '/home/cc/Repositories/ion',
  network: 'development',
  networks: 
   { development: 
      { host: 'localhost',
        port: 8545,
        gas: 4503599627370495,
        network_id: 1234,
        from: '0xf8DEE77490590ec7c9856CAa79bf680474Cac8AF' },
     clique: { host: 'localhost', port: 8501, network_id: '*' },
     coverage: 
      { host: 'localhost',
        port: 8555,
        network_id: '*',
        gas: 268435455,
        gasprice: 1 } },
  verboseRpc: false,
  build: null,
  resolver: 
   Resolver {
     options: 
      TruffleConfig {
        _deepCopy: [Array],
        _values: [Object],
        truffle_directory: [Getter/Setter],
        working_directory: [Getter/Setter],
        network: [Getter/Setter],
        networks: [Getter/Setter],
        verboseRpc: [Getter/Setter],
        build: [Getter/Setter],
        resolver: [Getter/Setter],
        artifactor: [Getter/Setter],
        ethpm: [Getter/Setter],
        logger: [Getter/Setter],
        compilers: [Getter/Setter],
        ens: [Getter/Setter],
        build_directory: [Getter/Setter],
        contracts_directory: [Getter/Setter],
        contracts_build_directory: [Getter/Setter],
        migrations_directory: [Getter/Setter],
        migrations_file_extension_regexp: [Getter/Setter],
        test_directory: [Getter/Setter],
        test_file_extension_regexp: [Getter/Setter],
        example_project_directory: [Getter/Setter],
        network_id: [Getter/Setter],
        network_config: [Getter/Setter],
        from: [Getter/Setter],
        gas: [Getter/Setter],
        gasPrice: [Getter/Setter],
        provider: [Getter/Setter],
        confirmations: [Getter/Setter],
        production: [Getter/Setter],
        timeoutBlocks: [Getter/Setter],
        mocha: [Object],
        solc: [Object],
        _: [],
        'show-events': false,
        showEvents: false,
        debug: false,
        'runner-output-only': false,
        runnerOutputOnly: false,
        'debug-global': 'debug',
        debugGlobal: 'debug' },
     sources: [ [Object], [Object], GlobalNPM {}, [Object] ] },
  artifactor: Artifactor { destination: '/tmp/test-1191019-11489-7ctub6.kmz2' },
  ethpm: 
   { ipfs_host: 'ipfs.infura.io',
     ipfs_protocol: 'https',
     registry: '0x8011df4830b4f696cd81393997e5371b93338878',
     install_provider_uri: 'https://ropsten.infura.io/truffle' },
  logger: 
   Console {
     log: [Function: bound consoleCall],
     debug: [Function: bound consoleCall],
     info: [Function: bound consoleCall],
     warn: [Function],
     error: [Function: bound consoleCall],
     dir: [Function: bound consoleCall],
     time: [Function: bound consoleCall],
     timeEnd: [Function: bound consoleCall],
     trace: [Function: bound consoleCall],
     assert: [Function: bound consoleCall],
     clear: [Function: bound consoleCall],
     count: [Function: bound consoleCall],
     countReset: [Function: bound countReset],
     group: [Function: bound consoleCall],
     groupCollapsed: [Function: bound consoleCall],
     groupEnd: [Function: bound consoleCall],
     Console: [Function: Console],
     dirxml: [Function: dirxml],
     table: [Function: table],
     markTimeline: [Function: markTimeline],
     profile: [Function: profile],
     profileEnd: [Function: profileEnd],
     timeline: [Function: timeline],
     timelineEnd: [Function: timelineEnd],
     timeStamp: [Function: timeStamp],
     context: [Function: context],
     [Symbol(counts)]: Map {} },
  compilers: { solc: { settings: [Object] }, vyper: {} },
  ens: { enabled: false, registryAddress: null },
  build_directory: '/home/cc/Repositories/ion/build',
  contracts_directory: '/home/cc/Repositories/ion/contracts',
  contracts_build_directory: '/tmp/test-1191019-11489-7ctub6.kmz2',
  migrations_directory: '/home/cc/Repositories/ion/migrations',
  migrations_file_extension_regexp: /^\.(js|es6?)$/,
  test_directory: '/home/cc/Repositories/ion/test',
  test_file_extension_regexp: /.*\.(js|ts|es|es6|jsx|sol)$/,
  example_project_directory: '/home/cc/Repositories/ion/node_modules/truffle/example',
  network_id: 1234,
  network_config: 
   { gas: 4503599627370495,
     gasPrice: 20000000000,
     from: '0xf8DEE77490590ec7c9856CAa79bf680474Cac8AF',
     host: 'localhost',
     port: 8545,
     network_id: 1234 },
  from: '0xf8DEE77490590ec7c9856CAa79bf680474Cac8AF',
  gas: 4503599627370495,
  gasPrice: 20000000000,
  provider: 
   HttpProvider {
     host: 'http://localhost:8545',
     httpAgent: 
      Agent {
        domain: null,
        _events: [Object],
        _eventsCount: 1,
        _maxListeners: undefined,
        defaultPort: 80,
        protocol: 'http:',
        options: [Object],
        requests: {},
        sockets: {},
        freeSockets: {},
        keepAliveMsecs: 1000,
        keepAlive: false,
        maxSockets: Infinity,
        maxFreeSockets: 256 },
     withCredentials: false,
     timeout: 0,
     headers: undefined,
     connected: false,
     send: [Function],
     _alreadyWrapped: true },
  confirmations: undefined,
  production: undefined,
  timeoutBlocks: undefined,
  mocha: 
   { useColors: true,
     enableTimeouts: false,
     reporterOptions: undefined,
     files: 
      [ '/home/cc/Repositories/ion/test/clique.js',
        '/home/cc/Repositories/ion/test/helpers/encoder.js',
        '/home/cc/Repositories/ion/test/helpers/utils.js',
        '/home/cc/Repositories/ion/test/ibft.js',
        '/home/cc/Repositories/ion/test/integration-base_fabric.js',
        '/home/cc/Repositories/ion/test/integration-clique_ethereum.js',
        '/home/cc/Repositories/ion/test/ion.js',
        '/home/cc/Repositories/ion/test/patricia_trie_test.js',
        '/home/cc/Repositories/ion/test/storage-ethereum.js',
        '/home/cc/Repositories/ion/test/storage-fabric.js' ] },
  solc: { optimizer: { enabled: true, runs: 200 } },
  _: [],
  'show-events': false,
  showEvents: false,
  debug: false,
  'runner-output-only': false,
  runnerOutputOnly: false,
  'debug-global': 'debug',
  debugGlobal: 'debug',
  test_files: 
   [ '/home/cc/Repositories/ion/test/clique.js',
     '/home/cc/Repositories/ion/test/helpers/encoder.js',
     '/home/cc/Repositories/ion/test/helpers/utils.js',
     '/home/cc/Repositories/ion/test/ibft.js',
     '/home/cc/Repositories/ion/test/integration-base_fabric.js',
     '/home/cc/Repositories/ion/test/integration-clique_ethereum.js',
     '/home/cc/Repositories/ion/test/ion.js',
     '/home/cc/Repositories/ion/test/patricia_trie_test.js',
     '/home/cc/Repositories/ion/test/storage-ethereum.js',
     '/home/cc/Repositories/ion/test/storage-fabric.js' ] }
/home/cc/Repositories/ion/node_modules/truffle/node_modules/mocha/lib/runner.js:726
  err.uncaught = true;
               ^

TypeError: Cannot create property 'uncaught' on string 'abort({"name":"AssertionError","message":"expected promise to be rejected but it was fulfilled with { Object (tx, receipt, ...) }","showDiff":true,"actual":{"tx":"0xeb51cef6708fe4a17c6a9f4eca4de0178795c3ae1738e3d1fc74931fe7ea72a7","receipt":{"transactionHash":"0xeb51cef6708fe4a17c6a9f4eca4de0178795c3ae1738e3d1fc74931fe7ea72a7","transactionIndex":0,"blockHash":"0xd0f18a5f18a03001e7b63554834712634578bc4ab9c10dcf3fe3d5b0bc553fe2","blockNumber":172,"from":"0xf8dee77490590ec7c9856caa79bf680474cac8af","to":"0x0e60af7ee2e62e24f39de38ee052907357cfa07e","gasUsed":50648,"cumulativeGasUsed":50648,"contractAddress":null,"logs":[],"status":true,"logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","v":"0x1c","r":"0x3de661523731aa289b925600eebe1dd7e2a33ffe201dd1b9102b1d0d6351abf7","s":"0x32346a8b7b0345a5660258059b734066cf58c5b8d83058646becc41b7d2a5949","rawLogs":[]},"logs":[]},"stack":"AssertionError: expected promise to be rejected but it was fulfilled with { Object (tx, receipt, ...) }"}). Build with -s ASSERTIONS=1 for more info.'
    at Runner.uncaught (/home/cc/Repositories/ion/node_modules/truffle/node_modules/mocha/lib/runner.js:726:16)
    at process.uncaught (/home/cc/Repositories/ion/node_modules/truffle/node_modules/mocha/lib/runner.js:839:10)
    at emitOne (events.js:116:13)
    at process.emit (events.js:211:7)
    at process.emit (/home/cc/Repositories/ion/node_modules/truffle/build/webpack:/~/source-map-support/source-map-support.js:465:1)
    at process._fatalException (bootstrap_node.js:375:26)

Changing some of the functions from async to not changes where this error happens but it still occurs with the same error message.

[gnidan]

@Shirikatsu no way this is the same issue as the original. Mind opening your issue as a new one, to reduce the noise? Thanks!

[Shirikatsu]

Sure! Sorry :)

[Nicca42]

Has this issue been opened with Ganache? I don't use truffle at all (using Etherlime) and I get the same inconsistent failing tests. (Same issues as mentioned by others above, a big contract ecosystem, complex testing and every couple of times when testing a random test will fail).