Truffle: Avoid compilation when doing tests

[JustinDrake]

I am writing tests (leaving the contract code intact) but truffle will recompile every time I run truffle test. Ideally truffle test would be smart about recompiling only files that have changed. As a quick fix, a flag to skip compilation would do for me 👍

[JustinDrake]

I see there's the --compile-all option. I wonder why the default contracts generated by truffle init get recompiled every time.

--compile-all: Compile all contracts instead of intelligently choosing.

[jellegerbrandy]

I have the same problem:

jelle@:~/projects ★ truffle version
Truffle v3.3.0, bundle version: 3.3.1
Solidity v0.4.11 (solc-js)
jelle@:~/projects/daostack/play ★ mkdir play
jelle@:~/projects/daostack/play ★ cd play
jelle@:~/projects/daostack/play ★ truffle init
Downloading project...
Project initialized.

  Documentation: http://truffleframework.com/docs

Commands:

  Compile: truffle compile
  Migrate: truffle migrate
  Test:    truffle test

jelle@:~/projects/play ★ truffle compile
Compiling ./contracts/ConvertLib.sol...
Compiling ./contracts/MetaCoin.sol...
Compiling ./contracts/Migrations.sol...
Writing artifacts to ./build/contracts

jelle@:~/projects/play ★ truffle test
Using network 'development'.

Compiling ./contracts/ConvertLib.sol...
Compiling ./contracts/MetaCoin.sol...
Compiling ./test/TestMetacoin.sol...
Compiling truffle/Assert.sol...
Compiling truffle/DeployedAddresses.sol...


  TestMetacoin
    ✓ testInitialBalanceUsingDeployedContract (44ms)
    ✓ testInitialBalanceWithNewMetaCoin (38ms)

  Contract: MetaCoin
    ✓ should put 10000 MetaCoin in the first account
    ✓ should call a function that depends on a linked library (64ms)
    ✓ should send coin correctly (120ms)


  5 passing (594ms)

jelle@:~/projects/play ★ truffle test
Using network 'development'.

Compiling ./contracts/ConvertLib.sol...
Compiling ./contracts/MetaCoin.sol...
Compiling ./test/TestMetacoin.sol...
Compiling truffle/Assert.sol...
Compiling truffle/DeployedAddresses.sol...


  TestMetacoin
    ✓ testInitialBalanceUsingDeployedContract
    ✓ testInitialBalanceWithNewMetaCoin

  Contract: MetaCoin
    ✓ should put 10000 MetaCoin in the first account
    ✓ should call a function that depends on a linked library (59ms)
    ✓ should send coin correctly (120ms)


  5 passing (572ms)

[dstreppa]

Not the solution for you, but maybe the following command can be helpful:

$ truffle test test/TestScrooge.sol 

Only TestScrooge.sol and its imported contracts will be compiled by Truffle, and only the tests in TestScrooge.sol will be executed.

[jellegerbrandy]

i resolved this for my case: apparently, the problem was due to outdated solc or solc-cli packages.

[travs]

@tcoulter
Is this a duplicate of the second part of #343 ?

[dkent600]

You could also try running truffle compile separately before running truffle test. Has worked for me.

[PaulLaux]

Had the same problem, fixed by compiling in advance for the same network that is used for the tests:

first:
truffle.cmd compile --network testrpc
then:
truffle.cmd test --network testrpc test\bigTest.test.js

[zulhfreelancer]

Running truffle compile followed by truffle test [path] did not work for me. I'm using Truffle v4.1.13.

Everytime I ran truffle test [path], all other contracts get compiled and deployed.

I'm expecting only the contracts related in my test will get compiled and deployed. Not everything.

Update

I found a workaround for this. I create an empty folder in my Truffle project called migrations_null and I put nothing inside it.

Next, I ran the test with --migrations_directory flag and it worked. Other contracts didn't get compiled and migrated.

$ truffle tests ./test/MyTest.js --migrations_directory migrations_null

[eduardonunesp]

@gnidan any fix for this issue? beyond the workaround? for version 4.1.15

[ferittuncer]

There should be an option to disable compiling also.

I needed to debug a flaky test situation so I wanted to run a test a hundred times to be sure. But compiling a hundred times is totally unnecessary and a waste of time.

[robertmagier]

I have exactly same problem. Would be nice if one can avoid compilation every time test is executed.

[gnidan]

We will look into this. Thanks!

[AC0DEM0NK3Y]

Adding to this, with v4 I was able to work around this with these steps:

truffle compile --network test --all
truffle test --network test

With v5 if you compile first as above, the test runs/hangs indefinitely with no output. I have to wipe the build folder to get back into a working state and then have truffle test do the compilation every time.

[CruzMolina]

Hmm, so I don't believe this is a problem any more on the latest truffle (v5.0.10).

If anyone has a repo they could share to reproduce, I'll get right to debugging. 😎

[rogerioyuuki]

I also have this problem on truffle v5.0.13. I used a modified version of the updated function to debug this issue.

On line 42, it finds all contracts and creates an empty array using the path as a key.

On line 83, it should look at the artifacts json files and pushes the json in the empty array created before, using data.sourcePath as a key. However, in my case, line 42 was getting relative paths and data.sourcePath contained absolute paths, so instead of using the previosly created arrays, it was creating another key in the object.

So, for the rest of the function, truffle was believing that some contracts didn't have artifacts json files, and the result of that is the recompilation of these contracts.

I don't know why they were different since it seems to work for some people and they were only different for some contracts. I'm on Windows, but the same thing happened inside a Docker container (node:8 image). Maybe some path normalization can fix this issue?

[CruzMolina]

Thanks for that info @rogerioyuuki , looking into this. Hopefully there's a quick fix.

[CruzMolina]

@rogerioyuuki , I'm on macOS and when using v5.0.13, if I truffle compile and then truffle test, the only contracts compiled/recompiled are any solidity unit tests (i.e. TestContract.sol).

Can you verify what the behavior is on your end when doing the above?

[rogerioyuuki]

@CruzMolina I think my problem might be Windows related.

Instead of running a single script similar of that updated function, I added a few console.log directly in my installed truffle cli.bundled.js to get more accurate information.

What I found is similar to what I described, with one further addition. I also found another type of path mismatch, but instead of absolute-relative mismatch, there is a mix of paths with inverted slash and forward slash and it causes the same problem. It happens even on files that I didn't create (from openzeppelin dependency, installed via npm). I don't understand why the slashes are getting mixed up. My guess is that contracts that are found via import works differently than not imported contracts.

> Compiling openzeppelin-solidity/contracts/math/SafeMath.sol
> Compiling openzeppelin-solidity/contracts/ownership/Ownable.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/ERC20.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/ERC20Burnable.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/ERC20Detailed.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/ERC20Mintable.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/ERC20Pausable.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/IERC20.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/SafeERC20.sol
> Compiling openzeppelin-solidity\contracts\access\Roles.sol
> Compiling openzeppelin-solidity\contracts\access\roles\MinterRole.sol
> Compiling openzeppelin-solidity\contracts\access\roles\PauserRole.sol
> Compiling openzeppelin-solidity\contracts\lifecycle\Pausable.sol
> Compiling openzeppelin-solidity\contracts\math\SafeMath.sol
> Compiling openzeppelin-solidity\contracts\token\ERC20\ERC20.sol
> Compiling openzeppelin-solidity\contracts\token\ERC20\IERC20.sol

---

Reproduction
I was able to create a brand new truffle project and break this behaviour.

On a new project ( created with truffle init), I added these contracts:
contracts/HelloWorld.sol (copied from https://www.ethereum.org/greeter)

pragma solidity >=0.4.22 <0.6.0;

contract Mortal {
    /* Define variable owner of the type address */
    address owner;

    /* This constructor is executed at initialization and sets the owner of the contract */
    constructor() public { owner = msg.sender; }

    /* Function to recover the funds on the contract */
    function kill() public { if (msg.sender == owner) selfdestruct(msg.sender); }
}

contract Greeter is Mortal {
    /* Define variable greeting of the type string */
    string greeting;

    /* This runs when the contract is executed */
    constructor(string memory _greeting) public {
        greeting = _greeting;
    }

    /* Main function */
    function greet() public view returns (string memory) {
        return greeting;
    }
}

contracts/Test.sol

pragma solidity >=0.4.22 <0.6.0;

import "./HelloWorld.sol";

contract Test is Mortal {
    function greet() public view returns (uint) {
        return 1;
    }
}

When you first run truffle compile, I can see it compiled HelloWorld.sol twice:

> Compiling .\contracts\HelloWorld.sol
> Compiling .\contracts\Migrations.sol
> Compiling .\contracts\Test.sol
> Compiling .\contracts\HelloWorld.sol

And when running truffle test it recompiles HelloWorld.sol even without modifications. I logged the paths on that updated function on lines 43 and 90 and the return of that function. Here is what I got:

43: C:/Projetos/truffle-debug2/contracts/HelloWorld.sol
43: C:/Projetos/truffle-debug2/contracts/Migrations.sol
43: C:/Projetos/truffle-debug2/contracts/Test.sol
90: C:\Projetos\truffle-debug2\contracts\HelloWorld.sol
90: C:/Projetos/truffle-debug2/contracts/Migrations.sol
90: C:\Projetos\truffle-debug2\contracts\HelloWorld.sol
90: C:/Projetos/truffle-debug2/contracts/Test.sol
[ 'C:/Projetos/truffle-debug2/contracts/HelloWorld.sol' ]

And looking at the artifacts json files, both contracts Greeter and Mortal have the "sourcePath" with \, and I think it's related with that twice compilation of HelloWorld.sol.

I guessed that it had to do with imports because you can change this project to have the absolute and relative path mismatch I mentioned earlier. Change the import to import "contracts/HelloWorld.sol"; and the result now is:
truffle compile

> Compiling .\contracts\HelloWorld.sol
> Compiling .\contracts\Migrations.sol
> Compiling .\contracts\Test.sol
> Compiling contracts/HelloWorld.sol

updated function:

43: C:/Projetos/truffle-debug2/contracts/HelloWorld.sol
43: C:/Projetos/truffle-debug2/contracts/Migrations.sol
43: C:/Projetos/truffle-debug2/contracts/Test.sol
90: contracts/HelloWorld.sol
90: C:/Projetos/truffle-debug2/contracts/Migrations.sol
90: contracts/HelloWorld.sol
90: C:/Projetos/truffle-debug2/contracts/Test.sol
[ 'C:/Projetos/truffle-debug2/contracts/HelloWorld.sol' ]

---

Obs: I'm on v5.0.13

$ truffle version
Truffle v5.0.13 (core: 5.0.13)
Solidity v0.5.0 (solc-js)
Node v8.12.0
Web3.js v1.0.0-beta.37

PS: To add more context why this matters, it's not only an optimization issue. I was trying to generate coverage reports to my tests, but some tools rely on intermediate steps between the compilation and the tests that gets overridden if truffle recompiles during truffle test. My current workaround is to generate coverage reports inside a special Docker container with a patched truffle that never recompiles the test. I think this may be a valid use case for a flag that forces no recompilation.

[CruzMolina]

Hey @rogerioyuuki , thanks again for looking into this.

Can you go into this section of the code locally https://github.com/trufflesuite/truffle/blob/develop/packages/truffle-compile/index.js#L58-L79

and replace it with:

  Object.keys(sources).forEach(function(source) {
    var replacement = path.resolve(source);

    // Save the result
    operatingSystemIndependentSources[replacement] = sources[source];

    // Just substitute replacement for original in target case. It's
    // a disposable subset of `sources`
    if (hasTargets && options.compilationTargets.includes(source)) {
      operatingSystemIndependentTargets[replacement] = sources[source];
    }

    // Map the replacement back to the original source path.
    originalPathMappings[replacement] = source;
  });

Hoping this might help. Unfortunately I don't have access to Windows to debug this locally.

[CruzMolina]

An alternative suspect is in truffle-resolver and the truffle test resolver.

Here https://github.com/trufflesuite/truffle/blob/develop/packages/truffle-resolver/fs.js#L13-L15
and here https://github.com/trufflesuite/truffle/blob/develop/packages/truffle-core/lib/testing/testresolver.js#L18-L20

[AC0DEM0NK3Y]

Still an issue with this setup (using windows):

Truffle v5.0.20 (core: 5.0.20)
Solidity - 0.5.9 (solc-js)
Node v10.15.0
Web3.js v1.0.0-beta.37

Seems to be possibly unrelated to tests as I get the same recompile if I just run "truffle compile" one after the other after wiping the build folder.

[flockonus]

Still an significant _nice to have_ in big projects where there is a lot of source code to compile.

If I run truffle build just before testing, still some files are compiled and it takes roughly the same time as compiling all the files.

The motivation is that often I have my Solidity source but don't actually alter it that at all, just the javascript test files, many times, as I re-run the tests.

[pizzarob]

Just an update - still happens on 5.0.30

[wjmelements]

Because we are working around another issue we are redoing truffle test once per test file. Not having this feature means that we recompile the contracts for each test file, which almost doubles test time.

[eggplantzzz]

Truffle no longer compiles your contracts if they are up-to-date with your built artifacts during testing. It is mandatory, however, that Truffle compile any Solidity tests that you have as it does not store those as artifact files. If you want to ensure that none of your non-test artifacts are compiled you can also use the --compile-none flag to skip the check to see if the built files are up-to-date.

[cbruguera]

This is absolutely still happening in Truffle 5.1.40... @eggplantzzz Which version is this supposed to be fixed?

[drbullock]

I have All Issues described by @rogerioyuuki in Truffle v5.1.49

• Mixed slashes
• Most of the contracts are double compiled
• truffle compile re-compiles unchanged contracts

Compiling @openzeppelin/contracts-ethereum-package/contracts/GSN/Context.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/Initializable.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/access/AccessControl.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/access/Ownable.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/math/SafeMath.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/token/ERC20/ERC20.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/token/ERC20/ERC20Burnable.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/token/ERC20/SafeERC20.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/utils/ReentrancyGuard.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\GSN\Context.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\Initializable.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\math\SafeMath.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\token\ERC20\ERC20.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\token\ERC20\IERC20.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\utils\Address.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\utils\EnumerableSet.sol
Compiling .\contracts\Migrations.sol
Compiling .\contracts\Modules\AddressProvider.sol
Compiling .\contracts\Modules\SchoolModule.sol
Compiling .\contracts\Modules\OracleModule.sol
Compiling .\contracts\Modules\ClassModule.sol
Compiling .\contracts\City\City.sol
Compiling .\contracts\inc\Base.sol
Compiling .\contracts\inc\CoreLibrary.sol
Compiling .\contracts\Modules\AddressProvider.sol
Compiling .\contracts\Modules\SchoolModule.sol
Compiling .\contracts\Modules\OracleModule.sol
Compiling .\contracts\Modules\ClassModule.sol
Compiling .\contracts\City\City.sol
Compiling .\contracts\inc\Base.sol
Compiling .\contracts\inc\CoreLibrary.sol

[drbullock]

@eggplantzzz please reopen this issue

[cbruguera]

Am I missing something? (Truffle v5.1.40)

Note: this is probably way more important than currently regarded. Development times are severely hindered during test implementation due to unnecessary frequent compiling.