Solidity-coverage: Cannot find generated artifacts

[wighawag]

I am using hardhat-deploy with solidity-coverage so there might be some incompatibilities but cannot find any obvious one

The issue I have is that there seems to be no generated artifacts in the temp folder, even though the .coverage_cache contains solidity-files-cache.json

I commented the line that delete the temp folder and after execution of hardhat coverage the folder exist but is empty.

Any idea ?

[cgewecke]

Hi @wighawag,

Do you have an example I could try to reproduce with?

There is a basic e2e test at https://github.com/sc-forks/hardhat-e2e that we're using in CI here.

To run solidity-coverage with it you would

git clone https://github.com/sc-forks/hardhat-e2e.git
cd hardhat-e2e
npm install
npm install --save-dev solidity-coverage
npx hardhat coverage

[wighawag]

I have this repo (branch: solidity-coverage) : https://github.com/wighawag/template-ethereum-contracts/tree/solidity-coverage

[cgewecke]

It works when running...

npx hardhat coverage --temp artifacts

--temp specifies an alternate temporary folder to store coverage-instrumented artifacts in.

Does the deploy plugin also run the compile task?

---

[wighawag]

interesting

It seems that when not using --temp artifacts, the temp artifacts do not contains any files and hardhat hre.artifacts.readArtifact works because it have the path of artifact set before solidity-coverage is executed
while the plugin hardhat-deploy use config.paths.artifacts to get the artifacts

[wighawag]

I guess the artifacts are suposed to be generated in the temp folder , right ?

[wighawag]

note that generating temp in the default artifacts folder (but saving the cache in a different folder than the default cache folder) will make hardhat caching fails as it will keep using soldity-coverage artifacts unless changes to file happen

[cgewecke]

It seems that when not using --temp artifacts, the temp artifacts do not contains any files and hardhat hre.artifacts.readArtifact works because it have the path of artifact set before solidity-coverage is executed
while the plugin hardhat-deploy use config.paths.artifacts to get the artifacts

Yes I think you're right that something is going wrong with the sequence that things are read in.

solidity-coverage resets the path values (including cache) here:

https://github.com/sc-forks/solidity-coverage/blob/8660166ae09963e0b27f3ff1a7c1ed88bee2038a/plugins/hardhat.plugin.js#L138-L139

[wighawag]

yes, hardhat-deploy read the path for artifacts from these newly update paths but because there is no files there it fails to get them (unless --temp artifacts
on the other hand anything based on hre.artifacts.readArtifact will read from the path originally set in hardhat.config so no temp folder
And I guess solidity-coverage expect that hardhat will write the artifacts in config.paths.artifacts but hardhat seems to already have decided where it is based on the value set in hardhat.config.

So hardhat should allow plugin to update the path, either by having hardhat read config.paths each time or by having an api for the plugins to update it

[cgewecke]

So hardhat should allow plugin to update the path, either by having hardhat read config.paths each time or by having an api for the plugins to update it

Mmm, I think under normal circumstances it does allow this and the problem is coming from plugin interaction. Normally the temporary artifacts are saved to the temp folder and HH reads from them correctly.

[wighawag]

Without hardhat-deploy I can see that the temp artifact folder are empty too.
I think what is happening is that solidity-coverage overwrite the artifacts in the artfiacts folder, not in the temp folder
and it works only because hre.artifacts.readArtifacts also read from there

[cgewecke]

Oh I see. You might be right that this is hre.artifacts.readArtifact specific. The e2e test I have and other testing I've done is using artifacts.require via the truffle-v5 plugin.

[wighawag]

I guess artifacts.require use hre.artifacts.readArtifacts hence why it works

see here where the artifacts is set at env construction time : https://github.com/nomiclabs/hardhat/blob/6ffb8baf0ede8c8c73b502bba826b0f8196604b2/packages/hardhat-core/src/internal/core/runtime-environment.ts#L77

[wighawag]

and this is used for writing too : https://github.com/nomiclabs/hardhat/blob/6ffb8baf0ede8c8c73b502bba826b0f8196604b2/packages/hardhat-core/src/builtin-tasks/compile.ts#L801

[cgewecke]

@wighawag To be clear, you don't set the paths anywhere in your code? Only read?

[wighawag]

yes only read

[cgewecke]

Ah ok ok I just re-ran my e2e an now I see exactly what you're saying. Something's seriously wrong.

Thanks for helping with all of this, sorry.

[wighawag]

No problem, glad to be of help. solidity-coverage is a great tool :)

[cgewecke]

@wighawag This should be fixed with 0.7.13 in combination with Hardhat 2.0.4. There's no longer any need to use the --temp flag and solidity-coverage has stopped trying to modify the paths artifacts & cache values.

Have also added your a fork of your ethereum-template-contracts project as a smoke-test in CI here and everything seems to work as expected.

Just lmk if you see any further problems on your side.