Solidity: Swarm hash in the middle of the bytecode prevents verification with etherscan

[yaronvel]

I compiled a file that imports several files (including one library).
Looking at the bytecode I see to occurrences of bzzr0. One roughly in the middle of the code, and one in the end of the bytecode.

I am guessing the first bzzr0 might be the result of contacting a bytecode of some imported file, but I am not sure.

The relevant solidity code can be found here.

EDIT: Some more details. It seems to happen because in the contract I deploy another contract. Without it there is only one swarm hash.

[veox]

Are you sure this is due to the swarm hash, and not mismatch of compiler version, or other issue on etherscan's side?

Is this on main-net, or one of the testnets?

Could you link the deployed contract?

[veox]

Oh, and which Solidity version? EDIT: I mean the actual compiler binary. OS info could also come useful.

[yaronvel]

@veox as I said, it is not a bug, and I think I understand why it happens.
Eventually I was able to verify it on etherscan with some help I got and some unofficial script that concat files in the exact order the compiler does.

The code can be found here
https://etherscan.io/address/0xd6cd31f283d24cfb442cba1bcf42290c07c15792#code

If you look at its binary you will see that bzzr appears twice. Meaning that extra spaces could cause verification to fail.

Again, I think the 1st bzzr is because I deploy a contract from the contract code. So the deployed contract must have swarm hash as well. However, it makes it hard to verify in etherscan.

[axic]

Why does it make it hard to verify on etherscan?

Every contract has its own metadata and metadata hash, that includes contracts created by other contracts.

[yaronvel]

@axic afaik if there is only one contract, then etherscan just ignores last 32 bytes (swarm hash).
Otherwise, extra whitespaces in the code could fail the verification.
However, this approach fails if there is a metadata hash also in the middle of the code (because contract A deploys contract B).

I am not saying it is a bug. Just that it makes life harder.

[axic]

I don't think Etherscan should ignore the metadata hash. Why is it hard to keep the same source code (without whitespace changes) which was used for compilation?

[yaronvel]

@axic I also think ignoring metadata hash other than in the end of the code could be risky.
It is hard as simple user have no idea on the order of the files during compilation.
And if it is not arranged in the exact order, verification will fail.

[axic]

Can you please elaborate how the order matters?

I'd imagine Etherscan should support uploading a zip file with the directory structure and use Solidity to resolve the imports.

[yaronvel]

oh, not at all. etherscan supports only a single file.
So I have to concat everything manually.
However, as swarm hash is embedded, different order of concatenation results in different hash, which results in different bytecode.

[axic]

I mean Etherscan should support uploading a zip file, that is the real solution. Have you tried raising a feature request with them?

[yaronvel]

I agree that the problem is not in solc. But in the past solc made efforts/changes to assist etherscan verify code.
Supporting multiple files in a known and notorious problem in etherscan.

Solc could help by adding a --output_concat_src flag for example.
The current situation, where user have to use some unofficial script to do it is a bit awkward.
Again, I am only raising here because such issues with etherscan were discussed in solidity repo in the past.

[axic]

It is always useful raising it here so we can get a better understanding of the problem. In this particular instance I think Etherscan should improve their verification tool to support packages. @mtbitcoin, would that be possible?

[mtbitcoin]

We need to be able to compile not only with the latest version of solc but with all other version previous versions of solc. And we need to be able to do that programmatically server side, if someone can suggest and advice on how we can do that while also supporting multiple source files and producing a deterministic result that will generate the same code regardless of the naming/path/location of those multiple source files i am open to suggestions.

Edit: At the moment we are using https://www.npmjs.com/package/solc for the backend server side compilation, if there any better suggestions i am open to hear them

[yaronvel]

assuming solc works in a deterministic way, I would imaging it can just glue it for you, and it might also match the order of old solc versions. But even if it doesn't it would be great if it would work in future versions.

[axic]

@mtbitcoin there a couple of different things here:

• I propose that Etherscan should support accepting a zip or a tar file with all the sources and a field to define the main contract name. Unfortunately solc-js doesn't yet support imports, but the regular solc does.
• The best solution on top of this would be to use the standard JSON interface because that works via both solc and solc-js without issues and you have full control over the files. Again, accepting a zip / tar and creating the appropriate JSON is the right way to go.

[veox]

Eventually I was able to verify it on etherscan with some help I got and some unofficial script that concat files in the exact order the compiler does.

For ref, I'd guess the tool mentioned by @yaronvel is BlockCatIO/solidity-flattener.

[yaronvel]

@veox yes, thank you.

[chriseth]

I don't see how this issue is about solidity. Please reopen if I'm wrong.