Web3.js: Cannot install on Node v13

[adrianmcli]

Description

Installing on Node v13.0.1 fails. It seems to be failing because of two reasons:

node-gyp failed to build the [email protected] package and the [email protected] package.

Keccak

This is a problem that seems to persist even if you use the latest version of keccak (i.e. 2.0.0).

Sha3

This problem doesn't exist if we use the latest sha3 package (i.e. 2.0.7).

If you just type npm install sha3, everything works fine. But if you type npm install [email protected], you'll get the same error (as you'd get when trying to install Web3.js).

Expected behavior

Should install without errors.

Actual behavior

Install fails, see error below.

Steps to reproduce the behavior

1. Make sure you are on Node v13 (i.e. v.13.0.1 in my testing)
2. In a fresh NPM project folder, run npm install web3

Error Logs

See gist here: https://gist.github.com/adrianmcli/434d60842c70f8829faa445a3d0fa9e6

Versions

• web3.js: v1.2.2
• nodejs: 13.0.1

[nivida]

Thanks for opening this issue!
We will fix this issue ethereumjs and web3.js wide with the packages @alcuadrado is preparing here.
(This is related to https://github.com/ethereum/web3.js/issues/3151)

[alcuadrado]

We will fix this issue ethereumjs and web3.js wide with the packages @alcuadrado is preparing here.

When we finish and adopt ethereum-cryptography this kind of issue will be a thing of the past. We are working on a security audit now, so that will take some time.

We should fix it here in the meantime. Is this just a matter of upgrading the dependencies? Or should it be fixed in ethereumjs?

[adrianmcli]

@nivida @alcuadrado thanks for the response, nice to know that this will be a thing of the past eventually.

In terms of what we need to do now, the sha3 problem seems to go away with an upgrade, but the keccak problem persists. I'm not familiar enough with Web3.js internals to make a judgement as too what we need to do unfortunately, just reporting the problem for visibility.

[nivida]

@alcuadrado

When we finish and adopt ethereum-cryptography this kind of issue will be a thing of the past. We are working on a security audit now, so that will take some time.

💪

We should fix it here in the meantime. Is this just a matter of upgrading the dependencies? Or should it be fixed in ethereumjs?

The dependency we in web3.js would have to upgrade is the ethereumjs-tx package. Web3.js itself is using the scrypt-shim to fix the scryptprimitive in the web3-eth-accounts package until the ethereum-cryptography package is ready.
The fixes for the ethereumjs related packages would be to already switch over to pure JS implementations of those primitives, to update the related packages if it does fix it, or to create shims for them as we were creating with the scrypt-shim package.

[alcuadrado]

The new release of ethereumjs-util fixes the problem with keccak.

The situation with sha3 is more complex. It is depended on via:

web3 > web3-bzz (exact version 1.2.2) > swarm-js (exact version 0.1.39) > eth-lib (^0.1.26) > keccackjs (^0.2.1) > sha3 (^1.2.2)

(Note that keccakjs is not the same than pacakge keccack)

It's unclear what should be modified to fix this as widely as possible. We could remove (and we should) all the pinned versions from web3 dependencies, but in the meantime users with older versions won't see any fix. Contrast this with the ethereumjs situation, where everything is defined with a caret, so the fix gets automatically available to everyone.

I'm inclined to think that the best would be to release a new patch version of eth-lib (i.e. 0.1.28) that doesn't use keccackjs. @evertonfraga this is one of the libraries I mentioned in Osaka. Can you help with this? I'm happy to prepare a PR.

[knoxcard2]

What's the status on this?

[GiovanniCapizzi]

Hello, any news about it?

[nivida]

@knoxcard2 @GiovanniCapizzi Yes, I'm currently in contact with Maia the developer behind the swarm-js package and do try to get the permissions to publish the updated package.json.

[evertonfraga]

Hi all, I got publish access from the maintainer and published [email protected], thus fixing the issue.

Please let me know if that worked for you all.

[nivida]

@evertonfraga I can approve the installation of web3.js. It does work now on node v12 and 13.

@all Be aware to update your package-lock.json as well and don't panic about the warnings from secp256k1 they do not have an impact and it will be fixed by the maintainer of this package as soon as possible.