Web3.js: "Module did not self-register" (Scrypt) in node.js worker thread

[7-of-9]

Description Web3 fails to initialize in node.js worker thread

Expected behavior

const web3 = require('web3') // expected: ok

Actual behavior

Error: Module did not self-register.
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:717:18)
    at Module.load (internal/modules/cjs/loader.js:598:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)
    at Module.require (internal/modules/cjs/loader.js:636:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (C:\SCP\src\scpx-app\ext\wallet\node_modules\scrypt\index.js:3:20)
    at Module._compile (internal/modules/cjs/loader.js:688:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
    at Module.load (internal/modules/cjs/loader.js:598:32)
Emitted 'error' event at:
    at Worker.[kOnErrorMessage] (internal/worker.js:334:10)
    at Worker.[kOnMessage] (internal/worker.js:344:37)
    at MessagePort.Worker.(anonymous function).on (internal/worker.js:281:57)
    at MessagePort.emit (events.js:182:13)
    at MessagePort.EventEmitter.emit (domain.js:442:20)
    at MessagePort.onmessage (internal/worker.js:86:8)

Steps to reproduce the behavior

run node with --experimental-workers
create a worker thread:

const { Worker, isMainThread, parentPort } = require('worker_threads')
new Worker('./cpu-worker/worker.js')

require web3 in the worker thread file (worker.js)

#

Versions

"web3": "1.0.0-beta.52",

• nodejs: v10.14.1
• browser: N/A
• ethereum node: N/A

[7-of-9]

Platform is Windows; I've tried npm rebuild, deleting node_modules and reinstalling, also have tried npm upgrade - to no avail.

If anybody knows the root cause or any workaround, would be very much appreciated. Will try downgrading web3 to see if it makes any difference.

[nivida]

Could you create a demo GitHub repository where I could do some more tests?

[sshelton76]

You got an extra . in your settings somewhere. Look carefully at your error message...

Object.Module._extensions..js

Not saying that's the problem, but I'd start there.

[7-of-9]

Thanks @nivida @sshelton76 for the input; I'll try packaging what I have into github (also will take a look at the extra .!) -- will revert ASAP.

[7-of-9]

@nivida - https://github.com/Scoop-Tech/scpx-wallet/blob/master/README.md

This repro's for me on a clean checkout. The offending line is in ./actions/wallet-account.js (line 6): const Web3 = require('web3'). Without this line commented out, it fails on npm start.

There's no packaging in this project, just babel-core on the entry point file (./sw-cli.js) -- note that the worker file (which ref's wallet-account.js) is resolved at runtime (./cli-workers.js) and isn't transpiled as far as I can tell.

NB: git checkout 56e5a01a8ab04b5fe7deb868b8a83e14c9d468d5 for the repro commit; subsequent commits are patching over this, as below.

[7-of-9]

Worth noting: it fails in the same way if I move the require web3 to the top-level worker file (./cpu-worker/worker.js).

[7-of-9]

UPDATE: if I defer the require('web3') and move it inside a function instead of putting it at the module/file scope, it works! Maybe some timing condition or similar, in trying to reference the web3 module immediately. Happy to have a workaround at least.

[nivida]

Did you also test it with another package than Web3.js? (It doesn't look like an issue from our side)

[7-of-9]

Did you also test it with another package than Web3.js? (It doesn't look like an issue from our side)

Yea, the other require references all work fine. As mentioned, there does seem to be a workaround (just don't require web3 at the top of the file); I'll double check that the instantiated web3 does actually work as expected, then will close.

[nivida]

@dmorris99 Did you check this?:)

[7-of-9]

I’m still trying to resolve it — the behavior is very strange; depending on where I require web3, it fails. Some places it’s fine, others it’s not.

There doesn’t seem to be much pattern - in some places it fails in the main thread and works in the worker_thread, other times it’s the inverse.

I did manage to check that if it gets past the require line, then any created Web3 instances seem to work just fine.

From the stack trace it looks to me like some issue with scrypt dependency under certain conditions.

[7-of-9]

@nivida -- I noticed that the module failing to initialize (https://github.com/barrysteyn/node-scrypt) is marked as deprecated on the maintainer's github page.

It says https://nodejs.org/api/crypto.html#crypto_crypto_scrypt_password_salt_keylen_options_callback is preferred instead, although I guess that might not be much use in the browser.

Investigation continues. The same codepath in the browser all works ok. Would like to find an environmental fix rather than hack my code to work around it. Will try upgrading Node from 10.14 to 12.

[noureddinel]

I am facing the same issue:

events.js:174

  throw er; // Unhandled 'error' event
  ^

ReferenceError: transaction_gas is not defined

at calculateResults (/home/core/test.js:207:48)
at NewHeadsSubscription.<anonymous> (/home/core/test.js:240:17)
at NewHeadsSubscription.emit (/home/node_modules/eventemitter3/index.js:181:35)
at WebsocketProvider.<anonymous> (/home/node_modules/web3/node_modules/web3-core-subscriptions/dist/web3-core-subscriptions.cjs.js:70:18)
at WebsocketProvider.emit (/home/node_modules/eventemitter3/index.js:181:35)
at WebsocketProvider.onMessage (/home/node_modules/web3/node_modules/web3-providers/dist/web3-providers.cjs.js:304:12)
at WebsocketProvider.onMessage (/home/node_modules/web3/node_modules/web3-providers/dist/web3-providers.cjs.js:404:77)
at W3CWebSocket._dispatchEvent [as dispatchEvent] (/home/node_modules/yaeti/lib/EventTarget.js:115:12)
at W3CWebSocket.onMessage (/home/node_modules/websocket/lib/W3CWebSocket.js:234:14)
at WebSocketConnection.<anonymous> (/home/node_modules/websocket/lib/W3CWebSocket.js:205:19)

Emitted 'error' event at:

at Worker.[kOnErrorMessage] (internal/worker.js:176:10)
at Worker.[kOnMessage] (internal/worker.js:186:37)
at MessagePort.<anonymous> (internal/worker.js:118:57)
at MessagePort.emit (events.js:196:13)
at MessagePort.onmessage (internal/worker/io.js:68:8)

Using node js version v12.2.0

[nivida]

I'll test and if required fix it asap. Thanks for all the additional information!

[gorgos]

Issue still exists. I didn't even realize it was related to Web3, but thanks to Google I found this here.

I can confirm that

defer the require('web3') and move it inside a function instead of putting it at the module/file scope

is a functioning workaround.

[nivida]

Does this issue still exist also when scrypt is defined as an optional dependency? (v1.2.2 of web3.js)

[cgewecke]

Believe this native dependency related and fixed with 1.2.8 / #3536.

Now using scrypt-js which does not require a node-gyp build.

Please ping if anyone is still seeing this problem.