Ethers.js: ES modules?

[bpierre]

Is there any plan to provide an ES modules version of ethers.js? It would really help to bundle an optimized build for browsers (using rollup, webpack or similar tools).

Also, with the current version, the main field points to index.js, which is the CommonJS (Node) version. A solution could be to add a module field for the ES modules build, making it compatible with Node and ESM bundlers out of the box.

What do you think?

[ricmoo]

The plan to migrate to TypeScript, which will be used to output to ES modules, TS modules and the dist files.

I will look more into CommonJS; I definitely want this to be compatible with as many systems out-of-the-box as possible. :)

[bpierre]

Awesome, thanks! Feel free to close this issue depending on how you manage issues / discussions.

[rsaccon]

@ricmoo have you considered to migrate to ReasonML instead of TypeScript ? It is statically typed, has extremely fast compiler to es6 (or commonjs) modules, and the compiled JS is also highly optimized and soon it will also automatically generate Flow and TypeScript declarations. It can be mixed with JS, therefore migration can happen gradually. If you are interested, I would love to help with it. Currently I am writing bindings to ReasonML (but just for those functions I need right now)

[pkieltyka]

@ricmoo btw, ping me whenever you working on the TS stuff and want some help with the build tools to package commonjs, umd and as es6 modules (which is nice for tree-shaking by other es6 projects).

[ricmoo]

Question regarding the ES module; should the module field point to an ES6 module compiled for the browser, or for node? Is there a way to specify something like module.node and module.browser? Obviously not those keys, but the browser version requires several transforms that are non-trivial to do otherwise.

The other option is to make the module version perform run-time checks, since the ES6 engine, with tree-shaking, may be able to at compile-time create the correct module?

[MicahZoltu]

Any progress on this? I just tried to integrate ethers.js into an ES modules project and immediately ran into trouble because ethers.min.js doesn't export anything.

[MicahZoltu]

Question regarding the ES module; should the module field point to an ES6 module compiled for the browser, or for node?

Personally, I think the module field in package.json should point to a _standards compliant_ ES module, which means "browser" since node still doesn't actually support ES modules and it is up in the air how they will support them when the time comes. That being said, I wouldn't want to hold up getting an ES module build out on this since a browser directly loading an ES module doesn't look in package.json anyway.

[MicahZoltu]

While definitely not elegant, I was able to get ethers.js working in an ES modules project by tacking this onto the end of ethers.min.js:

const Signer = window.ethers.Signer;
const Wallet = window.ethers.Wallet;
const VoidSigner = window.ethers.VoidSigner;
const getDefaultProvider = window.ethers.getDefaultProvider;
const providers = window.ethers.providers;
const Contract = window.ethers.Contract;
const ContractFactory = window.ethers.ContractFactory;
const constants = window.ethers.constants;
const errors = window.ethers.errors;
const utils = window.ethers.utils;
const wordlists = window.ethers.wordlists;
const platform = window.ethers.platform;
const version = window.ethers.version;
const ContractFunction = window.ethers.ContractFunction;
const ContractTransaction = window.ethers.ContractTransaction;
const Event = window.ethers.Event;
const EventFilter = window.ethers.EventFilter;
export { Signer, Wallet, VoidSigner, getDefaultProvider, providers, Contract, ContractFactory, constants, errors, utils, wordlists, platform, version, ContractFunction, ContractTransaction, Event, EventFilter };

Open to ideas that are better, but that is something simple enough that I can do in a one-line(ish) script and doesn't require waiting on a new version of ethers.js. Would _love_ any improvements though!

[MicahZoltu]

Perhaps as a first step towards ES module support, the build process could simply generate a copy of dist/ethers.js that contains that tacked on the end and then minify that as well so in dist you would have ethers.min.js and ethers.es.min.js (and their respective .map files). Then the published package can set the module property to dist/ethers.es.min.js. I think this will at provide a kind-of-terrible-but-functional stop-gap while you can figure out the correct full solution, as is the JavaScript way.

[ricmoo]

This is a lot to digest/play with. Maybe I can jump in tomorrow. I was really hoping the .mjs or whatever choice would have been made by now... le sigh...

I think lerna (suggested by @pkieltyka; thanks!) solves a lot of this, since I no longer need a link to the minified file to make React Native work with their hijacked require.

I need to create test cases too, for each weird platform, and make sure it works across them all, sans modifications. Maybe that would be a good GitCoin task.

[MicahZoltu]

I tried to run ethers.js through rollup, unfortunately ethers.js depends on some libraries that don't follow good internal patterns (like elliptic and uuid). If these dependencies can be swapped out, it is possible that rollup could work for generating ES modules. If anyone goes down this path, this is how far I got:
rollup.config.js

import commonjs from 'rollup-plugin-commonjs'
import nodeResolve from 'rollup-plugin-node-resolve'
import json from 'rollup-plugin-json'
import globals from 'rollup-plugin-node-globals'
import builtins from 'rollup-plugin-node-builtins'

module.exports = {
    input: 'index.js',
    output: {
        file: 'dist-es/bundle.js',
        format: 'esm'
    },
    plugins: [
        nodeResolve({
            preferBuiltins: true
        }),
        commonjs(),
        json(),
        globals(),
        builtins()
    ]
}

npm install --save-dev rollup rollup-plugin-commonjs rollup-plugin-node-resolve rollup-plugin-json rollup-plugin-node-globals rollup-plugin-node-builtins
npx rollup --config rollup.config.js

[ricmoo]

Elliptic is about the most important package in the usiverse. :)

What about it makes it not play well? Just curious. I haven’t touched rollup yet.

[MicahZoltu]

They have circular dependencies all over the place. I created an issue for theme to see if they are interested in fixing it, it is just a matter of a programming style change on their end, no functional changes necessary.

elliptic.js

const utils = require('./utils')
const hash = require('./hash')

utils.js

...

hash.js

const elliptic = require('./elliptic')
const utils = elliptic.utils

The fix is easy, just make hash.js have const utils = require('./utils') instead. Their current style is to have every file require('./elliptic') and then have elliptic.js have a reference to every file, so when file A needs to access stuff in file B, you do elliptic.B.

[d14na]

unfortunately ethers.js depends on some libraries that don't follow good internal patterns

__+1,__ I've seen first hand how some of those packages don't play nicely with others.

Would absolutely love es module support. I "think" I could then use Ethers exclusively, instead of having to use BOTH web3js AND ethers in virtually every "bundled" project.

[lastmjs]

I just landed here after struggling to get web3.js to work in the browser...which I have not been able to do (I don't use bundlers like webpack for my entire application). This project seems friendlier in that regard, and I'm excited for module support. I've run into some of the issues mentioned already in this post as well, so if you need help, I might be able to provide some guidance from my experience with this all at least.

[ricmoo]

@lastmjs Awesome! It doesn't seem like TypeScript is ready for mjs yet... But I'm working on getting a Lerna version working, which should hopefully make the process a whole lot easier. :)

Anything you can chime in on though, is greatly appreciated. :)

[lastmjs]

Are you authoring the entire library in TypeScript? I don't see the need to wait for .mjs support, do you? My thought would be that from TypeScript source code you should be able to generate distribution builds for every platform that you want to support.

[ricmoo]

The problem is the distribution builds need to be properly referenced in package.json. Every tools does its own little thing. What works in RN/Expo breaks in Lerna/Rollup/Webpack, what works in the browser for ES6 does not work in the browser for ES5, and for ES3 browsers the world adds complications, and so on. Once you start dealing with other JS engines, more complications enter, Otto vs V8, etc...

There are too few options. There really needs to be a "module' and "browser-module" key in the package.json.

There are a lot of different platforms that use ethers, and each has made their own custom (and incompatible magic)...

Keeping everyone working has been a lot of work. :)

[lastmjs]

Wow, extremely ambitious, good on you. Is there a reason for not
distributing the source files on npm? For me personally, I think that'd be
good enough, then I can pull them into my own build system which consumes
TypeScript directly, and it wouldn't affect anything else as far as I know.

On Fri, Mar 8, 2019 at 5:40 PM Richard Moore [e-mail]
wrote:

The problem is the distribution builds need to be properly referenced in
package.json. Every tools does its own little thing. What works in RN/Expo
breaks in Lerna/Rollup/Webpack, what works in the browser for ES6 does not
work in the browser for ES5, and for ES3 browsers the world adds
complications, and so on. Once you start dealing with other JS engines,
more complications enter, Otto vs V8, etc...

There are too few options. There really needs to be a "module' and
"browser-module" key in the package.json.

There are a lot of different platforms that use ethers, and each has made
their own custom (and incompatible magic)...

Keeping everyone working has been a lot of work. :)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ethers-io/ethers.js/issues/145#issuecomment-471126081,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGrSj_QwoRbhHJ1j8IxWVk-7AhbvXEwqks5vUwL4gaJpZM4S5sMy
.

--
Jordan Last

819 N 500 E
Logan, UT 84321
801-709-1860

https://www.linkedin.com/in/lastmjs
https://github.com/lastmjs

If you don't want me emailing you again for whatever reason, just reply and
let me know, thanks!

[ricmoo]

The TypeScript source files? I just followed the convention which seems to be to drop them from npm... Are there populate packages you can point me to that do not? I completely just deferred to the community. :)

I do seem to recall not .npmignore-ing them did lead to build issues, with people having to add paths for node_modules, or something like that, because with npm link, there were now multiple implementations (in actuality, the exact same file, but TS seemed confused), which resulted in nothing building without very complicated (e.g. nested node_modules exclusions) tsconfig.json files... :s

[lastmjs]

Doesn't seem to be the convention, sounds like it would be trouble...I kind
of hope things turn out that way though, or at least that libraries ship
JavaScript written with es modules, but I suppose Node.js and the whole
.mjs thing is holding that back for now.

On Fri, Mar 8, 2019 at 6:03 PM Richard Moore [e-mail]
wrote:

The TypeScript source files? I just followed the convention which seems to
be to drop them from npm... Are there populate packages you can point me to
that do not? I completely just deferred to the community. :)

I do seem to recall not .npmignore-ing them did lead to build issues,
with people having to add paths for node_modules, or something like that,
because with npm link, there were now multiple implementations (in
actuality, the exact same file, but TS seemed confused), which resulted in
nothing building without very complicated (e.g. nested node_modules
exclusions) tsconfig.json files... :s

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ethers-io/ethers.js/issues/145#issuecomment-471129449,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGrSj_HDmhxBc1TAVgi0e0T4OmyKTFFtks5vUwhWgaJpZM4S5sMy
.

--
Jordan Last

819 N 500 E
Logan, UT 84321
801-709-1860

https://www.linkedin.com/in/lastmjs
https://github.com/lastmjs

If you don't want me emailing you again for whatever reason, just reply and
let me know, thanks!

[MicahZoltu]

In all of my projects I include the .ts files in the NPM package, though I don't reference them, they are there for source mapping and so users can browse the original source locally.

It doesn't seem like TypeScript is ready for mjs yet

I don't think it is TS that is the problem, but rather Node that is the problem. Node's support for ES modules is still pretty immature. The _one_ problem I have run into with TS generated ES modules is that you have to suffix your relative import statements with .js extension because ES module loaders do not have the ability to "find" the right file and the module loader won't add an extension for you. This means if you compile import {} from './foo' into an ES module, you'll end up with exactly that import line and the browser will try to fetch ./foo which doesn't exist. You instead need to write your TS import line like import {} from './foo.js', which TS will realize you "probably mean './foo.ts' during compilation, and the emitted code will also be ./foo.js which a browser can successfully fetch.

[MicahZoltu]

FWIW, I have a TS library that depends on ethers and has both an ES module and a CommonJS tsconfig file, and both of them work. I use the ES module in another project that has no bundler or minifier and the "build" process is a few lines of an NPM script.

Dependency management is definitely not elegant as I have to manually add a line to my build script and to my ES module path finder, but it all does "work".

[ricmoo]

@MicahZoltu Do you use npm link or Lerna? I've found that is when things go awry with including the ts files...

Also, completely agree on the TS/node front... IT'd be nice if at some point things "just worked". I'd also be happy if main and company were taken as a root. Like, if you do a require("foo/bar/baz") on a ES6-friendly env, and you had module: "es6" in your package.json, it would automatically look for es6/foo/bar/baz, and if it was a browser-target, with browser: "browser" in your package.json, it would automatically look for browser/foo/bar/baz... The whole space seems like a mess right now... Hopefully something awesome will fall from the woodwork soon though. :)

[MicahZoltu]

I used to use NPM link and didn't run into any trouble, these days I do "package-name": "file:../other-module/" as npm link caused me _other_ problems (related to filesystem links causing problems when you recursively delete a directory via something like git clean -dfx).

[MicahZoltu]

The whole space seems like a mess right now...

There has never been a time in my life where I looked at the JS ecosystem and didn't think that...

[cellog]

just as an FYI, ethers as a whole minimizes to 295k, which is pretty large. module is only needed for browser tree shaking, node does not suffer from loading the whole thing from disk. Is there a plan to work on this? I know you're working on v5, so I don't want to step on that, but right now, I have a service worker that weighs in at 335k where 295k of that is ethers, so it would be pretty nice to be able to tree shake :)

[janus]

FWIW, I have a TS library that depends on ethers and has both an ES module and a CommonJS tsconfig file, and both of them work. I use the ES module in another project that has no bundler or minifier and the "build" process is a few lines of an NPM script.

Dependency management is definitely not elegant as I have to manually add a line to my build script and to my ES module path finder, but it all does "work".

@MicahZoltu I want to reduce the size of ethers because what I am doing just needs a couple of functions from utils module. How do I accomplish this ? I am using TS libraray(ethers).

[MicahZoltu]

@janus What I mentioned above wouldn't help with that I think. My recommendation is to find a smaller library that provides the functions you need if you don't need much.

[janus]

@ricmoo @MicahZoltu Which smaller library would you recommend for the below:

utils.toUtf8String
utils.hexlify
utils.toUtf8Bytes
utils.bigNumberify
utils.arrayify
utils.getAddress
utils.isHexString

[MicahZoltu]

Personally, for those I would just vendor them (copy/paste the code into your project). Alternatively, you can use native equivalents for most of them, with some caveats for. For example, isHexString can be achieved with /^(?:0x)?([a-fA-F0-9]*)$/.test(x) (you can wrap that up in a function named isHexString if you want). bigNumberify can be achieved with BigInt(x) if you are using a modern version of NodeJS/browser (stage 3 ES proposal). toUtf8String I'm assuming just decodes a byte array into a string, in which case you can use new TextDecoder().decode(x).

Specific solutions to each for your project will depend on how you are using them.

[ricmoo]

You can easily pull in the v5 packages for those. Even the beta is pretty stable for those:

• @ethersproject/address
• @ethersproject/bignumber
• @ethersproject/bytes
• @ethersproject/strings

Note: bigNumberify is now BigNumber.from(value) in v5.

In v5, each package is broken out with more granularity thanks to tools like Lerna. :)

P.S. isHexString is a bit more complex, as it can verify data lengths as an optional parameter and the UTF-8 library in ethers uses strict mode (by default), which I recommend for security focused applications. I would imagine TextDecoder uses the replace-heuristic for both bad surrogates and overlong sequences, while toUtf8String will throw. This protects you (to some extent) from reading mangled values from chain, which can have the consequence that the hash Solidity computes and what your client computes as the hash being different.

[janus]

@ricmoo How do I install it? I tried npm install --save ethers but landed v4.0.27, I check the branch for direction as per installation but nothing. https://github.com/ethers-io/ethers.js/tree/ethers-v5-beta

sudo npm install git://github.com/ethers-io/ethers.js.git#ethers-v5-beta --save
npm ERR! Can't install git://github.com/ethers-io/ethers.js.git#03c97259c46de10dbe6ce62921de2f32ffff0522: Missing package version

[ricmoo]

You need to specify the version or next tag. For example:

/home/ricmoo> npm install ethers@next

Or to install a specific package:

/home/ricmoo> npm install @ethersproject/bignumber@next

It is still beta, but would love to get some feedback.

[janus]

You need to specify the version or next tag. For example:

/home/ricmoo> npm install ethers@next

Or to install a specific package:

/home/ricmoo> npm install @ethersproject/bignumber@next

It _is_ still beta, but would love to get some feedback.

Thanks. Sure you would hear from me. And I would be hanging out here. Great!

[janus]

@ricmoo @MicahZoltu
I noticed two third reduction in my code size. However, I still noticed that tree shaking is not working .. I am seeing some functions which should not be imported in the first place.
Is there something wrong with my webpack file?

```const path = require('path')

module.exports = {
mode: 'production',
entry: {
index: './src/index.ts'
},
output: {
path: path.resolve(__dirname, 'lib'),
filename: '[name].js',
libraryTarget: 'umd',
library: 'Bvcc',
umdNamedDefine: true,
globalObject: 'this'
},
resolve: {
extensions: ['.ts', '.tsx', '.js']
},
devtool: 'source-map',
optimization: {
minimize: true
},
module: {
rules: [{ test: /.tsx?$/, loader: 'ts-loader'}]
}
}
```

[Alonski]

Wondering if they has been any progress on this? Might have missed something :)

[ricmoo]

@Alonski heya! All packages have an ES module and should work fine. The ethers/dist files also includes an esm file, and it is run inside a chrome instance with all the tests, so it works fine using the import syntax in browsers.

I haven’t looked into whether TypeScript has fixed their tree-shake issues recently though. I’ll check it out this week.

[ricmoo]

Looking through (and closing old issues). I believe this is taken care of, in v5 everything is available as both CJS and ESM. Please feel free to re-open though if you have problems, or open a new issue.

Thanks! :)

[rymnc]

Doesn't work with deno :(

[wighawag]

Not sure if it is related, but regarding es modules, some fails at runtime in browser because they depends on node builtins.
Is there a list of these and which one are used when generated the ethers.min.js file ?

[ricmoo]

@wighawag do you have an example? The ES modules are built assuming it is for the browser...

The whole ES browser/node ‘verse is all over the place. :s

[wighawag]

I guess the es module assume there are polyfills in place for node builtin module used.