Faker.js: Reduce package size

[alexander-akait]

Package contains unnecessary directories and files - locale/.publish, .jshintrc and maybe others. I can do PR if your accept this issue.

[joeyfigaro]

Worth mentioning there doesn't seem to be a clear way to import only the methods you need. Including the following is 1.2mb raw/768kb compressed:
internet, name, random, address

Increasing my lib's size by 768kb to get email addresses, first and last names, random numbers, and addresses is hard to swallow.

[joeyfigaro]

Going to have to try out chance.js or roll our own helpers after seeing this. Everything highlighted there is from faker.

[Marak]

@joeyfigaro

Not sure what you mean by import here. The faker package contains data for a bunch of APIs as well as localities ( outside of en ). If you install the package, you will get all that data.

If you need to reduce your libraries size ( perhaps it's for the browser ), you'll probably want to only include a subset of this data. We've already got custom builds per locality, but not per method. If you need to further optimize the size of the package, you could start removing require statements from inside faker lib and re-run the build process.

Not sure what else you would expect or how else this could work. I'm open to suggestion.

I also am not sure certain the sizes you report are 100% accurate. I am seeing faker.en.js browser file at 433kb uncompressed and 306kb minified. That is for all of en data. I suspect you may be looking at all data for all localities.

[Marak]

@evilebottnawi -

Sounds good, let us know what you can do.

[alexander-akait]

@joeyfigaro you can use tree shaking for avoid unnecessary method in you app, it is not related to faker.js, look at webpack and rollup libraries, thanks!

[jeandat]

Sorry guys, I don't really understand how I should import faker to reduce my bundle size. Like @joeyfigaro my bundle has increased by 1.3 MB. With modern imports (typescript), how should I import faker if I only want the uuid() method from the random lib for instance?

For now I'm only using this method but the project is just starting so there will be a few more.

[Marak]

@jdat82

You can try tree-shaking with something like webpack and rollup to automatically remove un-used codepaths. You can also manually edit your local faker code to remove modules, methods, and locale data which you don't require ( and rebuild ).

There is probably a way to use import the way you want, but I'm not exactly sure how to do it. It might require a few changes to the way we are constructing faker instances.

[Marak]

@evilebottnawi - Any updates? If not I'll be closing this issue for inactivity.

[luanmuniz]

How about the locale/.publish folder. Can we remove it? It's triggering my security alerts because of the eval inside locale/.publish/scripts/docstrap.lib.js

[ghaagsma]

For anyone curious, I developed a workaround for this issue in a create-react-app bootstrapped application that uses faker in tandem with miragejs. It utilizes code splitting to ensure the modules only required for development environments are never requested in production.

The Mirage team recently published a release to enable automatic tree shaking for consuming applications by indicating the package has no side effects. Ideally, faker contributors can publish a similar change to provide the same benefit; although, I'm not sure it would be as simple depending on how the modules are exported in this solution.

[ohcibi]

@ghaagsma to make faker tree Shake-able is not that easy. Currently faker is a huge object that had all its namespaces assigned as properties. This is from before es6modules were a thing. I started porting faker to a esmodules based library written in typescript which is pretty much straight forward but there is a few places where internal dependencies will basically disable proper tree shaking like the fake function. Every function that uses it internally cannot be treeshaked at all but will instead pull the entire lib into the tree no matter how hard you shake 😁. I didn’t had the time to finish the port yet. I’m just telling because trying it gave me the insights how complicated it will be. But I just pushed what I have so far to my fork anyways. Check the faker.ts branch in case you're interested. Please note that @Marak is not (yet) involved at all with that so don’t bug him about issues/problems with this port. As of now my port is not more than a poc. I also know that this port is way behind upstream master also because I didn't had time to continue with that.

I'd also like to note that the pain is not as hard as it seems. That is if you only use faker for testing/developing purposes. The build size here doesn't really matter as long as build time is not increased too much, so its not too bad that we can't tree shake it.

[Marak]

@ohcibi seems to be correct in their assessment. The original implementation for generating the faker.js bundles was created before tree shaking tools or es6modules were a thing.

After updating all the build scripts dependencies we seem to have lost the ability to generate individual browser packages ( something with new tree walking has gone awry ).

I don't have the exact solution worked out yet, but I believe we will have to restructure faker.js a bit in order to allow for minimal / incremental builds of faker methods and data.

The general idea will be to implement a plugin type module system where there is a core faker lib and a series of core faker modules. Then, we can have a user-land exist of many other modules and localities.

Tracking: #919

[yiga2]

I have forked https://github.com/pdunn/bq_fake_data to update the faker version used, and being able to use new methods like zipCodeByState().
Unfortunately Google BigQuery does not support packages > 1MB for javascript user-defined functions, and faker 5.1.0 comes out at 1.3MB.

Is there any way to webpack only for a list of locales e.g. en and en_US w/o removing locales from the base package ?