Truffle: Truffle contract artifacts are too big and contain too much sensitive detail to actually deploy on the web

[interfect]

• [X] I've asked for help in the Truffle Gitter before filing this issue.

---

Issue

The .json build artifacts from Truffle are supposed to be put onto a web server, where in-browser dapps can download them and use them to interact with the blockchain.

But my contract files are all huge, ranging from 11 KB at the smallest to almost 2 MB at the largest. These are way too big to make my users download before my app can work, and way bigger than they need to be for what the client needs out of them.

It looks like this is partly due to the inclusion of the contract source, but mostly due to the inclusion of two syntax trees (ast and legacyAst). The contract source probably doesn't need to go out to clients (unless they are going to be deploying the contract and then verifying it on Etherscan), and the syntax tree definitely doesn't need to go out, and super definitely doesn't need to go out twice.

Also, in that syntax tree, I see full paths to the directories where Truffle found the source code. That's good for debugging, but that's not really something I want to be revealing to end users of my dapp.

Am I missing something I need to do to prepare these .json files for distribution and strip out all this extra debug data? Is there some kind of "release" build I am supposed to be building?

I'm using Truffle v4.1.3 (core: 4.1.3); has this been fixed in newer versions?

If not, the format needs to be revised to have less stuff in it (maybe by breaking out Truffle debugging metadata from what truffle-contract actually needs to see), or a concept of a "release build" for an artifact that is small enough to distribute needs to be created.

Steps to Reproduce

git clone https://github.com/NovakDistributed/macroverse.git
cd macroverse
npm install
truffle compile
ls -lah build/contracts
grep $USER build/contracts/*

Expected Behavior

The contract files prepared for distribution over the web should be about a few kb each, holding the contract ABI, addresses, and binary. The user name of the last person to build the artifact should not appear in any of them.

Actual Results

The contract files are ~7 MB total, and contain a bunch of syntax tree stuff that will be of no use in my Dapp front-end.

total 7.2M
drwxr-xr-x 25 anovak  800 Sep 13 19:37 .
drwxr-xr-x  3 anovak   96 Jul  5 21:56 ..
-rw-r--r--  1 anovak  11K Sep 13 19:37 AccessControl.json
-rw-r--r--  1 anovak  85K Sep 13 19:37 BasicToken.json
-rw-r--r--  1 anovak  37K Sep 13 19:37 CanReclaimToken.json
-rw-r--r--  1 anovak  49K Sep 13 19:37 ControlledAccess.json
-rw-r--r--  1 anovak  39K Sep 13 19:37 ERC20.json
-rw-r--r--  1 anovak  26K Sep 13 19:37 ERC20Basic.json
-rw-r--r--  1 anovak  29K Sep 13 19:37 HasNoContracts.json
-rw-r--r--  1 anovak  39K Sep 13 19:37 HasNoEther.json
-rw-r--r--  1 anovak  24K Sep 13 19:37 HasNoTokens.json
-rw-r--r--  1 anovak 505K Sep 13 19:37 MRVToken.json
-rw-r--r--  1 anovak 1.5M Sep 13 19:37 MacroverseStarGenerator.json
-rw-r--r--  1 anovak 300K Sep 13 19:37 MacroverseStarRegistry.json
-rw-r--r--  1 anovak 1.9M Sep 13 19:37 MacroverseSystemGenerator.json
-rw-r--r--  1 anovak  52K Sep 13 19:37 Migrations.json
-rw-r--r--  1 anovak  49K Sep 13 19:37 MinimumBalanceAccessControl.json
-rw-r--r--  1 anovak 603K Sep 13 19:37 OrbitalMechanics.json
-rw-r--r--  1 anovak  57K Sep 13 19:37 Ownable.json
-rw-r--r--  1 anovak 279K Sep 13 19:37 RNG.json
-rw-r--r--  1 anovak 1.3M Sep 13 19:37 RealMath.json
-rw-r--r--  1 anovak  64K Sep 13 19:37 SafeERC20.json
-rw-r--r--  1 anovak  92K Sep 13 19:37 SafeMath.json
-rw-r--r--  1 anovak 273K Sep 13 19:37 StandardToken.json
-rw-r--r--  1 anovak  16K Sep 13 19:37 UnrestrictedAccessControl.json

Also, my user name appears 66 times in the files.

Environment

• Operating System: OS X
• Ethereum client: N/A
• Truffle version (truffle version): v4.1.3
• node version (node --version): v9.4.0
• npm version (npm --version): 5.6.0

[gnidan]

Thanks for raising this issue!

We plan to address this concern as part of a major feature upcoming on our roadmap. We will keep you informed here as this progresses.

[benjaminmbrown]

@gnidan Is there a publicly accessible roadmap?

[kyriediculous]

Sorry but this is not an issue.

You do never ever ever have to put your JSON ABI's fully on a webserver.

You never ever ever have a webserver when running a dappclient (ideally).
Because that means centralizing the architecture again.

The necessary configuration will be included upon building with webpack.

[interfect]

Sorry, I'm completely unfamiliar with the build process you are referring to @kyriediculous.

I've been building my dapp with Browserify to package up the truffle-contract module, and then invoking the module's constructor function that it exports according to https://github.com/trufflesuite/truffle/tree/next/packages/truffle-contract#usage which says to pass in the JSON data for the contract, which is precisely what the built .json files are, as far as I can tell. I could try and require them into the build instead of manually loading each file separately with a fetch, but all that will do is cram the JSON into my app's massive JS bundle somewhere; I don't think Browserify is actually clever enough to remove any of the unused information.

My intended deployment process is to put the resulting Browserify-built static site with Javascript up on a web server, and have that be the front-end for my Dapp. Users are supposed to use it with Metamask or Mist or some other web3-enabled web browsing solution.

What build and distribution system/process are you using where there's no web server involved? Are you just writing a native app that talks to an Ethereum node?

[sulliwane]

@kyriediculous not familiar with your workflow neither.

Most projects run truffle compile to get /build directory containing all ABI.json files, and then publish to NPM. Then in js file, { abi } = require('packagename/build/contracts/ABI.json')

That's indeed a BIG issue when size matters, ie: frontend applications (Single Page App for instance).

We can't reasonably import +500KB json to just use the abi field in it (probably < tenth of the total size), neither manually remove the ast fields from the json.

What's the recommended solution?

Many thanks!

EDIT: looking at this comment : https://www.reddit.com/r/ethdev/comments/9egprt/how_to_reduce_trufflecompiled_json_files_to/e5ovtuz/ seems we'll have to wait for truffle v5.1 for this feature.

[kyriediculous]

I guess I took a very long sentence to say that webpack removes unused JSON if you import JSON files.

https://webpack.js.org/guides/tree-shaking/

You can upload your static bundle onto IPFS/Swarm

My bundle after building/bundling is 5.2mb and is more than a basic app.

[sulliwane]

@kyriediculous oh cool, didn't suspect webpack to be that clever ;) thanks for making your point clear

[stale[bot]]

Thank you for raising this issue! It has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If you would like to keep this issue open, please respond with information about the current state of this problem.

[interfect]

The truffle contract JSONs are still huge.

[stale[bot]]

Thanks for your response! This issue is no longer considered stale and someone from the Truffle team will try to respond as soon as they can.

[gnidan]

Actively working on this problem.

Currently looking at GraphQL to provide a flexible means for customizing the contract artifact format.

Teaser screenshots

[hellwolf]

I created a gist that minimize the contracts using jq utility: https://gist.github.com/hellwolf/473d598b7963079e32af98ab168bc78a

[valerioleo]

thanks for your effort guys. is there any progress on this issue?

[kyriediculous]

https://github.com/webpack/webpack/pull/6184

[jackoelv]

I created a gist that minimize the contracts using jq utility: https://gist.github.com/hellwolf/473d598b7963079e32af98ab168bc78a

thanks so much for hell wolf. you script is useful for reduce the size!
great job!

[jackoelv]

I created a gist that minimize the contracts using jq utility: https://gist.github.com/hellwolf/473d598b7963079e32af98ab168bc78a

great job!
when json file reduced, app.js resized from 5M to 2M. could you reduce it to less than 100K?
how to reduce web3 and then? thanks a lot!

[FabioBonfiglio]

Hello there.
Just wanted to point out that another aspect of the issue originally described here is the fact that the json files produced by truffle compile contains too much sensitive data (local paths mentionning usernames).

I'm currently looking for a way to source control the informations related to the migrations, so that other developers of the project can fetch and build the front-end webapp and connect to the correct version of the deployed contracts, but these sensitive informations simply prevent me to include the json files in the source control.