Grpc-web: Reduce compiled JS file size

[stanley-cheung]

As it stands, our npm package index.js is 218KiB (which is being compiled by the closure compiler here).

With the Echo example echo.proto, that has about 7 message types, 1 service, 3 unary methods and 5 streaming methods:

• The Closure example final compiled.js compiled by the Closure compiler: 317KiB
  
  
  
  • The CommonJS example final dist/main.js compiled by webpack: 415KiB
  
  
  • The same CommonJS example compiled by browserify: 479KiB
  
  
  • The CommonJS example with .d.ts typings generated, and then compiled by webpack: 499KiB
  
  

I tried babel-minify and uglify-js on some of these JS files and the output are not any particularly smaller.

There should be some other things we can do to further minimize the compiled JS code size.

Reference: #79

[stanley-cheung]

A bit more info, as it stands:

• the generated protobuf messages echo_pb.js produced by --js_out: 35KiB
• the generated gRPC Web stub echo_grpc_web_pb.js produced by --grpc-web_out: 9KiB

A couple more observations:

• Just by looking at compiled.js, dist/main.js produced by the closure compiler, webpack, etc, the compiled JS files are already reasonably minimized. Most variables are already renamed, whitespace removed, etc.
• From a cursory look, the largest increase of code size is caused by 1. proto messages pulling in google-protobuf runtime classes and 2. grpc-web service stub pulling in goog.net.streams and closure library code. But again, after those dependencies are pulled in, the resulting code are already reasonably minimized. So how does 35KiB/9KiB get to 317KiB?

[wenbozhu]

For the protobuf messages, could you check the compiled.js with only the protobuf messages?

I.e. it is useful to find out the respective sizes of google-protobuf v.s. goog.net.streams

[codesuki]

Would be super interesting if you could share any news here. The biggest hurdle to adopt grpc-web for our front end people is the size of the dependencies / generated code.

[wenbozhu]

@codesuki could you share the generated code size v.s. dependencies respectively for your front-end project? Optimizing protobuf-JS is not something we could do easily but grpc-web stub and runtime may have room for optimization.

[jjbubudi]

Will need to dig into the bundled code but at a glance I believe it's because of the same reason as this:
https://github.com/protocolbuffers/protobuf/pull/5509

Currently in build.js, it looks like closure compiler's advanced optimizations feature is not turned on, so the bundle ended up containing redundant library code because there is no tree shaking. To fix this we will need to:

1. Turn on advanced optimizations
2. Add @export to the public APIs and let closure compiler advanced mode safely performs tree shaking

Will get back on this thread once I confirmed this. If it works it would drastically reduce the bundle size.

[Yannic]

I think @jjbubudi is right, I'm seeing smaller bundle sizes and we use advanced optimizations for all our code (I don't have exact numbers to share here because there is more compiled into our bundles, but our files are smaller than the sizes mentioned above).

If we go ahead and add @export annotations to the public APIs, can we do that in separate files, or at least add an option to turn that on/off? Otherwise, those of us who use it directly from closure-style code will export these symbols when there is no need to.

[jjbubudi]

IIUC, Closure Compiler will ignore @export unless --generate_exports is explicitly used when running the compiler, as per https://github.com/google/closure-compiler/wiki/Flags-and-Options. Would that resolve your concern @Yannic? (Please correct me if I am wrong)

Finally got it working locally and passing all mocha tests. Here is the result so far:

index.js is now only ~24KB (uncompressed) after enabling advanced optimizations with proper annotations.

Will send a PR soon for review and further testing if anyone is interested.

[Yannic]

Wow, that's great news @jjbubudi!
Unfortunately, --generate_exports won't help in my case. We use @export in our library to export public parts of the API.

However, @export is translated to goog.export{Symbol,Property}. Manually writing these and guarding them by a compile-time constant would be an option. It's a tiny bit more work, but doesn't increase the compiled size because of dead code elimination. I think it's worth it if we can avoid exporting unnecessary symbols.

/** @const {boolean} */
const EXPORT_GRPC_WEB_SYMBOLS = true;

function foo() {}
if (EXPORT_GRPC_WEB_SYMBOLS) {
  goog.exportSymbol('foo', foo);
}

Let me know when you submit your PR.

[jjbubudi]

Ah, ok. I've submitted PR #422. Looks like it needs to be further updated to avoid affecting existing Closure users.

[jjbubudi]

@Yannic Actually, looking deeper at the optimized code, Closure Compiler has stripped away all the goog.exportX calls and merely leaving 1 little line of prototype alias code. I'd think when using @export the increase in code size should be negligible but we gain a lot in maintainability.

Here is the pretty printed version of the new bundle FYI: https://gist.github.com/jjbubudi/86549bd597ffc4d74b30990e7a385f44#file-grpc-web-js-L1287

[nehbit]

Did @jjbubudi's fix ever end up being available? I've encountered the same issue — with a trivial hello world proto, the smallest my bundle got was 180~Kb with Webpack, of which, only 8Kb is actual application code. The rest appears to be grpc-web. This is a problem because our web app targets mobile devices, we simply don't have the ability to add +180Kb to our payload, our landing page in total is smaller than that.

I've also tried using Closure Compiler via Webpack, however, this predictably fails at google-protobuf on the advanced mode required to get the size reduction.

Thanks for the hard work — it otherwise works very well. I would just love to be able to strip away the google-protobuf's unused parts somehow, preferably with Webpack.

[lePereT]

https://github.com/grpc/grpc-web/pull/469#issue-254871907

[jjbubudi]

@nehbit It went into 1.0.4, see https://bundlephobia.com/[email protected]

[mleonhard]

I built six single-field protos and my bundle.js grew by 332 KiB. This is unacceptable. So far, it seems to me that the only solution offered is to learn a new language (Closure) and translate our code to it. Is this correct?

Now I will try Protobuf.js.

[clehene]

I built six single-field protos and my bundle.js grew by 332 KiB. This is unacceptable. So far, it seems to me that the only solution offered is to learn a new language (Closure) and translate our code to it. Is this correct?

Now I will try Protobuf.js.

I believe the references are to the Closure compiler