Aws-xray-sdk-node: Can the bundle size be reduced?

[mrchief]

According to bundlephobia, the core package weighs at 98kb minified+gzipped. Is there any way to reduce this bundle size?

lodash and winston seem to be obvious targets. lodash supports tree shaking (as well as has individual packages that are lighter). winston is a fatty package in itself with a lot of lighter & faster replacements.

I think it'd help to bring the size down especially for AWS Lambda packages which themselves can be very lightweight.

Thoughts?

[chrisradek]

@mrchief

99 was just merged to remove the lodash dependency. We have to be more careful about replacing winston as we would need a logger that output the same way winston does today but otherwise I think that's a fair change to make.

Longer-term we'll also be able to replace the aws-sdk with @aws-sdk/client-xray-node which should be a lot smaller, but will want to wait until that project moves out of developer preview first.

[mrchief]

99 was just merged to remove the lodash dependency.

@chrisradek This is great news!

We have to be more careful about replacing winston as we would need a logger that output the same way winston does today but otherwise I think that's a fair change to make.

Yes, I totally understand. I'm not deeply familiar with the subject, so I can't say for certain whether this would work out, but I've found pino to be very modular (in addition to being fast and lightweight). It has a big ecosystem of modules so you can pick and choose the ones you actually need.

Ultimately, I'd leave up to the maintainers to make the call but I'm happy that things are proceeding in the right direction!

[jplock]

When can a release be made that moves the aws-sdk dependency to a peerDependency?

[willarmiros]

Hi @jplock,

I explained in #187 why making aws-sdk a peer dependency is not a trivial matter. We would like to create a separate module for centralized sampling, which consumes aws-sdk, so it does not need to be imported by default. However, the way the package is currently designed makes sampling logic tightly coupled with many other parts of the X-Ray SDK, so making such a separate module is not an easy task. For that reason I cannot give a solid timeline on when such a release will be made.

[vertti]

date-fns dependency is huge. It adds over 4MB to the size of my lambdas. I'm guessing you are using like 0,5% of that. date-fns claims to be modular and treeshakeable etc, but if I just use aws-xray-sdk-core as a dep in my Lambda, that 4.5MB is what I get.. :(

[willarmiros]

Hi @vertti,
Can you explain what you mean a little further? date-fns is a tiny dependency, adding only about 3.2 kB to the SDK package, according to both the above image and bundlephobia.

[vertti]

@willarmiros Sure. I would love to hear that I'm doing something silly here.

I have a bunch of lambdas that have a shared dependency. This dep is instrumented with x-ray for it's Postgresql calls.

In my common library I run:

rm -rf node_modules
npm i --production

then doing du -h node_modules gives me:

---clipped for brewity---
20K node_modules/date-fns/sub_years
20K node_modules/date-fns/start_of_day
20K node_modules/date-fns/is_wednesday
20K node_modules/date-fns/start_of_month
20K node_modules/date-fns/end_of_minute
4,6M    node_modules/date-fns
..
--clipped for brewity--
..
12K node_modules/emitter-listener/test
36K node_modules/emitter-listener
64K node_modules/semver/test
8,0K    node_modules/semver/bin
204K    node_modules/semver
72K node_modules/pg-pool/test
116K    node_modules/pg-pool
40K node_modules/isstream
20K node_modules/cycle
24K node_modules/postgres-array
8,0K    node_modules/pg-connection-string/test
36K node_modules/pg-connection-string
59M node_modules/

and checking where the date-fns comes from with npm ls date-fns tells me:

@kesko/[email protected] /home/vertti/dev/kesko/k-ruoka-product-service/common
└─┬ [email protected]
  └── [email protected] 

But I'm guessing this is just the difference of using npm dependencies and not webpacking/treeshaking/uglifying?

[willarmiros]

Interesting, I reproduced that just fine on my end. We shouldn't have to require our customers to use treeshaking before deploying their Lambda functions to reduce the size of a dependency - especially because we only use date-fns in our logger, which is ignored in Lambda 🙃

Since its only use could be accomplished with native JS, I'll open a PR to remove date-fns.

[khaledosman]

Are there any updates on this? moving aws-sdk to peerDependencies to be more specific

I'd rather not use Xray at all than add the entire aws-sdk to my bundle, especially after doing all these workarounds in my serverless environment to remove it like:

1. moving "aws-sdk" to devDependencies
2. using serverless-webpack's includeModules
3. using webpack node-externals plugin
4. setting serverless-webpack to forceExclude aws-sdk
5. setting up yarn --autoclean to remove unnecessary files from node_modules

And yet it still ends up in my package due to aws-xray-sdk-core, which makes my packaged lambdas size too big and slows down my deployments, very frustrating.

[willarmiros]

Hi @khaledosman,
Thanks for raising this concern. The aws-sdk dependency will be removed entirely in version 3.0.0 of the SDK as stated in #157, and development on that removal is underway.

[willarmiros]

As of 3.0.0-alpha.2, aws-sdk, winston, lodash, and date-fns have been removed from this package. We do not plan on removing any further dependencies, so I will resolve this issue.