Tfjs: Support aggressive tree-shaking to remove code not needed to execute a model for inference.

[PiMastah]

Describe the problem or feature request

Let me start out by appreciating all the effort that is going into this project!

I am currently trying to build an agent in order to play a game on a coding site. Their limit for the size of your uploaded code seems to be around 100 kb. Using webpack and including the whole library, I am already at around 1,7 mb. I was therefore wondering if it is possible to export a standalone activation function of a given model in order to save some bytes by not including the whole library? Looking through the documentation, I did not find a way to do just this but I remember that e.g. synaptic allows to do exactly that.

Thanks in advance for your effort!

[bileschi]

Hi PiMastah,

It sounds like you are looking for a way to export a library that contains just enough code & data to run my_model.predict to minimize the footprint of the code. Is that right? Something like a TensorFlow Lite for TensorFlow.js?

[PiMastah]

Yes, @bileschi - what you describe would fit my needs very well.

[bileschi]

Hrm., it looks like tf.Lite is 300KB minimum, so that probably won't meet your needs. I expect at 100kb you would be looking at a pretty small model. Do you have a model which meets your performance quality expectations, or are you more at the technology exploratory phase?

[PiMastah]

It is definitely the latter at the moment. If it can't be done, nothing much is lost. I know the limitations are pretty severe (after talking to one of the platform devs, the upload limit is actually 100k characters) but I was more trying to picture the whole task as a maybe somewhat unusual challenge which would however allow me to dive deep into a lot of things related to TF and thus serve as a learning opportunity :) I will still be using TF.js to build a sophisticated model which will run locally either way.

[nsthorat]

This is something we've thought about and might pursue in the future, so keeping this open. For now we don't have any near term plans to do this.

[mohammedzamakhan]

@igorminar @stephenfluin can probably guide you guys. Angular Team has done a great job, and tree shaking and ahead of time compilation has helped a lot of Angular Projects

[fredyagomez]

+1. I think this is really important if anybody wants to consume this. No as a final consumer but as a vendor or middle consumer.

import { tensor2d } from "@tensorflow/tfjs"; imports the whole library! 2.6MB.
@nsthorat why the low priority for this?

ok, after looking at the dependencies I ended up doing this one.
import { tensor2d } from '@tensorflow/tfjs-core'; which is 1.37 MB.

[ydennisy]

Hello All,

Any idea why tree shaking as implemented by webpack does not work for tfjs-core?

Do you guys plan to support the mentioned use case of picking and choosing objects from the library?

import { tensor2d } from '@tensorflow/tfjs-core';

Thanks in advance!

[ydennisy]

Any update here?

[nsthorat]

We’re starting to work on this this quarter!

[bwnodak]

How's this coming along? Would love to start using this at scale soon. Thanks so much for the effort!

[tafsiri]

We are working on it! I hesitate to give a timeline right now as its quite a big change that involves modularizing all our ops and kernels as well as some internal architecture changes. This github label will track issues/PRs related to this.

[ydennisy]

Hi @tafsiri @nsthorat I can see work on some of the issues labelled - would you happen to have an estimate yet?

Or is there any pre-release stuff one can play with?

[tafsiri]

@ydennisy we hope to have something out by the end of the year, at the very least in pre-release form if not finalised.

As for pre-release stuff, we don't have any documentation yet, but if you are adventurous and don't mind looking at source code/tests, you can see the workflow we are targeting here https://github.com/tensorflow/tfjs/tree/master/e2e/custom_bundle/dense_model. The one tip I can give if you look at that is that we use the result of tf.profile to get the list of kernels in the config file.