Tfjs: Make onesLike and zerosLike bindable ops

[caisq]

TensorFlow.js version

latest (0.15.0)
tfjs-node latest: 0.2.3

In tfjs-node, onesLike and zerosLike are not ops. As a result, the tensors are created on the JavaScript side, as TypedArray buffers, and get uploaded to the TF kernel (e.g,. the CUDA backend), which can be very expensive if the size of the tensor is large. This happens a lot in both the forward and backward passes, as they are used in many places in tfjs-layers functions and the associated gradients.

onesLike and zerosLike do exist as C++ TensorFlow op kernels. We should make them bindable ops in tfjs-core. This should give us significant boost in tfjs-node performance.

[dsmilkov]

Also while tf.ones and tf.zeros are not kernels, in Python they call into tf.fill which is a kernel. This avoids the allocation of data on the Python side. We should do the same in tfjs-core, that is have ones and zeros delegate to fill

[dsmilkov]

To summarize the following ops should become kernels:

• randomNormal
• randomUniform
• linSpace
• range
• onesLike
• zerosLike
• fill (called by zeros and ones)

[hpssjellis]

@dsmilkov @caisq

Any idea why zerosLike() is not working since TFJS version 1.0.0. Or what I should do to fix the code?

This TFJS-GAN by @mwdchang works pre-version 1.0.0

my TFJS-GAN post version 1.0.0 doesn't. My fork of his github is here

The error message says: Uncaught (in promise) Error: Cannot compute gradient: gradient function not found for zerosLike.

The only use of zerosLike() in the working original github is here

[hpssjellis]

This looks relevant for onesLike()

https://github.com/tensorflow/tfjs/issues/1398#issuecomment-490806579

Its because onesLike and zerosLike were converted to KERNEL version. replace the functions with their deprecated versions while we wait for an update.

@BenjaminWegener

Can someone give an example, or should this be fixed fairly soon. I am in no rush.

Fairly sure tf.onesLike_() does not work.

[BenjaminWegener]

Heres a working 1.0.0
tf.js.txt

[dsmilkov]

https://github.com/tensorflow/tfjs-core/pull/1753 fixes the onesLike, zerosLike problem.