Transformers: Is there support for TensorflowJs?

[hamletbatista]

❓ Questions & Help

I managed to save my tenforflow 2.0 model and I see keras .h5 and config.json files.

When I run the tensorflowjs converter it seems to run with no issues.

!tensorflowjs_converter --input_format=keras save/tf_model.h5 save/tfjs_model

I see the output as expected in the generated files. But, when I try to load them from Javascript, I get these errors:

models.ts:287 Uncaught (in promise) TypeError: Cannot read property 'model_config' of null
at models.ts:287
at common.ts:14
at Object.next (common.ts:14)
at a (common.ts:14)

I found some Github issues like this one https://github.com/tensorflow/tfjs/issues/931, that mention the issue is that the .h5 file only includes the weights and they provide a workaround which involves saving the model with the weights, but it is not clear to me how to do that with the HF library.

Is this something you support or is there a way to get the Keras model with the weights?

[julien-c]

My understanding is that tfjs is still kinda unstable so you’d be better off bringing that issue there.

That being said, @Pierrci has tried to do similar stuff so might be able to chime in.

[hamletbatista]

thanks, @julien-c I will repost there. Do you think I could have better luck if I try this with torchjs instead?

I tried ONNX and faced multiple roadblocks. I didn't anticipate running transformer models in JavaScript would be so challenging 😅

[Pierrci]

I found some Github issues like this one tensorflow/tfjs#931, that mention the issue is that the .h5 file only includes the weights and they provide a workaround which involves saving the model with the weights, but it is not clear to me how to do that with the HF library.

Is this something you support or is there a way to get the Keras model with the weights?

Yes the first step is actually to convert the Keras model into a SavedModel format, you can see this notebook as an example: https://colab.research.google.com/drive/1p1Nifh1P-vqAZ1gHsNSCXAHzVWzl5YPP (from my experiments it doesn't work on all models).

Once you have the SavedModel then you can use (in another environment with TF 1.15 since it's the TFJS converter requirement) the tfjs.converters.convert_tf_saved_model method to convert to TFJS format. But then you might run into exceptions like Unsupported Ops (it seems a lot of operators are yet to be implemented in TFJS).

Feel free to cross-reference this issue if you post another issue in the TFJS repo!

[hamletbatista]

thanks, @Pierrci let me try this out

[hamletbatista]

@Pierrci the conversion to savedmodel works, but now I get an error when converting to tfjs:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Input 0 of node StatefulPartitionedCall/tf_bert_for_sequence_classification/bert/embeddings/position_embeddings/embedding_lookup was passed float from Func/StatefulPartitionedCall/input/_3:0 incompatible with expected resource.

I will try changing the input tensor spec to float32

[Pierrci]

@hamletbatista which version of TensorFlow did you use to convert to SavedModel format? Is it the nightly or an older one like 2.0?

[hamletbatista]

@Pierrci I used 2.0. Then, I created a second version using the nightly, but my Colab crashed. Trying it now. I'll let you know.

[hamletbatista]

Hi @Pierrci made a copy of your notebook and tried my model there and got it to export fine. Thanks a lot for your help! Now, let's see if it works in JavaScript :)

[hamletbatista]

Now I get a missing operator AddV2 in TFJS

Uncaught (in promise) Error: Tensorflow Op is not supported: AddV2

I will take a break and look into this.

[hamletbatista]

Got it to work with tfjs 1.4.0.

[Pierrci]

Wonderful! Can I ask you what is the model you're working with @hamletbatista?

[hamletbatista]

@Pierrci Sure. I wrote a couple of articles about this. See https://www.searchenginejournal.com/automated-intent-classification-using-deep-learning-part-2/318691/

I'm trying to get this to work from within Excel and need it working in JavaScript while keeping things simple. I tried Ludwig, but it doesn't support this. See https://github.com/uber/ludwig/issues/575

[Pierrci]

Thanks!