Transformers: Cannot import transformers with TF version 2.1.0

[amogkam]

The installation README says that transformers library requires Tensorflow version >2.0, but I can't seem to import the latest transformers 3.2 release even with Tensorflow v2.1.

>>> import transformers
wandb: WARNING W&B installed but not logged in.  Run `wandb login` or set the WANDB_API_KEY env variable.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/amog/dev/ray/lib/python3.7/site-packages/transformers/__init__.py", line 121, in <module>
    from .pipelines import (
  File "/Users/amog/dev/ray/lib/python3.7/site-packages/transformers/pipelines.py", line 47, in <module>
    from .modeling_tf_auto import (
  File "/Users/amog/dev/ray/lib/python3.7/site-packages/transformers/modeling_tf_auto.py", line 45, in <module>
    from .modeling_tf_albert import (
  File "/Users/amog/dev/ray/lib/python3.7/site-packages/transformers/modeling_tf_albert.py", line 24, in <module>
    from .activations_tf import get_tf_activation
  File "/Users/amog/dev/ray/lib/python3.7/site-packages/transformers/activations_tf.py", line 53, in <module>
    "swish": tf.keras.activations.swish,
AttributeError: module 'tensorflow_core.python.keras.api._v2.keras.activations' has no attribute 'swish'

Upgrading to TF 2.2 works fine, but I think this should be made more clear in the docs.

cc @jplu @sgugger

Environment info

• transformers version: 3.2.0
• Platform: Mac OS
• Python version: 3.7.7
• PyTorch version (GPU?):
• Tensorflow version (GPU?): 2.1.0. On CPU only.
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help

Information

Model I am using (Bert, XLNet ...):

The problem arises when using:

• [ ] the official example scripts: (give details below)
• [ ] my own modified scripts: (give details below)

The tasks I am working on is:

• [ ] an official GLUE/SQUaD task: (give the name)
• [ ] my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

1.
2.
3.

Expected behavior

[jplu]

Hello !

Indeed, the requirements have to be updated.

[Wufang1997]

Has the problem been solved? I met the same issue when loading the transformers.

[jplu]

Hello, you have to have TF 2.3 at min. This will be fixed in the next release.

[EvanZ]

This breaks at least a couple of the tutorial notebooks. Even with TF 2.3.0 I get the same error.

[jplu]

If you get this message error:

AttributeError: module 'tensorflow_core.python.keras.api._v2.keras.activations' has no attribute 'swish'

It means you don't have at least TF 2.2 installed.

[avialfont]

The problem can be seen as Transformers uses the tf _swish activation function_ by default (that does not exists in tf 2.1: https://www.tensorflow.org/versions/r2.1/api_docs/python/tf/keras/activations).

A workaround, instead of upgrading tf to 2.2 (unavailable at this time with conda), is to downgrade Transformers to a version that was developed with tf 2.1.

For example, I had this warning with TF 2.1.0 and transformers 3.5.1 and it desappeared with transformers 3.0.2 [Warning : this version installs a specific version of pytorch so it is better to use a virtual environment].