Transformers: Missing xlm-mlm-100-1280

[andompesta]

🐛 Bug

For some reason I can't download the xlm-mlm-100-1280 model for tensorflow 2.0

Model I am using (Bert, XLNet....): XLM

Language I am using the model on (English, Chinese....): 100 languages

The problem arise when using:
TFXLMForSequenceClassification.from_pretrained("xlm-mlm-100-1280")

Expected behavior

Being able to download the model as for the other configuration

Environment

• OS: Ubuntu 16.04
• Python version: 3.7.5
• Using GPU : yes
• Distributed of parallel setup : distributed
• Tensorflow 2.0
• transformers version 2.1.1

[TheEdoardo93]

It works with PyTorch, but not with TensorFlow. I'm using Python 3.6.9, Transformers 2.2.1 (installed with pip install transformers), PyTorch 1.3.1 and TensorFlow 2.0.0.
With TensorFlow, the stack trace is the following:

> from transformers import TFXLMForSequenceClassification
> model = TFXLMForSequenceClassification.from_pretrained("xlm-mlm-100-1280")
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 41478/41478 [00:00<00:00, 365198.30B/s]
304B [00:00, 133069.13B/s]
2019-12-04 10:44:05.684050: W tensorflow/python/util/util.cc:299] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/transformers/modeling_tf_utils.py", line 289, in from_pretrained
    model.load_weights(resolved_archive_file, by_name=True)
  File "/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 181, in load_weights
    return super(Model, self).load_weights(filepath, by_name)
  File "/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/network.py", line 1171, in load_weights
    with h5py.File(filepath, 'r') as f:
  File "/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/h5py/_hl/files.py", line 408, in __init__
    swmr=swmr)
  File "/home/<user>/anaconda3/envs/huggingface/lib/python3.6/site-packages/h5py/_hl/files.py", line 173, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 88, in h5py.h5f.open
OSError: Unable to open file (file signature not found)

If you want, with TensorFlow, it works the XLM model with config xlm-mlm-17-1280, which is a Masked Language Modeling with 17 languages.

> from transformers import TFXLMForSequenceClassification
> model = TFXLMForSequenceClassification.from_pretrained("xlm-mlm-17-1280")
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3311066864/3311066864 [05:40<00:00, 9737775.86B/s]

Bug

For some reason I can't download the xlm-mlm-100-1280 model for tensorflow 2.0

Model I am using (Bert, XLNet....): XLM

Language I am using the model on (English, Chinese....): 100 languages

The problem arise when using:
TFXLMForSequenceClassification.from_pretrained("xlm-mlm-100-1280")

Expected behavior

Being able to download the model as for the other configuration

Environment

• OS: Ubuntu 16.04
• Python version: 3.7.5
• Using GPU : yes
• Distributed of parallel setup : distributed
• Tensorflow 2.0
• transformers version 2.1.1

[andompesta]

Yes I'm refearing to TF2 and I'm currently using xlm-mlm-17-1280, but I wanted to use the bigger model to see if I was able to achieve better performances.

At the moment I'm quite disappointed with xlm-mlm-17-1280, but it might be my fault.

[TheEdoardo93]

If you suspect that you're in trouble, please copy and paste your code here and discuss together

Yes I'm refearing to TF2 and I'm currently using xlm-mlm-17-1280, but I wanted to use the bigger model to see if I was able to achieve better performances.

At the moment I'm quite disappointed with xlm-mlm-17-1280, but it might be my fault.

[thomwolf]

Indeed, this one is missing from the S3. Adding it now!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.