Transformers: Can't run convert_roberta_original_pytorch_checkpoint_to_pytorch.py

[csliangchen]

❓ Questions & Help

when i want to convert my model which prertrained in fairseq, the error like follows:

2019-11-16 23:53:48.119139: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-11-16 23:53:48.148610: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2097740000 Hz
2019-11-16 23:53:48.151302: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x557dc08b8900 executing computations on platform Host. Devices:
2019-11-16 23:53:48.151342: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): Host, Default Version
loading archive file ./checkpoint_best.pt
extracting archive file ./checkpoint_best.pt to temp dir /tmp/tmpz46rlxeu
Traceback (most recent call last):
File "./transformers/convert_roberta_original_pytorch_checkpoint_to_pytorch.py", line 178, in
args.classification_head
File "./transformers/convert_roberta_original_pytorch_checkpoint_to_pytorch.py", line 46, in convert_roberta_checkpoint_to_pytorch
roberta = FairseqRobertaModel.from_pretrained(roberta_checkpoint_path)
File "/home/liangchen/.conda/envs/py36lc/lib/python3.6/site-packages/fairseq/models/roberta/model.py", line 139, in from_pretrained
**kwargs,
File "/home/liangchen/.conda/envs/py36lc/lib/python3.6/site-packages/fairseq/hub_utils.py", line 33, in from_pretrained
model_path = file_utils.load_archive_file(model_name_or_path)
File "/home/liangchen/.conda/envs/py36lc/lib/python3.6/site-packages/fairseq/file_utils.py", line 80, in load_archive_file
with tarfile.open(resolved_archive_file, 'r:' + ext) as archive:
File "/home/liangchen/.conda/envs/py36lc/lib/python3.6/tarfile.py", line 1588, in open
raise CompressionError("unknown compression type %r" % comptype)
tarfile.CompressionError: unknown compression type 'pt'

Ask for help. Thanks a lot!

[LysandreJik]

Hi! Do you succeed in loading your pre-trained model in Fairseq?

[csliangchen]

Yes, it work well in Fairseq~

[csliangchen]

My script:
python ./transformers/convert_roberta_original_pytorch_checkpoint_to_pytorch.py --roberta_checkpoint_path ./checkpoint_best.pt --pytorch_dump_folder_path ./race_base/roberta-convert-checkpoint/

[pratyay-banerjee]

you will need --classification-head to load the final layer i guess

[AdaCheng]

Hi, I also come across this issue, have you solved it?

[csliangchen]

No, I give up :(

[AdaCheng]

Hi, I found that --roberta_checkpoint_path should be ./ not ./checkpoint_best.pt

[paulthemagno]

Is there a way to convert from a PyTorch bin (obtained with Huggin Face) to a Roberta model? When I run this script:

from fairseq.models.roberta import RobertaModel
roberta= RortaModel.from_pretrained('/path/to/checkpoint/folder/',checkpoint_file='pytorch_model.bin')

I got this error:

File "/usr/local/lib/python3.6/dist-packages/fairseq/checkpoint_utils.py", line 162, in load_checkpoint_to_cpu
    args = state["args"]
KeyError: 'args'

I think I have the opposite problem, but I don't find a script for this.

[julien-c]

@paulthemagno https://github.com/pytorch/fairseq/issues/1514#issuecomment-567934059

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.