Simpletransformers: ImportError: cannot import name 'amp' from 'torch.cuda'

[OriJonathan]

Description
When running from simpletransformers.classification import MultiLabelClassificationModel I am met with ImportError: cannot import name 'amp' from 'torch.cuda'

Details
Running on Anaconda Jupyter Notebook.
My imported modules:

import pandas as pd
import numpy as np
import sklearn
from sklearn.model_selection import *
import transformers
import simpletransformers
import torch
import apex

Importing modules are all good. Installed apex using conda install -c conda-forge nvidia-apex and pytorch using conda install pytorch torchvision cpuonly -c pytorch

Tried to troubleshoot with solutions in this thread: https://github.com/NVIDIA/apex/issues/621 but nothing has worked so far,

Full Error

ImportError                               Traceback (most recent call last)
<ipython-input-3-de513398c64a> in <module>
----> 1 from simpletransformers.classification import MultiLabelClassificationModel

~\AppData\Local\Continuum\anaconda3\lib\site-packages\simpletransformers\classification\__init__.py in <module>
----> 1 from simpletransformers.classification.classification_model import ClassificationModel
      2 from simpletransformers.classification.multi_label_classification_model import MultiLabelClassificationModel
      3 from simpletransformers.classification.multi_modal_classification_model import MultiModalClassificationModel
      4 from simpletransformers.config.model_args import ClassificationArgs
      5 from simpletransformers.config.model_args import MultiLabelClassificationArgs

~\AppData\Local\Continuum\anaconda3\lib\site-packages\simpletransformers\classification\classification_model.py in <module>
     43 from simpletransformers.custom_models.models import ElectraForSequenceClassification
     44 from tensorboardX import SummaryWriter
---> 45 from torch.cuda import amp
     46 from torch.utils.data import DataLoader, RandomSampler, SequentialSampler, TensorDataset
     47 from torch.utils.data.distributed import DistributedSampler

ImportError: cannot import name 'amp' from 'torch.cuda' (C:\Users\...\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\cuda\__init__.py)

Thank you so much for your help!

[ThilinaRajapakse]

This should be fixed now in 0.46.4. The amp import should only be attempted when fp16 is True.

Running these models on a CPU is not usually practical though.

[mbin999]

Now the same problem appered during the model.train_model(train_df)
ImportError: cannot import name 'amp' from 'torch.cuda'
Is there some one else find that?
@ThilinaRajapakse

[caprone]

try to install Pytorch with last cudatoolkit available:

conda install pytorch torchvision cudatoolkit=10.2 -c pytorch

[ThilinaRajapakse]

Set fp16 to False.

[cbi4]

@ThilinaRajapakse
Why is the default value for fp16 set to True . . ? As you point out "Running these models on a CPU is not usually practical though"

[sunyongke]

Set fp16 to False.

How to set the fp16 to False

[sunyongke]

@ThilinaRajapakse

Excuse me, How to set the fp16 to False?

[pulkitmehtawork]

you have to pass that as an argument while training the model e.g.
model.train_model(train_df,args={'fp16': False,'num_train_epochs':3,'overwrite_output_dir':True})

[ThilinaRajapakse]

@ThilinaRajapakse
Why is the default value for fp16 set to True . . ? As you point out "Running these models on a CPU is not usually practical though"

fp16 is for when you are training on GPUs. It doesn't work on CPUs.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jmugan]

For the Classifier model, I had to pass args={'fp16': False} to the constructor.