Simpletransformers: When running the model with cuda, "cant pickle addon objects"

[ssaeed-rpx]

model = ClassificationModel("roberta", "roberta-base",use_cuda=True)

Train the model

model.train_model(train_df)

Evaluate the model

result, model_outputs, wrong_predictions = model.eval_model(eval_df)

TypeError Traceback (most recent call last)
in ()
10
11 # Train the model
---> 12 model.train_model(train_df)
13
14 # Evaluate the model

9 frames
/usr/lib/python3.6/multiprocessing/reduction.py in dumps(cls, obj, protocol)
49 def dumps(cls, obj, protocol=None):
50 buf = io.BytesIO()
---> 51 cls(buf, protocol).dump(obj)
52 return buf.getbuffer()
53

TypeError: can't pickle AddedToken objects

[NaRuecker]

Same issue here. Follwing

[kanik4agg]

Same Issue.
Following.

[ThilinaRajapakse]

This is likely caused by the latest tokenizers update. Please try tokenizers=0.7

[arqgroup-IsaacAsamoah]

This is likely caused by the latest tokenizers update. Please try tokenizers=0.7

Hey @ThilinaRajapakse, tried the above and received the same error msg
This first happened on my gpu instance, this test was run on AWS EC2 ml.t3.xlarge

Also tokenizers=0.7 is not compatible with latest transformers
ERROR: transformers 3.0.0 has requirement tokenizers==0.8.0-rc4, but you'll have tokenizers 0.7.0 which is incompatible.

TypeError: can't pickle AddedToken objects.

Code:
```!which python
!python --version (Python 3.6.10 :: Anaconda, Inc.)
!pip install simpletransformers==0.40.1
!pip install transformers==2.11.0
!pip install tokenizers==0.7.0
!pip install wandb
!pip install sentencepiece==0.1.91
!pip install torch==1.5.1
!pip freeze > requirements.txt

import pandas as pd
import numpy as np
import sklearn
import logging
import os
import torch
import wandb
from sklearn.model_selection import train_test_split
from simpletransformers.ner import NERModel, NERArgs

train_data = [
[0, "Harry", "B-PER"],
[0, "Potter", "I-PER"],
[0, "was", "O"],
[0, "a", "O"],
[0, "student", "B-MISC"],
[0, "at", "O"],
[0, "Hogwarts", "B-LOC"],
[1, "Albus", "B-PER"],
[1, "Dumbledore", "I-PER"],
[1, "founded", "O"],
[1, "the", "O"],
[1, "Order", "B-ORG"],
[1, "of", "I-ORG"],
[1, "the", "I-ORG"],
[1, "Phoenix", "I-ORG"],
]
train_data = pd.DataFrame(
train_data, columns=["sentence_id", "words", "labels"]
)

eval_data = [
[0, "Sirius", "B-PER"],
[0, "Black", "I-PER"],
[0, "was", "O"],
[0, "a", "O"],
[0, "prisoner", "B-MISC"],
[0, "at", "O"],
[0, "Azkaban", "B-LOC"],
[1, "Lord", "B-PER"],
[1, "Voldemort", "I-PER"],
[1, "founded", "O"],
[1, "the", "O"],
[1, "Death", "B-ORG"],
[1, "Eaters", "I-ORG"],
]
eval_data = pd.DataFrame(
eval_data, columns=["sentence_id", "words", "labels"]
)

Configure the model

model_args = NERArgs()
model_args.train_batch_size = 16
model_args.evaluate_during_training = True
model_args.overwrite_output_dir = True

cuda_available = torch.cuda.is_available()
print(cuda_available)

model = NERModel(
"roberta", "roberta-base", use_cuda = cuda_available, args=model_args
)

Train the model

model.train_model(train_data, eval_data=eval_data)

Evaluate the model

result, model_outputs, preds_list = model.eval_model(eval_data)

Make predictions with the model

predictions, raw_outputs = model.predict(["Hermione was the best in her class"])```

[arqgroup-IsaacAsamoah]

Hey @ThilinaRajapakse, my bad - some weird kernel issues on my end. Started a fresh instance and the code above now works.
Only note is transformers==2.11.0 since 'tokenizers==0.7.0' is not compatible with transformers==3.0.0

ERROR: transformers 3.0.0 has requirement tokenizers==0.8.0-rc4, but you'll have tokenizers 0.7.0 which is incompatible.

[h-kouame]

I got the same issue with simpletransformers 0.41.1.

It was resolved after I downgraded transformers to 2.11.0 and re-installed simpletransformers 0.41.1.

[wassim-henia]

I got the same issue with simpletransformers 0.41.1.

It was resolved after I downgraded transformers to 2.11.0 and re-installed simpletransformers 0.41.1.

Thanks, it worked for me

[h-kouame]

The title of the issue may be a bit misleading. The issue occurred for me without cuda.

[crispin-nosidam]

I think it happens when you install simpletransformer (using tokenizers 0.7) to trigger the dependency of transformers (using tokenizers 0.8.1rc1).
If you install transformers before you install simpletransformers, then it will not happen.
If you did it the other way around, you can:
pip install simpletransformers (with that error)
pip uninstall tokenizers
pip install transformers

This should fix it.

[akhatami]

I got the same issue with simpletransformers 0.41.1.

It was resolved after I downgraded transformers to 2.11.0 and re-installed simpletransformers 0.41.1.

It didn't work for me. Using following model:
ClassificationModel('roberta', 'roberta-base', use_cuda=False, args={'reprocess_input_data': True, 'overwrite_output_dir': True})

[crispin-nosidam]

I got the same issue with simpletransformers 0.41.1.
It was resolved after I downgraded transformers to 2.11.0 and re-installed simpletransformers 0.41.1.

It didn't work for me. Using following model:
ClassificationModel('roberta', 'roberta-base', use_cuda=False, args={'reprocess_input_data': True, 'overwrite_output_dir': True})

Did you try to uninstall tokenizer then use the transformers' dependency to install it? The main point is to ensure 0.8.1rc1 is installed. (or perhaps you can install the tokenizer 0.8.1.rc1 manually. I am using roberta as well.

[ThilinaRajapakse]

Simple Transformers 0.46.6 _should_ install the correct (which is the latest) versions of both transformers and tokenizers.

[akhatami]

I got the same issue with simpletransformers 0.41.1.
It was resolved after I downgraded transformers to 2.11.0 and re-installed simpletransformers 0.41.1.

It didn't work for me. Using following model:
ClassificationModel('roberta', 'roberta-base', use_cuda=False, args={'reprocess_input_data': True, 'overwrite_output_dir': True})

Did you try to uninstall tokenizer then use the transformers' dependency to install it? The main point is to ensure 0.8.1rc1 is installed. (or perhaps you can install the tokenizer 0.8.1.rc1 manually. I am using roberta as well.

No, So probably this was the case.

[brightgems]

I solved the same issue after install latest verion of python==1.6, transformers==3.0.2 simpletransformers 0.46 in a clean conda enviroment

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.