Simpletransformers: ValueError: too many dimensions 'str' for Multilabel-classification

[vanh17]

Hi @ThilinaRajapakse

I run through your tutorial on Medium and found this bug. My dataFrame structure is like you described but has the number of labels 11 instead of 6. I also created two additional column called "labels" and "text" for the train_data DataFrame.

/work/vnhh/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
I1118 08:14:10.053511 140433564665600 file_utils.py:39] PyTorch version 1.3.0 available.
I1118 08:14:10.217245 140433564665600 modeling_xlnet.py:194] Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex .
I1118 08:14:10.847334 140433564665600 tokenization_utils.py:374] loading file https://s3.amazonaws.com/models.huggingface.co/bert/roberta-base-vocab.json from cache at /work/vnhh/.cache/torch/transformers/d0c5776499adc1ded22493fae699da0971c1ee4c2587111707a4d177d20257a2.ef00af9e673c7160b4d41cfda1f48c5f4cba57d5142754525572a846a1ab1b9b
I1118 08:14:10.847580 140433564665600 tokenization_utils.py:374] loading file https://s3.amazonaws.com/models.huggingface.co/bert/roberta-base-merges.txt from cache at /work/vnhh/.cache/torch/transformers/b35e7cd126cd4229a746b5d5c29a749e8e84438b14bcdb575950584fe33207e8.70bec105b4158ed9a1747fea67a43f5dee97855c64d62b6ec3742f4cfdb5feda
I1118 08:14:11.280796 140433564665600 configuration_utils.py:151] loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/roberta-base-config.json from cache at /work/vnhh/.cache/torch/transformers/e1a2a406b5a05063c31f4dfdee7608986ba7c6393f7f79db5e69dcd197208534.9dad9043216064080cf9dd3711c53c0f11fe2b09313eaa66931057b4bdcaf068
I1118 08:14:11.282683 140433564665600 configuration_utils.py:168] Model config {
  "attention_probs_dropout_prob": 0.1,
  "finetuning_task": null,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-05,
  "max_position_embeddings": 514,
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "num_labels": 11,
  "output_attentions": false,
  "output_hidden_states": false,
  "output_past": true,
  "pruned_heads": {},
  "torchscript": false,
  "type_vocab_size": 1,
  "use_bfloat16": false,
  "vocab_size": 50265
}

I1118 08:14:11.620726 140433564665600 modeling_utils.py:337] loading weights file https://s3.amazonaws.com/models.huggingface.co/bert/roberta-base-pytorch_model.bin from cache at /work/vnhh/.cache/torch/transformers/228756ed15b6d200d7cb45aaef08c087e2706f54cb912863d2efe07c89584eb7.49b88ba7ec2c26a7558dda98ca3884c3b80fa31cf43a1b1f23aef3ff81ba344e
I1118 08:14:15.541236 140433564665600 modeling_utils.py:405] Weights of RobertaForMultiLabelSequenceClassification not initialized from pretrained model: ['classifier.dense.weight', 'classifier.dense.bias', 'classifier.out_proj.weight', 'classifier.out_proj.bias']
I1118 08:14:15.541777 140433564665600 modeling_utils.py:408] Weights from pretrained model not used in RobertaForMultiLabelSequenceClassification: ['lm_head.bias', 'lm_head.dense.weight', 'lm_head.dense.bias', 'lm_head.layer_norm.weight', 'lm_head.layer_norm.bias', 'lm_head.decoder.weight']
Features loaded from cache at cache_dir/cached_train_roberta_512_binary
Traceback (most recent call last):
  File "nn.py", line 54, in <module>
    test_predictions = train_and_predict(train_data, test_data)
  File "nn.py", line 20, in train_and_predict
    model.train_model(train_data)
  File "/work/vnhh/anaconda3/lib/python3.6/site-packages/simpletransformers/classification/multi_label_classification_model.py", line 106, in train_model
    return super().train_model(train_df, multi_label=multi_label, output_dir=output_dir, show_running_loss=show_running_loss, args=args)
  File "/work/vnhh/anaconda3/lib/python3.6/site-packages/simpletransformers/classification/classification_model.py", line 173, in train_model
    train_dataset = self.load_and_cache_examples(train_examples)
  File "/work/vnhh/anaconda3/lib/python3.6/site-packages/simpletransformers/classification/multi_label_classification_model.py", line 115, in load_and_cache_examples
    return super().load_and_cache_examples(examples, evaluate=evaluate, no_cache=no_cache, multi_label=multi_label)
  File "/work/vnhh/anaconda3/lib/python3.6/site-packages/simpletransformers/classification/classification_model.py", line 458, in load_and_cache_examples
    all_label_ids = torch.tensor([f.label_id for f in features], dtype=torch.long)
ValueError: too many dimensions 'str'

[ThilinaRajapakse]

The features are being loaded from cache. Cached features might not be in the correct format if you change the configuration. Set reprocess_input_data to True and see if it fixes the issue.

model = MultiLabelClassificationModel('roberta', 'roberta-base', args={'reprocess_input_data': True})

[vanh17]

yes it works. Thank you for your reply!

[efperez23]

How is it going. I just add that config in True, but keep the same error. :(

[ThilinaRajapakse]

Are you using the latest version of the library?

Please share the code you are running and the error you get.

[tsavinho]

I've run into the same issue. My Dataset looks as follows:
text label
342 The HQ2 Scam: How Amazon Used a Bidding War to... [0, 0, 0, 1, 0]
343 Every Parent Should Know The Scandalous Histor... [0, 0, 0, 1, 0]
344 Chiquita Brands, Part III-A — What (Allegedly)... [0, 0, 0, 1, 0]
345 Looking Back: 25 Years Ago, PETA Ended Car-Cra... [1, 0, 0, 0, 0]
346 Victory! Pharma Company Roche Bans Forced Swim... [1, 0, 0, 0, 0]

model = MultiLabelClassificationModel('roberta', ''roberta-base', num_labels=5, args={'reprocess_input_data': True, 'overwrite_output_dir': True})

The error occurs whilst training the model:
model.train_model(training_df)

Any guidance on how to fix this?

[ThilinaRajapakse]

The column names should be text, labels.

[tsavinho]

Changed it to your suggestions, still no success sadly... Is there any limitations on the length of the text or something?

[ThilinaRajapakse]

There is, but any text longer than the maximum length will be truncated so it won't cause any issues. Can you check the data type of the labels. Make sure they are all python lists and not strings. Saving and loading pandas DFs seems to convert them to strings.

[tsavinho]

Thanks a lot! This was it, panda converted it into a string 🙄

[ThilinaRajapakse]

That damned panda!

[azamatolegen]

I am facing the same error:
`df['labels']

0 (0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0)
1 (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1)
2 (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1)
3 (0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0)
4 (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1)
...
1698 (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1)
1699 (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1)
1700 (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1)
1701 (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1)
1702 (0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1)

Name: labels, Length: 1703, dtype: object
`
df['labels'].astype(int) doesn't help:
ValueError: setting an array element with a sequence.
How you guys fixed that?

[tsavinho]

@azamatolegen I looped over it and split it again^^ not the most beautiful solution, but it worked.

for count,value in enumerate(df["labels"]):
df["labels"][count]= list(map(int,value.strip('][').split(', ')))

But now I'm using a different method to transform it:

from sklearn.preprocessing import MultiLabelBinarizer
mlb = MultiLabelBinarizer()
mlb.fit([['1','2','3','4','5']])
transformedData = mlb.transform(categoryData[:,1])

for i, transformedEntry in enumerate(transformedData):
  categoryData[:,1][i] = transformedEntry

trainCatData = categoryData[:int(len(categoryData)*0.7)]

train_cat_df={}

train_cat_df["text"]=[]
train_cat_df["labels"]=[]

for x in trainCatData:
  train_cat_df["labels"].append(x[1])
  train_cat_df["text"].append(x[2].replace('"',''))

train_cat_df = pd.DataFrame.from_dict(train_cat_df)

[hammad26]

I used this to fix it.

from ast import literal_eval
train_df['labels'] = train_df['labels'].apply(literal_eval)