Transformers: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.

[RoderickGu]

When I finetune Bert with simple_lm_finetuning.py, there seems an error:
"UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector."

Will it influence the performance of the finetuning process ? Thanks in advance for any suggestion.

[thomwolf]

It should be fine. There are probably your output losses.

[RoderickGu]

@thomwolf Thanks for your reply. Could you explain more about why this happens ? I am still confused though

[xinsu626]

Maybe it is caused by calculating the loss in the model's forward function.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[calusbr]

I have the same problem in version 2.2.1, what can it be?

[RoderickGu]

In my case, it doesn’t influence the results

On Mon, Jan 6, 2020 at 5:19 AM calusbr [e-mail] wrote:

I have the same problem in version 2.2.1, what can it be?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/huggingface/transformers/issues/852?email_source=notifications&email_token=AKRMVV4J4UT6HCXEEMOYBGDQ4MVV3A5CNFSM4IFUKP3KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIFNJAI#issuecomment-571135105,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKRMVV37AGO5AXJNPSGUVLTQ4MVV3ANCNFSM4IFUKP3A
.

[sociengineer]

In my case, the speed of training with 4 GPU is the same as 1 GPU. How solve the speed issues?