Flair: TypeError: function takes exactly 5 arguments (1 given) during CharLM training

[amrayach]

Describe the bug
As showen in the picture, I start the script to train my own CharLM forward embeddings and face this TypeError everytime at the same epoch(1) and same split (depends on the batch size) even with
Minimal Arguments like:
(hidden_size=128, nlayers=1, sequence_length=10, mini_batch_size=10, max_epochs=10, patience=25)
or
Big Arguments like:
(hidden_size=1024, nlayers=1, sequence_length=250, mini_batch_size=100, max_epochs=10, patience=25)

To Reproduce
from pathlib import Path

from flair.data import Dictionary
from flair.models import LanguageModel
from flair.trainers.language_model_trainer import LanguageModelTrainer, TextCorpus
import torch

is_forward_lm = True

dictionary: Dictionary = Dictionary.load('chars')

corpus = TextCorpus(Path(''),
dictionary,
is_forward_lm,
character_level=True)

language_model = LanguageModel(dictionary,
is_forward_lm,
hidden_size=1024,
nlayers=1)

torch.set_num_threads(6)

trainer = LanguageModelTrainer(language_model, corpus)

trainer.train(base_path='resources/taggers/language_model', sequence_length=250, mini_batch_size=100, checkpoint=True, patience=25)`

Screenshots

Environment (please complete the following information):

• OS: Linux Ubuntu 18 server Version
• Version: latest version 0.4.1

Sorry if something is not clear this is my first issue hope i'll get some answers.
Thanks in advance.
Best Regards

[stefan-it]

Hi @amrayach,

this error could be related to #654.

Could you please check if you have no hidden files in your training folder 🤔 You could just use find . -name ".*" for that 😀

[amrayach]

@stefan-it Thanks for your fast response I tried find . -name ".*" and ls -a and got no hidden files in my training Directory but I tried to change the general structure as you suggested in #654 and started the training again hope it'll resolve the issue, I will get back to you until I reach the point where the issue always occurs.

Screen-Shot:

Best Regards.
Ammer

[amrayach]

@stefan-it
The suggested structure in #654 did not work and the bug still exists and as I mentioned above there are no hidden files or folders in my training folder.

[amrayach]

Thank you alot, I had to leave this part because I couldn’t resolve this
issue currently I am busy with other stuff but later on I will try this way
an report back.
Thanks for your support. :D

On Fri 21. Jun 2019 at 12:18, laotao [e-mail] wrote:

I ran into the same problem. It turns out that the xxx in train_split_xxx
should be strictly ranged. Mine were 00,01,02,...,89, 9001,9002, ... ,
which produces the bug. After I changed xxx to 00, 01, 02, ..., 89, 90,
91,..., the bug disappear.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zalandoresearch/flair/issues/665?email_source=notifications&email_token=AGCH6W744DGSISJRHTI5RHLP3STGHA5CNFSM4HGHVT42YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYICJBY#issuecomment-504374407,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGCH6WYMOGS42LZA6YT5GPLP3STGHANCNFSM4HGHVT4Q
.

[laotao]

Sorry this is not what actually fixed the bug. I drew the wrong conclusion.

The real problem was how I splitted the training file. I used split -C,
which seemed to produce some illegal inputs to flair. After changed to
split -l, it becomes OK.

amrayach [e-mail] 于 2019年6月23日周日 下午10:43写道：

Thank you alot, I had to leave this part because I couldn’t resolve this
issue currently I am busy with other stuff but later on I will try this way
an report back.
Thanks for your support. :D

On Fri 21. Jun 2019 at 12:18, laotao [e-mail] wrote:

I ran into the same problem. It turns out that the xxx in train_split_xxx
should be strictly ranged. Mine were 00,01,02,...,89, 9001,9002, ... ,
which produces the bug. After I changed xxx to 00, 01, 02, ..., 89, 90,
91,..., the bug disappear.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/zalandoresearch/flair/issues/665?email_source=notifications&email_token=AGCH6W744DGSISJRHTI5RHLP3STGHA5CNFSM4HGHVT42YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYICJBY#issuecomment-504374407
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/AGCH6WYMOGS42LZA6YT5GPLP3STGHANCNFSM4HGHVT4Q

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/zalandoresearch/flair/issues/665?email_source=notifications&email_token=AAYVERA4ZRAO6F763RQTS7LP36DYPA5CNFSM4HGHVT42YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYLACYQ#issuecomment-504758626,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYVEREVI4VS64HTDN23QXLP36DYPANCNFSM4HGHVT4Q
.

[pkreissel]

@laotao could you elaborate on your process on splitting your training data in a way that will not cause the error? I have the same problem but I can't get it to work. Thanks.

>>python train_embeddings.py

2019-07-24 15:14:00,410 read text file with 117899 lines
2019-07-24 15:15:12,251 read text file with 118383 lines
Traceback (most recent call last):
  File "train_embeddings.py", line 30, in <module>
    max_epochs=10)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/flair/trainers/language_model_trainer.py", line 307, in train
    training_generator, self.split
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 637, in __next__
    return self._process_next_batch(batch)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 658, in _process_next_batch
    raise batch.exc_type(batch.exc_msg)
TypeError: function takes exactly 5 arguments (1 given)

[laotao]

@pkreissel try something like the following:
split -l 1000000 --numeric-suffixes corpus.txt train/train_split_

[amrayach]

@laotao Thanks a lot this:
split -l 1000000 --numeric-suffixes corpus.txt train/train_split_

worked for my case.

Cheers :D