Hi, I am trying to do classification fine tuning using bert-base-uncased. I am using examples from master and pytorch_pretrained_bert==0.6.2. Here are my repro steps:
I create a train.tsv and dev.tsv file with my own domain data. The files contain sentences and labels separated by a tab. I put these files in /tmp/bertdata
I do fine tuning using: python run_classifier.py --data_dir /tmp/bertdata --bert_model bert-base-uncased --task_name sst-2 --do_lower_case --do_train --output_dir tmp. This works fine and a model, config json, and vocab.txt are placed in tmp
I try to use the fine tuned model on the dev.tsv set: python run_classifier.py --data_dir /tmp/bertdata --bert_model tmp --task_name sst-2 --do_lower_case --do_eval --output_dir tmp_result. When I do that, I get this error:
Traceback (most recent call last):
File "run_classifier.py", line 1024, in
main()
File "run_classifier.py", line 794, in main
t_total=num_train_optimization_steps)
File "/home/ec2-user/anaconda3/lib/python3.6/site-packages/pytorch_pretrained_bert/optimization.py", line 215, in __init__
schedule = schedule_type(warmup=warmup, t_total=t_total)
File "/home/ec2-user/anaconda3/lib/python3.6/site-packages/pytorch_pretrained_bert/optimization.py", line 45, in __init__
if t_total < 0:
TypeError: '<' not supported between instances of 'NoneType' and 'int'
Anything obvious I am doing wrong? Thanks!
I also get this problem when predicting,Did you solved the problem?
Here is the problem during initialization of the optimizer:
t_total=num_train_optimization_steps)
This var is initialized with None
for the first time num_train_optimization_steps = None
and it's initialized correctly only when --do_train
flag is passed to the script
if args.do_train:
train_examples = processor.get_train_examples(args.data_dir)
num_train_optimization_steps = int(
len(train_examples) / args.train_batch_size / args.gradient_accumulation_steps) * args.num_train_epochs
if args.local_rank != -1:
num_train_optimization_steps = num_train_optimization_steps // torch.distributed.get_world_size()
but in case of --do_eval
this var is None
and you got an error from description.
It's a bug, I think, and should be fixed. For you local needs, just initialize the optimizer "somehow" - it's not used while evaluating.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Most helpful comment
Here is the problem during initialization of the optimizer:
t_total=num_train_optimization_steps)
This var is initialized with
None
for the first timenum_train_optimization_steps = None
and it's initialized correctly only when
--do_train
flag is passed to the scriptbut in case of
--do_eval
this var isNone
and you got an error from description.It's a bug, I think, and should be fixed. For you local needs, just initialize the optimizer "somehow" - it's not used while evaluating.