Transformers: TypeError: '<' not supported between instances of 'NoneType' and 'int'

Created on 26 Apr 2019  ·  3Comments  ·  Source: huggingface/transformers

Hi, I am trying to do classification fine tuning using bert-base-uncased. I am using examples from master and pytorch_pretrained_bert==0.6.2. Here are my repro steps:

  1. I create a train.tsv and dev.tsv file with my own domain data. The files contain sentences and labels separated by a tab. I put these files in /tmp/bertdata

  2. I do fine tuning using: python run_classifier.py --data_dir /tmp/bertdata --bert_model bert-base-uncased --task_name sst-2 --do_lower_case --do_train --output_dir tmp. This works fine and a model, config json, and vocab.txt are placed in tmp

  3. I try to use the fine tuned model on the dev.tsv set: python run_classifier.py --data_dir /tmp/bertdata --bert_model tmp --task_name sst-2 --do_lower_case --do_eval --output_dir tmp_result. When I do that, I get this error:

Traceback (most recent call last):
File "run_classifier.py", line 1024, in
main()
File "run_classifier.py", line 794, in main
t_total=num_train_optimization_steps)
File "/home/ec2-user/anaconda3/lib/python3.6/site-packages/pytorch_pretrained_bert/optimization.py", line 215, in __init__
schedule = schedule_type(warmup=warmup, t_total=t_total)
File "/home/ec2-user/anaconda3/lib/python3.6/site-packages/pytorch_pretrained_bert/optimization.py", line 45, in __init__
if t_total < 0:
TypeError: '<' not supported between instances of 'NoneType' and 'int'

Anything obvious I am doing wrong? Thanks!

wontfix

Most helpful comment

Here is the problem during initialization of the optimizer:
t_total=num_train_optimization_steps)

This var is initialized with None for the first time num_train_optimization_steps = None
and it's initialized correctly only when --do_train flag is passed to the script

    if args.do_train:
        train_examples = processor.get_train_examples(args.data_dir)
        num_train_optimization_steps = int(
            len(train_examples) / args.train_batch_size / args.gradient_accumulation_steps) * args.num_train_epochs
        if args.local_rank != -1:
            num_train_optimization_steps = num_train_optimization_steps // torch.distributed.get_world_size()

but in case of --do_eval this var is None and you got an error from description.

It's a bug, I think, and should be fixed. For you local needs, just initialize the optimizer "somehow" - it's not used while evaluating.

All 3 comments

I also get this problem when predicting,Did you solved the problem?

Here is the problem during initialization of the optimizer:
t_total=num_train_optimization_steps)

This var is initialized with None for the first time num_train_optimization_steps = None
and it's initialized correctly only when --do_train flag is passed to the script

    if args.do_train:
        train_examples = processor.get_train_examples(args.data_dir)
        num_train_optimization_steps = int(
            len(train_examples) / args.train_batch_size / args.gradient_accumulation_steps) * args.num_train_epochs
        if args.local_rank != -1:
            num_train_optimization_steps = num_train_optimization_steps // torch.distributed.get_world_size()

but in case of --do_eval this var is None and you got an error from description.

It's a bug, I think, and should be fixed. For you local needs, just initialize the optimizer "somehow" - it's not used while evaluating.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Was this page helpful?
0 / 5 - 0 ratings