Transformers: Running squad_convert_examples_to_features causes warnings.

[dominichillier]

Running function squad_convert_examples_to_features from data/processors/squad.py causes warning. Very annoying to disable warnings when running function many times :)

Warned that tokenization_base_utils.py, max_len is depricated.
tokenization_utils_base.py:1320: FutureWarning: The max_len attribute has been deprecated and will be removed in a future version, use model_max_length instead.

Code to reproduce.

from transformers import AutoTokenizer
from transformers.data import SquadExample, squad_convert_examples_to_features
tokenizer =  AutoTokenizer.from_pretrained('a-ware/roberta-large-squadv2')
example = SquadExample(None,'what is test','test is good',None,None,None)
features = squad_convert_examples_to_features(
            examples=[example],
            tokenizer=tokenizer,
            max_seq_length=512,
            doc_stride=128,
            max_query_length=64,
            is_training=False,
            tqdm_enabled=False
        )

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Ierezell]

Found this issue looking for the same problem.

I'm using a question_answering pipeline (which call the squad_convert_examples_to_features under the hood) and it's really annoying when doing an information retrieval system to get the warning each time the QnA infrence is made...

I tried to disable it with logging.getLogger("transformers.tokenization_utils_base").setLevel(logging.ERROR) without success.

Do you have any clues how to disable it or to change the code so it's not deprecated anymore ?

Thanks in advance

[LysandreJik]

The version v4.0.0-rc-1 that will be released today or tomorrow will not have this warning anymore.

[vinicius-cleves]

You can also disable this warning with

import warnings
warnings.simplefilter("ignore")