Transformers: Unclear error message when unable to cache the model

[czyzby]

I encountered the following error:

[2019-05-07 11:06:51,904: ERROR/ForkPoolWorker-1] Model name 'bert-base-uncased'
was not found in model name list (bert-base-uncased, bert-large-uncased, bert-base-cased,
bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased,
bert-base-chinese).
We assumed 'https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz'
was a path or url but couldn't find any file associated to this path or url. 

After some debugging, I found that the root cause of the issue was the fact that the application is unable to cache the model in the home directory. It was a simple I/O error rather than an issue with the model name or file downloading, as the message suggests. I think it would be worth it to handle this case with an appropriate message, and what's more important - throwing an exception.

In my case, I did get the error logs, but the application initiated "successfully" - with the tokenizer and model set to None. If the library is not able to load the model for any reason, I'd expect it to throw an exception rather than just (almost) silently return a None.

[thomwolf]

Yes, this error message hides several potential sources, I'll see if I can disentangle the error messages :)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[czyzby]

This is still an issue. I suggest to improve the message and raise an exception if unable to load any of the models, instead of silently returning None.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[czyzby]

Still an issue, as far as I know.

[twothousand]

@czyzby Hello，How do you solve the cache problem? I have the same problem but can't fix it

[czyzby]

@twothousand To be honest, I don't remember, but I think the directory did not exist or lacked write permission. Are you sure the cache is causing the problem? If you changed the cache directory, make sure the folder exists and has appropriate permissions - otherwise I'd debug model loading and see which exception is being ignored.

@thomwolf It seems that it's still an issue, will you look into proper error handling?

[thomwolf]

Yes, it should have been improved on the latest release 2.1.1 (with the merge of #1480).

Maybe open a new issue with clear details of the current issue?