Transformers: LXMERT imports

[ecekt]

❓ Questions & Help

Hello, I was very happy to see that LXMERT is being integrated into this library and I wanted to try it out. I got a KeyError in configuration_auto.py, as in CONFIG_MAPPING there was no 'lxmert'. Then, I re-installed transformers from source. The KeyError went away, but this time I encountered issues in modeling_auto.py as below:

model = AutoModelWithLMHead.from_pretrained("unc-nlp/lxmert-base-uncased")

/transformers/src/transformers/modeling_auto.py", line 841, in from_pretrained
raise ValueError(

ValueError: Unrecognized configuration class for this kind of AutoModel: AutoModelWithLMHead.
Model type should be one of T5Config, DistilBertConfig, AlbertConfig, CamembertConfig, XLMRobertaConfig, MarianConfig, BartConfig, LongformerConfig, RobertaConfig, BertConfig, OpenAIGPTConfig, GPT2Config, MobileBertConfig, TransfoXLConfig, XLNetConfig, FlaubertConfig, XLMConfig, CTRLConfig, ElectraConfig, EncoderDecoderConfig, ReformerConfig.
`

In the example here, AutoModelWithLMHead is imported. However, because of the error above, I also tried LxmertPreTrainedModel, which led to some other error concerning the initialization of the weights. Importing LxmertModel instead seems to work out without errors. Would that be the correct way if I want to extract features from a pretrained model?

I would appreciate any help regarding this issue!

Best regards,
Ece

[patrickvonplaten]

Hey @ecekt,

Thanks for your issue!
Yes, you are correct "unc-nlp/lxmert-base-uncased" should only be imported with the AutoModel or LxmertModel class.

@julien-c - the config looks correct I'm not sure why it states AutoModelWithLMHeadModel in the examlpe here: https://huggingface.co/unc-nlp/lxmert-base-uncased

[ecekt]

Hi @patrickvonplaten, thank you for the reply! I will explore its use with the LxmertModel import then.

Best.

[julien-c]

This is now fixed on https://huggingface.co/unc-nlp/lxmert-base-uncased, thanks for the heads up