Captum: Questions about the IMDB sentiment analysis tutorial

[Jethro85]

❓ Questions and Help

Hello,

I'm new to captum.
I download the IMDB sentiment analysis tutorial and run it in jupyter notebook. I download the model and the Glove according to the tutorial and run the codes step by step. However, I get different results:

As you can see, the prediction results for the fourth and sixth sentences are totally different from the results shown by the turtorial.
Are there any errors in my operation?

[NarineK]

Hi @Jethro85, thank you for the question! We had a similar question asked here:
https://github.com/pytorch/captum/issues/399
It could be different versions of IMDB dataset, perhaps also related to different versions of torchtext. What is the vocabulary size in your example ? What version of torchtext do you use ?

[Jethro85]

Hi @Jethro85, thank you for the question! We had a similar question asked here:

399

It could be different versions of IMDB dataset, perhaps also related to different versions of torchtext. What is the vocabulary size in your example ? What version of torchtext do you use ?

I see. My torchtext version is 0.6.0, and the vocabulary size is 101513. Maybe I have to train a new model.

[NarineK]

Thank you @Jethro85, I tested it locally and I also see vocabulary Size: 101513 and the same result as you see. However as you can see in the github version the vocabulary Size is 101982:
https://github.com/pytorch/captum/blob/master/tutorials/IMDB_TorchText_Interpret.ipynb
I tried lower versions of torchtext, such as 0.5.0 and 0.4.0 and I see the same results as you do.
It can be that the version of spacy has changed that caused different number of vocabulary size.

I unfortunately do not remember what version of spacy I used for training the model.

[Jethro85]

Thank you @Jethro85, I tested it locally and I also see vocabulary Size: 101513 and the same result as you see. However as you can see in the github version the vocabulary Size is 101982:
https://github.com/pytorch/captum/blob/master/tutorials/IMDB_TorchText_Interpret.ipynb
I tried lower versions of torchtext, such as 0.5.0 and 0.4.0 and I see the same results as you do.
It can be that the version of spacy has changed that caused different number of vocabulary size.

I unfortunately do not remember what version of spacy I used for training the model.

Ok. Thanks for your reply. I'll train a new model.

[NarineK]

Sounds great! If you train the model do you mind sharing the notebook with me ? I think I haven't preserved the original one.

[Jethro85]

Sounds great! If you train the model do you mind sharing the notebook with me ? I think I haven't preserved the original one.

No problem. I will give you one version after finishing it.