Haystack: Slow computation on google colab.

[zshnhaque]

I have been using Haystack for a quite long time. Last few days, I am experiencing slow computation on Google colab given I am using GPU. So everytime I need to download GitHub repository and install Haystack and urllib3. Earlier, for question-answer using Roberta or mini-lm takes only 3-4 seconds (with top-k-retriever = 5 and top-k-reader=3), but from last few days, it is taking around +1 minute. Also, during fine-tuning, these 2 readers take only 6-7 minutes for 1 epoch where my annotator file has 300 questions, but now with the same annotation file it shows 15-16 hours for training. So my query is regarding time computation in latest version of Haystack. Also, I would like to add that I am using elasticsearch as my document store and the number of documents are the same both cases.

[tholor]

Hey @zshnhaque ,

Can you maybe share a Colab Notebook where you see this slowdown? A speed difference of this magnitude rather sounds like the CPU is used instead of the GPU.

[zshnhaque]

Hello @tholor , I have check the runtime configuration, also I have checked I am using GPU, for your reference I am sharing the colab notebook with you.
Finetuning_pre_trainded_model.zip

[cupcoder]

I am having similar problem. It looks like the use_gpu parameter is not being considered. It always uses CPU which makes it super slow.

[tholor]

Ok, is this only happening for training or also for inference via the Finder?

@brandenchan Can you please investigate this?

[Arunast2812]

I am having the same problem, slow for FAISS embeddedings, use_gpu is not taken into account. Goes for CPU instead of GPU

[brandenchan]

Hey @zshnhaque, thanks for that notebook. I am running it in colab and I see that the GPU is not being engaged. Am looking into a fix now.

11/10/2020 10:06:49 - INFO - farm.utils - device: cpu n_gpu: 0, distributed training: False, automatic mixed precision training: None

[zshnhaque]

Hey @tholor , yes I see slow computation in training as well as in Finder.get_answer() .

[brandenchan]

Ok so this is a Torch versioning issue. Haystack uninstalls Torch1.7.0+cu101 (Colab's new default) and installs Torch1.5.0. This version is not compatible with Colab's GPUs. PR #576 should fix our tutorials. For your own notebooks, you'll need to add this line, after the installation of Haystack.

!pip install torch==1.6.0+cu101 torchvision==0.6.1+cu101 -f https://download.pytorch.org/whl/torch_stable.html

If one of you can confirm that this solution works, then I will close this issue!

[zshnhaque]

Thank you @brandenchan for the solution, I will check this solution on Colab and share my findings here.

[zshnhaque]

Hello @brandenchan , thanks again for the solution, I checked fine-tuning the reader and tried finder.get_answer(). It is working fine now , getting the results in 3-4 seconds like before.

So you can close this issue. Once again thank you very much.