Mediaelch: [TheTvDb] IMDb rating not loaded for some TV shows due to timeout

[bugwelle]

Describe the bug
Reported on the Kodi forum: https://forum.kodi.tv/showthread.php?tid=136333&pid=2914165#pid2914165

For some (but not all) shows the rating is loaded from TheTvDb even though IMDb is selected in the TV scraper settings.

Example show where it works well: Scrubs
Show where it doesn't work: The Big Bang Theory

To Reproduce
Steps to reproduce the behavior:

1. Go to TV show section
2. Scrape a TV show
3. Check rating

Expected behavior
Rating is loaded from IMDb if it is selected in scraper settings.

MediaElch Version:

• [x] 2.6.3-dev (nightly)
• [ ] 2.6.2 stable

Operating System:

• all

[bugwelle]

My answer from the forum:

Oh, today I was able to reproduce this issue. And it's a stupid one. MediaElch sets the timeout for any network request to 3 seconds and then aborts the request.
That means when data from TheTvDb is successfully loaded but the request to IMDb is aborted, the rating from TheTvDb is shown. And it probably only happens on certain pages (i.e. shows) because the page is either larger (which actually shouldn't matter; it's just a few KB; no images) or has higher latency (maybe takes longer to be generated on IMDb's server). Or maybe my internet connection is slower today than on other days. Either way, MediaElch behaves poorly. No error message is shown to the user at all. It is logged but then again: Who reads the log? I myself often don't and I'm the current maintainer...

As a "workaround" I'll set the timeout to 6 seconds. That way IMDb data should be loaded. That's not a real solution but correct error handling will take me a while.

[bugwelle]

I'll create a PR soon that adds some error reporting.

[hewwra]

It is still not working, I checked the log file and this is what it said:

[MediaElch] INFO: [TheTvDb] Load TV show with id: "80379"
[MediaElch] DEBUG: [TheTvDb] TV show with ID "80379" loaded
[MediaElch] DEBUG: [TheTvDb] Load IMDb TVShow with id: ImdbId("tt0898266")
[MediaElch] WARNING: [TheTvDb] Network Error (load imdb): "Operation canceled"

[bugwelle]

(writing from mobile)

Operation canceled only happens if the request is aborted...
Which means that the request must have taken more than 6s if you're using the latest version from today. An ui error should also have been shown.

I'll come back to this issue tomorrow. The timeout is annoying. :)
For me it worked well after increasing the timeout.

Maybe making network timeouts a setting would help.

[hewwra]

My bad I was using 2020-01-18_20-32 but there was a later version called 2020-01-18_21-24 and when I use that one it correctly scrapes the imdb ratings.

However I have discovered another bug :)
If I go to the individual episode and scrape that it will not scrape the imdb rating.
This does not matter that much to me since I will always scrape at show/all episodes level but it should still be fixed.

[bugwelle]

Thank you for verifying the workaround. Regarding your second issue: I had a look at the current TheTvDb scraper as well as the "old" one used in MediaElch v2.6.2. Both don't load data from IMDb at all if a single episode is scraped. And it never did...
That is unexpected. I've opened another issue for that: #871