Bazarr: Provider Betaseries several errors

[kxzaon]

Similar to #680

I have a lots of "HTTPError. Exception info: '404 Client Error: Not Found for url" for the provider Betaseries and of course it throttles for 10mn.

01/04/2020 00:10:08|INFO |root |Throttling betaseries for 10 minutes, until 20/04/01 00:20, because of: HTTPError. Exception info: '404 Client Error: Not Found for url: https://subtitles.betaseries.com/uploaded/Avenue%205%20-%20S01E04%20-%20Wait%20a%20Minute,%20Then%20Who%20Was%20That%20on%20the%20Ladder??%20(iNTERNAL.WEB.H264-AMRAP).srt'| 01/04/2020 01:00:43|INFO |root |Throttling betaseries for 10 minutes, until 20/04/01 01:10, because of: HTTPError. Exception info: '404 Client Error: Not Found for url: https://subtitles.betaseries.com/addic7ed/The%20Big%20Bang%20Theory%20-%2012x24%20-%20The%20Stockholm%20Syndrome.AVS-SVA.French.HI.updated.Addic7ed.com.srt'|

I was digging a bit and it seems that Betaseries has some problems with this character "?" for instance.
https://subtitles.betaseries.com/uploaded/Avenue%205%20-%20S01E04%20-%20Wait%20a%20Minute,%20Then%20Who%20Was%20That%20on%20the%20Ladder??%20(iNTERNAL.WEB.H264-AMRAP).srt
But in the case of the second one, it seems that it simply doesn't exist.
https://subtitles.betaseries.com/addic7ed/The%20Big%20Bang%20Theory%20-%2012x24%20-%20The%20Stockholm%20Syndrome.AVS-SVA.French.HI.updated.Addic7ed.com.srt

Is it possible to bypass or return an error instead of throttling ?
I saw on your discord that I'm not the only one who have these errors.

But I have also another problem sometimes and I have no clue where it comes from :
01/04/2020 02:04:44|INFO |root |Throttling betaseries for 10 minutes, until 20/04/01 02:14, because of: AttributeError. Exception info: "'BetaSeriesProvider' object has no attribute 'video'"|

• Bazarr: v0.8.4.2

[morpheus65535]

First: try dev branch. We fixed a lot of providers issues.

[kxzaon]

I updated to the dev docker image.
I will see for the "attribute video" error.
But for the HTTPError of course, it is not better cause it's the provider that failed. I'm wondering if it's possible to bypass the error instead of throttling the provider.
Thanks.

[kxzaon]

Other example :
HTTPError. Exception info: '400 Client Error: Bad Request for url: https://api.betaseries.com/shows/episodes?key=39e31da34c2f&thetvdb_id=73141&season=15&episode=21&subtitles=1&v=3.0'
There isn't any subtitles available for this episode on the provider website so of course the api request return an error.
In this case too, it would be preferable to tell Bazarr that the provider hasn't any subtitles for this episode instead of throttling the provider.

[morpheus65535]

The problem is on their side.

{  "errors": [{"code":0,"text":"An exception occurred while executing '\n            SELECT\n                sub.rec_id AS id,\n                sub.episode_id,\n                sub.langue AS language,\n                sub.source,\n                sub.fichier AS file,\n                sub.date AS timestamp\n            FROM shows_subs_liste sub\n            WHERE sub.episode_id IN ()\n            ORDER BY sub.rec_id DESC\n        ':\n\nSQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ')\n            ORDER BY sub.rec_id DESC' at line 9"}] }

[kxzaon]

Yeah I saw that. I've opened a bug report on their ticket service.

[morpheus65535]

Yeah I saw that. I've opened a bug report on their ticket service.

Thanks a lot for doing this! I'm quite busy and have difficulty finding time to work on that. Your help is appreciated. Keep me informed!

[morpheus65535]

@kxzaon The issue on provider seems to be fixed now, could you confirm if it's working fine now?

[kxzaon]

@kxzaon The issue on provider seems to be fixed now, could you confirm if it's working fine now?

Kind of. We can continue to report every problem with their subs storage server and they fix the problem from the provider side but it seems that this provider has a lot of errors like this. I don't know if it exists with other provider but we can think of a way to ignore these errors and instead of throttling. What do you think ?

Thank you for your work.

[morpheus65535]

You're right, we could probably better manage provider exceptions Ideally, they would be documented and it would make it easier. Let see if @GermanG can do something about it. :-)

[kxzaon]

You're right, we could probably better manage provider exceptions Ideally, they would be documented and it would make it easier. Let see if @GermanG can do something about it. :-)

Thanks a lot.

[GermanG]

@kxzaon can you attach a full log in debug mode? That way I will be able to reproduce it. I'm guessing the subtitle language is French, am I right?

[GermanG]

@morpheus65535 sorry for lagging behind, my days are a chaos lately.

[kxzaon]

@kxzaon can you attach a full log in debug mode? That way I will be able to reproduce it. I'm guessing the subtitle language is French, am I right?

Exactly. Subs wanted are in french.
Debug log > https://hastebin.com/oqewuqelaj.php

[GermanG]

Other example :
HTTPError. Exception info: '400 Client Error: Bad Request for url: https://api.betaseries.com/shows/episodes?key=39e31da34c2f&thetvdb_id=73141&season=15&episode=21&subtitles=1&v=3.0'
There isn't any subtitles available for this episode on the provider website so of course the api request return an error.
In this case too, it would be preferable to tell Bazarr that the provider hasn't any subtitles for this episode instead of throttling the provider.

The provider is being throttled because it's lying about having a subtitle it doesn't have ;-)
If your curl
curl 'https://www.betaseries.com/srt/818786'
You'll get a redirect to
https://subtitles.betaseries.com/uploaded/Better Things - S03E04 - Monsters in the Moonlight? (WEB.H264-MEMENTO).srt">https://subtitles.betaseries.com/uploaded/Better Things - S03E04 - Monsters in the Moonlight? (WEB.H264-MEMENTO).srt
Which is a 404!
Urlencoding the request to not have a 404 results in:
curl 'https://subtitles.betaseries.com/uploaded/Better%20Things%20-%20S03E04%20-%20Monsters%20in%20the%20Moonlight%3F%20%28WEB.H264-MEMENTO%29.srt'
And the answer is kinda scary:
/mnt/nfs/betaseries/production/releases/20191205160547/www/data/srt/uploads/Better Things - S03E04 - Monsters in the Moonlight? (WEB.H264-MEMENTO).srt

So, I can catch the 404 and log the error, so the throttling won't happen, but I guess that "silent" failure will raise other complaints about why is that happening.

[morpheus65535]

So, I can catch the 404 and log the error, so the throttling won't happen, but I guess that "silent" failure will raise other complaints about why is that happening.

Can you catch the redirect followed by a 404 and log it as an error with this provider?

[GermanG]

I've pushed it.
I've tested in my testbed and seems to work, at least it won't break the provider.
@kxzaon if you're able to test my branch in https://github.com/GermanG/bazarr/tree/betaseriesfix please do, if not I'll make a PR to have it in bazarr/development branch.

[morpheus65535]

@GermanG no answer from @kxzaon. Can you PR this to dev and we'll close the issue?

[kxzaon]

I'm sorry guys but I'm using bazarr as container so it is complicated for me to test out your fix branch...
But if I'm reading well the code and the comment from @GermanG it seems it's a good progress.
Thanks

[morpheus65535]

I'm sorry guys but I'm using bazarr as container so it is complicated for me to test out your fix branch...
But if I'm reading well the code and the comment from @GermanG it seems it's a good progress.
Thanks

linuxserver/bazarr:development

[kxzaon]

I'm using the last unstable of hotio docker image but are you sure that the fix is included in the development branch ?
I'm not seeing the changes made by @GermanG for this file

[morpheus65535]

No he didn't PR it yet. I hope he'll be able to do it soon.

[morpheus65535]

@GermanG doesn'T seems to be available to PR so I made it. @kxzaon can you test dev branch and report please?

[kxzaon]

Seems good.
https://subtitles.betaseries.com:443 "GET /uploaded/Better%20Things%20-%20S03E04%20-%20Monsters%20in%20the%20Moonlight?%20(WEB.H264-MEMENTO).srt HTTP/1.1" 404 None
but the betaseries provider status is still "Good".
Thanks a lot for your hard work both of you.