Sonarr: Release wrongly classified as foreign due to ep title

[thezoggy]

release wrongly classified as foreign due to ep title:

need to update regex/eliminate ep title from that decision.. (use the tvdb data you have. cross reference release regex data)

[CatmanIX]

Also does this when multiple languages are listed, even when english is one of them.

[thezoggy]

if any of the blacklisted words are found.. then yes. just because another is in there doesnt exclude it. sonarr knows the ep info.. thus it knows what the ep title is.. so easily should be able to eliminate false positives in that part of the naming.

[thezoggy]

just got hit again with this:
Anthony.Bourdain.Parts.Unknown.S07E03.The.Greek.Islands.HDTV.x264-MiNDTHEGAP
result is: English is wanted, but found Greek

[l3uddz]

Encountered this issue with last nights episode of Below Deck Mediterranean. It was rejected for not being an english release.

Release name:
Below.Deck.Mediterranean.S01E13.That.Was.Very.Greek.of.Us.HDTV.x264-CRiMSON

[krin-san]

I have slightly different issue with Sonarr ignoring RSS-Feed from russian tracker http://www.lostfilm.tv because of incorrect language mapping.

From debug logs below I see release was rejected by 2 reasons: language and quality. I can deal with quality but I can't do anything with language mapping.

RSS feed (part):

<rss version="0.91">
<channel>
<title>LostFilm.TV</title>
<description>Свежачок от LostFilm.TV</description>
<link>https://www.lostfilm.tv/</link>
<lastBuildDate>Tue, 03 Oct 2017 15:39:29 +0000</lastBuildDate>
<language>ru</language>
<item>
    <title>Бойтесь ходячих мертвецов (Fear the Walking Dead). Эта земля — твоя земля (S03E13) [1080p]</title>
    <category>[1080p]</category>
    <pubDate>Tue, 03 Oct 2017 15:37:46 +0000</pubDate>
    <link>http://tracktor.in/rssdownloader.php?id=24772</link>
</item>
...
</channel>
</rss>

Debug log:

17-10-3 19:09:01.0|Debug|Parser|Parsing string '   (Fear the Walking Dead).      (S03E13) [1080p]'
17-10-3 19:09:01.0|Debug|Parser|Episode Parsed. (Fear the Walking Dead) - S03E13 
17-10-3 19:09:01.0|Debug|Parser|Language parsed: English
17-10-3 19:09:01.0|Debug|QualityParser|Trying to parse quality for    (Fear the Walking Dead).      (S03E13) [1080p]
17-10-3 19:09:01.0|Debug|Parser|Quality parsed: HDTV-1080p v1
17-10-3 19:09:01.0|Debug|Parser|Release Group parsed: 
17-10-3 19:09:01.0|Debug|AcceptableSizeSpecification|Beginning size check for:    (Fear the Walking Dead).      (S03E13) [1080p]
17-10-3 19:09:01.0|Debug|AcceptableSizeSpecification|Release has unknown size, skipping size check.
17-10-3 19:09:01.0|Debug|LanguageSpecification|Checking if report meets language requirements. English
17-10-3 19:09:01.0|Debug|LanguageSpecification|Report Language: English rejected because it is not wanted, wanted Russian
17-10-3 19:09:01.0|Debug|ReleaseRestrictionsSpecification|Checking if release meets restrictions:    (Fear the Walking Dead).      (S03E13) [1080p]
17-10-3 19:09:01.1|Debug|ReleaseRestrictionsSpecification|[   (Fear the Walking Dead).      (S03E13) [1080p]] No restrictions apply, allowing
17-10-3 19:09:01.1|Debug|QualityAllowedByProfileSpecification|Checking if report meets quality requirements. HDTV-1080p v1
17-10-3 19:09:01.1|Debug|QualityAllowedByProfileSpecification|Quality HDTV-1080p v1 rejected by Series' quality profile
17-10-3 19:09:01.1|Debug|MinimumAgeSpecification|Not checking minimum age requirement for non-usenet report
17-10-3 19:09:01.1|Debug|RetentionSpecification|Not checking retention requirement for non-usenet report
17-10-3 19:09:01.1|Debug|DelaySpecification|Profile does not require a waiting period before download for Torrent.
17-10-3 19:09:01.1|Debug|HistorySpecification|Performing history status check on report
17-10-3 19:09:01.1|Debug|HistorySpecification|Checking current status of episode [2815] in history
17-10-3 19:09:01.1|Debug|DownloadDecisionMaker|Release rejected for the following reasons: [Permanent] Russian is wanted, but found English, [Permanent] HDTV-1080p is not wanted in profile

[markus101]

@krin-san please open a thread on the forums for support. Your issue is different.

[speedyflipper]

Encountered this while trying to download the show Greek. Every episode is thought to be in Greek, not english

[tedrock]

Currently occurring with the new show “Russian Doll”

[Taloth]

PS, the merged commit a few days ago deals with languages in SxxExx series titles only, not episode titles and not other number formats. Therefore I kept the issue open. (It's also only on the v3 beta, not v2)

[Rumpa112]

It's the same with NORDIC in title. Also a "bug" from around 2015 or so. Which classifies the release as Norwegian. I can't do anything more then advanced batch files myself. But even I can create these ûber simple regex. So perhaps put Taloth is it? on some other tasks, peace.

[Thirrian]

I'm sure the devs appreciate a pull request to fix this if it's so simple...

[Rumpa112]

I'm sure the devs appreciate a pull request to fix this if it's so simple...

Tell me what regular-expression engine they use and a list of all the patterns to match, I'll do it. As well as current regex strings being used.

[dhaos21]

i think this may belong here, This is the cutoff unmet tab, even though the webdl-1080p has been met it is still trying to upgrade the release due to it thinking 2 episodes are Russian and one is Swedish due to the episode name. V3 Sonarr.

[darthShadow]

Considering the recent changes for fetching resolution from mediainfo, would it not be possible to do the same for the language too?

[thezoggy]

language 'tags' vs spoken language vs language of metadata.. these do not align.
You can only guess so much when all you have is a filename/release name to go off of.
Which is what this issue is all about.

Now, @dhaos21 issue is a little different.
At that point there is a physical file and lets say the user has mediainfo installed and the option enabled in sonarr...
You run into a different set of problems because maybe you think the language for the release is x due to the filename/release name. But then per mediainfo you see there is different languages for subs/audio. Which do you believe.. or better what do you classify it as.

As you can see, the 'language' topic is vague and has different meanings in different parts of the app.
Again to bring this all back, sonarr is rejecting releases due to the assumptions it makes via the release name.
All I want is that sonarr to use the metadata it has, such as the episode title, to eliminate the false positives.
This wont solve everything but this would greatly reduce them.

[Rumpa112]

Considering the recent changes for fetching resolution from mediainfo, would it not be possible to do the same for the language too?

You don't even know wtf you are talking about so why vote on my comment!? stupid. The ticket is about releases being classified incorrectly (due to improper regex, google it) and what you are talking about has todo with the language spoken, or rather what language is stored in file metadata, if any. It might contain several language tags making the information useless for this issue.

[tedrock]

What’s with the attitude? I subscribe to this ticket for updates, not you ranting and raving about votes (whatever that means) and calling someone stupid. And then all you did was repeat what thezoggy already said but more hostile and less articulate.

[Taloth]

I'll reply to avoid letting it devolve into a rantfest.

Language from mediainfo is indeed not useful in this case. I admit it has some applications (such as discern between audio and sub languages) but not for this ticket.

Rumpa, the regex is not easy. Even for pure scene titles because the same space is used for episode title and language. It's possible to patch some scenarios out but not all of them.
At this stage in the process the series/episode isn't known yet, so it doesn't have access to the metadata. But ideally one would remove the known episodetitle before trying to parse out the language.
This is also what we did for the partial fix earlier, removing the text before S07E07 to clear out the series title. That was an important fix because it dealt with scenarios where the entire series would have to be handled manually, rather than individual episodes.

You can see some of the troublesome cases here:

My.Series.S07E07.The.Slabside.Redemption.German.Custom.Subbed.720p.HDTV.x264.iNTERNAL-VhV
my.series.s01e07.the.arrow.of.god.french.dvdrip.xvid-pdtvfr

This ticket is here to track a low priority issue, it's not on anybodies active todo list, it's simply not important enough. It's squarely in 'not anytime soon, maybe never' territory. So you'll just have to deal with the individual episode manually.

PS: I'm not going to have a discussion about this, got other things to do. But I felt it useful to reiterate where the problem lies.

Have a nice day y'all, enjoy your series!

[beyondmeat]

Why not handle these cases. Most releases that are in a different language have it after the rls group now.

Language +"Custom.Subbed" = Treat as it's foreign.

For this case
ies.s01e07.the.arrow.of.god.french.dvdrip.xvid-pdtvfr

Language + .Quality, look at the other results, if 98% of releases in the results are similar, treat it as episode title)

Right now "The Substitute Spanish Prisoner" is treated as foreign audio.

Language + Not-A-Known-Word/tag = NOT foreign language
Not-A-Known-Word/tag + Language = NOT foreign language

Wouldn't be perfect, but it would be a lot better than it is now.

Of course, you have the TVDB, it's as easy as checking of Episode.Title.ContainsAny(Languages[]) == True then English.

[beechfuzz]

Just got hit with this one, forcing me to manually download the episode: