Sonarr: Advanced Parser

[SonarrBot]

Right now we rely on a very specific order when parsing files, this works reasonably well for most cases, but has some conflicts with similar naming standards:

Series Title - 101 - Episode Title - This could either be abs 101 or s01e01 depending on the type of the series.

When we're scanning series folders we know the content type of the episodes, so we could use the anime regexes followed by the standard regexes in order to make the best decision, for daily we could scan for daily followed by standard and for standard just use standard.

When importing files we would have to determine the series first (from the folder), then use the more specific parsers.

Parsing the release group from a file that ends in the episode title that contains a dash will cause the 2nd part of the title to be treated as the release group: Season 24/The Simpsons - S24E03 - Adventures in Baby-Getting.mp4 has the release group Getting.

[SonarrBot]

Comment from markus101 on Trello:

A proper full re-write would be much more valuable, so moving this down in priority.

[SonarrBot]

Comment from stevebennett12 on Trello:

Please can this be prioritised, currently anime in sonarr is annoying me royally!

But once this is working Anime in sonarr would be amazeballs!

[CatmanIX]

Another improvement that could be made with parsing release groups, recognizing groups that are in [] brackets at the end of file name, such as this:

I would have thought it could do that, since it can recognize it at the beggining of a filename, but i guess there's just not many release groups who put it at the end for it to have been accounted for?

[markus101]

Another case: https://github.com/Sonarr/Sonarr/issues/861

[b0bbert]

Currently it would appear that the parser doesn't handle my preferred format at all:
{Series Title} - [{season:00}x{episode:00}] - {Episode Title}

It fails for any file which contains numbers in the Series title or Episode, this is especially problematic for Castle (2009):

[markus101]

iTunes insists on using an awful naming convention, meaning things like:

11 404.mp4 and 07 The Big 1-2.mp4 aren't parsed well at all, if we can improve things here as well, that would be great.

[rglink]

Since sonarr knows the episode title is it possible to parse on episode title as well? iTunes always adds the exact episode number and title to the filename, this might be a solution when no usable file naming convention is used in a season folder when the exact title has been added.

By the way, I recon that nobody takes issue with a manual setting to iTunes parsing as a work around. Of course this isn't the best solution for users that might for some reason combine iTunes with non-iTunes folders.

[markus101]

Anime use case: Naruto Shippuden 433 VOSTFR 720p HDTV x264-Resistance where 433 is ambiguous due to conflicting with SEE numbering.

[markus101]

A couple more titles that get treated as anime:

11.22.63.S01E01.720p.HDTV.x264-DIMENSION[brassetv].mkv
11.22.63.S01E01.720p.HDTV.x265.10bit.opus-DIMENSION[brassetv].mkv

[markus101]

Another case: Yukon.Gold.S04E08.400p.249mb.MQwebrip.x264-][ Double Down ][ 07-Apr-2016 ]

[markus101]

The {MediaInfo Full} renaming token puts the language in square brackets at the end which triggers the anime parser in some cases.

The 100 - s02e12 - Rubicon - (1080p BluRay) - x264 DTS [EN]

[daiken002]

I also made a comment under #448.
I have been able to setup Sonarr and iTunes to work together happily except for the following.

When scanning files, Sonarr detects iTunes 720p files as SDTV due to the (HD) marker, which is iTunes only puts the resolution in the 1080p files. eg.:

SD = 1-01 Pilot.m4v
720p = 1-01 Pilot (HD).m4v
1080p = 1-01 Pilot (1080p HD).m4v

I thought if this is not something that can be automatically done, then maybe since people often have their own naming conventions there could be a place on the Quality Definitions settings page (on the right of each line after the size limit sliders) for people to specify their only quality filename markers or naming convention for each specific quality.

[markus101]

Mini series format that gets picked up as season 20 episode 6 and conflicts with absolute number parsing.

Planet.Earth.Ep05.Deserts.2006.720p.BluRay.DTS.x264-ESIR

[lolgithub]

Not sure if this is the right place to make this suggestion, please forgive if inappropriate.

I propose adding "--" as a valid separator string. While for example {MediaInfo Simple/Full} and {Episode Title} sometimes contain a single dash, AFAICT they almost never contain a double dash.

At the cost of one character per separator, one could greatly improve sonarr's behavior in some of these edge cases, for example "The Simpsons - S24E03 - Adventures in Baby-Getting".

[markus101]

@lolgithub I don't see what -- instead - would change in that particular scenario and I don't see the overall reasoning for that, but this would be better discussed in another issue.

[markus101]

Another Beauty: J:\TV DLing\Blindspot.S02E14.REPACK.HDTV.x264-LOL\blindspot.214r.hdtv-lol.mkv, falling back to the folder name would work here.

[hbooo]

Hello,

Don't know if I'm in the correct issue, sorry if it's not the case.

I have an example of naming that doesn't match number :

Dragon.Ball.Super.E81.FANSUB.720p.WEBRip.x264-Mystic.Z-Team, sonarr is unable to parse episodes from release name

EDIT : The show is already set to anime

[markus101]

Leading number in episode title get's treated as range of episodes: Some Series - S01E03 - 33.mp4

[markus101]

Some release groups don't use a dash to separate the release group (BTN and NTB, possibly others), we'll need to investigate how we can support them as well.

[Delerium]

Every week I've been having issues with Sonarr detecting "The 100". I realize this is an edge case where the show title closely resembles a season/episode format, but the parser should be smarter. An example filename is "The.100.S04E13.HDTV.x264-SVA[eztv].mkv" which fails to import with the message "Invalid Season or Episode". When manually importing it, the title seems to be recognized, but it doesn't fill in the season or episode. To me this file has a clear exact match to the title, as well as clear season and episode indicators. I'm not sure what the solution is, other than to first do a scan for the "SxxExx" format, and if it finds it, ignore any other numerical data for season/episode.

[markus101]

A couple more cases are in https://github.com/Sonarr/Sonarr/issues/1951

• Saekano S2 - 06 (1280x720 HEVC2 EAC3) - an anime release without a hash or release group in square brackets
• Additionally relying on the folder structure when parsing existing files (using the season folder to prefer matching to episodes in that season if it's not part of the filename)

Another case in: https://github.com/Sonarr/Sonarr/issues/1088 where an episode in the wrong series was updated because the filename matched to another series with ssee style naming. Pulling release information from the folder/download client would allow us to match this better.

[elementalest]

Here's another example where name parsing fails. Sonarr will not recognise files already on disk with this naming scheme:

[LostYears] Attack on Titan S2 - 01v3 (26) (WEB 720p Hi10 AAC) [ECFEACE8]

Or when searching for a file:

[LostYears] Attack on Titan S2 (Shingeki no Kyojin S2) - 01v3 (26) (WEB 720p Hi10 AAC) [Dual-Audio]

However, if I add an E in the file name before the episode number sonarr picks it up in both occasions (I was using season folders).

[markus101]

Currently Dara.O.Briains.Go.8Bit.S02E10.WEB.h264-TASTETV gets it's title parsed to Dara.O.Briains.Go due to the SimpleTitleRegex cleaning out 8Bit, for now there is an alias to handle that, but it's not ideal.

[Protektor-Desura]

I should add that I use:
/New-TV/The 100 (2014)/Season 1/The 100 - S01E01 - Pilot [Bluray-1080p HEVC AAC].mkv

and it works just fine. If I do this:
/New-TV/The 100 (2014)/Season 1/The 100 - S01E01 - Pilot [1080p HEVC AAC].mkv

It does not work and isn't imported.
So there is definitely a bug if simply adding "Bluray-" makes it work. I should note that none of my "The 100" episodes would import until I added "Bluray-" and "WEBDL-"

I have noticed that if you have episode titles with numbers in the them, they also fail.
/TV/Night Rider/Night Rider - S03E05 - Door 23 [1080p x264 AAC].mkv

fails to be detected.

/TV/Night Rider/Night Rider - S03E05 - Door 23 [WEBDL-1080p x264 AAC].mkv

is detected.

I can also remove the episode name, have it scan, then rename the episode and it will show up properly with number in the episode title as well.

[ShadowJonathan]

Why not add "custom regex" support? To make your own list of regex parsers that can be shared, made, edited, and used by loads of people.

Make some regexes priorities, to check against it and see if it fits.

/^(?P<title>[\w\s])s(?P<seasonnumber>\d+)e(?P<episodenumber>\d+)(?P<extra>.+?)\.(?P<ext>\w+)$/
(Just a sketch, throwing it in)

[Sonii]

Hello,

Why it cannot handle a such easy name ?

regards

[markus101]

A poorly named 720p release: bring.it.3x17_720

[devolution2409]

seems parser has issue with dot aswell:
[ www.Torrent9.PH ] The.100.S05E05.FRENCH.720p.HDTV.x264-SH0W.mkv
isn't recognized and must be imported manually.

[joska1993]

The Flash - Temporada 5 [HDTV 720p][Cap.519][AC3 5.1 Castellano][www.pctnew.com].mkv
Cap.519 = 5x19

is this possible?

[Taloth]

Isn't 'Cap' basically 'episode'/'chapter'? Temporada 5 Cap 19. So how would Sonarr differentiate between Cap 19 and Cap 519? or rather between Cap 119 (1x119) and Cap 119 (1x19)

_Just asking so we have all info if we ever get to build the advanced parser._

[mail0r]

@Taloth yes, that is the case. The number is always 3 or 4 digits long. I'm using this re to differentiate: "Cap.([0-9]{1,2})([0-9]{2})".

@joska1993 You might find this code useful if you're using transmission. If you're using linux and Sonarr3, save the file somewhere with execution permissions and then go to Settings > Connect > Custom Script. Just tick the on grab box and it will be executed when Sonarr sends a torrent to transmission. It will rename the video files so that Sonarr's postprocessor will understand them.

#!/usr/bin/env python3

# transmissionrpc python reference: https://pythonhosted.org/transmissionrpc/reference/transmissionrpc.html#mutators
# Sonarr Reference: https://github.com/Sonarr/Sonarr/wiki/Custom-Post-Processing-Scripts

import transmissionrpc
import os
import re

videoFilePattern = re.compile( "\.(mkv|avi|mp4)$" )
renameRE = re.compile( "Cap\.([0-9]{1,2})([0-9]{2})" )

# Check if sonarr_eventtype is defined and if it's equal to Grab
if not 'sonarr_eventtype' in os.environ or os.environ['sonarr_eventtype'] != 'Grab':
    exit(0)


tc = transmissionrpc.Client('localhost', port=9091 )

# Obtener ID de torrent


#print "Release title: " +  os.environ['sonarr_release_title']
torrent_id = os.environ['sonarr_download_id'].lower()
#print "Download ID: " +  torrent_id

torrent = tc.get_torrent( torrent_id )
files = torrent.files()
#print files




for i in files:
    file = files[i]
    fullPath =  file['name']
    fileName =  fullPath.split('/')[1]
    # If it's a video file
    if videoFilePattern.search( fileName ):
        # If it's named like "Cap.XXYY"
        if renameRE.search( fileName ):
            newName = renameRE.sub( '\\1x\\2', fileName )
            print( "Renaming " + fileName + "to " + newName )
            tc.rename_torrent_path( torrent_id, fullPath, newName )

exit(0)

[mcronce]

Just throwing in another couple use cases

• Show Title/XX/YY - Episode Title.ext (common case, handled by Plex/Emby/Jellyfin/Kodi/etc)
  
  
  
  • e.g. Scrubs/07/08 - My Manhood.avi
  
  
• Show Title/XXXX/YY/ZZ - Episode Title.ext (for daily shows, seems to have sporadic client support)
  
  
  
  • e.g. The Daily Show/2013/01/09 - Jeff Bridges.mkv
  
  

I use these because both are conducive to being browsed with a file manager.

Custom regexes as an advanced setting would definitely work for me, just as long as they receive either an absolute path to the file, or a path relative to a top-level media container, e.g. I mount /tv into the container. Either way is easy to handle with a regex.