Radarr: Limitless Loop: DownloadedMovieImportService (Import failed, path does not exist or is not accessible by Radarr)

[rkmfl]

Description:
The DownloadedMovieImportService will loop, endlessly, when it cannot find a file. It will continue forever. I've watched as 15x log files per day are created and filled with this single log-item as Radarr continues to locate a missing file.

Sidenote:
Until this is resolved, is there a way to reset this data manually?

Radarr Version:
0.2.0.696
0.2.0.596

Logs:
I've pasted the relevant portion of the log (rather than entirety) so save you the excess noise.

In the logs, notice that __after Housekeeping completes__, this endless loop continues.

17-6-1 23:43:22.0|Info|HousekeepingService|Running housecleaning tasks
17-6-1 23:43:22.1|Info|Database|Vacuuming Log database
17-6-1 23:43:22.2|Info|Database|Log database compressed
17-6-1 23:43:25.2|Info|Database|Vacuuming Main database
17-6-1 23:43:25.4|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:43:25.7|Info|Database|Main database compressed
17-6-1 23:43:55.3|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:44:25.2|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:44:56.4|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:45:25.4|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:45:55.4|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:46:25.4|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:46:56.5|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:47:25.5|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:47:56.0|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:48:25.8|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:48:56.8|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:49:26.0|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:49:56.1|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
17-6-1 23:50:26.1|Error|DownloadedMovieImportService|Import failed, path does not exist or is not accessible by Radarr: G:\MoviesNew_Downloading\Distance HKG 2016 BluRay iPad 720p AAC x264-OurPad\
Radarr Ver. 0.2.0.696

[fryfrog]

I'm guessing G:\MoviesNew_Downloading\ is a network share. Use UNC paths instead, like \\server\share\. And I'm also guessing your Radarr is running as a service. They default to SYSTEM user, switch it to run as your own user instead.

[rkmfl]

@fryfrog - Thank you. Unfortunately that's an unrelated issue. I'm well aware of the issues in running as a service, and these are all addressed. Radarr works fine in my configuration where it has full rights on both local/remote systems. I would not have reported an issue if this was user error.

The problems is not with this accessing files on a share, as other files work just fine. The problem I'm reporting here is the outcome of what happens in Radarr when there is a problem with a file that has downloaded from SABnzbd, that goes missing (for example, due to corruption, or Radarr not matching the file due to some punctuation, etc.

I want to be sure this is clear; I don't want this issue closed thinking this is user error. The problem is that Radarr will make literally hundreds of thousands of attempts to access a file that does not exist (when everything is working properly and this is _not_ a network issue), logging every one and pushing good logs out.

You can emulate and recreate the issue by simply deleting a file that has downloaded from SABnzbd and watch your system do the exact same.

This is not the only place this occurs. Another area in Radarr where this happens is in the event that Radarr will download a movie that it will not "match" once downloaded. For example, a movie with punctuation in the name. Radarr will download it, not realize it's downloaded, and continue re-downloading it.

This happened to me one month ago and I only learned about the once Radarr re-attempted the same few movies hundreds of times and it consumed the remaining 400GB of my Usenet "block" account.

In short, Radarr should implement a limited number of retries, no matter the failed operation or where it occurs in the system.

[galli-leo]

Yeah, radarr tries to import the file everytime it scans your download client (every 30 seconds IIRC). Just remove the troubled item from your sab history and the logging will stop. The same happens in Sonarr.

[rkmfl]

You're right, that fixed this instance of the problem, thank you.

The reason this has never been a problem in Sonarr (yet it has been a problem many times in Radarr) -- is due to that single character-replace bug in Radarr. This is something that is not needed in Sonarr, which explains why it's wrecked such havoc in Radarr.

In other words, many times Radarr will believe a file is missing, when it's not, due to requiring a fairly rigid name match without much flexibility. This breaks every time you have a movie with a two-part name, for example:

John Wick: Chapter 2
xXx: Return of Xander Cage

Sonarr does not need to contend with TV show names that include colons, it seems. However this is a big one with movies. All of the metadata scanning tools seem to use a replace function where ": " becomes " - " (colon + space _becomes_ space + hyphon + space).

In Radarr, the colon is simply eliminated in the rename. This would be fine, except Radarr also will not recognize any file that is named with the hyphen.

With the two movies above, Radarr will _not recognize the movies_ when the hyphen is used.

Regardless of that separate "fuzzy naming" issue that needs to strip special characters, etc -- I still believe it's a good failsafe to have a limited number of retries on any operation that has failed in the past.

I know you're attempting to work within the confines of Sonarr, and this is not used in Sonarr. However, this is one very key difference between movies and TV series/episodes.

TV series/episodes are far more regimented due to being heavily automated (it's not realistic that people download too many TV shows manually).

Movies, on the other hand, people have have large collections, or import them from sources outside of Radarr. Or, in this early-alpha-stage of Radarr, even Radarr's own downloads are not always being recognized by itself, due to this issue.

I suspect Sonarr didn't focus as much on the "fuzzy match", and they did not worry about implementing a failsafe that does not loop infinitely, because these issues are exponentially less common with regimented TV.

As an aside - you guys are doing amazing work - I don't mean for this to sound critical, I'm only doing my best to try and explain the logic as to why I feel this is important despite the common reply that it's not something that will be done due to it being outside the scope of Sonarr.

Thanks for everything you do and all the hard work!

[hotio]

Everything you said above, I have not experienced a single time. @galli-leo can correct me on this, but I think that Radarr ignoring a release because of :- in the title is false. A limit on the amount of retries seems a bit pointless when your setup is setup properly and might cause even more problems when set too low in some cases.

[rkmfl]

@hotio - You may just be misinterpreting my steps to reproduce, because @galli-leo has already acknowledged the naming issue in the post I created to raise this issue.

Regarding a fail-safe on re-attempts potentially hurting something, that's just flat out wrong. It doesn't have to be user configurable, it doesn't have to be low, but it can avoid major catastrophies in situations where Radarr is configured properly.

I had Radarr use 400GB of my "block" account on Usenet, until there was no bandwidth remaining, simply because I was out-of-town and it continually attempted to re-download the same movie that it didn't recognize as already having been downloaded, for exactly the reason I mentioned above, which again, was acknowledged in the threat I posted.

It's great you have better fortune with your own use; however, I'm attempting to provide information to improve the software. Not for selfish reasons, because I've stopped using Radarr as an automated tool. I've gone back to CP.

I only run Radarr in a diagnostic fashion (all donwloads are set to manual and I occasionally set them to process, for testing purposes ) .. in hopes of seeing the progress and switching to Radarr at some point in the future.

And although you have had good luck, you shouldn't chalk someone else's problems to user-error or a bad configuration, simply because of your own good fortune. I've spent hours testing vanilla defaults vs special configs -- monitoring logs, and attempting to post clear details on my findings. Please don't cheapen that by claiming I simply don't how how to properly configure the utility.

[galli-leo]

@rkmfl Maybe I am kinda stupid, but where exactly do you experience the hyphen issue? Is it when a download is finished and Radarr tries to import it? Is it when you add a movie file to the directory Radarr sets for the movie and you click update and scan disk? Also what naming config are you using?

[rkmfl]

@galli-leo - I don't understand the need for the "stupid" comment. I'm trying to help. If any of this bothers you or justifies sarcasm, I apologize. Please let me know and I won't invest any more time in noting issues here.

There are the two issues I've outlined. The first is the one you've acknowledged and marked a "bug" here:

(1) Radarr Created Directory/Paths <> Radarr Named Movies https://github.com/Radarr/Radarr/issues/1432

(2) As part of that same thread, I mentioned "fuzzy" matching and clarified that I was referring to stripping articles (And, The, etc) and simple things like ignoring punctuation. This is in situations where Radarr must "match" movies that might already exist in the library, as I've explained above.

You stated that this is already being done by Radarr. However, I'm trying to explain that it is not being done in 0.2.0.696 as you've indicated.

As with both examples above, if there is a movie already in the library path, under the methodology I explained, it typically will not match that movie.

I provided two current samples that you can use to reproduce the issue:

John Wick: Chapter 2
xXx: Return of Xander Cage

Simple name the folder / files as they would be named by any common metadata renamer, which is most commonly with a hyphen. For example:

Root\John Wick - Chapter 2 (2017)\John Wick - Chapter 2 (2017).ext {mkv|mp4|etc}
Root\xXx - Return of Xander Cage (2017)\xXx - Return of Xander Cage (2017).ext {mkv|mp4|etc}

Using the {Movie Title} ({Release Year}) methodology in Radarr, this should get recognized. In my 0.2.0.696 it is not being recognized, simply because of the hyphen.

If I remove the hyphen, no problem. In short, if Radarr is working as you explained, ignoring articles and punctuation or other rudimentary issues (plurals, etc) - this would not be a problem. Therefore, I thought I was helping by reporting this problem to you.

Now .. adding complexity to the matter .. Technically in two instances these files were recognized initially, yet never moved out of the download-queue. Then later they were removed completely by Radarr, as if they had not matched (in other words, part of the fuzzy matching may be working at one level, and not working at another).

Again .. I'm not actively using Radarr, simply trying to report issues I find. So there is no issue if you don't feel these issues are of concern.

[hotio]

I had Radarr use 400GB of my "block" account on Usenet, until there was no bandwidth remaining, simply because I was out-of-town and it continually attempted to re-download the same movie that it didn't recognize as already having been downloaded, for exactly the reason I mentioned above, which again, was acknowledged in the threat I posted.

As far as I understand Radarr shouldn't attempt to re-download a different release if it can't find the path of the download, until the download client tells Radarr the download has failed...

Not sure where you see me telling your problem is user-error either?

I'm not getting your point about hyphens and colons either, 1) They are absent in most release names, so how can they have an impact? 2) AFAIK Radarr just interprets the path given by your download client, so if this would contain a hyphen, why would radarr give the error that it can't find the path?

[rkmfl]

Radarr shouldn't attempt to re-download a different release if it can't find the
path of the download, until the download client tells Radarr the download has failed...

That's great news. Just know that evidently things can happen, because in my case, it certainly did get stuck in some strange loop and continuously downloaded the same few movies hundreds of times in repetition. I understand that's not the way it's supposed to work, which goes back to my point of a "failsafe" (something that only exists in the event that something goes unexpectedly wrong).

Unfortunately I have no way to know why it happened because my logs were set to "info" level at the time, not having realized I'd stumble upon something like this. I could not trace-back to the original event because of the thousands of redundant entries.

None the less, the reason why something like that happens is less important than my point:

As a software developer myself, I realize that with tech ... Sh!t happens. Frequently. Sh!t you never planned for, and swear can't possibly happen .. that you swear the user has done something wrong, etc. Having a "failsafe" can prevent something from going "off the rails" so-to-speak.

The issue with eating my bandwidth/downloading was one of only two. The second was a similar issue where SABnzbd an Radarr didn't see eye-to-eye and it continually attempted to re-import a file that doesn't exist (an issue in Sonarr as well, but not something I've ever run into -- and I've used Sonarr since the first nzbDrone releases).

I'm not getting your point about hyphens and colons either,
1) They are absent in most release names, so how can they have an impact?

I think we just have a miscommunication between us.

When using "lists" to populate Radarr with desired movies for download, those lists include proper names. It has nothing to do with scene-group titling, it has to do with the names as Radarr adds them, and how Radarr creates those names in the file system based on that metadata --or-- what Radarr searches for, to determine if these releases already exist (the scene names are irrelevant in that process).

If you signed-up for a "list" that would populate your system with the two new movies I mentioned above, and those movies are already in your library, titled as I explained, you will run into the issue as I've explained it above. I really don't know any other way to explain it differently.

2) AFAIK Radarr just interprets the path given by your download client, so if

I'm guessing you didn't follow my link to the acknowledged bug relating to my point here, Issue# https://github.com/Radarr/Radarr/issues/1432 - which explains the exception to this, that likely results in the error I'm referring to.

If you re-read my very last post in this tread, once more, it may make more sense in that I'm guessing we use Radarr two different ways, somehow. But that post really does give you everything you need to identify the problem.

Thanks for looking into it.

[galli-leo]

@rkmfl Sorry my comment was not meant sarcastic in any way, I just wanted to apologize for you having to explain the problem again to me :/. Not a native speaker.

Anyways, I think I understand your problem now. To be clear, the following happened right?:

1. You downloaded John Wick Chapter 2 with another program.
2. Another program or you yourself create a folder called John Wick - Chapter 2 (2017) and placed the movie in there
3. Then you added the movie in Radarr by just searching for John Wick or via a list.

If this is what happened, then I know why it happened. When you add a Movie in Radarr (via lists or search), it stores a path for it to store movie files. The path it stores movies downloaded by it is in this case "John Wick Chapter 2 (2017)". It only looks for existing movies in this path. If you wanted to have it look in the path you created, you would either have to import existing or change the path of the movie within Radarr to point to your directory.

[rkmfl]

@galli-leo
Thank you for the confirmation. I'm sorry you have so many issues on the forum to divide your attention and surely it's maddening.

You're a little closer to understanding the steps I've explained to reproduce, and the purpose of the post.
But just a little off-topic.

I'll clarify further, based on what you've written so far:

First, remember there are two issues that I was suggesting are related to my suggestion for a "failsafe" (after all, this post is about the idea of a failsafe limiting the total attempts in the system to stop potential catastrophe that is not expected).

As part of my explaining a need for Failsafe, I touched on an example, the first example that's already documented in my other #1432 post, where Radarr does not register movies that have already downloaded. The second issue is one where "fuzzy matching" that you believe to be working in Radarr, is not.

By that I mean that you posted previously that some basic fuzzy matching is already being done (ignoring articles, etc). However, one of the most simple things that needs to be done (ignoring punctuation) is currently not being done.

To clarify the examples for each of these issues:

(1) Issue#1432 where Radarr actually creates a different directory than it uses to search for the file once it's downloaded. Let me clarify with an exact, reproduceable copy from Issue 1432:

(First path shown is the path Radarr created, once downloaded)
(Second path show is where Radarr searches for the movie, once checking whether it exists in the library, after downloading)

Creates: C:\Movies\Crouching Tiger, Hidden Dragon - Sword of Destiny (2016)
Searches: C:\Movies\Crouching Tiger, Hidden Dragon Sword of Destiny (2016)

Creates: C:\Movies\Mechanic - Resurrection (2016)
Searches: C:\Movies\Mechanic Resurrection (2016)

This is Radarr acting alone, and does not involve me using an outside program, existing file, etc.

The second example, is where I have a movie that's already downloaded.

Imagine I have 10 movies in my library, which is seen by Radarr.

Radarr will recognize those movies and it will not re-download those movies, even though they were downloaded by SABnzbd -- providing they are named correctly. No problem there.

Now imagine that 2 of those movies are the examples I gave above:

John Wick: Chapter 2
xXx: Return of Xander Cage

Simple name the folder / files as they would be named by any common metadata renamer, which is most commonly with a hyphen. For example:

Root\John Wick - Chapter 2 (2017)\John Wick - Chapter 2 (2017).ext {mkv|mp4|etc}
Root\xXx - Return of Xander Cage (2017)\xXx - Return of Xander Cage (2017).ext {mkv|mp4|etc}

Using the {Movie Title} ({Release Year}) methodology in Radarr, this should get recognized. In my 0.2.0.696 it is not being recognized, simply because of the hyphen.

In summary of the #2 issue Radarr is matching existing titles and not downloading them, and it works great. Just one problem - Radarr is not ignoring something as simple as a hyphen, which I believe you are under the impression that it is (based on another one of your replies). I also believe that this is half-implemented, because I've witnessed Radarr match the hyphen-movies in the "files" tab, yet claim it's not downloaded.

Simple summary
(1) Radarr, on its own, is mixing up the hyphen issue
(2) Call it a feature-request if you'd like, but I think it's a bug that fuzzy-matching is partially implemented, but not working perfect.

Does this help at all?

Thanks again for your time to investigate!

[hotio]

Short summary

1000's of people not experiencing this bug, the one person who experiences it has some frankenstein test bench running.

_Conclusion: 1000's of people are wrong._

If it's so easy to replicate, why don't you start with a fresh log file, add John Wick and show us the log start to finish where radarr messes up.

[gigq]

FWIW, I've also ran into issues where a single movie can't be processed and it blocks all other downloads from being processed, so this guy is not alone.

I'd love to have an option to skip movies that can't be processed after X failed attempts.

[galli-leo]

@rkmfl Can you post a screenshots (for both 1) and 2)) of the movies in question loaded in the radarr ui with the files tab open?

[rkmfl]

@galli-leo I would love to. Unfortunately I uninstalled Radarr and went back to CP.

I am a donor and I would much prefer to use Radarr at some point ... I just became frustrated with being trolled and insulted on by hotio in every post where I tried to explain the details of my experiences -- and so I resigned to wait until a more stable release in hopes of avoiding further ridicule.

I'm sorry.

[hotio]

Just because somebody tells you anything that you prefer not to hear doesn't mean it's ridicule. IIRC you still have not produced any logs... but hey, no hard feelings and the best of luck with CouchPotato, I hear it's great.

[galli-leo]

Closing this issue, as there are a lot of other one's about large log files and this one is more about #1432

Duplicate of #1432