Sonarr: exFAT import permissions failure on Linux

[MichaIng]

Describe the bug
When mounting an exFAT file system on Linux (tested on Debian Stretch and Buster) with any other user then the one which Sonarr runs as, and configuring Sonarr to import series to this file system, the import fails, regardless of sufficient UNIX permissions. E.g. when running Sonarr as sonarr:sonarr

1. Mounting exFAT with default mount options root:root 777
   Sonarr is able to import series FROM this file system, but not TO this file system, as long as it does not run as root. Testing write permissions via sudo succeeds, Sonarr is even able to create the series sub directory, but not to import the file itself.
2. Mounting exFAT with root:sonarr and explicit 775 permissions
   Same as above
3. Mounting exFAT with sonarr:sonarr
   Import succeeds.

Since the user has definitely write permissions, it must be some Mono or Sonarr internal "authentication" failure in combination with how exFAT is mounted via fuse.

Logs

Couldn't import episode /mnt/exfat/deluge/downloads/XXXX/XXXX.mkv: Access to the path is denied.

System.UnauthorizedAccessException: Access to the path is denied. ---> System.IO.IOException: Operation not permitted
   --- End of inner exception stack trace ---
  at Interop.ThrowExceptionForIoErrno (Interop+ErrorInfo errorInfo, System.String path, System.Boolean isDirectory, System.Func`2[T,TResult] errorRewriter) [0x00014] in <285579f54af44a2ca048dad6be20e190>:0 
  at Interop.CheckIo (System.Int64 result, System.String path, System.Boolean isDirectory, System.Func`2[T,TResult] errorRewriter) [0x0000a] in <285579f54af44a2ca048dad6be20e190>:0 
  at Interop.CheckIo (System.Int32 result, System.String path, System.Boolean isDirectory, System.Func`2[T,TResult] errorRewriter) [0x00000] in <285579f54af44a2ca048dad6be20e190>:0 
  at System.IO.FileSystem.CopyFile (System.String sourceFullPath, System.String destFullPath, System.Boolean overwrite) [0x0005c] in <285579f54af44a2ca048dad6be20e190>:0 
  at System.IO.File.Copy (System.String sourceFileName, System.String destFileName, System.Boolean overwrite) [0x00062] in <285579f54af44a2ca048dad6be20e190>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.CopyFileInternal (System.String source, System.String destination, System.Boolean overwrite) [0x00000] in <4d680ffa10414c3ab7e1372769e11d73>:0 
  at NzbDrone.Mono.Disk.DiskProvider.CopyFileInternal (System.String source, System.String destination, System.Boolean overwrite) [0x00076] in <9c1bc51989834744a8646618373e03e7>:0 
  at NzbDrone.Common.Disk.DiskProviderBase.CopyFile (System.String source, System.String destination, System.Boolean overwrite) [0x000bc] in <4d680ffa10414c3ab7e1372769e11d73>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TryCopyFileTransactional (System.String sourcePath, System.String targetPath, System.Int64 originalSize) [0x00108] in <4d680ffa10414c3ab7e1372769e11d73>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x0034a] in <4d680ffa10414c3ab7e1372769e11d73>:0 
  at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, System.Boolean verified) [0x0000e] in <4d680ffa10414c3ab7e1372769e11d73>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.TransferFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Tv.Series series, System.Collections.Generic.List`1[T] episodes, System.String destinationFilePath, NzbDrone.Common.Disk.TransferMode mode) [0x0012c] in <96f52d5e5b95442a89ae3bbf62e59664>:0 
  at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.CopyEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode) [0x000a6] in <96f52d5e5b95442a89ae3bbf62e59664>:0 
  at NzbDrone.Core.MediaFiles.UpgradeMediaFileService.UpgradeEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode, System.Boolean copyOnly) [0x00167] in <96f52d5e5b95442a89ae3bbf62e59664>:0 
  at NzbDrone.Core.MediaFiles.EpisodeImport.ImportApprovedEpisodes.Import (System.Collections.Generic.List`1[T] decisions, System.Boolean newDownload, NzbDrone.Core.Download.DownloadClientItem downloadClientItem, NzbDrone.Core.MediaFiles.EpisodeImport.ImportMode importMode) [0x00281] in <96f52d5e5b95442a89ae3bbf62e59664>:0 

• To avoid confusion, it is indeed not the access to the Deluge download (importing exactly the same file to an ext4 file system with proper UNIX permissions works fine), but access to /mnt/exfat/... Sonarr import target.

System Information

• Sonarr Version: 2.0.0.5338
• Operating System: Debian Stretch + Buster
• .net Framework (Windows) or mono (macOS/Linux) Version: Mono 6.4.0.198

Additional context
Reference: https://github.com/MichaIng/DietPi/issues/3179

[markus101]

Does the same happen with mono 5.20? Best guess it's mono 6's change to disk access, which has most commonly been an issue with SMB shares. Sonarr doesn't do any "authentication" and this is failing within mono based on the stack trace.

[MichaIng]

@markus101
Jep, since at least we never faced this issue in the past (no report from users), I is most likely related to mono v6. It would fit if import to SMB shares has similar issues, since those as well do not support UNIX permissions natively but need to be mounted with wanted modes/ownership, just like exFAT mounts.

Then it seems like, if the mount does not support UNIX permissions (chown/chmod), mono does not respect the modes applied via mount options (and permissions inherited via group membership), but requires to either run as root or as user that owns the mount. This would include exFAT, SMB, FAT(32), NTFS (without ntfs-3g) and such.

I'll do some tests to verify this and test the same with mono v5. Something we could then forward to mono devs. However for Sonarr it is probably not a high priority topic anymore due to Sonarr v3 without mono?

[Taloth]

Sonarr v3 uses mono.

[MichaIng]

@Taloth
Thanks for info. Looks like I mixed things up 😉.

[MichaIng]

@markus101
Okay tested with mono 5.18.0.240 and indeed it worked well then. Switched back to mono 6.4.0.198 and import failed on exactly the same files on exFAT, FAT32 (vfat) and NTFS (without "permissions" flag but umask 000) and import failed.

Not sure if it helps, but when adding mount option "permissions" to the NTFS mount, to have UNIX permissions emulated, it works. So indeed it seems to be related to the ability to change/apply UNIX ownership/permissions, regardless if this is natively supported by the file system or emulated.

---

Shall we forward this to mono developers? I have no deep insights into mono, e.g. which function is used/affected and if this is a bug or by design (security "feature"), but IMO quite important when using any Windows-compatible file system on UNIX systems with mono.

[Taloth]

@MichaIng What we need is an exact reproduction scenario, preferably with a Dockerfile for one or more docker contains that can be used to reproduce it.
It would allow us to check whether it's a sonarr or dotnet core issue (it's not a mono issue per say, since as of 6.x mono ported over the dotnet core filesystem layer code)

[markus101]

Not sure if it helps, but when adding mount option "permissions" to the NTFS mount, to have UNIX permissions emulated, it works. So indeed it seems to be related to the ability to change/apply UNIX ownership/permissions, regardless if this is natively supported by the file system or emulated.

As far as workarounds, is that acceptable?

Not sure if mono was emulating permissions before or it just worked because of what mono was doing under the hood.

[MichaIng]

@Taloth
I sadly have no Docker experience nor too much time to dig into it and set all of this up currently 🤔. I'll see if I can write up a script/shell code that creates and mounts an .img file as loop device, format with mentioned file system and... but no idea how to automate download and Sonarr import directory then.

What I could do quicker is sending you a link to an export of my VirtualBox testing machine where everything is setup and configured, ready to fail.

it's not a mono issue per say, since as of 6.x mono ported over the dotnet core filesystem layer code

Sounds like this could be the root of the issue, but my knowledge ends here 😉.

@markus101
permissions mount option only exists for NTFS with ntfs-3g package (Debian) installed, for FAT variants this option does not exist. So this has nothing to do with Mono and I hope that it indeed is not able to emulate UNIX permissions by itself, on a drive that was mounted by another user. That would be a huge security issue, OS/kernel-wise 😉.

[MichaIng]

Found a related issue on mono: https://github.com/mono/mono/issues/16573

[Taloth]

@MichaIng Interesting, so it might be reproducible with different group permissions and explains why my earlier straight up reproduction attempts failed.
I'll see if I can reproduce it when playing with that, if I can't repro I'll get that virtualbox image.

[Taloth]

Upon inspection, the linked mono issue is about overwrite failing, which I can reproduce but isn't applicable to Sonarr. However, the underlying issue appears to be similar and there are a few issues and PRs about it on the mono github, but none truly address the underlying issue.
So it definitely lead to me understand _why_ this happens.

Can you do one test for me?

Does cp --preserve=mode work or not on exfat? I expect it to fail as well.

[MichaIng]

@Taloth
Okay test done, cp --preserve=mode fails, and indeed also what you said in the other topic totally makes sense and now all the test results start to make sense for me:

• Even if a file mode is 777, only its owner (or root) can change permissions (chown/chmod) reasonably.
• cp --preserve=mode always includes a mode (re)set, but since the file is not owned by sonarr (in our case), this fails with "Operation not permitted".
• Usually, if a file system supports UNIX permissions, when a users copies a file, it is always created with himself as owner of course, hence setting the mode always succeeds. So this is unique to file systems without UNIX permissions or dirs with forced permissions flag. Then, as long as a user has write permissions, the file is copied, but the owner will be the fixed mount/dir owner, hence setting mode fails.

[MichaIng]

PR is up to solve the issue: https://github.com/mono/mono/pull/17870

[vukomir]

hi,

not sure this is also related, but i have the same issue

```[Debug] MediaInfoFormatter: Formatting audio channels using 'AudioChannelPositions', with a value of: '2/0/0'
[Debug] EpisodeFileMovingService: Hardlinking episode file: /downloads/Younger.S06E09.1080p.WEB.x264-TBS[TGx]/younger.s06e09.1080p.web.x264-tbs.mkv to /tv/Younger/Season 6/Younger - S06E09 - Millennial's Next Top Model WEBDL-1080p.mkv
[Debug] DiskTransferService: HardLinkOrCopy [/downloads/Younger.S06E09.1080p.WEB.x264-TBS[TGx]/younger.s06e09.1080p.web.x264-tbs.mkv] > [/tv/Younger/Season 6/Younger - S06E09 - Millennial's Next Top Model WEBDL-1080p.mkv]
[Debug] DiskProvider: Hardlink '/downloads/Younger.S06E09.1080p.WEB.x264-TBS[TGx]/younger.s06e09.1080p.web.x264-tbs.mkv' to '/tv/Younger/Season 6/Younger - S06E09 - Millennial's Next Top Model WEBDL-1080p.mkv' failed.

[v2.0.0.5338] Mono.Unix.UnixIOException: Invalid cross-device link [EXDEV].
at Mono.Unix.UnixMarshal.ThrowExceptionForLastError () [0x00005] in <4a040cc44eb54354b3d289eb2bbc1e23>:0
at Mono.Unix.UnixMarshal.ThrowExceptionForLastErrorIf (System.Int32 retval) [0x00004] in <4a040cc44eb54354b3d289eb2bbc1e23>:0
at Mono.Unix.UnixFileSystemInfo.CreateLink (System.String path) [0x0000c] in <4a040cc44eb54354b3d289eb2bbc1e23>:0
at NzbDrone.Mono.Disk.DiskProvider.TryCreateHardLink (System.String source, System.String destination) [0x00013] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Mono\Disk\DiskProvider.cs:182

[Warn] ImportApprovedEpisodes: Couldn't import episode /downloads/Younger.S06E09.1080p.WEB.x264-TBS[TGx]/younger.s06e09.1080p.web.x264-tbs.mkv

[v2.0.0.5338] System.UnauthorizedAccessException: Access to the path "/downloads/Younger.S06E09.1080p.WEB.x264-TBS[TGx]/younger.s06e09.1080p.web.x264-tbs.mkv" or "/tv/Younger/Season 6/Younger - S06E09 - Millennial's Next Top Model WEBDL-1080p.mkv.partial~" is denied.
at System.IO.File.Copy (System.String sourceFileName, System.String destFileName, System.Boolean overwrite) [0x00192] in <254335e8c4aa42e3923a8ba0d5ce8650>:0
at NzbDrone.Common.Disk.DiskProviderBase.CopyFileInternal (System.String source, System.String destination, System.Boolean overwrite) [0x00000] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Common\Disk\DiskProviderBase.cs:208
at NzbDrone.Mono.Disk.DiskProvider.CopyFileInternal (System.String source, System.String destination, System.Boolean overwrite) [0x00076] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Mono\Disk\DiskProvider.cs:129
at NzbDrone.Common.Disk.DiskProviderBase.CopyFile (System.String source, System.String destination, System.Boolean overwrite) [0x000bc] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Common\Disk\DiskProviderBase.cs:203
at NzbDrone.Common.Disk.DiskTransferService.TryCopyFileTransactional (System.String sourcePath, System.String targetPath, System.Int64 originalSize) [0x00108] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Common\Disk\DiskTransferService.cs:462
at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, NzbDrone.Common.Disk.DiskTransferVerificationMode verificationMode) [0x0034a] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Common\Disk\DiskTransferService.cs:289
at NzbDrone.Common.Disk.DiskTransferService.TransferFile (System.String sourcePath, System.String targetPath, NzbDrone.Common.Disk.TransferMode mode, System.Boolean overwrite, System.Boolean verified) [0x0000e] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Common\Disk\DiskTransferService.cs:196
at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.TransferFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Tv.Series series, System.Collections.Generic.List1[T] episodes, System.String destinationFilePath, NzbDrone.Common.Disk.TransferMode mode) [0x0012c] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Core\MediaFiles\EpisodeFileMovingService.cs:119 at NzbDrone.Core.MediaFiles.EpisodeFileMovingService.CopyEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode) [0x0006b] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Core\MediaFiles\EpisodeFileMovingService.cs:94 at NzbDrone.Core.MediaFiles.UpgradeMediaFileService.UpgradeEpisodeFile (NzbDrone.Core.MediaFiles.EpisodeFile episodeFile, NzbDrone.Core.Parser.Model.LocalEpisode localEpisode, System.Boolean copyOnly) [0x00167] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Core\MediaFiles\UpgradeMediaFileService.cs:72 at NzbDrone.Core.MediaFiles.EpisodeImport.ImportApprovedEpisodes.Import (System.Collections.Generic.List1[T] decisions, System.Boolean newDownload, NzbDrone.Core.Download.DownloadClientItem downloadClientItem, NzbDrone.Core.MediaFiles.EpisodeImport.ImportMode importMode) [0x00281] in M:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Core\MediaFiles\EpisodeImport\ImportApprovedEpisodes.cs:108

[Debug] Api: [GET] /api/queue?sort_by=timeleft&order=asc: 200.OK (2 ms)
[Debug] Api: [GET] /api/queue?sort_by=timeleft&order=asc: 200.OK (3 ms)

i'm using CIFS to mount my share, and using latest docker image for sonarr linuxserver/sonarr

Version
2.0.0.5338
Mono Version
5.20.1.34
AppData directory
/config
Startup directory
/opt/NzbDrone ```

[Taloth]

@vukomir No it's not, this issue only affects mono 6.x. You have a simple permission issue. If you need assistance, ask on IRC or our Discord.

[paddatrapper]

I can confirm this happens with CIFS mounted file systems too. I'm running into this on Debian Buster with mono 6.12.0.90. The exact same configuration worked on Debian Stretch with an older mono version. cp --preserve=mode fails too

[MichaIng]

I tried to track where the solution has gone:

• PR on mono got closed: https://github.com/mono/mono/pull/17870
• I think in favour or PR on .NET core, which got closed as well: https://github.com/dotnet/corefx/pull/42458
• Since the repo was merged with https://github.com/dotnet/runtime: https://github.com/dotnet/announcements/issues/119
• Related PRs there got closed as topic moved to other teams and such: https://github.com/dotnet/runtime/pull/1495
• Finally a fix was merged this August: https://github.com/dotnet/runtime/pull/40753

This means .NET 5 is what we are waiting for, currently RC 2 has been released: https://github.com/dotnet/runtime/releases
Of course then mono need to be updated based on this .NET runtime, and then Sonarr (v2) needs to run on this new .NET core/mono version, not sure if breaking changes have been done?
Probably Sonarr v3 with a workaround included a long time ago will be officially released earlier? 😃

That was a long way. If Microsoft really aims to better integrate/work with Linux, such issues, which exactly affect the way how Windows and Linux can play together, really need to be addressed faster, although I'm no .NET dev, no idea how complex this issue really was, it just looked so trivial 😅.

[bakerboy448]

closing; when sonarr switches to .netcore this will be resolved (and that is a ways off)

[Taloth]

@bakerboy448 The bug was introduced by dotnet core and ported back to mono 6.x. dotnet 3.x will never get that fix.
Sonarr v3 got a workaround to deal with it by handling the failure and falling back to a Copy+Delete operation. In fact, a similar workaround is implemented for radarr v3 as well.

Btw. I suspect it still happens in .net 5, based on that pr 40753, because a failure to copy permissions is considered a failure of the copy operation itself.