Flutter_file_picker: iOS dumps picked files after 60 seconds because of the tmp dir usage

[DebugTheCode]

Hi there,

I came across an unpleasant "feature" or "bug" on the iOS side of the app. Because the picked files are stored in the tmp directory of the phone, the OS doesn't keep these files cached even while using the app.

The StackOverflow link below is explaining this:
https://stackoverflow.com/questions/33767256/how-long-do-files-stay-in-the-documents-directory-ios

If I understand it correctly, instead of using the tmp folder, use the cache folder in the Library.

Issue details

1. Platform iOS
2. Platform 13.5.1
3. Detailed steps to reproduce: Pick any file with the FilePicker and wait 60 seconds, then try to do something with the File object -> it'll say it doesn't exists.

Obviously, I do not experience this on any Android device.

[miguelpruivo]

Hi @DebugTheCode, as it is, if you need persistent storage, you may want to make a File copy for your app's application folder. I understand your use case, but can you imagine a lot of other use cases where users only pick files to send it to a server, for example, and after a while your app has a huge size because it wasn't purged automatically by the OS?

Basically, as it is, you can play around it by just creating a copy after the file being picked whereas the other way, it will be much trickier to play around it and devs would have to explicitly call removeTemporaryFiles().

Let me know what you think.

[DebugTheCode]

@miguelpruivo Thanks for your quick response.

I'm also picking files to send them to a server, but not within 60 seconds after selecting them. I understand what you're saying with the trickiness of letting a dev explicitly call removeTemporaryFiles() but in a lot of use cases like these it would be great to have control over it by yourself.

The thing I don't understand is how Android handles this different and therefor a lot better.

[miguelpruivo]

Yes, typically files will be purged by OS whenever it needs more space but usually never during app execution. That's the oddly iOS side.

I need to dig it a bit more around this to understand what would be best way to please every use case.

[DebugTheCode]

Exactly. Let's stay in touch on this problem. If I come up with an idea, I'll let you know.

Also a side note: on the physical iOS device I still have 30GB left on storage, so it's not full.

[DebugTheCode]

As I've read here: https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW4

The OS would not remove the files from the memory while the app is running. Any idea what it could be that this is causing?

[miguelpruivo]

@DebugTheCode ah, so it seems to be as I suspected. No, actually I don't. Are you removing the files somewhere else? May you want to try with a sample app.

[DebugTheCode]

@miguelpruivo I've tested it again, and it seems that probably the flutter_ffmpeg package is causing this, but I'm still not 100% sure.

[miguelpruivo]

Ok, may be it, as it could potentially have a clean up routine for compressed files where it actually completely wipes tmp directory (it shouldn't anyways, if so, I suggest you to fill an issue on its project repo).

[DebugTheCode]

@miguelpruivo I've made a workaround. I came across some interesting things. I'll try to explain it to you.

In my app, picking files with the iOS Photos app won't issue the problem, only the Files app. I think it's because picking any file with the Files app changes the AppLifecycleState to .inactive and .paused while the Photos app isn't doing this (because it's a modal probably?).
Or could it be something else? The Files app is temporarily storing the picked files in the tmp folder, but is the Photos app also doing this?

[miguelpruivo]

@DebugTheCode you can easily see where they are both being cached by printing its file paths. However, both photos and files app are being displayed in a modal view so they should behave the same as a matter of fact.

[DebugTheCode]

@miguelpruivo Of course. Well, I've tested it multiple times, and it doesn't. Using the Files app the following AppLifecycleState sequence appears: (opening the app) inactive, paused, (picked a file) inactive, resumed. Using the Photos app, no change in the AppLifecycleState appears.

[miguelpruivo]

What about the picked files folder? Where are pictures from photos being stored?

[DebugTheCode]

@miguelpruivo some on the iPhone itself and some on iCloud. It's also happening on a Simulator.

[miguelpruivo]

No, I mean, after being picked, print the paths so you can be sure whether they are being stored on tmp folder or not.

[DebugTheCode]

@miguelpruivo

Files: /private/var/mobile/Containers/Data/Application/CB817C75-CAC1-4255-B2D2-4DD5FEFCE95D/tmp/[package_name]-Inbox/IMG_0669-146 2.jpg

Photos: /private/var/mobile/Containers/Data/Application/CB817C75-CAC1-4255-B2D2-4DD5FEFCE95D/tmp/DKImageAssetExporter/QTAxODkzNkEtMzFBOS00MTBGLUI0RkUtQUJGREQxNDIyQTFEL0wwLzAwMQ==/615711848.9446771/0/d5642c98-d928-454c-8702-787e86572376.jpg

[miguelpruivo]

Both are being kept on tmp folder so it seems (like I suspected).

[DebugTheCode]

@miguelpruivo Yup. But they're stored in different folders in the tmp folder. Maybe there's something going wrong? Otherwise, why would the OS delete the files from the package name directory, and not the DKImageAssetExporter folder.

[miguelpruivo]

Do you have replicable steps for this issue? Or it will just happen randomly?

[DebugTheCode]

@miguelpruivo Just pick files with the Files method of the package. After around 60 seconds you'll see the tmp doesn't contain the picked files anymore. I've not experienced anything random.

[miguelpruivo]

Does waiting 60 seconds with the app open without any interaction will be enough to reproduce it? Does that happen on real device, simulator or both?

[DebugTheCode]

@miguelpruivo Yes, without any interaction the file will be disposed out of the tmp folder. Both physical device and simulator.

[miguelpruivo]

@DebugTheCode I'll try to debug it as soon as I can.

[miguelpruivo]

@DebugTheCode I've spent some hours around this and I must admit that I don't feel like moving to caches is a solution here as it will make the files live for a longer period that tmp and as every developer knows, if we want to persist data, we should take care of it and move to app's content folder (or even cache folder, through getTemporaryDirectory() of path_provider which will actually give you access to caches dir — this can actually be a solution for you, after picking, moving/copying it to caches directory.

However, what surprises me and where I took more time around it, is why it actually removes the files from tmp after a minute or so without having any other action. I'm just using my example app and I can confirm that after picking a file, it will stay on tmp but after some time it's gone without anything else happening.

According to docs, the files should only be purged when the app isn't running or at most, on its startup.

[DebugTheCode]

@miguelpruivo Thank you for your effort. As you're saying, the solution you are suggesting is working fine. Already made that workaround.

I'm still wondering why it removes that package name directory in the tmp folder, and not the Photos picked files...

[miguelpruivo]

@DebugTheCode yes me too! I couldn’t find any relevant information regarding this online, it’s intriguing.

[yy1300326388]

@DebugTheCode
@miguelpruivo

First of all, thank you both for your hard work, I also encountered this problem. I am trying the solution provided.