Flutter_file_picker: OutOfMemory crash when picking a video

[Extazx2]

Describe the bug
When opening a video of 271 MB, the app crashes.
When I open a video of only 108 MB, it's working fine.

Issue details
1-2. Android 10 on OnePlus 6

1. File type (filter) — FileType.video
2. Detailed steps to reproduce
   Try to upload a 4k 60fps video. Mine is 18 seconds long, 271 MB

Here is my picker code :

_pickVideo() async {
    FilePickerResult result = await FilePicker.platform.pickFiles(type: FileType.video);
    print("RESULT");
    print(result);
    if (result != null) {
      File file = File(result.files.single.path);
      final defaultThumbNail = await _getDefaultThumbnailForVideo(result.files.single.path);

      setState(() {
        _videoFile = file;
        _thumbnailFile = defaultThumbNail;
      });
    }
  }

Error Log
Please, post the _full_ console log of your issue, if applicable.

I/ideopublicatio(21902): Background concurrent copying GC freed 388774(13MB) AllocSpace objects, 3(2560KB) LOS objects, 49% free, 10MB/20MB, paused 34us total 101.641ms
D/FilePickerDelegate(21902): Selected type video/*
D/FilePickerDelegate(21902): [SingleFilePick] File URI:content://com.android.providers.media.documents/document/video%3A90149
I/FilePickerUtils(21902): Caching from URI: content://com.android.providers.media.documents/document/video%3A90149
D/DecorView(21902): onWindowFocusChangedFromViewRoot hasFocus: true, DecorView@35dada[MainActivity]
W/libEGL  (21902): EGLNativeWindowType 0x7e2911a010 disconnect failed
I/flutter (21902): INFO: 2020-09-16 17:41:00.138012: tag Repository: finished inserting data
I/ideopublicatio(21902): Starting a blocking GC Alloc
I/chatty  (21902): uid=10536(com.ouestfrance.videopublication) Thread-3 identical 1 line
I/ideopublicatio(21902): Starting a blocking GC Alloc
I/ideopublicatio(21902): Alloc concurrent copying GC freed 6(16KB) AllocSpace objects, 1(64MB) LOS objects, 15% free, 129MB/153MB, paused 26us total 12.944ms
I/ideopublicatio(21902): Forcing collection of SoftReferences for 256MB allocation
I/ideopublicatio(21902): Starting a blocking GC Alloc
I/ideopublicatio(21902): Alloc concurrent copying GC freed 181(33KB) AllocSpace objects, 0(0B) LOS objects, 15% free, 129MB/153MB, paused 27us total 5.911ms
W/ideopublicatio(21902): Throwing OutOfMemoryError "Failed to allocate a 268435468 byte allocation with 25165824 free bytes and 126MB until OOM, target footprint 160871832, growth limit 268435456" (VmSize 5970424 kB)
I/ideopublicatio(21902): Starting a blocking GC Alloc
I/chatty  (21902): uid=10536(com.ouestfrance.videopublication) Thread-3 identical 1 line
I/ideopublicatio(21902): Starting a blocking GC Alloc
I/ideopublicatio(21902): Alloc concurrent copying GC freed 3(16KB) AllocSpace objects, 0(0B) LOS objects, 15% free, 129MB/153MB, paused 24us total 6.169ms
I/ideopublicatio(21902): Forcing collection of SoftReferences for 256MB allocation
I/ideopublicatio(21902): Starting a blocking GC Alloc
I/ideopublicatio(21902): Alloc concurrent copying GC freed 4(16KB) AllocSpace objects, 0(0B) LOS objects, 15% free, 129MB/153MB, paused 64us total 5.817ms
W/ideopublicatio(21902): Throwing OutOfMemoryError "Failed to allocate a 268435472 byte allocation with 25165824 free bytes and 126MB until OOM, target footprint 160888224, growth limit 268435456" (VmSize 5970424 kB)
E/AndroidRuntime(21902): FATAL EXCEPTION: Thread-3
E/AndroidRuntime(21902): Process: com.ouestfrance.videopublication, PID: 21902
E/AndroidRuntime(21902): java.lang.OutOfMemoryError: Failed to allocate a 268435472 byte allocation with 25165824 free bytes and 126MB until OOM, target footprint 160888224, growth limit 268435456
E/AndroidRuntime(21902):    at java.util.Arrays.copyOf(Arrays.java:3161)
E/AndroidRuntime(21902):    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
E/AndroidRuntime(21902):    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
E/AndroidRuntime(21902):    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
E/AndroidRuntime(21902):    at com.mr.flutter.plugin.filepicker.FileUtils.openFileStream(FileUtils.java:156)
E/AndroidRuntime(21902):    at com.mr.flutter.plugin.filepicker.FilePickerDelegate$2.run(FilePickerDelegate.java:122)
E/AndroidRuntime(21902):    at java.lang.Thread.run(Thread.java:919)
I/Process (21902): Sending signal. PID: 21902 SIG: 9

Screenshots and/or video
If applicable, add screenshots or video to help explain your problem.

Flutter Version details

[✓] Flutter (Channel stable, 1.20.0, on Linux, locale fr_FR.UTF-8)
    • Flutter version 1.20.0 at /home/matouin/Android/flutter
    • Framework revision 840c9205b3 (il y a 6 semaines), 2020-08-04 20:55:12 -0700
    • Engine revision c8e3b94853
    • Dart version 2.9.0

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
    • Android SDK at /home/matouin/Android/Sdk
    • Platform android-29, build-tools 29.0.3
    • Java binary at: /snap/android-studio/91/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Android Studio (version 4.0)
    • Android Studio at /snap/android-studio/91/android-studio
    • Flutter plugin version 49.0.2
    • Dart plugin version 193.7547
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

[✓] Connected device (1 available)
    • ONEPLUS A6003 (mobile) • a0e57af8 • android-arm64 • Android 10 (API 29)

• No issues found!
Process finished with exit code 0

[Extazx2]

When using Image_picker, I can upload the file in question and it's working fine, but as I need your fix on ".png" extension for video, and it is not yet merged in the image picker plugin, I switched to this plugin instead.

[miguelpruivo]

@Extazx2 this Will likely to happen only with few devices when not enough memory (ram) is available for buffer writing. I’ll probably update the buffer to handle these scenarios, but right now I’m out on vacations, I can only look into it next week.

Sorry for any inconvenience.

[Extazx2]

@miguelpruivo thanks for the answer, I'll put that on hold.

I can work with my other files for the moment, so it isn't "blocking".

I'll be able to help if needed.

[miguelpruivo]

Thank you for your patience @Extazx2. I’ll keep you posted.

[atrope]

It is also happening with us when trying to upload big files.

[miguelpruivo]

@atrope that’s because if you set withData flag to true, it will preload all picked files into memory. So you must use it with caution unless you’re certain that user’s phone will be able to handle the load.

[atrope]

@miguelpruivo I've tried putting withData:false and have a good phone (S10) and even with that when trying to select the file it throws the out of memory.

E/AndroidRuntime(15532): java.lang.OutOfMemoryError: Failed to allocate a 268435472 byte allocation with 25165824 free bytes and 121MB until OOM, target footprint 166567976, growth limit 268435456
E/AndroidRuntime(15532):    at java.util.Arrays.copyOf(Arrays.java:3161)
E/AndroidRuntime(15532):    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
E/AndroidRuntime(15532):    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
E/AndroidRuntime(15532):    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
E/AndroidRuntime(15532):    at com.mr.flutter.plugin.filepicker.FileUtils.openFileStream(FileUtils.java:156)
E/AndroidRuntime(15532):    at com.mr.flutter.plugin.filepicker.FilePickerDelegate$2.run(FilePickerDelegate.java:122)
E/AndroidRuntime(15532):    at java.lang.Thread.run(Thread.java:919)

Is there a way to only return the paths without opening it?

[miguelpruivo]

@atrope using withData: false doesn't loads it into memory, however, the file is cached. This is, if you are picking it from a remote provider (google drive, cloud, etc) it needs to be locally saved before you can use it, even from local storage, since scoped storage was introduced (Android 10) it needs to be cached into your apps content.

You are trying to load a 200MB+ file, I might need to tweak the sampling size when caching files, hence this issue.

If you want an immediate solution, try adding android:largeHeap="true" under your manifest Application tag.

[atrope]

@miguelpruivo even with largeheap it is giving me the crash.

Also tested on Android 9 (without the scoped storage) and it crahed also. Do we need to open the file? Because for example, in my case i just need the path chosen and then I create a File reference in Flutter. Shouldn't this be happening and not needng to open the file?

[miguelpruivo]

@atrope you can't have a path without first caching it. If you want that behavior, you can use any of <2.0.0 versions of the plugin. However, that's not how Android team wants you to access files. There are 2 recommended ways to use files on Android, specially since scoped storage:

• Use the URI natively to CRUD operations (which is not helpful for Flutter, as you need an absolute path to use File instances);
• Open a file stream for the given URI and clone it into your app's scope for your own usage — the approach used with the plugin since 2.0.0 and that allows you to have an absolute path that can also be forced deleted if you want for some reason (use clearTemporaryFiles() method).

I'll try tweaking the sampling size when caching the file to make it easier for bigger files.

[atrope]

We def need to allow users to upload big files so for now i will go with your suggestion of downgrading.

When you think you have something tweaked i can test i will leave here subscribed :)

Thanks @miguelpruivo

[miguelpruivo]

Fixed with 2.0.3.
Feel free to reopen if you still encounter something related to this issue.

Thank you!

[atrope]

@miguelpruivo I can confirm that the solution worked :)
Only one question the cache dir from the app is cleaned by the OS or we should clean when we no longer need the file?

[miguelpruivo]

@atrope you shouldn’t have to worry with it, but if you know beforehand that your users will pick big files for upload for example, you are free to call clearTemporaryFiles(); afterwards. It’s really up to you.