React-native-fetch-blob: RNFetchBlob Memory issues when uploading large files

[Asleepace]

Environment

• RNFetchBlob (0.10.6)

• React Native (0.45.1)

• iOS (10.3.3)

Description

Hi, recently I have been using RNFetchBlob for my files uploads, and it has been working great. Unfortunately when I try uploading a file (such as a large video) around 500MB+ it will cause the application (iOS) to climb in memory until the system kills the app.

I have tried this both on the simulator and physical devices with no luck. I have looked at other issues which suggested adding the 'Chunked' transfer encoding and removing the progress percentage, but neither seem to have any effect on memory usage.

From what I've seen so far the request hasn't even begun to send by the time the system crashes from a memory warning. (i.e. the percentage uploaded never shows).

Also as a side not I've noticed that once the memory climbs for smaller files, it doesn't seem to be released after the request has finished.

Any suggestions or help would be greatly appreciated, and I'll post a sample of the code I'm using below, thanks!

multiPartPost(data, fileSize, callback)
{
  Request.get("/uploads/start",(result, error) => {

    var headers = this.headers();
    headers['Content-Type'] = 'multipart/form-data';
    headers['Transfer-Encoding'] = 'Chunked';

    const { fields } = result;

    var b1 = { name:'uploader_id', data:fields.uploader_id };
    var b2 = { name:'app_id', data:fields.app_id };
    var b3 = { name:'transcode', data:fields.transcode };
    var b4 = { name:'max_file_size', data:fields.max_file_size };
    var b5 = { name:'key', data:fields.key + '/' + data.name };
    var b6 = { name:'file_size', data:fileSize };

    const uri = this.removeFilePrefix(data.uri);

    var body = {
      name:data.name,
      filename:data.name,
      data:RNFetchBlob.wrap(uri),
    };

    RNFetchBlob.config({ })
    .fetch('POST', result.url, headers, [b1, b2, b3, b4, b5, b6, body])
    //.uploadProgress({interval:250},(written, total) => {console.log('uploaded', written / total)})
    .then((response) => response.json())
    .then((response) => callback(response))
    .catch((err) => callback(err))
  });
}

Also here is an image of the memory usage for a file around 100MB, the first spike is from reading the file size, but as you can see after the request is done a lot of the memory isn't being released either. The slope between the spikes seems to be from building the request, and it is also where larger files tend to fail for me.

[Asleepace]

I solved this issue with file streams, the solution is in depth but if anyone wants more info just message me and I'll send you the details.

[lll000111]

You should consider adding your solution to the Wiki - it is editable by everyone:

https://github.com/wkh237/react-native-fetch-blob/wiki

[Asleepace]

Maybe I will, but right now it's actually a separate solution from RNFetchBlob and only works on iOS. If I get some time in the near future I will try and format it so it matches the same API, but as of right now it's completely different.

[Shannanigans]

On android you would have to use a fore-ground service. https://developer.android.com/guide/components/services.html

We ended up using https://github.com/Vydia/react-native-background-upload to handle large file uploads without the os killing the app.

[tstrand]

Anyone else still experiencing this? I'm getting an NSMalloc Exception on large file uploads. It says "Failed to grow Buffer". It's unhandled too so causing my app to crash

[radreamer]

@tstrand i also experiencing this

[AugustoAleGon]

@Asleepace how did you handle it?