Stacks.js: putFile crashes Safari on iOS when uploading many encrypted files.

[nerdic-coder]

I'm using the putFile to upload encrypted images to Gaia storage.
It works great on most platforms, but on iOS Safari if I try to upload a list of more than 3 images the browser often crashes and reloads the page. It works fine if I turn off encryption in the putFile method.
I analyst the memory usage and it spikes up to about 1 GB memory usage before crashing and these images is only around 2-MB in size.

Any ideas what the issue could be?

I'm using blockstack.js version 18.1.0 and tested on an iPhone 8, iOS 12.
The issue can be recreated by uploading images from an iPhone here: app.block-photos.com

The code I use to upload the images: https://github.com/nerdic-coder/block-photos/blob/master/src/services/PictureService.js#L60

[larrysalibra]

@nerdic-coder I was able to upload 6 images at app.block-photos.com without encountering the problem.

I'm using iPhone XS max with iOS 12.1.

Can you share console logs for your browser when the problem happens?

[nerdic-coder]

Can’t share the log since the browser crashes and the inspector as well.
Did you upload all 6 pictures at the same time?

I made a video with the issue here: https://youtube.com/watch?v=h3ObE25Bm7E&feature=youtu.be

[zone117x]

This issue looks pretty bad. It looks like without encryption we use the appropriate APIs for upload-streaming content directly from disk (like an image) to the hub.
With encryption it looks like we make several in-memory copies of the content. At a cursory overview it looks like around 4 (😬!) copies. Probably more depending on implementation of the crypto APIs we pass the data through). Some of these copies are string/b64 encoding the binary data into a massive JSON object upload.

Potential steps to fix:

• Use a crypto lib that will stream encrypt the data during upload. Need to investigate if the http upload APIs like fetch can do this. Otherwise we'd need to chunk it ourselves or something.
• Use one of the json-streaming libs that exist for this purpose. Or use a binary format for encrypted data - although, json is nice for other devs/implementations not having to deal with parsing some binary format (but then they would run into the same sizing problems too).
• If we like keeping the json format I'd propose only using it for string content-types, and using binary for binary content-types.

@kantai @yknl @hstove @larrysalibra Thoughts? If I have the bandwidth I'd be interested in taking this on.

[muneebm]

@zone117x Any updates on this issue? I am using v19.1.0 and noticing that the encrypted data size is always ~4 times the actual size. Is there any way to work around this issue until we have a fix?

[larrysalibra]

During the public weekly engineering two weeks ago, @zone117x indicated he’d made some progress on investigating more efficient encryption implementations. Will let him update us as to the exact status.

[markmhx]

@zone117x should we assign this issue to you and move it into "In Progress" if you're investigating?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed. Please reopen if needed.