Sdwebimage: NSAllocateMemoryPages Failed crash

[attheodo]

New Issue Checklist

• [X] I have read and understood the CONTRIBUTING guide
• [X] I have read the Documentation
• [X] I have searched for a similar issue in the project and found none

Issue Info

Info | Value |
-------------------------|-------------------------------------|
Platform Name | iOS
Platform Version | 11.3.1
SDWebImage Version | 4.4.0
Integration Method | Cocoapods
Xcode Version | XCode 9
Repro rate | Rare
Repro with our demo prj | No
Demo project link | Working on that

Issue Description and Steps

Fatal Exception: NSInvalidArgumentException
0  CoreFoundation                 0x1822d2d8c __exceptionPreprocess
1  libobjc.A.dylib                0x18148c5ec objc_exception_throw
2  CoreFoundation                 0x1822d2c6c -[NSException initWithCoder:]
3  Foundation                     0x182c8e218 NSAllocateMemoryPages
4  Foundation                     0x182c79004 _NSDataCreateVMDispatchData
5  Foundation                     0x182c115ec -[_NSPlaceholderData initWithBytes:length:copy:deallocator:]
6  SDWebImage                     0x1057e2c2c -[SDWebImageDownloaderOperation URLSession:task:didCompleteWithError:] (SDWebImageDownloaderOperation.m:407)
7  SDWebImage                     0x1057e0370 -[SDWebImageDownloader URLSession:task:didCompleteWithError:] (SDWebImageDownloader.m:380)
8  CFNetwork                      0x182a27760 __51-[NSURLSession delegate_task:didCompleteWithError:]_block_invoke.207
9  Foundation                     0x182cfae88 __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__
10 Foundation                     0x182c3c8d0 -[NSBlockOperation main]
11 Foundation                     0x182c3bcac -[__NSOperationInternal _start:]
12 libdispatch.dylib              0x181bc4ae4 _dispatch_client_callout
13 libdispatch.dylib              0x181bcc1f4 _dispatch_block_invoke_direct$VARIANT$mp
14 libdispatch.dylib              0x181bc4ae4 _dispatch_client_callout
15 libdispatch.dylib              0x181bcc1f4 _dispatch_block_invoke_direct$VARIANT$mp
16 libdispatch.dylib              0x181bcc0e0 dispatch_block_perform$VARIANT$mp
17 Foundation                     0x182cfc750 __NSOQSchedule_f
18 libdispatch.dylib              0x181bc4ae4 _dispatch_client_callout
19 libdispatch.dylib              0x181bccf18 _dispatch_continuation_pop$VARIANT$mp
20 libdispatch.dylib              0x181bcb850 _dispatch_async_redirect_invoke$VARIANT$mp
21 libdispatch.dylib              0x181bd1d30 _dispatch_root_queue_drain
22 libdispatch.dylib              0x181bd1a80 _dispatch_worker_thread3
23 libsystem_pthread.dylib        0x181ef7fac _pthread_wqthread
24 libsystem_pthread.dylib        0x181ef7b08 start_wqthread

Crashes with Fatal Exception: NSInvalidArgumentException*** NSAllocateMemoryPages(133801) failed Raw Text
 

[dreampiggy]

Your application is run out of memory. The image data is a NSMutableData, and it should be called copy before decoding to make sure thread-safe. However, if the image raw data is really huge and your application current is run out of memory. This will cause memory allocation failed. Seems no good solution for this.

[dreampiggy]

Maybe one choice, we can map the in-memory image data to the disk files during download. So it does not triggert a memory peak (where NSMutableData as well as copied one exist at the same time). After all the image data is downloaded, we load the disk data into memory again. Then using the image data to feed the decoder.

This can be considered using a thread-hold for image data size. For example, if the image data is larger than 5MB, we store it to disk instead of using NSMutableData in memory.

But anyway, this need some works to do it. Keep it here and I'll find a better solution later

[huanghe810229530]

This situation has led to my APP hundreds of times crash。 crying

[bpoplauschi]

@attheodo @huanghe810229530 we need to see what types and sizes of images do you deal with. There are some things we can do to handle large images (that cause those OOM issues) - see similar OOM issues.

[huanghe810229530]

I change the option from SDWebImageDownloaderProgressiveDownload to SDWebImageDownloaderLowPriority, so that this bug hasn't appeared.
When I use option SDWebImageDownloaderProgressiveDownload, the picture of the 1 MB or more may cause crash, I don't know why. @bpoplauschi

[dreampiggy]

@huanghe810229530 Maybe related to progerssive decoding and been fixed by #2475

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is still an issue, please make sure it is up to date and if so, add a comment that this is still an issue to keep it open. Thank you for your contributions.

[bpoplauschi]

Closing per lack of response. Feel free to reopen/continue the conversation if still applicable.

[dreampiggy]

More same issue like #2620 .

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is still an issue, please make sure it is up to date and if so, add a comment that this is still an issue to keep it open. Thank you for your contributions.

[wanliming11]

Same crash on 4.4.3. Would it be fixed on 4.4.7？

[dreampiggy]

@wanliming11 There are one MR maybe related to this #2624 . You can upgrade to have a try. This MR is available in v4.4.6

[wanliming11]

@dreampiggy 👌，try it， thanks！

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is still an issue, please make sure it is up to date and if so, add a comment that this is still an issue to keep it open. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is still an issue, please make sure it is up to date and if so, add a comment that this is still an issue to keep it open. Thank you for your contributions.

[kylebrowning]

Im seeing this today in my crash logs as well.

[Hesse-Huang]

Same crash, same issue in my app.
Users said it crashes with Wi-Fi, but NOT IN LTE. @dreampiggy

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is still an issue, please make sure it is up to date and if so, add a comment that this is still an issue to keep it open. Thank you for your contributions.

[rahulvyas]

Got the same issue 6 days ago

[majid701]

Got the same issue here as well. The user had 245mb free RAM too.

[dreampiggy]

Got the same issue here as well. The user had 245mb free RAM too.

https://developer.apple.com/documentation/foundation/1416210-nsallocatememorypages?language=objc

It's documented behavior that when no enough memory left, the allocate will raise an exception.

Currently, since SDWebImage have to supports progressive image rendering, it have to keep all the data buffer into RAM. Not using NSURLSessionDownloadTask which write the data to file, so this may be in risk when your Web URL have large data size.

[cai3804007]

I ran into this problem in a very early version, and then I added a piece of code that I hadn't run into for a long time, but recently. How to change it.
It's been modified before：
[SDImageCacheConfig defaultCacheConfig].maxMemoryCost = 300 * 1024 * 1024 ; // 300M

[atistank]

Same issue for me. within 7 days

[Lobster-King]

git the same issue .

[ZhangDihan]

Same issue for me.