Sdwebimage: Memory problem or leak on iOS 7 ?

[kefon]

Hello everybody,
I'm sorry in advance as maybe this is a dumb and noob question...

I'm using SDWebImage to display pictures in a UITableView in my cellForRowAtIndexPath method, using the classic

[cell.pointPicture setImageWithURL:[NSURL URLWithString:thePoint.imageURL] placeholderImage:[UIImage imageNamed:POINT_DEFAULT_IMAGE]];

(the displayed pictures are light and well compressed jpgs, just some ko)
When I inspect my app with "Instrument - Allocations", and just scroll down my UITableView (with 40 cells containing picture, a bit like Instagram), I got a huge amount of memory used ! (see screenshot)

But it seems to be "VM", and especially "VM: CG raster data" from the coreGraphics library.

So the questions are :

• Is it normal?
• Is that a serious problem?
• Is there a way to avoid this?

I'm sorry but after few search on the web I can't find any relevant information concerning the "VM: CG raster data"... Any idea? Thanks in advance !

[ivanbruel]

I am indeed having the same problem on an iOS7-only app. Images don't seem to be recycled. Seems to be related with [SDWebImageDownloaderOperation connectionDidFinishLoading:] and the ForceDecode of the UIImage.

[kefon]

@ivanbruel nice to know that I'm not the only one ;) I still looking to understand the problem, but unsuccessfully for now...

[kefon]

I also have the same issue while running the SDWebImage demo project (on iOS7 / iPhone 5)

[rs]

SDWebImage cache images is using NSCache. It is discardable memory. See Apple documentation: https://developer.apple.com/library/ios/documentation/cocoa/reference/NSCache_Class/Reference/Reference.html

[kefon]

@rs Ok ! So if I understand, this is a perfectly normal behavior and the memory is released if needed. Thank you for your answer, now I think I understand what's going on. Sorry for the noob question ! (So I can close this issue right?)

[ivanbruel]

Well, from my particular issue, I'm URLing into the documents folder, the memory keeps growing no matter how many passes I make through the list, it will most of the time fetch them from disk instead of memory. So does this mean the discardable memory is being discarded even while it grows?

[cloay]

I have this issue only on iOS7. How to avoid this?

[mrtristan]

i see this too, are we thinking this isn't an issue? @rs are you saying the memory will grow and grow until it actually needs to be reclaimed?

[rs]

Yes, it's caching

[willclarke]

I am seeing a lot of crashes as the CG raster data grows - it doesn't look like it's actually discarding memory when it needs to be. I see memory warnings start to get generated, it kills background apps, and then the app crashes without ever discarding any of the CG raster data.

[phionardo]

I have exactly the same issue, too. @rs Is there a way to clear cache timely, or reduce using memory for caching?

In my case, the "All Anonymous VM" grows to 700MB and my app crashes, due to the huge amount of "CG raster data".

[rs]

We could play with the totalCostLimit of NSCache to implement a maximum used memory barrier.

[willclarke]

I added this:

// Init the memory cache
_memCache = [[NSCache alloc] init];
[_memCache setTotalCostLimit:41943040];
_memCache.name = fullNamespace;

But I'm still getting this as I scroll through a UITableView with lots of images:

Note the CG Raster Data is still blowing up and it's killing other apps but never reducing the raster data.

Is there something I should be doing to release the images in the table cells when the cells are dequeued and reused? Sorry if that's a stupid question.

[rs]

I guess you have to set the cost of each entry at insertion time otherwise it can't work.

[willclarke]

Looks like right now it's setting a cost for each entry of the image height * width * scale as an approximation for image size... I've changed this to:

    NSData *png_representation = UIImagePNGRepresentation(image);
    [self.memCache setObject:image forKey:key cost:png_representation.length];

In the three places where setObject happens - but memory usage is still exploding. I have it capped at 80 MB but CG Raster Data continues to increase unbounded... I suspect this could be the way CoreGraphics is storing rendered images in the view and has nothing to do with the cache that SDWebImage is keeping. But I just have no idea how to tell CG how to free this raster data when it's not on screen.

[rs]

A lot of apps are using SDWebImage caching a lot of images with no memory issue. You may have better chances searching for a leak in the rest of your app yes :)

[willclarke]

Definitely possible. I'm not seeing any memory actually leak in Instruments, but I'll keep trying to track it down. Thanks for your responses @rs !

[kaishin]

@rs Also reported in #548. This is clearly a leak that has to do with GIFs. Removing GIFs or using AFNetworking which doesn't support GIFs 'fixes' the problem.

[wujie0919]

I also encountered such a problem

[andrewprocter]

I'm encountering this as well. I believe @kaishin is onto something re: GIF's.

[pradeepfission]

I'm encountering ImageIO_PNG_Data memory problem. I'm seeing following in instruments

CGRaster Data and ImageIO_PNG_Data are taking a lot of memory.

[jxdwinter]

:+1:
Maybe some issues in iOS7 with this GIF stuff. I build a simple project in both iOS6 and iOS7. Check this out.Same code.

iOS6

iOS7

[myell0w]

seeing the same problem and lots of crashes related to SDWebImage. @rs: I think that's an issue worth looking into and isn't related to the affected apps.

[daveluong]

We're having the same memory problem with SDWebImage too. Seem to happen only on iOS7 with CGRaster Data keep growing to hundred of MBs just by swiping the table view, to the point that the app will get memory warning and be terminated.

I personally believe this got something to do with iOS than the library itself.

[jxdwinter]

Hi,guys, just use UIWebView to show a gif right now.

[donholly]

How is that related to this topic?

[daveluong]

Seem like this is a bug in iOS Simulator only. I've just tested on 7.0.3 device and saw a huge drop in memory usage. Looks like the system is claiming back the memory when it's needed.

[kaishin]

@huyduc Are you testing with GIFs? You don't seem to be.

[KetchupMoose]

I also have this issue, but I am seeing the same problem on a device running ios 6.1 but the ios SDK 7.0.

I am using GIFs.

[Pei116]

I'm facing the same issue. It gets memory warning and finally crashes due to memory pressure when I set image with url.

[tscheepers]

Anymore thoughts where this build up could come from? I have a lot of memory related crashes on actual devices. I am clearing the cache om memory warnings [SDImageCache.sharedImageCache clearMemory];. And have set a cache limit: SDWebImageManager.sharedManager.imageCache.maxMemoryCost = 3000 * 3000; but it does not seem to be working.

[AcidicSkittles]

I was having memory problems too. I ended up just calling
[[SDImageCache sharedImageCache] setValue:nil forKey:@"memCache"];
periodically and that seemed to work. I found this answer here:
https://github.com/rs/SDWebImage/issues/391

[tscheepers]

My problem seemed to originate from something else than the SDImageCache but since it did contribute to the memory buildup I asked my question here.

Perhaps other people with the same problem could try what I tried. Keep in mind that I was using ARC.

• Run the profiler with the Leaks template and check for the allocation of your own classes in the Allocation Summary.
• Dive into them and check how they are allocated, if there are leaks. Mind that a leak doesn't show up in the Leaks instrument because you are using ARC. So Instruments could think everything is going oké but there still could be a leak somewhere. By diving in to the allocation of your own classes you could figure out what is going wrong.
• Keep in mind that the retain/release count information is only provided when using the Leaks template and not while using the Allocations template.

My problem was that I was referencing instance variables and self directly from within blocks without reassigning them to __weak variables. When self is used within a block it will automatically be retained by ARC and sometimes never released. A weak reference prevents that from happening.

For example, this is _wrong_:

[[NSNotificationCenter defaultCenter] addObserverForName:UIKeyboardDidShowNotification object:nil queue:nil usingBlock:^(NSNotification *note) {
    [self.view setContentOffset:CGPointMake(0.0, kKeyboardOffset) animated:YES];
}];

You should call self using a __weak reference like this:

__weak YourViewControllerClass *weakSelf = self;
[[NSNotificationCenter defaultCenter] addObserverForName:UIKeyboardDidShowNotification object:nil queue:nil usingBlock:^(NSNotification *note) {
    [weakSelf.view setContentOffset:CGPointMake(0.0, kKeyboardOffset) animated:YES];
}];

Since my app uses a lot of block's I had a ton of leaks the Leaks instrument could not detect. When I fixed them the memory problem was gone.

I hope this helps.

[pradeepfission]

@rs Is this behaviour normal?

I am using setimagewithurl method to display images(url).

What can i do, to free the cgrasterdata which is growing enormously and crashes after 10 min usage of app.

And i am using 3.2 version on sdwebimage.

I observed this behaviour starting iOS 7 and still continuing in iOS 7.1 too.

[peterjenkins]

+1. I'm seeing the exact same behavior as everyone else.

[kaishin]

After further investigation, I found out that this problem has less to do with SDWebImage and more to do with UIKit. Calling the method UIImage animatedImageWithImages:duration: is what causes the memory hike. Any GIF implementation using this method is affected.

My thinking is that Apple could have done a far better job for animating images. Currently, it seems like all the frames (images) are preloaded in memory, which is not a bright idea.

That said, there are some clever GIF implementations out there that use CADisplayLink and partial frame loading/deallocation to keep the memory footprint low. See https://github.com/liyong03/YLGIFImage for more details.

[marinh]

I tried implementing YLGIFImage instead of current GIF implementation in SDWebImage. It doesn't crash at all now and GIFs load even faster. It is really noticeable on lower end devices (iPhone 4 or older).

[andrewprocter]

@marinh that's awesome, any way we can get our hands on your implementation?

[voronianski]

@marinh @andrewprocter +1

struggling from this issue as well..

[voronianski]

@kaishin @kefon @marinh @andrewprocter

that's the screen of memory usage in my app.. just two GIF images were loaded -

is there a workaround?

or should I try another lib like ImageView+AFNetworking??

[eightytwo]

I use the following library for loading gifs:
https://github.com/mayoff/uiimage-from-animated-gif

It is available as a cocoapod here:
http://cocoapods.org/?q=UIImage%2BanimatedGif

[voronianski]

@eightytwo actually I don't need to load gifs separately, my app is kind of feed and there are images loaded from different resources, they can be gifs, png, jpg and so on.. I'm looking for the way to disable this veeeery huge memory leak on SDWebImage

[kaishin]

@voronianski Unfortunately GIF support is baked into SDWebImage. So you either patch it, or use a separate library (see my comment above).

I personally went for the more time-consuming approach of separating concerns (networking, caching, animating) using 3 separate libraries. This beats the purpose of SDWebImage (takes care of everything for you), but it's a decision you should make based on how central images are in your app. In my case, the entire app is about displaying images, so it just makes sense that I have total control over the flow.

[voronianski]

@kaishin I'm just wondering is it possible to replace UIImage+GIF with some solution like YLGIFImage (btw it's not ideal too, was crashing for me when using image from url - see https://github.com/liyong03/YLGIFImage/issues/5)..

[kaishin]

@voronianski No, you will have to create a UIImageView subclass if you want to use the YLGIFImage approach. A category on UIImage simply won't work. See my comment on the issue you mentioned above for a little bit more details.

[ahui2823]

I didn't use GIF in my app, but it seems the same problem, When I scroll the tableView with image in cell, the VM(CG raster data) live bytes growing huge, and then I take photo, app crashes. Does anyone has a good way to solve this problem?

[pradeepfission]

+1 ahui2823 i too facing same problem

[zkarcher]

Similar issue here, after upgrading my project (and iPad) from iOS 6 to 7.

When my application launches, it loads 50 images (PNG format) using this code:

SDWebImagePrefetcher * prefetcher = [SDWebImagePrefetcher sharedImagePrefetcher];
prefetcher.options = SDWebImageRetryFailed | SDWebImageRefreshCached | SDWebImageLowPriority;
prefetcher.delegate = self;
prefetcher.maxConcurrentDownloads = 1;
[prefetcher prefetchURLs:urls];

Note the 313.47 MB of "CG raster data" which is never released:

Possible fix: Insert this line:
return image;
at the beginning of SDWebImageDecoder's +(UIImage *)decodedImageWithImage:(UIImage*)image method, so that images are not redrawn. After inserting this line and clearing my cache, the "CG raster data" leaks are gone:

What is the purpose of the decodedImageWithImage method?

[ws513]

+1

[algrid]

facing the same problem (I don't use gifs)

[Serlight]

encounter the same issue

[macelangelo]

+1

[briansunter]

+1

[DDany]

In my app(iOS7 only), SDWebImage works very well. But GIF supporting is also on the schedule now. After a few hours investigation, I decide to use YLGIFImage for GIF function. SDWebImage is really not good at GIF.

[holetse]

I am also seeing this issue.

@rs I understand that these images are in a NSCache and that NSCache SHOULD release objects as memory pressure increase, however the background decoded images are never freed. I can push the app to the point of being terminated due to memory pressure and it never yields substantive memory back. If I disable the background decode by returning immediately, I still see memory pressure increase but at a reasonable point it starts to release memory back to the OS and the app will run indefinitely.

In fact, if there are any GIFs in the cache (which aren't decoded) they are released as memory pressure increases but the actual decoded images remain.

This leaked memory persists even if I manually clear the cache using [SDImageCache clearMemory].

[futuretap]

Did you file a radar for this?

Ortwin

Am 03.07.2014 um 02:42 schrieb Edward Holets [email protected]:

I am also seeing this issue.

@rs I understand that these images are in a NSCache and that NSCache SHOULD release objects as memory pressure increase, however the background decoded images are never freed. I can push the app to the point of being terminated due to memory pressure and it never yields substantive memory back. If I disable the background decode by returning immediately, I still see memory pressure increase but at a reasonable point it starts to release memory back to the OS and the app will run indefinitely.

In fact, if there are any GIFs in the cache (which aren't decoded) they are released as memory pressure increases but the actual decoded images remain.

This leaked memory persists even if I manually clear the cache using [SDImageCache clearMemory].

\
Reply to this email directly or view it on GitHub.

[holetse]

@futuretap I have not. I have yet to have the time to conclusively investigate if the bug is related to NSCache and images containing CG Raster Data, or if its related to SDWebImage's usage of NSCache.

[welsonla]

I solved this by add clearMemory code when didReceiveMemoryWarning,and when the controller disappear call the clear code too: [[SDImageCache sharedImageCache] clearMemory];

- (void)didReceiveMemoryWarning
{
    [super didReceiveMemoryWarning];

    //clear memory cache
    [[SDImageCache sharedImageCache] clearMemory];
}

[albsala]

+1 I have the same problem.

[kidGuo]

I have the same problem. app crash because high virtual memory.

[zkarcher]

My app uses SDWebImagePrefetcher to preload images. (The app is an interactive installation, and preloads several gigabytes of image data.)

When I uninstall the app, then reinstall & launch it, the app runs OK. It does not crash.

But when I launch the app _when it's already installed_, then some images are already cached on disk. ... And in this case, memory consumption grows quickly during the prefetching. I don't know why. Is it retrieving images from the disk cache, decompressing them, and not releasing them? Very strange.

My solution: don't prefetch images with the same URLs as cached images.

// URLs to preload:
NSMutableArray * urls = [NSMutableArray array]; // of NSURLs

SDWebImageManager * manager = [SDWebImageManager sharedManager];
for( NSURL * url in [page imagePreloaderURLs] ) {
    if( [manager diskImageExistsForURL:url] ) {
        //NSLog(@"SKIPPING: %@", url.absoluteString);
        continue;
    }
    [urls addObject:url];
}

if( [urls count] > 0 ) {
    SDWebImagePrefetcher * prefetcher = [SDWebImagePrefetcher sharedImagePrefetcher];
    prefetcher.options = (SDWebImageRetryFailed | SDWebImageRefreshCached);
    prefetcher.delegate = self;
    prefetcher.maxConcurrentDownloads = 1;  // ??? Not sure what's best here
    [prefetcher prefetchURLs:urls];
} else {
    // All images already in disk cache.
}

This has fixed my memory pressure crashes.

Unfortunately, if any of the image assets change, they won't be reloaded. The outdated image (in the disk cache) will be used instead.

Hope this helps someone. I'm using version 3.7.1.

[zzontheway]

+1, i have this problem too，how to solve this bug，my app crash frequently

[harishkashyap]

Its the memory issue again. decodedImageWithImage takes up huge memory and causes the app to crash. I have added an option to put this off in the library but defaulting to YES so there aren't any breaking changes. If you put off the decodeImageWithImage method in both image cache and image downloader then you shouldn't be seeing the VM: CG Raster data on the top consuming lots of memory

decodeImageWithImage is supposed to decompress images and cache them so the loading on tableviews/collectionviews become better. However, with large set of images being loaded, the experience worsened and the memory of uncompressed images even with thumbnails can consume GBs of memory. Putting this off only improved performance.

[[SDImageCache sharedImageCache] setShouldDecompressImages:NO];
[[SDWebImageDownloader sharedDownloader] setShouldDecompressImages:NO];

https://github.com/harishkashyap/SDWebImage/tree/fix-memory-issues

[foundry]

@harishkashyap - I am experiencing the staircase memory pattern using SDWebImage with jpeg images. I was encouraged to see your patch appear _in realtime_ as i am wrestling with this right now. Unfortunately it does not fix the issue.

• Your patch applied with shouldDecompress set to YES:

• Your patch applied with shouldDecompress set to NO:

• Here is a run _without_ SDWebImage - using [NSData dataWithContentsOfURL:url] for crude comprarison.

This seems to confirm that SDWebImage is the cause of the issue. Perhaps I am not using it correctly.

Here is the offending code:

- (void)downloadImage:(NSURL*)url withCompletion:(Block)completion {
    @weakify(self)
    self.imageDownloadOperation =
    [[SDWebImageManager sharedManager] downloadImageWithURL:url
                                                options:0
                                               progress:nil
                                              completed:^(UIImage *image
                                                          , NSError *error
                                                          , SDImageCacheType cacheType
                                                          , BOOL finished
                                                          , NSURL *imageURL) {
                                                  @strongify(self)
                                                  self.imageURL = url;
                                                  self.rotationZoomView.image = image;
                                                  if (completion) completion();
                                              }];
}

And the comparison version using dataWithContentsOfURL:

- (void)downloadImage:(NSURL*)url withCompletion:(Block)completion {
    @backgroundqueue(^{
    NSData* data = [NSData dataWithContentsOfURL:url];
    UIImage* image = [UIImage imageWithData:data];
    if (image) {
        @mainqueue(^{
            self.imageURL = url;
            self.rotationZoomView.image = image;
            if (completion) completion();
        })

    }
})
 }

[harishkashyap]

@foundry not sure how you have configured it. You seem to have leaks there; there shouldn't be leaks with/without the shouldDecompress. That seems like the code may be leaking somewhere else.

[foundry]

@harishkashyap - you are correct. I don't think I was testing with your patched version. Here are better result. Your fix _has_ done the trick, many thanks for this timely intervention!

Your patch applied with shouldDecompress set to YES (crashes):

CG Raster data -~ 200MB+

Your patch applied with shouldDecompress set to NO (does not crash):

CG Raster data - ~ 3MB

[harishkashyap]

@foundry : thanks for letting me know!

[delannoyk]

Had the same issue in my app (which use a lot of large JPEG images).

Thanks to @harishkashyap for the patch, works well!

[rs]

@harishkashyap could you please provide a PR?

[harishkashyap]

@rs I have put in a PR for this: https://github.com/rs/SDWebImage/pull/996. Note that this also needs the configuration to be set:
[SDImageCache sharedImageCache].shouldDecompressImages = NO;
[SDWebImageDownloader sharedDownloader].shouldDecompressImages = NO;

I would favor defaulting to these options in the library but that is your decision. I added these as options externally so not to have breaking changes.

[donholly]

I'm a little confused...

The entire point of the decompression step when serving images via SDWebImage is to avoid that expense when the image is drawn to screen (and to keep it off of the main thread).

You can read more about this here (the TLDR is that decompressing JPEGs is expensive):
http://www.cocoanetics.com/2011/10/avoiding-image-decompression-sickness/

What you're likely going to see when you disable this functionality is that images will take longer to appear (and may cause main thread hiccups) when you're drawing them (this will be more noticeable in a UICollectionView with many images).

Per my understanding, this won't actually prevent CGRasterData accumulation (since this same process will occur when you draw the images to screen). At best, you're just delaying the decompression costs to the point of drawing - which has its merits, but doesn't solve the underlying memory issue in the long run. The only reason I could think you might not want to decompress the images on a background thread upon retrieving them from the cache is if you're not planning on drawing them immediately (pre-fetching for later use).

UIImage has its own memory management techniques and should purge unused image data when it is no longer needed (or when memory warnings occur). SDWebImage also uses an NSCache to store in-memory images, which will also purge its contents in the event of a memory warning.

I do _not_ think image decompression should be off by default. This is one of the key benefits of using SDWebImage in the first place (decompressing JPEGs in a background thread is noticeably more performant than doing so JIT on the main thread).

I would also argue that having two separate places for setting that option (both in the cache and image downloader) is poor design and not very intuitive. Perhaps moving this option to the SDWebImageManager (which can then set it in both places on the user's behalf) is a better way to do this?

[harishkashyap]

It does prevent CGRasterData accumulation as the amount of memory persisted during infinite scrolling only increases and infact increases tremendously to a point that causes the crash. @foundry has demonstrated it and that is exactly what we found as well. Also we didn't see any stutter while testing; on the contrary it remarkably improves. Also patch allows choosing the queue and it was using the main queue to prefetch that was adding a bad performance to the already existing high memory due to decompression. The idea of having different options is to allow granular level of choice that was consciously done not a poor design. This reduced memory by almost as much as 2/3rd. The earlier implementation didn't perhaps consider inifinite scrolling with large images and that was a gross error.

[rromanchuk]

@donholly i can objectively say i have found no performance degradation turning off decompression using @harishkashyap patch. Not using the patch causes my app to crash in under one minute as the application reaches it's memory pressure limits. You can reproduce this very easily, just use larger images. UIImage does _not_ clean up its CGRasterData

To be honest, the article you posted may be completely invalid in today's market. I have ran many different UICollectionViews with decompression off and have noticed zero difference. The current state of SDWebImage using decompression with images any larger than 100x100 is going to cause ios system shutdowns as RasterData balloons to device limitations.

I honestly don't think you will see any performance degradation or FPS frame drops using instruments when turning off decompression. Without this patch we need to put a huge warning that SDWebImage will only not crash for users using small thumbnails (but will still leak unless the client listens for memory warnings and manually wipes sdwebimage)

[donholly]

Okay - perhaps I'm just not seeing this b/c of the size of the images I'm loading. What size images do you guys see these problems with? And you don't notice _any_ difference in drawing time when you turn decompression off? Even with older devices (iPhone 4/4S)?

You may be right that the article I referenced is out of date. If so, I may actually want to use this patch and save CPU cycles for decompression.

[harishkashyap]

I would be happy to move the configs to as sdwebimagemanager and default to not decompressing if everyone concurs.

[delannoyk]

@donholly I had the memory issue causing crashes on iPad 2. I didn't notice any performance issue applying @harishkashyap 's patch as an user of the app.

Images I'm loading are quite large (some of them are bigger than 1000 x 1000).

[donholly]

I didn't mean to criticize the design, sorry :)

What was the reason for separating them? Why would you want them to be
decompressed from the cache but not when downloaded (or vice versa)?

On Wed, Feb 25, 2015 at 11:19 AM, Harish Krishnamurthy <
[email protected]> wrote:

I would be happy to move the configs to as sdwebimagemanager and default
to not decompressing if everyone concurs.

—
Reply to this email directly or view it on GitHub
https://github.com/rs/SDWebImage/issues/538#issuecomment-76034597.

[donholly]

Got it... is the consensus that background image decompression is a thing
of the past now? Perhaps ImageIO/CoreGraphics are just better/smarter about
this now?

On Wed, Feb 25, 2015 at 11:20 AM, Kevin Delannoy [email protected]
wrote:

@donholly https://github.com/donholly I had the memory issue causing
crashes on iPad 2. I didn't notice any performance issue applying
@harishkashyap https://github.com/harishkashyap 's patch as an user of
the app.

Images I'm loading are quite large (some of them are bigger than 1000 x
1000).

—
Reply to this email directly or view it on GitHub
https://github.com/rs/SDWebImage/issues/538#issuecomment-76034888.

[harishkashyap]

The other way around, to have compressed in cache and decompressed in rendering just to control cache. But I agree that is minor and will be happy to move it to the webmanager

[donholly]

Got it. Sounds good!

[holetse]

@donholly The big issue here seems to be that NSCache has a memory leak when it's used to cache CGRasterData. All these solutions are just ways of working around that central bug.

[harishkashyap]

I think decompression methods may have a leak; not sure if it is NSCache. There are alternatives to + (UIImage *)decodedImageWithImage:(UIImage *)image on iOS 7+. Someone can see if the new decompression methods resolves that. However, infinite scrolling with large images + caching on decompressed images (no matter what method you use) can consume huge memory at any given time that can affect performance.

[donholly]

@harishkashyap - What alternative methods are you referring to?

[harishkashyap]

http://www.objc.io/issue-5/iOS7-hidden-gems-and-workarounds.html has decompression methods that I haven't had time to investigate as an alternative method to what is currently used.

[donholly]

Ah yes, I do remember coming across that awhile back now.. I didn't investigate further due to what @steipete noted:

I was very excited when I first found out about this, but you really shouldn’t be. In my tests, performance actually decreased when I enabled immediate caching. Either this method calls up to the main thread (unlikely) or perceived performance is simply worse because it locks in copyImageBlockSetJPEG, which is also used when showing a non-decrypted image on the main thread.

[harishkashyap]

Thanks @donholly! That was the only thing I was going to look at.

[mallorypaine]

NSCache is definitely the problem. I added a PR #1130 which eliminates the memory spikes by allowing the in-memory cache to be disabled.

[rs]

would implementing the mem cache without nscache fix the issue in you opinion?

[benbonnet]

@rs sorry to ask as I'm not that experienced in using sdwebimage;
but what's not obvious here to me, is wether there's a fix to apply to sdwebimage to play well with newer iOS version, or of if this whole discussion is just a long list of people using bad patterns, leading sdwebimage to misbehave ?

[rs]

My understanding is that most people here see a leak where it is just cached images. Memory caching of images used to be a big deal to get 60fps on old hardware. It's certainly less the case today, and maybe disk only caching can be enough for some use cases.

[harishkashyap]

@rs that's right. This reminds me. The default should be changed to put off decompression so there is no patch that needs to be applied.

[benbonnet]

@harishkashyap thing is with this tons of posts of different flavors here; sorry if I bother but I'm lost…

What would be the simplest config to apply to get this to work properly, in order to apply your "put off decompression" statement ?

[rs]

@harishkashyap can you send a pull request? @mallorypaine if you disable decompression, does it fix your issue without disabling memory caching?

[foundry]

@bbnnt: this is how to disable decompression -
change:
_shouldDecompressImages = YES
to:
_shouldDecompressImages = NO
in:
SDImageCache.m
SDWebImageDownloader.m
SDWebImageDownloaderOperation.m

This assumes use of current SDWebImage which has @harishkashyap fixes included. In my case, this fixes the issue. Without this fix, ballooning memory that resulted in crashes every time (context is infinite scrolling tableview/collectionview cells). See my post further up the page

[mallorypaine]

@rs Yes, implementing an in-memory cache with a cost feature that is actually never exceeded would also fix this issue. For my use case, I _do_ want my images forcibly decoded on a background thread. That way, I don't get scrolling stutters when the images are ultimately displayed in a UIImage View.

If I turn off decompression (which I need), then the spikes appear to go away. However, the decompression is still unrelated to this issue. As I'm sure you know, a UIImage's memory consumption balloons the first time that image is actually drawn as that's when decompression + bitmap buffer allocation takes place. If I turn off decompression, then the UIImages that are in the NSCache are just lightweight pointers referencing file data, so the spike does appear to be fixed. However, as each UIImage is actually displayed via a UIImageView, it will be decompressed and its memory consumption will balloon significantly, and the NSCache will still be holding onto these larger images. If you actually displayed enough UIImages that were referenced by the NSCache, then the memory spike would be the same as if forced decompression were enabled.

[benbonnet]

@foundry thx a lot for pointing this out ! I'll have a further look
@mallorypaine you're saying there is no real solution to all this, hence this issue (debate) will never gets closed ?

[mallorypaine]

@bbnnt there is definitely a solution to keeping unwanted memory from piling up. However, if you want to cache every image in memory then there your app's memory consumption will reflect the caching strategy. As I mentioned, I'm able to get precisely the behavior I need by disabling the NSCache. IMHO, NSCache is basically never worth using since its behavior is buggy and imprecise. It's also fairly trivial to build your own cache object that tracks cost of each item and then evicts based on LRU when the total cache cost exceeds what the app has configured. Maybe if I can find a spare hour or two I'll whip that up as it would be a true solve for this issue.

[mythodeia]

@mallorypaine so according to your pull request you turn NSCache off.
Do you disable decompression as well in your apps?

[benbonnet]

@mallorypaine so it might not be really an sdwebimage problem; but more or less a memory management strategy if i get it well ? I'm not expert enough; an app with lot of images developed without this experience/knowledge would at some point always crash then ?

[holetse]

This is fundamentally an NSCache issue as I understand it. Implementing a
custom cache a la @mallorypaine would address the issue. So would Apple
fixing NSCache, but that seems to not be happening.
On Mon, Apr 27, 2015 at 12:41 PM bbnnt [email protected] wrote:

@mallorypaine https://github.com/mallorypaine so it might not be really
an sdwebimage problem; but more or less a memory management strategy if i
get it well ? I'm not expert enough; an app with lot of images developed
without this experience/knowledge would at some point always crash then ?

—
Reply to this email directly or view it on GitHub
https://github.com/rs/SDWebImage/issues/538#issuecomment-96792685.

[rromanchuk]

I noticed SDWebImage is not using NSCache countLimit, any reason? This is probably the worst documentation i've read in a while so it could be completely worthless, and i'm not even sure what the default is? @mallorypaine do you have any insight to this setting?

https://developer.apple.com/library/mac/documentation/Cocoa/Reference/NSCache_Class/index.html#//apple_ref/occ/instp/NSCache/countLimit

[rromanchuk]

Actually,

The default value is 0, which means no limit.

Could this be causing - (void)setMaxMemoryCost:(NSUInteger)maxMemoryCost to be completely ignored? I understand it's a black box, but maybe we can help coach it a bit more?

[rromanchuk]

I'm going to do a few tests with instruments with these changes, and i'll report back

https://github.com/rs/SDWebImage/compare/master...rromanchuk:feature/add-count-limits

[trungp]

I disable decompression by changing:
_shouldDecompressImages = YES
to:
_shouldDecompressImages = NO
in:
SDImageCache.m
SDWebImageDownloader.m
SDWebImageDownloaderOperation.m

But it looks like the memory is still so high. After scrolling the table view down to some large images, the app take up huge memory.

Can any one can help me to explain what happened here and write down some points about this issue ?

[burakkilic]

I have exactly the same issue although I made the same changes. I have some jpg images (20) in a gridview, but 80kb for each. But app crashes after one or two scroll.

[mythodeia]

@burakkilic you get a crash with just 80kb images on your 2nd scroll? thats weird.
i have a tableview as well with 300x250 imageview and get no crashes whatsoever while scrolling.
are you sure its memory pressure thats causing your crashes?

[trungp]

@burakkilic : In your case, its weird. My app just lagging when I'm scrolling, it still happens on iPhone 5s but get no crashes. I don't know why the memory never get down or I miss somethings here.

[burakkilic]

Here is the profiler screenshot. Images are like:

I don't know what happens

[mythodeia]

@burakkilic how large are your images in resolution?

[burakkilic]

1536*1024 and 300 Pixel per Inch

[mythodeia]

oh come on...thats huge. Thats why it crashes. Even if your size is 80kb your images are large in resolution. You should also experience laggy scroll. Generally speaking you do not add large res images as thumbs in a tableview/collection view. You add smaller ones. Do you actually need 300 ppi for thumb images?

[burakkilic]

Actually no. My client uploaded them. What is the optimum resolution?

Adil Burak Kılıç

On Wed, May 13, 2015 at 11:13 AM, Konstantinos K. [email protected]
wrote:

oh come on...thats huge. Thats why it crashes. Even if your size is 80kb
your images are large in resolution. You should also experience laggy
scroll. Generally speaking you do not add large res images as thumbs in a
tableview/collection view. You add smaller ones. Do you actually need 300
ppi for thumb images?

—
Reply to this email directly or view it on GitHub
https://github.com/rs/SDWebImage/issues/538#issuecomment-101565312.

[mythodeia]

In my apps i use 300x300 max as thumbs while scrolling. You could use lower if you want.
And you can load the large image when you tap the thumb.
There is no need to make them square like i said but you get my point

[benbonnet]

@burakkilic "300 Pixel per Inch" is default for print, not digital

[burakkilic]

Thank you. I am downgrading to 72

Adil Burak Kılıç

On Wed, May 13, 2015 at 1:11 PM, bbnnt [email protected] wrote:

@burakkilic https://github.com/burakkilic "300 Pixel per Inch" is
default for print, not digital

—
Reply to this email directly or view it on GitHub
https://github.com/rs/SDWebImage/issues/538#issuecomment-101609330.

[rromanchuk]

Guys, the images you are posting do not show any memory issues. If your apps are lagging it's most likely because you are doing silly things on the main thread. @trunglee you say huge memory growth but what exactly is your _persisted_ memory after scrolling?

[trungp]

I have a table view with a list of image urls. When the table scrolling down, it's show image on every row. That's all. For some first pages (about 20 rows per page), the table view is still smooth in an acceptable way. But from the page 3, the table view is going to lagging. I just keep a list of urls as text string for table view metadata and use SDWebImage to load image for imageView on table view cell.

[pajapro]

It seams like a long discussing around this particular bug, but is anyone still experiencing a huge growth in memory allocation when loading high resolution images? I experience this issue primary on iPhone 6. I tried the suggested solution (from fork https://github.com/harishkashyap/SDWebImage/tree/fix-memory-issues) to turn off
[[SDImageCache sharedImageCache] setShouldDecompressImages:NO];
[[SDWebImageDownloader sharedDownloader] setShouldDecompressImages:NO];
but that does not solve the memory growth. For other low resolution images I see no significant growth

[bpoplauschi]

@pprochazka72 are you using static images with high res or animated ones (GIF or other format)?

[pajapro]

@bpoplauschi static images with high res.

[CavalcanteLeo]

+1

[weng1250]

[[SDImageCache sharedImageCache] setShouldDecompressImages:NO];
[[SDWebImageDownloader sharedDownloader] setShouldDecompressImages:NO]; is good for me to avoid huge memory cost when it is processing GIF format pics.

[bpoplauschi]

We have replaced our GIF component with FLAnimatedImage. You can try our 4.x branch (instructions).

[songshp39]

image = [[UIImage alloc] initWithData:data];
This init method will bring in a memory spike when the image data is larger than 1M.

[bpoplauschi]

Closing. The FLAnimatedImage solution replacing our own GIF is integrated into the 4.x branch.

[bpoplauschi]

@mythodeia @rs it seems that 3 different people created PR's with the same idea of fixing some of the memory issues we have: scaling down the large images: #787 #884 #894. Do you think it is a good idea? Which one do you prefer? If we merge any of them, I would go with a SDWebImageScaleDownLargeImage option so people can go around it if they really need to.

[mythodeia]

@bpoplauschi #884 and #894 are almost the same.From these two i prefer #884.
Now for #787 i would like to see some benchmarks and the code needs some cleanup.
However i like the idea of making it optional like you said

[bpoplauschi]

@mythodeia I agree. Now since I saw that #787 was the most solid PR and it also included the Apple sample for downscaling, I considered including it is best. I merged it via 959d965 (of course it had conflicts and I updated the code). Then I added another commit 00bf467 cleaning stuff up in the Decoder class. Take a look and let me know what you think. We still need to bechmark it.

Everyone, #787 and 00bf467 should give you a good solution for dealing with large images. Together with the GIF handled by FLAnimatedImage, this ticket should be done now.