Magento2: Excessive number of product images generated (sometimes causing server crashes)

[hewgim]

Preconditions (*)

1. Magento 2.3.4
2. PHP 7.2
3. Server with 6 vCPU, 8GB RAM, PHP memory limit 2GB
4. Blank theme

Steps to reproduce (*)

1. Create a configurable product with 4 colours and 3 sizes
2. Add 7 images to Configurable product and 7 images to each of the simple products
3. Clear out product image cache
4. Go to product view page for the configurable page
5. Choose each colour and size variation in turn
6. Look to see how many images are in the cache directory.

Expected result (*)

1. I would hope that Magento only generates the number of images it needs. By going through each of the variations I would expect the following images to be generated
   Main images: 1 configurable and 12 simple = 13 images
   Gallery images: 7 for configurable + 7 * 12 simple products = 91 images
   Total number of miages I would hope to be generated = 104 or thereabouts

Actual result (*)

In the cache folder there were 1512 images generated and that's for 1 configurable product !!!
On our server if I choose the colors and sizes quickly on the product view page, I can actually max out the cpu and crash the server, presumably because of the overhead of creating so many images.
It seems to me that Magento is creating way way too many images for configurable products, and I can't believe that they are all necessary. I run a small Magento 1 shop which has around 1000 active configurable products with typically 12 color/size variants each with around 7 images. By my maths that means Magento 2 could generate around 1.5 million products images. And thats just active products for a small shop !!!

[m2-assistant[bot]]

Hi @hewgim. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• [ ] Summary of the issue
• [ ] Information on your environment
• [ ] Steps to reproduce
• [ ] Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

@hewgim do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• [ ] yes
• [ ] no

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[hewgim]

@magento give me 2.4-develop instance

[magento-engcom-team]

Magento instance deployment temporary unavailable due to service maintenance.

[gwharton]

It creates all the different sizes of each image as per the definitions in your view.xml. you can vastly reduce this by tweaking your view.xml

[hewgim]

@gwharton I'm guessing it creates all the different sizes of images for all the configurable child product gallery images to come up with such a large number of images. If that is the case (I'll test this shortly when I can get a 2.4 develop instance), I think that is sub optimal. Configurable product child products are supposed to be not visible individually so most of the sized images would never be used. When we were creating a configurable product with 12 varients on Magento 2.3.4, each with 7 images, the server would never complete the job because of the amount of work the cpus had to do to create that vast number of images. Inevitably only some of the child product were saved, and strangely they were often duplicated. In the end we had to saved the configurable product and its children without images and then add the images individually to each product as a workaround. I just don't know if it's a great idea to create every size of image for every uploaded product image up front. It hammers the server and seems a little wasteful if it's creating images that will never be seen. I need to do a bit of testing to see exactly what it going on. Regards Mark

[hewgim]

@magento give me 2.4-develop instance

[magento-engcom-team]

Magento instance deployment temporary unavailable due to service maintenance.

[hewgim]

Okay I see what is going on. In our theme (pretty much the blank theme) there are 21 different sizes of images specified.

If I create a configurable product with 4 colours and 3 sizes and both the configurable and simple products have a set of 6 images, the number of images uploaded is:

(1 configurable + 12 simple) * 6 images = 78 images uploaded

And the number of images Magento tries to create is 78 images uploaded * 21 different sizes = 1638 images.

All these images are created when the configurable product is created and saved and maxes out our CPU and causes the server to crash.

So how many images would I expect Magento to create for a configurable product with 12 simple products and 6 images on each product (main, small_image, thumbnail image all the same image) with a theme with 21 different sized images.

For the configurable product I would expect 21 different sized images for the image that is the main, small and thumbnail image.
For the configurable product I would also expect the other media gallery images to generate a small medium and large image for the product view media gallery, but I wouldn't expect any images to be generated for these images for things like the shopping cart, recently viewed block etc. So that's 5 media images * 3 = 15 images.
For the simple products, (assuming that the simple product images are not displayed in the cart), I would expect a small medium and large image to be generated for the simple products for the product view media gallery. So thats 12 products * 6 images * 3 different sizes = 216

Total number of images that Magento should generate if optimised = 21 + 15 + 216 = 252

Magento actually generates 1638 images, and thats 6.5 times as many as are required.

Magento makes 2 mistakes in my view. It generates all sizes of images specified in view.xml for media gallery images that aren't marked as thumbnail, small_image, or main image when it doesn't need to and it also generates all sizes of images specified in view.xml for configurable child product images when it doesn't need to.

[gwharton]

I cut my view.xml down to 5 images. Covered the main sizes i needed. catalog:images:resize runs much quicker too.

[hewgim]

@gwharton I followed your advice and reduced the number of distinct sizes to 5 also and it certainly cures the problem of crashing the server, so thanks for that tip.

I still think it's wrong for Magento to create all the sized images for the other media gallery images that aren't thumbnail, small_image or main images. These images only really appear on the product view page and not in other places around the site.

[sedax90]

Magento's behavior is absolutely insane. it is unthinkable to recreate all the caches of an image especially when you have extremely large catalogs.

Why not just generate the image cache that serves as Drupal does? What did he think whoever wrote this stuff?

[gwharton]

            <image id="new_products_images_only_widget" type="small_image">
                <width>78</width>
                <height>78</height>
            </image>
            <image id="product_stock_alert_email_product_image" type="small_image">
                <width>76</width>
                <height>76</height>
            </image>
            <image id="sendfriend_small_image" type="small_image">
                <width>75</width>
                <height>75</height>
            </image>

Why reuse when you can define your own images. 👎

[hewgim]

I've changed the title of this issue because it appears on further investigation that there are a number of problems going on here resulting in way way more presized porduct images generated and cached than are ever required.

Problem 1) - Magento generates every resized image defined in a theme's view.xml file for EVERY product image uploaded even when the images uploaded are only required for the media gallery. The view.xml appears to specify what image sizes are required using something like this:

            <image id="cart_cross_sell_products" type="small_image">
                <width>240</width>
                <height>300</height>
            </image>

This would imply that Magento should create a pre-sized image 240x300 from the small_image but Magento actually creates 240x300 presized images for every media gallery image. I'e it creates pre-sized images that would never be required by the theme.

Problem 2) - Magento creates tons of pre-sized images for all simple product children of a configurable product even though these products are generally "Not visible individually" and these presized images would not generally be used.

Problem 3) - The default blank and luma theme arguably creates more presized images than it really needs. For every image uploaded Magento creates 21 presized images if the luma theme is being used and many of these images are of very similar size. @gwharton mentioned that he could cut this down to 5 images.

The net affect of these issues can be demonstrated as follows.

Imagine using the luma theme and creating a configurable product with 1 image as the main, small_image and thumbnail and 5 other gallery images. The configurable product has 12 children (4 colors and 3 sizes say) and each child has one image representing the main,small_image and thumbnail and 5 gallery images.

Magento would generate the following presized images:
For the configurable: 6 images x 21 presizes = 126 images
For the children: 12 children x 6 images x 21 presizes = 1512 images
Total: A WHOPPING 1638 IMAGES for just one product line

So how many images would I expect Magento 2 to create if the problems above were addressed and the view.xml file was optimised to create 5 presizes ?

For the configurable: 1 image (the main, small_image and thumbnail) x 5 presizes and 5 gallery images x 3 presizes for the media gallery = 20 images
For the children: 12 children * 6 images * 3 presizes for the media gallery = 216 images
Total: 236 images

If you went a step further and decided not to create presized images for the media gallery, the total number of presized images created would be
For the configurable: 1 image (the main, small_image and thumbnail) x 5 presizes
For the children: none
Total: 5 images

So for this particular scenario, of configurable products and simple product children all with media gallery images it may be possible to reduce to number of presized images generated by over 300 times !! For 1000 product lines that's 5,000 images instead of 1,638,000 images

[craig-bartlett]

It's also worth noting that in multi-store environments it will do this per store too, whether the product is enabled in that store or not.