Magento2: X-Magento-Tags header too large

[katef]

Preconditions

I've been reading about people using Magento2 and the size of the X-Magento-Tags header.

Steps to reproduce

For example http://www.maxbucknell.com/blog/2016/2/10/troubleshooting-varnish-with-magento-2 writes:

The rub here is that Magento adds a tag for every product in a category. And the tag is of the form "catalog_product_{{PRODUCT_ID}}".

Actual result

That's an integer and 20 bytes of string per product, which gives some users headers several kB in size, when many products are in one category. In that case, the author describes hitting Varnish's limit at 400 products.

Expected result

I work for a CDN where some of our users use Magento, and we cache their content with Varnish. Increasing Varnish's http_resp_hdr_len size may suit that one person's situation, but it is infeasible for us, because it would affect all of our users.

Proposed solution

I have a suggestion to offer, for how to use header space more efficiently:

Each product ID is an integer. The X-Magento-Tags header describes the presence or absence of products only. That information can be done with a single bit per product. The index of the bit would give the product ID.

That's usually known as a bit vector, or a bitfield, or a bitmap. My suggestion is to switch the header to use a bitmap, encoded in hex. That'd take one bit per product, instead of over 20 _bytes_.

More on bitmaps: https://en.wikipedia.org/wiki/Bit_array

[stamster]

Excellent point!
But I'm afraid such change is not going to happen anytime soon, maybe for Magento 3.0... and this is only one of the topics out there to discuss when it comes to optimization.

[katef]

Thank you - Although I'd request that you consider this a blocking item, rather than an optimisation, because this header is so large that it prevents certain users from using Varnish.

[melnikovi]

@katef thank you for reporting this issue, internal ticket created MAGETWO-57851

[stamster]

@katef : I'm not representative of Magento Commerce Inc. ... :) I completely agree with you, of course it is blocking for certain use cases out there.

[pynej]

Some more detail:
The catalog_category_product_ and catalog_category_ could easily be changed to 2 characters that would still be unique and cut down the length of this header by ~80%.

Of critical issue though is that in Magento 2.1 all versions of a configured product are incorrectly added to this header field. In 2.0 only the parent item was added. As a result the size of the field grows exponentially and makes varnish unusable. I am trying to find the code in question that is incorrectly adding all versions so I can comment it out and restore 2.0 functionality and and have an actual working site.

[pynej]

This workaround fixes but wastes lots or memory.

[pynej]

I did this workaround to get my site working again. Basically disabling the addition of the child products of configurable items to the tag. There are cons't defined that specify the catalog_product_ tag, so that should be pretty straightforward to shorten.

--- vendor/magento/module-configurable-product/Plugin/Model/Product.php.orig    2016-09-07 14:10:22.201010000 -0400
+++ vendor/magento/module-configurable-product/Plugin/Model/Product.php 2016-09-07 14:08:17.107250000 -0400
@@ -26,9 +26,9 @@
         /** @var Configurable $productType */
         $productType = $product->getTypeInstance();
         if ($productType instanceof Configurable) {
-            foreach ($productType->getChildrenIds($product->getId())[0] as $productId) {
-                $result[] = \Magento\Catalog\Model\Product::CACHE_TAG . '_' . $productId;
-            }
+            #foreach ($productType->getChildrenIds($product->getId())[0] as $productId) {
+            #    $result[] = \Magento\Catalog\Model\Product::CACHE_TAG . '_' . $productId;
+            #}
         }
         return $result;
     }

[pboisvert]

@choukalos can you ensure this item gets the magetwo ticket? pulled the improvement label

[choukalos]

Tracking internally - MAGETWO-58265 ; Thanks @pynej for diagnosing and the code diff.

[veloraven]

Internal ticket for 2.1 - MAGETWO-58362

[choukalos]

Closing; this was resolved in 2.1.3

[katef]

@choukalos What was the resolution?

[djfordz]

I'm still having issue with this in version 2.1.3

Error 503 Backend fetch failed

With Category Pages that have tons of categories.

[dcabrejas]

@choukalos I think this ticket should be reopen, I have encountered this issue on a Magento site running on 2.1.5. Category pages containing a large number of products return aX-Magento-Tag header which is bigger than the size allowed by the HTTP specification.

We have implemented this workaround for now http://devdocs.magento.com/guides/v2.0/config-guide/varnish/tshoot-varnish-503.html

But this is not a solution, just a workaround because the http header size is still larger than the limit set by the HTTp specification.

[redelschaap]

Any update on this? This is still present in 2.1.7.

[tedsalmon]

As previously stated, this is still an issue in 2.1.8. I have implemented a Plugin around Magento\Catalog\Model\Product like so:

<?php

namespace YourCompany\YourModule\Plugin;

class Product{

    protected $_request;

    public function __construct(\Magento\Framework\App\Request\Http $request){
       $this->_request = $request;
    }

    /*
     * Return no simple product tags when returning category pages
     */
    public function afterGetIdentities(\Magento\Catalog\Model\Product $subject, $tags){
        if(!$this->_request){
            return $tags;
        }
        $r = $this->_request;
        $fullRoute = sprintf(
            '%s_%s_%s', $r->getRouteName(), $r->getControllerName(), $r->getActionName()
        );
        if($fullRoute === 'catalog_category_view'){
            return [];
        }else{
            return $tags;
        }
    }
}

?>

I very much realize that this is a hack, but we simply have too many products (over 64kB of tags) for the tag shortening to work. This will still show any parent product tags, FWIW.

[satishgadhave]

Can you pls provide link to pull request?

[mpchadwick]

This also impacts Apache + PHP-FPM (https://maxchadwick.xyz/blog/http-response-header-size-limit-with-mod-proxy-fcgi). I see this was marked as done in 2.3-develop. Is there a link to a PR that fixes this?

[sambolek]

@mpchadwick

Believe this is the commit https://github.com/magento/magento2/commit/2529ec4fc035eec8c524e94c0b73efd8c3efbb28 , but it's just another workaround.

It only delays the error, which will happen anyways, if you have a category with large amount of products.

Edit: found another commit that affects tags, but it's related only to purge requests https://github.com/magento/magento2/commit/e1f8de3a9936987b83d052aa6228930978ab2981

[quickshiftin]

May I ask what is the purpose of the X-Magento-Tags header? I'm troubleshooting 'Varnish' issues on an M2 site with a large number of categories. The X-Magento-Tags header is absolutely massive!

Varnish obviously is not the culprit here, it's Magento. Why on earth does this header need to be so large? Could the data not be embedded on the page? I'm trying to download pages that are resulting in 503's through Varnish with tried-and-true CLI HTTP clients - guess what the results are when using

wget: HTTP request sent, awaiting response... Read error (Cannot allocate memory) in headers.
curl: curl: (27) Avoided giant realloc for header (max is 102400)!

I looked for options to expand memory allotment for headers in these two CLI clients, could not find them. It seems there are several pieces of software out there indicating this is a gross abuse of HTTP headers!

[sambolek]

Here's a potential solution which splits X-Magento-Tags header into multiple short ones that might help your issue https://github.com/magento/magento2/pull/12831

[tedsalmon]

@quickshiftin,

The purpose of the header is so the cache can be cleared on the fly for any page that contains the tag of the entity being updated in the Admin panel.

The data could always be gzipped and then base64 encoded into the header but at some point you're going to run into this issue again.

[sambolek]

That header is used to determine which pages need to be flushed from Varnish when e.g. a product is saved or cms page is updated. Essentially, Magento tags each page with set of tags pulled in from content that's being displayed on page.

More info https://inviqa.com/blog/how-full-page-cache-works-magento-2

[quickshiftin]

@sambolek, @tedsalmon - Thank you guys for the lighting fast responses!

[quickshiftin]

Hi guys, after looking over the blog page about how the M2 FPC works, other comments on this thread and getting things working on my site I've got a couple things I'd like to share.

First off, merely increasing http_resp_hdr_len was not cutting it. There were several other settings I increased in Varnish to stop the 503's

• workspace_backend - 256k
• workspace_client - 256k
• workspace_session - 256k
• workspace_thread - 8k (max)

Not sure if all of those are necessary, but the pages are loading. Stumbled into this thread when working on possibly related 504 responses on the same site about a week ago. I also brought http_resp_hdr_len to 2MB, based on what I saw requesting pages directly from the webserver (that would load successfully through curl lol).

The other thing is I think the patch in 2.2.3 is reasonable. Pretty much I agree with @pynej, it's not a perfect solution, but it's definitely the low hanging fruit here, and should dramatically reduce the X-Magento-Tags header size. And for an exhaustive solution like what @sambolek has built, I would still suggest the patch in 2.2.3 go along with it, because why store a bunch of extra data if you don't need to!

[alexblueacorn]

I'm on 2.3.2 and we still have this issue. I can fix it on my local mac, just by increasing proxy memory buffer, but we have production on ubuntu , apache php-fpm and we having issues to configure it right,
For now we just disabled FPC , but I think do the same @pynej suggested. Just exclude childs from the list.

` /**
* Add child identities to product identities
*
* @param Product $subject
* @param array $identities
* @return array
*/
public function afterGetIdentities(Product $subject, array $identities): array
{
// foreach ($this->configurableType->getChildrenIds($subject->getId()) as $childIds) {
// foreach ($childIds as $childId) {
// $identities[] = Product::CACHE_TAG . '_' . $childId;
// }
// }

    return array_unique($identities);
}`

[Tjitse-E]

I had the same problem on Magento 2.3.2. In my opinion, the cleanest approach is just to disable the \Magento\ConfigurableProduct\Model\Plugin\Frontend\ProductIdentitiesExtender::afterGetIdentities plugin. I did this by creating a new module and then disable the plugin via DI.xml.

Create etc/frontend/di.xml with content:
<?xml version="1.0"?> <config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd"> <!-- disable \Magento\ConfigurableProduct\Model\Plugin\Frontend\ProductIdentitiesExtender::afterGetIdentities --> <type name="Magento\Catalog\Model\Product"> <plugin name="product_identities_extender" disabled="true"/> </type> </config>

[diazwatson]

I can confirm that this issue is still happening in 2.3.2.
Maybe we could rescue the work done in https://github.com/magento/magento2/pull/12831 and apply the changes suggested by @ishakhsuvarov to have this finally fixed?

[m2-assistant[bot]]

Hi @engcom-Echo. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• [ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• [ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!

• [ ] 5. Add label Issue: Confirmed once verification is complete.

• [ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

[magento-engcom-team]

@engcom-Echo Thank you for verifying the issue.

Unfortunately, not enough information was provided to acknowledge ticket. Please consider adding the following:

• [ ] Add "Reproduced on " label(s) to this ticket based on verification result
• [ ] Update issue content to follow format required by Issue Reporting Guidelines

Once all required information is added, please add label "Issue: Confirmed" again.
Thanks!

[magento-engcom-team]

@engcom-Echo Thank you for verifying the issue.

Unfortunately, not enough information was provided to acknowledge ticket. Please consider adding the following:

• [ ] Update issue content to follow format required by Issue Reporting Guidelines

Once all required information is added, please add label "Issue: Confirmed" again.
Thanks!

[magento-engcom-team]

:white_check_mark: Confirmed by @engcom-Echo
Thank you for verifying the issue. Based on the provided information internal tickets MC-29755 were created

Issue Available: @engcom-Echo, _You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself._

[hostep]

We are also running against this problem on our local machines (no Varnish, no Redis) which seems to be caused by a configurable product on which 510 simple products are linked.

Does anybody know what settings to tweak in php-fpm and/or apache2 to get around this?

The solution linked by @mpchadwick on his blog doesn't seem to resolve the issue over here unfortunately.

Disabling the plugin as mentioned by @Tjitse-E does seem to work, but doesn't sound like a good solution.

@engcom-Hotel: can you try to bump the priority of this issue internally, because this is an issue on which you can spend many hours staring at a blank screen, finding no decent errors in log files and then accidentally finding this thread here.

[mpchadwick]

@hostep I haven't found any way to get around this with Apache and PHP-FPM. We've had to move to nginx to get around this.

[engcom-Hotel]

Hello @hostep
I will try to advance this task.

Thanks for contributing!

[KevinMace]

We're also seeing this issue on a category containing a large configurable product (+1000 variations) along with a dozen simple products.

Magento Version 2.3.1

[engcom-Hotel]

Hello @sdzhepa

Could you raise the priority of this issue by talking to internal Magento teams?
The problem really needs a solution as @hostep wrote earlier.

Thanks for collaboration!

[dhorytskyi]

It looks like \Magento\ConfigurableProduct\Model\Plugin\Frontend\ProductIdentitiesExtender plugin, which adds child product identities to a configurable product, can be safely removed. Another plugin adds parent identities to child products. So if child product is changed, cache for configurable product will be flushed even if page doesn't have cache tag of child product.

[m2-assistant[bot]]

Hi @o-iegorov. Thank you for working on this issue.
Looks like this issue is already verified and confirmed. But if you want to validate it one more time, please, go though the following instruction:

• [ ] 1. Add/Edit Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 2. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!

• [ ] 3. If the issue is not relevant or is not reproducible any more, feel free to close it.

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[o-iegorov]

Internal Magento team is working on this issue right now.

[gianstraf]

It looks like \Magento\ConfigurableProduct\Model\Plugin\Frontend\ProductIdentitiesExtender plugin, which adds child product identities to a configurable product, can be safely removed. Another plugin adds parent identities to child products. So if child product is changed, cache for configurable product will be flushed even if page doesn't have cache tag of child product.

I can confirm that it's true.
The plugin is not useful and causes a varnish 503, when Magento is behind varnish.

[drew7721]

I believe this plugin is should not have been added to configurable products.

On stores with over 350k products this is what happens :

1. It increases the size of the tag header from 815b to 18k (showing 48 products per page)
2. Varnish RAM usage is increased because of the size of the tags.
3. Header buffer sizes need to be increased for nginx buffers

I really don't see the point of having all these tags. Note that this was introduced in this commit that was attempting to fix this issue https://github.com/magento/magento2/pull/19313

This was a fix for bundle products but was unfortunately also applied to the configurable products.

This might be a good solution for a store that has only a few products, not many children for configurable products and showing a small number of products per page in the category listings. Not so much for a big store that has configurable products with hundreds of children and shows many products per page.

This also seems to increase the PURGE requests size since it will have all the child tags. I believe that children products should simply return the parent tags and when a change is made to a child or a product the cache should be flushed by the parent tag. In other words, the child/parent relation should be checked when flushing the cache instead of adding all these tags to all the pages.

Maybe @sivaschenko can give us more insight on the reasoning for merging this and the testing that was done to ensure that this change does not break the infrastructure.

[danemacmillan]

I always see the same faces in Magento's tickets related to performance and huge catalogs. ✋

[sivaschenko]

@drew7721 the problem is that child products affect parent product representation while parent products do not affect child products representation on the frontend, so in order to be sure that the cache is valid - the cache of pages with parent products should be invalidated when a child product is changed.

[drew7721]

@sivaschenko I agree that the child products affect parent product representation. But I strongly believe that we are addressing this issue the wrong way, as far as configurable products are concerned.

Goal : clear parent product caches when the child product is modified.

---

First approach : _(the one implemented now)_

• Add all child product cache tags on parent product pages
• Add all child products as well as parent products on category pages.
• Clear caches by the child product cache tag and we get the wanted results.
  To me, this seems like a back-end/admin issue being fixed by adding non essential data to the front-end.

Pros: _I tried to find some_

• We can clear caches by child product tag.
• Makes the save operation faster (?) _- not certain of this and I don't want to benchmark this._
• No need to figure out the parent product of a child when changes are done on the back-end (not sure if this is a pro or con actually)

Cons:

• Header size is increased due to thousands of product tags
  
  
  
  • Think of category with 48 products per page & 5 size super-attributes & 5 colors super-attributes per product => translates into over 1248 tags; That's huge!
  
  
• Increased overhead processing for all category pages.
• Every time a category page loaded needs to figure out all the tags of all the child products
• Server config changes needed to support the increased header size.
• Higher use of RAM by Varnish
• Increased bandwidth and page sizes
• Decreased performance overall - _IMO_

---

Second approach: _(what I suggest as a more viable solution)_

• Always us the parent product tags on categories, configurable product pages and even the child product pages.
• When a change is made to a child product, figure out the parent(s) and flush caches based on that tag.
• When changes are made to a parent product, clear caches by the parent product cache tag.
  In other words, approach the issue when the caches are getting flushed (back-end), not when the category is getting generated for the front-end.

Cons:

• Changes to a child product will flush caches for all siblings (if they're visible individually).
• Might make back-end operation less performant (save and maybe indexing) _- again, not certain of this and I don't want to benchmark this but feel free to and share the result._

Pros:

• Fixes all the _cons_ of the first approach.
• It seems a more viable solution overall.

Now, this might not be a good approach for bundled products. I'm not very familiar with this type of products but I get the gist of it. I'm quite positive that a similar approach could be taken.

---

Please, let me know if this makes sense.

[ilnytskyi]

For those who faced this issue caused by custom code. e.g. import of product/categories
Make sure you have all indexers set up tu run "Update by Schedule"

Because after each save from entity repository in the loop it will add tags to
\Magento\Framework\Indexer\CacheContext or other storage and will cause reindexing and purging more entities, not only saved/updated ones

[MartinAarts]

It looks like \Magento\ConfigurableProduct\Model\Plugin\Frontend\ProductIdentitiesExtender plugin, which adds child product identities to a configurable product, can be safely removed. Another plugin adds parent identities to child products. So if child product is changed, cache for configurable product will be flushed even if page doesn't have cache tag of child product.

We fixed the problem of an entire category giving a 503 error with this solution on Magento 2.3.4. We just commented out a couple of lines in this file. This is a dirty fix, but this fix can easily be made without downtime, and for as far as I know without any consequences. Used this fixed, assuming this file will soon be changed in the update to 2.3.5 to really solve this problem.

[gamort]

A product[parent or not] knows what categories it was in and what categories it is now in when it is saved.
A child product can lookup what parent products it is in before and after it is saved.
for an edge case, a child product can lookup what categories it's parent product is in when it is saved.

Therefore when a product is saved, a product is able to trigger a flush for all related category pages.
When a child product is saved, it can trigger a flush for all it's parent products.

A category page only has to have the tags associated with it for the category.
A product page only has to have the tags associated directly with it.

There are some tricky problems here to address with the staging feature of Commerce in regards to indexing and caching - but they are already broken to begin with so they can be addressed with an EE specific fix.

[magento-engcom-team]

Hi @katef, @o-iegorov.

Thank you for your report and collaboration!

The issue was fixed by Magento team.

The fix will be available with the upcoming 2.3.5 release.

[magento-engcom-team]

Hi @katef, @o-iegorov.

Thank you for your report and collaboration!

The issue was fixed by Magento team. The fix was delivered into magento/magento2:2.3-develop branch(es).
Related commit(s):

• 8280a9a6478d27131e7a12fa4b112baaf84ca86f
• 1729dcc4d9172e2dd51d061e7d5f560f244ed304

The fix will be available with the upcoming 2.3.5 release.

[katef]

@magento-engcom-team Wonderful, thank you!

The commits you linked to 404 though. Was that branch rebased and now they have different hashes now, perhaps?

[magento-engcom-team]

Hi @katef, @o-iegorov.

Thank you for your report and collaboration!

The issue was fixed by Magento team. The fix was delivered into magento/magento2:2.4-develop branch(es).
Related commit(s):

• bde57a3a0c018cfb9924f150f4d651600ed92406
• b0071aff2c0289d72431e05f22ac353236242107
• d2e252cf2696247715b927833aad7c22d8565ab6
• 219024d611ebeef5234901d2f3d784c9fc355721
• b6206fec5a4a1c88dd3b46c512c1e6314e7451c9
• e85903535b71e3e6fe534123d60a259099ce6aa5
• ca59b15fe42e18cd26ea8cfc093fdd5e4c6773a8
• 520590a2db348a1f8e6d4c1a8274a4f248fcdcb8

The fix will be available with the upcoming 2.4.0 release.

[katef]

Hi @magento-engcom-team! That's great, thank you!
All your links 404 for me, though. Are those private perhaps?

[bmitchell-ldg]

Hi @magento-engcom-team we would also like to see the related commits for 2.4.0 if you would be able to resolve those 404 links please?

[ecoprince]

Hi @katef, @o-iegorov.

Thank you for your report and collaboration!

The issue was fixed by Magento team. The fix was delivered into magento/magento2:2.4-develop branch(es).
Related commit(s):

• bde57a3a0c018cfb9924f150f4d651600ed92406
• b0071aff2c0289d72431e05f22ac353236242107
• d2e252cf2696247715b927833aad7c22d8565ab6
• 219024d611ebeef5234901d2f3d784c9fc355721
• b6206fec5a4a1c88dd3b46c512c1e6314e7451c9
• e85903535b71e3e6fe534123d60a259099ce6aa5
• ca59b15fe42e18cd26ea8cfc093fdd5e4c6773a8
• 520590a2db348a1f8e6d4c1a8274a4f248fcdcb8

The fix will be available with the upcoming 2.4.0 release.

@magento-engcom-team All links goes to 404. Please update it.

[ihor-sviziev]

@ecoprince,
Now these commits are available. Here is merge commit that contains all these changes: https://github.com/magento/magento2/commit/3b611a7cc0040c9e5731cbc8e2080c2adb205d86

[katef]

Thank you!

[websecureNL]

Any way to get this fixed on 2.3.5? I understand the fix is made for 2.4 develop but for our production store we are using 2.3.5-p1 and having this problem.

[ihor-sviziev]

@websecureNL you could try to create patch and apply on top of 2.3.5.
You can use https://devdocs.magento.com/guides/v2.4/comp-mgr/patching.html or https://github.com/vaimo/composer-patches

[websecureNL]

@ihor-sviziev, I tried this and it seems to work properly. We will do some further testing. Since making patches was new to me a quick hint for other users. We used this file (based on merge commit 3b611a7) to patch using instructions https://devdocs.magento.com/guides/v2.4/comp-mgr/patching.html

[websecureNL]

The problem also occurs when using catalogsearch. No idea if this also impacts 2.4 but since the patch did not fix this it could be.

[ihor-sviziev]

Could you check on empty 2.4-develop installation, and if issue there -
report it with exact steps to reproduce?

On Thu, 2 Jul 2020 at 00:29, websecureNL [e-mail] wrote:

The problem also occurs when using catalogsearch. No idea if this also
impacts 2.4 but since the patch did not fix this it could be.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/magento/magento2/issues/6401#issuecomment-652655940,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAOJOUJWTKQRJSTOTHLVGO3RZOTCXANCNFSM4COCM5GA
.

[websecureNL]

@ihor-sviziev. Of course I can do this but isn't this originated from Magento 2.3? We are having problems with 2.3.5 and would like to get it fixed there. I will check 2.4-develop asap.

[elvinristi]

if you compare commits then actually nothing was done. was reverted probably something what didn't exist yet in M2.1 and therefor no matter of the version, issue continues.

temporary fix can be currently https://github.com/magento/magento2/issues/6401#issuecomment-325510002

but longer run should be similar what was done here but closed and ignored due inactivity https://github.com/magento/magento2/pull/12831 which splits headers.

[ihor-sviziev]

if you compare commits then actually nothing was done. was reverted probably something what didn't exist yet in M2.1 and therefor no matter of the version, issue continues.

temporary fix can be currently #6401 (comment)

but longer run should be similar what was done here but closed and ignored due inactivity #12831 which splits headers.

@elevinskii https://github.com/magento/magento2/issues/6401#issuecomment-325510002 is really hack and not a correct solution, it breaks automatic cache flushing for category pages when any product was changed in the admin. definitely don't recommend it.

https://github.com/magento/magento2/pull/12831 this solution is quite good, but it's not finalized in order to be accepted. You could create new PR based on this PR and apply requested changes. Only one concern - such change might break some services like Fastly (for instance used in Magento Cloud), so probably it should be double checked additionally

Also would like to add that there was a fix for configurable products that mostly caused this issue, this fix you can see in https://github.com/magento/magento2/issues/6401#issuecomment-639289963

BTW if you have steps to reproduce of this issue on 2.4-develop - please report it separately