Magento2: x-magento-vary and store cookie mismatch can cause a varnish multi-store cache collision

[maderlock]

Preconditions

1. Magento 2.1.5 with php7.1
2. Multi-store setup switched using cookies rather than different subdomains

Steps to reproduce

1. Visit a page and switch to a non-default store
2. Clear out store cookie
3. Clear varnish for page
4. Refresh page
5. Try to switch to non-default store

Expected result

1. On refresh you'll see the default store (as no store cookie set)
2. When switching store you see the non-default store version of page

Actual result

1. On refresh you'll still see the default store
2. When switching you'll still see the default store

What happens here is that the x-magento-vary cookie matches the non-default-store page, but the store is not set. Magento renders the default page, but varnish stores this against the non-default-store hash.

The basic issue is that there are two cookies here that should be synchronized, but as they are client-side there is nothing forcing them to be so.

This might seem like a real edge case, but in practice we're finding it happens quite a lot - perhaps because the cookies have different expirations! x-magento-vary is session expiry and store is a year by default. So in just this one instance you can see that you'd end up with one cookies and not the other. This example is round the other way to the one above, but you'd still get a collision.

[maderlock]

In order to fix this, I've added a check to Magento\Framework\App\Response\Http in sendVary to check that the X-Magento-Vary cookie matches the calculated vary string. If it doesn't match then it sends no-cache headers to ensure that varnish does not cache the page this time.

I've done this by a plugin to the class with the following contents:

<?php

namespace Me\MagentoFixes\Plugin\Framework\App\Response;

use Magento\Framework\App\Http\Context;
use Magento\Framework\App\ObjectManager;
use Magento\Framework\App\Request\Http as HttpRequest;

class Http
{
    /** @var Context */
    protected $context;

    /** @var HttpRequest */
    protected $request;

    /**
     * @param Context $context
     * @param HttpRequest $request
     */
    public function __construct(
        Context $context,
        HttpRequest $request
    ) {
        $this->request = $request;
        $this->context = $context;
    }

    /**
     * In case of mismatch between x-magento-vary and calculated vary, do not
     * allow page to be cached. Allow Magento's own cookie setting mechanism to
     * re-sync this information. Caching can continue after this.
     *
     * This is to fix https://github.com/magento/magento2/issues/9695
     *
     * @param \Magento\Framework\App\Response\Http $http
     */
    public function beforeSendVary($http)
    {
        // Calculate vary string for THIS call - can be empty if default
        $varyString = $this->context->getVaryString();

        // Get vary string passed in from cookie
        $cookieVal = $this->request->get(\Magento\Framework\App\Response\Http::COOKIE_VARY_STRING);

        // Compare the two strings. They match only if both do not exist, or both exist and are equal
        $doNotCache = ($cookieVal && $varyString && ($cookieVal != $varyString));
        $doNotCache |= (!$cookieVal && $varyString);
        $doNotCache |= ($cookieVal && !$varyString);

        // If mismatch, set headers so does not cache
        if ($doNotCache) {
            $http->setNoCacheHeaders();
        }
    }
}

[maderlock]

I'm not sure if we should be using setPrivateHeaders($ttl) instead of setNoCacheHeaders(), as I found that I had to edit sub vcl_fetch in the varnish default.vcl to add the following:

# Temporarily do not cache pages marked as no-cache
if (beresp.http.Cache-Control ~ "no-cache") {
    set beresp.ttl = 0s;
    return (hit_for_pass);
}

Without this the no-cache was being ignored by varnish. Not sure if there is another mechanism that we should be using.

[maderlock]

Nobody else bothered by this? This caused so many problems for us - but maybe not that many people are using Varnish with multiple-sites.

[KevinMace]

@maderlock We've been experiencing the same issue. We've been seeing Varnish randomly serve a page from "Store B", when the customer is currently navigating the site in "Store A".

Clearing the cache solves the issue for a short while, but eventually it's polluted with mismatched data again.

[maderlock]

In 2.1.8 my fix causes an error in some cases, so be aware that it might only be working before this. The fix was created for 2.1.5.

[AndreKlang]

@maderlock Can you please describe what kind of errors, and what cases this applies in?

I was just about to add it in 2.1.8, since we use the fix successfully in 2.1.5

[maderlock]

The problem I have in >= 2.1.8 is that the follow lines cause an error:

$http->setNoCacheHeaders();

So I've replaced this with:

$http->setHeader('pragma', 'no-cache', true);
$http->setHeader('cache-control', 'no-store, no-cache, must-revalidate, max-age=0', true);

[maderlock]

Slight issue here is that varnish ignores no-cache and caches anyway. I've had to add a custom no-varnish option and listen to that in the varnish vcl. It would appear that Magento have not added any way other than private pages to stop pages from being cached as they come through varnish.

[maderlock]

I've ended up swapping out the pragma and cache-control for:

$http->setHeader('varnish-control', 'no-cache', true);

And added this to my default.vcl's sub vcl_fetch:

if (beresp.http.varnish-control ~ "no-cache") {
    set beresp.ttl = 0s;
    return (hit_for_pass);
}

[magento-engcom-team]

Hi @maderlock
Thanks for reporting this issue.
Unfortunately, I could not reproduce the issue as you described it.
Please try reproduce the issue on 2.1.10 version. May be it was fixed in this version.
Thanks.

[magento-engcom-team]

@maderlock, we are closing this issue due to inactivity. If you'd like to update it, please reopen the issue.

[maderlock]

Seeing this issue in 2.2.2. My fix has stopped working. Looking on updating my fix code to work with 2.2.2. Reproduction steps still as before.

[domeglic]

I have the same problem. I don't know if it is possible to change this but Varnish will call vcl_hash only once, right after vcl_recv.
When a request comes back from the webserver it will not call vcl_hash and will be saved under the same hash data.
If X-Magento-Vary has changed then it should always set ttl=0, otherwise it is possible to cache pages under wrong conditions.

Magento v2.2.6
Steps to reproduce:

• Have two stores, storecode is NOT set to be in url
• Have FPC enabled
• Open a page in a non-default store. You now have cookies: store and x-magento-vary.
• Clear FPC
• Delete cookie x-magento-vary from the browser
• Open a page. Because the store cookie is set, the page is in the requested store
• Open the same page in another browser.

Expected

• Both browsers show the page in correct store

Actual

• The second browser will show the page in the wrong store

[adam-inco]

im having this issue with magento 2.3 & Varnish, did anyone every find a fix for this?

[domeglic]

I didn't have a problem with the store cookie, but with my own custom one. This should work for all of them. What I did was to set request as uncacheable if the Vary string changes.

Plugin for Magento\Framework\App\Response\Http:

public function afterSendVary(\Magento\Framework\App\Response\Http $subject)
    {
        $originalHttpContext = $this->request->getCookie(\Magento\Framework\App\Response\Http::COOKIE_VARY_STRING);

        if (strcmp($originalHttpContext, $this->httpContext->getVaryString()) != 0)
        {
            $subject->setNoCacheHeaders();
        }
    }