Magento2: Varnish cache not working

[werdlv]

Steps to reproduce

1. Install Magento 2.0.7 on Centos with Nginx, PHP 7.07 FPM and MariaDB 10.1
2. set $MAGE_MODE production; is set in Nginx server block
3. Some content is added
4. Varnish 4.03 is installed and configured. Note that Varnish was installed after Magento2 installation. All of that is currently running on the same server.
5. Magento is configured to use Varnish Caching
6. varnish.vcl generated by "Export VCL for Varnish 4" is being used

7. We have around 46k products imported - none of them are enabled yet however

   Expected result

8. Pages are cached with ~100ms response time

   Actual result

9. Pages are not cached - it actually takes ~3 seconds to load

10. Static files (CSS/JS) are cached
11. Sysadmin is saying that PHP (Magento) is sending "no cache" header and therefore Varnish ignores the request.
12. Following response headers are present:

Via:1.1 varnish-v4
X-Content-Type-Options:nosniff
X-Frame-Options:SAMEORIGIN
X-Magento-Cache-Debug:MISS
X-Powered-By:PHP/7.0.7
X-Varnish:262271
X-XSS-Protection:1; mode=block

What could be the issue?

[bwaters]

Thanks for reporting. We are currently testing the following fix for 2.1
Add the following at bottom of sub vcl_backend_response

# If page is not cacheable then bypass varnish for 2 minutes as Hit-For-Pass
   if (beresp.ttl <= 0s ||
        beresp.http.Surrogate-control ~ "no-store" ||
        (!beresp.http.Surrogate-Control && beresp.http.Vary == "*")) {
        # Mark as Hit-For-Pass for the next 2 minutes
        set beresp.ttl = 120s;
        set beresp.uncacheable = true;
    }
    return (deliver);

Does this resolve your issue?

[werdlv]

@bwaters no, it doesn't resolve the issue.

[bwaters]

This fix was delivered to mainline -- regenerate your vcl file, replace and restart your varnish server.

https://github.com/magento/magento2/commit/ce87aa48fff2fdb4ba13bb90fc3630893596c49d

[werdlv]

@bwaters we already did that - issues is still present

[werdlv]

@bwaters any update on this?

[bwaters]

It is working for others can you post your complete vcl and varnish version?
On Jun 14, 2016 3:51 AM, "Jānis Šakars" [email protected] wrote:

@bwaters https://github.com/bwaters any update on this?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/magento/magento2/issues/4966#issuecomment-225818864,
or mute the thread
https://github.com/notifications/unsubscribe/AAb4JhInnvUTNaRzsFdAS2DlPxzKr2Atks5qLmuBgaJpZM4Iy-8f
.

[werdlv]

@bwaters here is complete VCL - https://gist.github.com/werdlv/b7b0a7247bac7444c3e1c1389887cd69
We re using Varnish 4.0.3

[bwaters]

You need to remove the return (deliver) above the hit for pass block. The
hit for pass code will not execute with the return there and it will slow
down non cacheable requests.

That error shouldnt disable caching though.

Are you in development mode? If so you can view the Hitt or Miss http
headers to easily determine the caching.
On Jun 14, 2016 7:53 AM, "Jānis Šakars" [email protected] wrote:

@bwaters https://github.com/bwaters here is complete VCL -
https://gist.github.com/werdlv/b7b0a7247bac7444c3e1c1389887cd69
We re using Varnish 4.0.3

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/magento/magento2/issues/4966#issuecomment-225871557,
or mute the thread
https://github.com/notifications/unsubscribe/AAb4Jt-oCMN1tTSZJWA_DnQYt_5N5DU-ks5qLqRbgaJpZM4Iy-8f
.

[werdlv]

@bwaters I did that however issue is still there. We are in production mode.
I tried with developer mode - result is the same with X-Magento-Cache-Debug:MISS

[Kraeutergarten]

Same problem here

[m0zziter]

Wanted to chime in here and say we were experiencing the exact symptoms and in our case it was a third party module.

Specifically this one: https://github.com/YotpoLtd/magento2-plugin

Long story short this line was the issue. When cacheable is set to false Magento sent X-Magento-Cache-Control: no-store, no-cache, must-revalidate, max-age=0 preventing Varnish from caching the response.

When cacheable is set to true (or if we disabled the module completely) Magento sent X-Magento-Cache-Control:max-age=86400, public, s-maxage=86400 and Varnish correctly cached the response.

Your mileage may vary, but these issues sounded close enough that I figure I'd offer up our findings. @rgoncharuk let me know if this sounds like a separate issue and I'll open up a ticket.

[caesarcxiv]

After spending hours trying to figure out why Magento was sending no-store I came up with this:

Search code folder for disabled cache tag

grep -rnw '/var/www/magento/app/code/' -e 'cacheable="false"'

This will output all files that have a cacheable="false". This makes it easy to identify which extension is the culprit.

[pantaoran]

I would recommend @werdlv to first confirm that it is a Varnish issue and not a general FPC issue. Configure Magento's built-in FPC, use developer mode and see if you get any cache hits. If the FPC always misses then the problem is not with Varnish but with your content, probably something like the commenters above me suggested (non-cacheable blocks etc.)

[werdlv]

@pantaoran Unfortunately currently I don't have time to do that

[20071313]

Same problem here

[20071313]

any update?

[kevinak941]

Same issue, solution from @caesarcxiv worked. Had a module with a cacheable="false" on a default.xml.

[magento-engcom-team]

@werdlv, thank you for your report.
The issue is already fixed in 2.2.0