Magento2: Varnish not working

[LS-Supriya]

Preconditions

1. Install Varnish
2. Magento Version :2.0.5

Steps to reproduce

1. Configured varnish as per http://devdocs.magento.com/guides/v2.0/config-guide/varnish/config-varnish-magento.html
2. After replacing vcl file generated by magento, I don't see X-Varnish response headers
3. I can see "X-Varnish response headers" when I use default vcl generated by varnish

Expected result

1. Varnish must work after replacing the vcl file generated by magento

Actual result

1. Varnish stops working after replacing vcl file generated by Magento

My Vcl File (generated by Magento):

vcl 4.0;

import std;

The minimal Varnish version is 4.0

backend default {
.host = "127.0.0.1";
.port = "8282";
}

acl purge {
"127.0.0.1";
}

sub vcl_recv {
if (req.method == "PURGE") {
if (client.ip !~ purge) {
return (synth(405, "Method not allowed"));
}
if (!req.http.X-Magento-Tags-Pattern) {
return (synth(400, "X-Magento-Tags-Pattern header required"));
}
ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
return (synth(200, "Purged"));
}

if (req.method != "GET" &&
    req.method != "HEAD" &&
    req.method != "PUT" &&
    req.method != "POST" &&
    req.method != "TRACE" &&
    req.method != "OPTIONS" &&
    req.method != "DELETE") {
      /* Non-RFC2616 or CONNECT which is weird. */
      return (pipe);
}

# We only deal with GET and HEAD by default
if (req.method != "GET" && req.method != "HEAD") {
    return (pass);
}

# Bypass shopping cart and checkout requests
if (req.url ~ "/checkout") {
    return (pass);
}

# normalize url in case of leading HTTP scheme and domain
set req.url = regsub(req.url, "^http[s]?://", "");

# collect all cookies
std.collect(req.http.Cookie);

# static files are always cacheable. remove SSL flag and cookie
    if (req.url ~ "^/(pub/)?(media|static)/.*\.(ico|css|js|jpg|jpeg|png|gif|tiff|bmp|mp3|ogg|svg|swf|woff|woff2|eot|ttf|otf)$") {
    unset req.http.Https;
    unset req.http.Cookie;
}

return (hash);

}

sub vcl_hash {
if (req.http.cookie ~ "X-Magento-Vary=") {
hash_data(regsub(req.http.cookie, "^._?X-Magento-Vary=([^;]+);_.*$", "\1"));
}

}

sub vcl_backend_response {
if (beresp.http.content-type ~ "text") {
set beresp.do_esi = true;
}

if (bereq.url ~ "\.js$" || beresp.http.content-type ~ "text") {
    set beresp.do_gzip = true;
}

# cache only successfully responses and 404s
if (beresp.status != 200 && beresp.status != 404) {
    set beresp.ttl = 0s;
    set beresp.uncacheable = true;
    return (deliver);
} elsif (beresp.http.Cache-Control ~ "private") {
    set beresp.uncacheable = true;
    set beresp.ttl = 86400s;
    return (deliver);
}

if (beresp.http.X-Magento-Debug) {
    set beresp.http.X-Magento-Cache-Control = beresp.http.Cache-Control;
}

# validate if we need to cache it and prevent from setting cookie
# images, css and js are cacheable by default so we have to remove cookie also
if (beresp.ttl > 0s && (bereq.method == "GET" || bereq.method == "HEAD")) {
    unset beresp.http.set-cookie;
    if (bereq.url !~ "\.(ico|css|js|jpg|jpeg|png|gif|tiff|bmp|gz|tgz|bz2|tbz|mp3|ogg|svg|swf|woff|woff2|eot|ttf|otf)(\?|$)") {
        set beresp.http.Pragma = "no-cache";
        set beresp.http.Expires = "-1";
        set beresp.http.Cache-Control = "no-store, no-cache, must-revalidate, max-age=0";
        set beresp.grace = 1m;
    }
}
return (deliver);

}

sub vcl_deliver {
if (resp.http.X-Magento-Debug) {
if (resp.http.x-varnish ~ " ") {
set resp.http.X-Magento-Cache-Debug = "HIT";
} else {
set resp.http.X-Magento-Cache-Debug = "MISS";
}
} else {
unset resp.http.Age;
}

unset resp.http.X-Magento-Debug;
unset resp.http.X-Magento-Tags;
unset resp.http.X-Powered-By;
unset resp.http.Server;
unset resp.http.X-Varnish;
unset resp.http.Via;
unset resp.http.Link;

}

[katmoon]

Hi @LS-Supriya

Which Magento mode do you use?

What is the difference between the default vcl file that you have and the file generated by Magento?

[LS-Supriya]

Hi ,

Magento is running in default mode.

My Default.vcl

vcl 4.0;

backend default {
.host = "127.0.0.1";
.port = "8282";
}

sub vcl_recv {
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
}

sub vcl_backend_response {
# Happens after we have read the response headers from the backend.

sub vcl_deliver {
# Happens when we have all the pieces we need, and are about to send the
# response to the client.
#
# You can do accounting or modifying the final object here.
}

[katmoon]

Hi @LS-Supriya

According to the docs, Varnish response headers are displayed only in Developer mode:
http://devdocs.magento.com/guides/v2.0/config-guide/varnish/config-varnish-configure.html

Before you can look at headers, you must set Magento for developer mode. There are several ways to do it, the simplest of which is to modify .htaccess in the Magento 2 root. You can also use the magento deploy:mode:set command.

Does this solve your problem?

[LS-Supriya]

@katmoon : I have enabled developer mode. Still I couldn't find those headers

[pantaoran]

The Magento2 Varnish config suppresses the header that you're looking for. Look at the end of the varnish.vcl there is a line
unset resp.http.X-Varnish;
That line removes the X-Varnish header. I don't know why the M2 developers chose to do this, maybe its for security reasons of not showing any system details about used software to the public.

The important thing is that you're looking for the wrong header and making the wrong conclusion. Not seeing an X-Varnish header does not mean that Varnish is not working. The header that you should be looking for is x-magento-cache-debug.

[peterzhao1981]

Hi @pantaoran ,

I also installed magento2 and varnish 5.0, but I can't see x-magento-cache-debug in the header, I have set the magento to developer mode, is there any other configuration for x-magento-cache-debug?

[magento-engcom-team]

@LS-Supriya, thank you for your report.
We were not able to reproduce this issue by following the steps you provided. If you'd like to update it, please reopen the issue.
We tested the issue on 2.3.0-dev, 2.2.0, 2.1.9