Magento2: Navigation in catalog view missing when Varnish is activated

[duerstd]

Hi there,

We currently installed and activated Varnish on our new Magento 2 solution. After the installation was successfully, we noticed that the navigation on the catalog view (just on the catalog view) is gone.

Our guess is that something in the varnish configuration file (default.vcl) which we exported from our Magento 2 shop is missing (http://devdocs.magento.com/guides/v2.0/config-guide/varnish/config-varnish-magento.html -> see step 7).
This because after the varnish installation on the server (before the activation in the Magento 2 backend) everything worked fine. Then we activated varnish in the Magento 2 backend, the hole navigation was gone. After updating the default.vcl file with the exported one from the Magento 2 backend, the navigation on the static pages came back - but not the navigation on the catalog view.

System information:
CentOS 6 and 7
Apache 2.2.15
Magento 2.0.2

Checkout the screenshots:

[erikhansen]

I have also experienced the same issue where the main navigation disappears when Varnish is enabled.

[erikhansen]

@duerstd I created a new issue with steps to reproduce this issue: #3897 Can you close this task since it reports the same issue as my newly created issue?

[mazhalai]

Closing as duplicate

[OmarFaruque]

I am facing same problem after install new Magento 2.1.1

[Jilco]

I removed the TTL attribute from /vendor/magento/module-theme/view/frontend/layout/default.xml and the menu bar was there again! But can that do any harm, if I'm correct the default TTL is enheritated. Or am I wrong?

[mkleim]

Hi, thanks a lot for that hint.
Worked really well after reindexing.
I´m happy about that

[ASirik]

hey @duerstd did you fix the issue ?

[jpcsguru]

hi,
having same issue any fix?

[kllee824]

I got the same issue and fixed it by removed TTL attribute.

Thanks a lot

[likeweb]

Information
The TTL attribute can be found under:
vendor/magento/module-theme/view/frontend/layout/default.xml
remove the ttl="3600"

After removing the attribute and clearing cache the varnish cache worked fine for me

[avadh88]

Enter Code in theme default.xml

<referenceBlock name="store.menu"> <block class="Magento\Theme\Block\Html\Topmenu" cacheable="false" /> </referenceBlock>

[erikhansen]

Beware of using cacheable="false" in layout XML. Any page that contains a block with cacheable="false" will never be cached in the Full Page Cache. So I would not recommend the solution that @vadh88 just posted.

[avadh88]

@erikhansen If we use full page varnish cache menu disappears. If we remove ttl from vendor file is not a feasible solution. are you know how to remove TTL from block ?

[erikhansen]

@avadh88 Check out this comment as it _might_ be relevant. Specifically, check to see if the view/base/layout/default.xml file in your installation contains this string layout="admin-1column" (it should not contain that string).

[bruno-serfe]

Hi Guys, Is there any other solution rather than remove the TTL attribute from the block? It looks that's not the proper solution:

I'm using:
Varnish (varnish-4.0.5 revision 07eff4c29)
Magento 2.2.3

Thanks

[Jilco]

Bruno,

I now use ultimo infortis theme and there is no problem in 2.2.3. I can really recommend that theme! It is highly adjustable and stable

Verzonden vanaf mijn Samsung Galaxy-smartphone.

-------- Oorspronkelijk bericht --------
Van: "Bruno Binci (Serfe.com)" [e-mail]
Datum: 16-04-18 21:00 (GMT+01:00)
Aan: magento/magento2 [e-mail]
Cc: Jilco [e-mail], Comment [e-mail]
Onderwerp: Re: [magento/magento2] Navigation in catalog view missing when Varnish is activated (#3421)

Hi Guys, Is there other any solution rather than remove the TTL attribute from the block? It looks that's not the proper solution:

I'm using:
Varnish (varnish-4.0.5 revision 07eff4c29)
Magento 2.2.3

Thanks

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmagento%2Fmagento2%2Fissues%2F3421%23issuecomment-381712651&data=02%7C01%7C%7Ca93c453562d14b4a9a8408d5a3cc483e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636595020133020123&sdata=3Kf4ukf4L5JXiAVt2rWFYW755x0Vy4x%2F1vBxi06RhjE%3D&reserved=0, or mute the threadhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAWoMGmUyjdbGYkoMPJMP56RTg0VW54Vjks5tpOo7gaJpZM4HbCQj&data=02%7C01%7C%7Ca93c453562d14b4a9a8408d5a3cc483e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636595020133020123&sdata=lRMmHfwP94e4gZLjTr4M6ZlKdqlyJyYcCGEnLNfSR9g%3D&reserved=0.

[Jilco]

Without varnish, but with nginx and redis. And its super fast with php 7.1

Verzonden vanaf mijn Samsung Galaxy-smartphone.

-------- Oorspronkelijk bericht --------
Van: "Bruno Binci (Serfe.com)" [e-mail]
Datum: 16-04-18 21:00 (GMT+01:00)
Aan: magento/magento2 [e-mail]
Cc: Jilco [e-mail], Comment [e-mail]
Onderwerp: Re: [magento/magento2] Navigation in catalog view missing when Varnish is activated (#3421)

Hi Guys, Is there other any solution rather than remove the TTL attribute from the block? It looks that's not the proper solution:

I'm using:
Varnish (varnish-4.0.5 revision 07eff4c29)
Magento 2.2.3

Thanks

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmagento%2Fmagento2%2Fissues%2F3421%23issuecomment-381712651&data=02%7C01%7C%7Ca93c453562d14b4a9a8408d5a3cc483e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636595020133020123&sdata=3Kf4ukf4L5JXiAVt2rWFYW755x0Vy4x%2F1vBxi06RhjE%3D&reserved=0, or mute the threadhttps://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAWoMGmUyjdbGYkoMPJMP56RTg0VW54Vjks5tpOo7gaJpZM4HbCQj&data=02%7C01%7C%7Ca93c453562d14b4a9a8408d5a3cc483e%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636595020133020123&sdata=lRMmHfwP94e4gZLjTr4M6ZlKdqlyJyYcCGEnLNfSR9g%3D&reserved=0.

[bruno-serfe]

@Jilco is not an option disregard varnish at this point for me. thanks anyway.

[gcampedelli]

I think they should reopen this issue. I found another partial solution using varnish which is here in the power point - https://pt.slideshare.net/TonyBrown44/magento-2-and-avoiding-the-rabbit-hole
Even if the solution results in menu displaying in checkout page, it does not solve the problem in home and catalog pages. I found a way to solve that but using Apache2+ Varnish. I now switch to Nginx + Varnish and I found the same problem and just arrived to solve to display menu in some pages. So, it is something in the server configuration which is missing in Magento2 Dev Docs which are a bit confusing once Varnish part is totally redirected to Apache 2 and Optimization suggests to use Nginx.

[bruno-serfe]

@gcampedelli I think is the same solution proposed here in this issue: https://github.com/magento/magento2/issues/3897

Do you have tried that? Is it working for you?

Thanks

[gcampedelli]

I solved it using the solution described in Varnish but for Magento 2.2.2. I didn't remove TTL from layout. But I managed to do it in Magento 2.2.2. When I switched to Magento 2.2.3 it happened again. Menu only displays in Checkout. I will try the Magento 2.2.3 install again, but I will check if the issue is not coming from .htaccess.

[bruno-serfe]

@gcampedelli yes, I'm on v2.2.3 and "-p feature=+esi_ignore_https" varnish option didn't work for me, I had to perform an override of the Topmenu block on my theme layout xml.

[gcampedelli]

Well, the solution just worked after changing some Apache 2 settings.
Here is my Apache 2 configuration

<IfModule mod_ssl.c>
<VirtualHost *:443>
RequestHeader set X-Forwarded-Proto "https" 
#RequestHeader set X-Forwarded-Port "443" #ProxyRequests Off 
DocumentRoot /var/www/html/magento2 
ServerName mydomain.com 
ServerAlias www.mydomain.com 
Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
<Directory /var/www/html/magento2>
Options Indexes FollowSymLinks
Allowoverride All
Require all granted
Include /etc/letsencrypt/options-ssl-apache.conf
ProxyPreserveHost On
ProxyPass / http://127.0.0.1:80/ retry=1 acquire=3000 timeout=600 Keepalive=On
ProxyPassReverse / http://127.0.0.1:80/
SSLCertificateFile /etc/letsencrypt/live/mydomain/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/mydomain/privkey.pem
</VirtualHost>
</IfModule>

[JoshuaFlood]

Hi @gcampedelli . Thanks for the solution! Unfortunately, my settings are much more basic than yours and I can't figure out which parts are important. I am having the same problem when connecting to only http, with SSL de-activated in the magento settings.

Which were the lines that fixed it for you?

Thanks.

[gcampedelli]

It is hard to help if I don't know your settings. But I had to change varnish and Apache 2 configuration as written above.
This lines - RequestHeader set X-Forwarded-Proto "https"

Include /etc/letsencrypt/options-ssl-apache.conf
ProxyPreserveHost On
ProxyPass / http://127.0.0.1:80/ retry=1 acquire=3000 timeout=600 Keepalive=On
ProxyPassReverse / http://127.0.0.1:80/
SSLCertificateFile /etc/letsencrypt/live/mydomain/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/mydomain/privkey.pem

And same for NGINX in other server which the configuration using varnish is like this

upstream fastcgi_backend {
server unix:/var/run/php/php7.0-fpm.sock;
}
server {
server_name meulindodominio.com.br;
listen 8080;
set $MAGE_ROOT /var/www/html/magento2;
set $MAGE_MODE production; # or developer

access_log /var/log/nginx/dmeulindodominio.com.br.com-access.log;
error_log /var/log/nginx/meulindodominio.com.br-error.log;

include /var/www/html/magento2/nginx.conf.sample;

}

server {

listen 443 ssl http2;
server_name meulindodominio.com.br;
proxy_headers_hash_bucket_size 128;
proxy_headers_hash_max_size 1024;
ssl_certificate /etc/letsencrypt/live/meulindodominio.com.br/fullchain.pem; # managed by Certbot    
ssl_certificate_key /etc/letsencrypt/live/meulindodominio.com.br/privkey.pem; # managed by Certbot
ssl_protocols              TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers               'AES128+EECDH:AES128+EDH:!aNULL';
#ssl_session_cache    shared:SSL:10m;
#ssl_session_timeout 24h;
#keepalive_timeout 300s;

location / {
    proxy_pass http://127.0.0.1:80;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    #proxy_set_header Ssl-Offloaded "1";
    proxy_set_header      X-Forwarded-Proto https;
    proxy_set_header      X-Forwarded-Port 443;
    #proxy_hide_header X-Varnish;
    #proxy_hide_header Via;
    proxy_set_header X-Forwarded-Proto $scheme;
    include fastcgi.conf;

}
location /phpmyadmin {
           root /usr/share/;
           index index.php index.html index.htm;
           location ~ ^/phpmyadmin/(.+\.php)$ {
                   try_files $uri =404;
                   root /usr/share/;
                   fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
                   fastcgi_index index.php;
                   fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                   include /etc/nginx/fastcgi_params;
           }
           location ~* ^/phpmyadmin/(.+\.(jpg|jpeg|gif|css|png|js|ico|html|xml|txt))$ {
                   root /usr/share/;
           }
    }
    location /phpMyAdmin {
           rewrite ^/* /phpmyadmin last;
    }

}

[JoshuaFlood]

No worries I have just noticed all sorts of problems with the varnish server configuration. Will let you know if the problem persists once I've cleaned it up. Thanks!

[iancassidyweb]

For anyone coming across this issue in the future please ensure you've read Magento's support article.

https://support.magento.com/hc/en-us/articles/360028757791-Top-navigation-panel-does-not-load-on-storefront