Magento2: Occasionally getting JSON strings on category pages

[dyuk1987]

We have encountered this issue on a few different Magento projects. We understand we may not have sufficient information or very clear procedures for reproducing the issue, but as this turned out to be quite annoying and we have no clue about the root cause at all, we still decided to raise this here. And we have seen similar issues on StackOverflow.

1. https://magento.stackexchange.com/questions/296351/magento-2-intermittent-cache-issue-showing-a-json-code-instead-of-homepage
2. https://magento.stackexchange.com/questions/296944/magento2-category-page-cached-as-json-data-in-varnish

Preconditions (*)

Case 1:

1. Magento 2.2.10 on AWS
2. Varnish 4.0.5

Case 2:

1. Magento 2.2.5 on Azure
2. Varnish 4.0.5

Steps to reproduce (*)

1. Browse any category page
2. Reach a cached page with JSON strings occasionally

Expected result (*)

1. Show a normal category page

Actual result (*)

1. Reach a cached page with JSON strings occasionally

It happened (seemingly) randomly, and after restarting the varnish service, the issue would be gone.

[jbclaudio]

I've exact the same issue at some specific category pages. Has anyone find a solution?

[pinedamg]

I'm experiencing the same issue here, I thought it was the Amasty Extension Layered Navigation, but is still happening without the extension, definitely is something related with Varnish and Magento Core, and clearly this happens on a random way, and there is not a pattern or clue about how to reproduce it.

Does someone have any clue about it? thank you very much in advance.

Apache version: Apache/2.4.6 (CentOS)
Magento: Commerce 2.3.4
Varnish: (varnish-5.2.1 revision 67e562482)

Update:
I have tried disabling the whole varnish, just making a bypass, and the issue still happening. (of course cleared cache before/after)

If you are having the same issue, please could you let us know your current request flow?, I'm pretty sure this is a forwarding/redirect request issue with Magento of course.

Apache Proxy SSL Termination + Varnish = Same Server
Magento Instance = Another Server same setup (Apache)

The current setup is, Client -> CloudFlare (free) -> Apache (Proxy SSL termination) -> Varnish -> Magento

Thanks,

[jbclaudio]

I have found the cause and the solution. The cause is that when Varnish is cleared and a user comes to a filtered page first, the JSON is cached. I have excluded this for Varnish and this solved the problem.

[dyuk1987]

I have found the cause and the solution. The cause is that when Varnish is cleared and a user comes to a filtered page first, the JSON is cached. I have excluded this for Varnish and this solved the problem.

@jbclaudio
O-M-G finally someone came up with a solution!
That's some amazing findings there mate.
Do you mind sharing the VCL rules you have added?

[ZIPITRO]

I have found the cause and the solution. The cause is that when Varnish is cleared and a user comes to a filtered page first, the JSON is cached. I have excluded this for Varnish and this solved the problem.

Wow, great job! Would you please share with us your VCL rules?

[pinedamg]

I have found the cause and the solution. The cause is that when Varnish is cleared and a user comes to a filtered page first, the JSON is cached. I have excluded this for Varnish and this solved the problem.

@jbclaudio That sounds great, but I'm curious because I've tried disabling Varnish and this still happening, are you completely sure is fixed? Did you were able to reproduce the issue to verify if the fix is real? I would like to think that you are our savior in this very rare issue :P, and as our bug-mates said, please could you show us that portion of the VCL file?, thanks

[dyuk1987]

I have found the cause and the solution. The cause is that when Varnish is cleared and a user comes to a filtered page first, the JSON is cached. I have excluded this for Varnish and this solved the problem.

On second thought, I could not figure out the relationship between the filtered page and the unfiltered page.
Aren't those filtered pages standalone cache entries in Varnish?

And I tried to follow your findings, but still could not find a way to 100% reproduce the issue as well.
Could you please also advise how you could do that?

[jbclaudio]

I've added this to the .vcl file:

    if (req.url ~ "shopbyAjax") {
        return (pass);
    }

This project uses the Amasty Shopby module. This added the parameter to the filter pages, which I could use. You may be able to use another parameter that Magento itself adds.

After this has been added to the .vcl file, the problem no longer occurs.

[TJEliot]

I just encountered this and was able to quickly resolve it, so I thought I would share my solution here: I just reran the catalog_product_category and catalog_category_product indexers