Wp-rocket: 404 issues on CSS & JS resources

[MuhammedSwalihct]

Currently, WP-Rocket uses a mechanism that deletes the old Minified files and replaces them with new ones to accomplish cache-busting. As someone who makes some kind of changes to the websites almost every week, the files get regenerated every few days.

While this cannot be avoided, the problem here is that the old resources return a 404 error whenever anyone requests that resource. As you may already know, not every request made to a website will ask for its HTML. There are cached requests such as "Search Engine caches", "Static page caches", etc. When such a request is made, they will not look for the new updated HTML version of the page. It will result in a completely broken web page due to missing resources.

I did a little search and found that there are mainly two types of cache-busting mechanisms in place. One is rewriting the file names as WP-Rocket does. The second one is using a query string like "example.css?v=1111". In the second method, even when the query string is changed (even while the older version is removed), there is an actual file returned instead of a 404.

So, coming to the matter. I understand that there should be a way for cache-busting, so we cannot use a single file name. But isn't it possible to implement a "Redirect" from the old deleted file requests to the new one with some form of patterns? Or use the version number method instead of the file renaming?

Backlog Grooming (for WP Media dev team use only)

• [ ] Scope a solution
• [ ] Estimate the effort

[GeekPress]

Hi @MuhammedSwalihct 🤚

Thanks for your suggestion.

We just had a discussion about that and we have a workaround to work with the Minify CSS & JS options. Unfortunately, it's going to be difficult to have a solution for Combine CSS & JS options.

* For @wp-media/php *

Here the first solution we discussed with @Tabrisrp and needs to be validated.

We will add a query string (e.g ?ver=) to set the value with the last modified time of the CSS or JS files. It means we won't rely anymore on the minify key and the filename of the minified file won't contain it too.

[MuhammedSwalihct]

That would be awesome. I can see that many of the top websites also using this method of version numbering for resources. So, it is definitely a good path to move forward. I do not know what challenges it would cause, but if we can make it work, it would definitely be a great improvement to this "almost serious" issue.

[jazir5]

Does this also explain why the files 404 in varnish when you clear the cache without the query string exception or the nginx cache? This would probably completely resolve those issues right?

[MuhammedSwalihct]

@jazir5 Yes, I was also getting these error messages in the NGINX error log. As I explained in the original post, this happens because NGINX cache/Varnish tries to access the old cached versions of the HTML where the removed resource is referenced. As it no longer exists on the server, it throws a 404 error. Applying the above fix might resolve these errors as well.

[jazir5]

Awesome!

On Sat, Feb 6, 2021, 4:25 PM Muhammed Swalih [e-mail]
wrote:

@jazir5 https://github.com/jazir5 Yes, I was also getting these error
messages in the NGINX error log. As I explained in the original post, this
happens because NGINX cache/Varnish tries to access the old cached versions
of the HTML where the removed resource is referenced. As it no longer
exists on the server, it throws a 404 error. Applying the above fix might
resolve these errors as well.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wp-media/wp-rocket/issues/3564#issuecomment-774565572,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHSHPL4H2CAAQ3WUV6XNWP3S5XMXNANCNFSM4XETOTQA
.

[MuhammedSwalihct]

@GeekPress @Tabrisrp Is there any update on this? I am having some serious troubles with the 404 issues. Recently Bing deindexed one of my sites because it was broken due to this issue. And I didn't even notice it. So, it's pretty serious at this point.

[engahmeds3ed]

Reproduce the problem
No need to reproduce it, we saw this issue on some of customers' sites.

Identify the root cause
As mentioned by @MuhammedSwalihct because we delete the old file and create a new one with different path/url so it may show 404 error when opened old ones.

Scope a solution

1. For CSS Replace the following lines
   https://github.com/wp-media/wp-rocket/blob/ad67f51569b43c60b82d12768e2ae203a5ed43fe/inc/Engine/Optimization/Minify/CSS/Minify.php#L138-L141

with

$filename      = ltrim( rocket_realpath( $parsed_url['path'] ), '/' );
$minified_file = rawurldecode( $this->minify_base_path . $filename );
$minify_url    = $this->get_minify_url( $filename, $url ) . "?ver=" . rocket_direct_filesystem()->mtime( $minified_file );

1. Do the same with JS

2. Adjust tests fixtures to remove hashed keys from urls and add ?ver at the end of each url.

Estimate the effort
[S]

[GeekPress]

@MuhammedSwalihct This issue has been added to our current sprint. It means the fix is scheduled for 3.8.6

[MuhammedSwalihct]

@engahmeds3ed @GeekPress Thank you...

[engahmeds3ed]

@webtrainingwheels I think this one needs documentation check

[seomarketingads]

Finally we have solution, I had a lot of issues with google search console " some resources couldn't be loaded " and this was affecting the mobile usability by marking the pages as not mobile friendly despite they show up fine with instant check.
a lot was blaming the host and the quota or even saying its a bug at google!

thanks for the transparency of the wprocket team