Wp-rocket: Enhancement - Integrate with Cloudflare's new Automatic Platform Optimization

[gevcen]

Is your feature request related to a problem? Please describe.
Cloudflare has released a new feature called Automatic Platform Optimization. However, a problem arises is that it requires the official Cloudflare plugin to be installed, which therefore requires to disable WP Rocket's Cloudflare addon.

This leads to a problem, that users will never be sure that Cloudflare's cache would be cleared when the cache is cleared in WP Rocket. Eventough their official plugin supports clearing cache on content update, it doesnt answer the problem that Cloudflare cache won't be cleared when it's manually cleared by the user in WP Rocket.

Describe the solution you'd like
WP Rocket has all the necessary information in the Cloudflare addon (APIs...) that is sufficient for enabling the new Automatic Platform Optimization and those data can be sent to Cloudflare when we enable the feature.

Describe alternatives you've considered
Alternatively, we could add a toggle within the Cloudflare addon in WP Rocket to enable the new APO, and in this case it's WP Rocket who will make the "call" by sending the APIs. Similar button already exist for Dev Mode.

In fact, this alternative is the only possible way to integrate this feature into WP Rocket. Why? Well the other way around is hardly achievable because if we enable APO from Cloudflare dashboard itself, then I wonder how Cloudflare will find the APIs of WP Rocket's Cloudflare addon.

So the integration must come from the Cloudflare addon within WP Rocket, which I think many of us will appreciate being taken as a new priority.

Additional context
This is a major enhancement.

[PatrickD1985]

For now it will only work without having the WP Rocket Cloudflare add-on enabled as it conflicts with the Cloudlfare plugin BUT ALSO you now need the Cloudflare official plugin.

And indeed invalidating cache is a thing here.

For APO in an ideal situation I would love to see this be supported from within the add-on of WP-Rocket as this means one less plugin is needed and it will help to keep control over how Cache/Cloudflare is handled.

[Basilakis]

@PatrickD1985 is APO works for WooCommerce?

[PatrickD1985]

@Basilakis I would not be able to confirm that. As I have personal webshops.
Perhaps others can confirm this.

Please note; As it stands there is an issue currently with APO;
https://community.cloudflare.com/t/automatic-platform-optimization-enabled-i-no-longer-see-visitor-ip/210241/29
Estimated fix is coming Wednesday.

[Basilakis]

Thank you for Sharing

[ScottTravisHartley]

I can confirm that APO does work with woocommerce because it does make exclusions for woocommerce related cookies and was included in the announcement post.

[PatrickD1985]

And I can confirm Cloudflare released a fix for https://community.cloudflare.com/t/automatic-platform-optimization-enabled-i-no-longer-see-visitor-ip/210241/45
Which has been tested by myself and multiple others not to break real ip checks anymore (security and other limitations kinda stuff).

So if the WP-Rocket team can look into an integration, that would be marvelous.

[CyberCr33p]

Lots of my clients need this.

[markllego]

Quick Question: May I know if I can use the Cloudflare APO with WP-Rocket (having the Cloudflare option disabled)? Thank you.

[PatrickD1985]

@markllego APO does work with the cloudflare integration off in WP-Rocket and using the official Cloudflare Wordpress plugin. (Currently that is the only way to use it with WP Rocket)

I am still in the process of seeing how much benefit this really gives. (Still not sure if it will yield any major advantage over our own setup, but I have not been able to test that yet due to the above mentioned issue). I will get back on that here in the upcoming week. From what I have seen so far it really seems to help with FCP, TTFB and the like.

But in the long run having WP Rocket support APO would be great whichever way you look at it.

For quick reference, after setting it up you should be able to see the below header on your requests;
x-via: speedwp/origin,cookie
That way you know it is being served through APO.

[markllego]

I can confirm that WP-Rocket is working fine with Cloudflare APO.

Header:

status: 200
x-cache: HIT
set-cookie: __cfduid=d47b0b49c7bf0f606541c64303647b1531602349346; expires=Mon, 09-Nov-20 17:02:26 GMT; path=/; domain=.teacherph.com; HttpOnly; SameSite=Lax; Secure
cf-cache-status: HIT
expires: Sat, 10 Oct 2020 14:53:22 GMT
vary: Accept-Encoding
x-via: speedwp/cache
last-modified: Sat, 10 Oct 2020 14:53:22 GMT
link: https://www.teacherph.com/wp-json/; rel="https://api.w.org/"
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?lkg-colo=22&lkg-time=1602349346"}],"group":"cf-nel","max_age":604800}
date: Sat, 10 Oct 2020 17:02:26 GMT
cf-ray: 5e01e8b6eeb40abc-NRT
cf-edge-cache: cache,platform=wordpress
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
nel: {"report_to":"cf-nel","max_age":604800}
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
content-encoding: br
age: 6532
strict-transport-security: max-age=31536000; includeSubDomains; preload
cf-request-id: 05b50fc65100000abc9ca4f200000001
server: cloudflare
cache-control: max-age=0
x-frame-options: SAMEORIGIN
content-type: text/html; charset=UTF-8
:status: 200

[webtrainingwheels]

Another request/inquiry: https://secure.helpscout.net/conversation/1309075581/202360?folderId=273766

[patche81]

Any update regarding this ? this is a major feature IMHO

[vmanthos]

Related ticket: https://secure.helpscout.net/conversation/1325982650/207868/

[patche81]

Folks we need this in CORE asap

[webtrainingwheels]

Related: https://secure.helpscout.net/conversation/1342363416/213604?folderId=4130683

[webtrainingwheels]

https://secure.helpscout.net/conversation/1337164234/211666?folderId=4130683

[tonydehnke]

How can we access/see what is in those urls @webtrainingwheels - as they require a login?

[webtrainingwheels]

@tonydehnke They are for our team's internal use only :)

[sejoker]

Hey, I'm an engineer working on Automatic Platform Optimization, let me know if you need any help with the integration. I know we haven't published docs on api.cloudflare.com for the feature. We are planning to finalize last parameters and publish it in the coming 2 weeks.

[viobru]

Related: https://secure.helpscout.net/conversation/1342665684/213710/

[sunnyt7]

I would like to see this as well. Running into a lot of issues using the two together - particularly with Nginx server-side and APO caching getting mixed up.

For example, if I use WP Rocket to minify CSS, the HTML often links to an outdated minified file (see: https://community.cloudflare.com/t/cloudflares-apo-with-wp-rockets-minified-css/225906)

Or changes to content won't be visible for quite some time, and requires a manual purge of all caches. Ideally, WP Rocket could purge them all at once like it can when not using APO.

[scorpionmit01]

Can we just turn off the WP-Rocket Cloudflare integration? Faster for no downside is a good one, but there appears to be an issue with CSS.

[vergosteak]

Hope we will see an integration soon ! It would be a major enhancement. I really want to use APO with WP rocket

[crystinutzaa]

@sejoker I am the developer of WP Rocket and I would love to get in touch and discuss about the integration with Automatic Platform Optimization. Do you have any initial documentation about it where I can see how should be integrated with WP Rocket?

You can reach me by email at: [email protected] and keep in touch on email 😄

[scorpionmit01]

Hi, thanks for reaching out! I am not an expert on Automatic Platform
Optimization, but Cloudflare is pushing it pretty hard. Happy to help and
give you my thoughts.

Requirements Document:

1. Website should have the Cloudflare Plugin v3.8.7 or higher
2. WP Rocket will need to let Cloudflare's plugin manage it
3. PHASE 2 (not MVP) - WP Rocket should have a "use Cloudflare Automatic
   Platform Optimization" button that sets up the Cloudflare Plugin for WP
   Rocket Sites

Here is where I see their Documentation:
https://support.cloudflare.com/hc/en-us/articles/360049822312-Understanding-Automatic-Platform-Optimization-APO-with-WordPress

The main thing I see is that Cloudflare will manage the Cloudflare plugins
based upon native hooks. WP Rocket would need to hop in prior to Cloudflare
and purge WP Rocket's cache
https://support.cloudflare.com/hc/en-us/articles/115002708027-Cloudflare-WordPress-Plugin-Automatic-Cache-Management-

I think that the main thing will be to effectively be running with Cache
Everything, at least as much as possible. You'd need to test, but I think
if you can hook into their cache clearing events first, we should be golden.

Alex

On Mon, Dec 14, 2020 at 5:42 AM Cristina Soponar [e-mail]
wrote:

@sejoker https://github.com/sejoker I am the developer of WP Rocket and
I would love to get in touch and discuss about the integration with
Automatic Platform Optimization. Do you have any initial documentation
about it where I can see how should be integrated with WP Rocket?

You can reach me by email at: [email protected] and keep in touch on
email 😄

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/wp-media/wp-rocket/issues/3180#issuecomment-744351656,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABYN4VJOS6X4SL7A4UD33ALSUXTX7ANCNFSM4SCREAKQ
.

--
Book a meeting with me
https://www.cloudhq.net/meeting/[email protected]

[viobru]

Related ticket: https://secure.helpscout.net/conversation/1370790935/224134?folderId=4130683

[luissimo]

Any progress on this? Much needed feature.. Can i help in any way?

[dreamjobma]

Please give us this feature asap

[Bistoman]

Hi @crystinutzaa , any update about integration between APO and WP-Rocket?

Do you have a WP-Rocket setup to act as a workaround to use APO till a final solution will be released by you (maybe disable CSS minification on WP-Rocket)?

thanks!

[webtrainingwheels]

another request: https://secure.helpscout.net/conversation/1400721336/232639?folderId=4130683

[camilamadronero]

another one: https://secure.helpscout.net/conversation/1400663145/232626/

[viobru]

Related ticket: https://secure.helpscout.net/conversation/1394893148/230946/

[sejoker]

I have published Automatic Platform Optimization API docs here:
https://api.cloudflare.com/#zone-settings-change-automatic-platform-optimization-for-wordpress-setting
https://api.cloudflare.com/#zone-settings-get-automatic-platform-optimization-for-wordpress-setting

Important notes:

• In order to enable APO all of the following value parameters requires to be set true: cf, wordpress, wp_plugin
• hostnames and cache_by_device_type parameters could be omitted, in that case existing values won't be overwritten.

[jazir5]

Yes please enable this, +1

[juricazuanovic]

Related ticket: https://secure.helpscout.net/conversation/1416058059/236883/

[webtrainingwheels]

https://secure.helpscout.net/conversation/1411063291/235529?folderId=377611

[patche81]

please enable this, +1

[jonasm82]

+1 for this

@crystinutzaa see above API doc links by sejoker

Thanks!

[ericger]

+1 we need this!

[scorpionmit01]

Need an update. It was implied this would be handled fast, but no update makes it look like never. Is it time to find a new solution?

[dreamjobma]

we need this asap, why is it taking so long, with no updates?

[Bistoman]

I agree, an update is needed asap. Next week I will start testing APO without wp-rocket to evaluate a drop of wp-rocket. I love wp-rocket but we can't wait for ever :( More than one month is gone without any info or plan

[GeekPress]

Hi guys,

We totally understand that Cloudflare APO is important for you. In the meantime, our team is actually fully focused on a new existing feature we are working on for a couple of months: Remove Unused CSS.

We can't provide any ETA when we could work on Cloudflare APO integration within our add-on.

Please note that WP Rocket is fully compatible with APO if you are using the official Cloudflare plugin instead of our add-on.

@wp-media/php Do we have enough info to make the grooming?

[scorpionmit01]

What does that mean?

If we turn off Cloudflare in WP Rocket and install the Cloudflare plugin it works fine?

That’s super relevant information that could have been shared when it was available.

Please verify that all we have to do is install their plugin and turn Cloudflare to off I. wP Rocket?

[GeekPress]

@scorpionmit01 Based on this comment https://github.com/wp-media/wp-rocket/issues/3180#issuecomment-705835317, yes it will work fine like that.

[markllego]

I have been using Cloudflare APO since it was launched last year. I can confirm that apo works fine with wp-rocket. Just disable the Cloudflare settings on wp-rocket and install the official Cloudflare plugin. To check if apo is working fine on your site you can use this curl command curl --request GET -I -H "Accept: text/html" https://www.yoursite.com

CF-Cache-Status: HIT
cf-apo-via: cache
cf-edge-cache: cache,platform=wordpress

[jhtjards]

WP Rocket + Cloudflare Plugin: While this does work most of the time, automatic cache purge is not reliable, so you sometimes end up with cloudflare-cached html referencing rocket's minified css that doesn't exist anymore.
So you end up with broken page layouts.

If you only edit your website yourself, your can purge cloudflare cache manually and often, if other editors are working on it, it easily gets messy.

[Basilakis]

@jhtjards U can put a hook or a cron job to do once in a while.

[GeekPress]

so you sometimes end up with cloudflare-cached html referencing rocket's minified css that doesn't exist anymore.

@jhtjards This kind of issues won't happen anymore once the fix #3564 will be released in WP Rocket 3.8.6

[crystinutzaa]

Reproduce the issue ✅

Cloudflare APO is not compatible with WP Rocket. 😅

Identify the root cause ✅

Issue is identified

Scope a solution ✅

APO is a paid plan of Cloudflare and this will display an error in case a customer tries to enable it on a free plan without having the APO option purchased.
1. Add a new option under Cloudflare settings: Enable APO which will enable / disable APO from Cloudflare.
- This operation will be done with PATCH zones/:zone_identifier/settings/automatic_platform_optimization
- https://api.cloudflare.com/#zone-settings-change-automatic-platform-optimization-for-wordpress-setting
- In case there is an error, it will be displayed to the user and will not enable / disable the APO functionality and the user will be redirected to Cloudflare website / account to fix the error (eg. Free plan without APO )
2. On upgrading WP Rocket to the new version, we will check if APO is enabled under Cloudflare (if credentials are ok) and automatically enable this option: https://api.cloudflare.com/#zone-settings-get-automatic-platform-optimization-for-wordpress-setting
3. On changing the Cloudflare credentials, we are going to recheck the APO option and display notices or automatically-enable APO option.
4. In WPMedia\Cloudflare\Subscriber::auto_purge() and WPMedia\Cloudflare\Subscriber::auto_purge_by_url() functions, in case APO is enabled automatically purge CF cache.

Estimate the effort ✅

Effort [M]

[crystinutzaa]

@wp-media/productrocket can you confirm the position where APO option needs to be placed under Cloudflare Addon and the exact text (field label + description).

Also we need to monitor https://github.com/wp-media/wp-rocket/issues/3564 and test with APO and when both issues are fixed and confirmed we shall get in touch with Cloudflare and move WP Rocket to Compatible Plugins. Right now WP Rocket is mentioned in the Plugins and Services which may cause issues with APO
https://support.cloudflare.com/hc/en-us/articles/360049822312-Understanding-Automatic-Platform-Optimization-APO-with-WordPress#h_01EQ44V6KRFM6Z0F06MM0EJGJ5

[GeekPress]

@crystinutzaa Can't this be done without an option? I mean, can we automatically detect that APO is activated on Cloudflare, and then do our magic for the compatibility?

[crystinutzaa]

@GeekPress yes, it can be done. However we need a transient to store when we got last time the APO option.
In case APO was enabled / disabled from Cloudflare directly, there will be a delay in WP Rocket to identify (the transient expiry date).
Another downside of this path, is that from WP Rocket, the customers cannot enable / disable APO option. They really need to go to Cloudflare account in order to enable / disable it (this could lead to a lot of confusion from customers - is enabled in Cloudflare, but in WP Rocket it seems is not working )
Also this will increase the no. of API calls to Cloudflare from WP Rocket.
Take into consideration these downsides of the automatic approach.

[scorpionmit01]

By edit the website you mean things that make JS and CSS changes?

Presumably updating content is t a problem? Either way this should be resolved in March 1?

Sent from my iPhone

On Feb 16, 2021, at 2:53 AM, jhtjards [e-mail] wrote:


WP Rocket + Cloudflare Plugin: While this does work most of the time, automatic cache purge is not reliable, so you sometimes end up with cloudflare-cached html referencing rocket's minified css that doesn't exist anymore.
So you end up with broken page layouts.

If you only edit your website yourself, your can purge cloudflare cache manually and often, if other editors are working on it, it easily gets messy.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

[crystinutzaa]

@scorpionmit01
Edit the website means that a post / page content was updated.
This means that WP Rocket cache is expired. If Cloudflare APO is enabled, when WP Rocket cache is cleaned, it will trigger also the cleaning of Cloudflare cache (to be in-sync), so there will not be any errors related to out-of-sync cache.

I would love to give you an ETA for this change, however I don't have a fixed deadline for it.

This task is groomed now and is waiting to be planned for a future sprint, however I cannot guarantee that it will be released before 1 March

[GeekPress]

@crystinutzaa Thanks for the extra info. It seems an option will be the best approach based on your last feedback.

Label
Automatic Platform Optimization (APO)

Description
Serve your entire site from Cloudflare edge network. More info.

@vmanthos @webtrainingwheels I ping you on to have your feedback. Does the wording sound good to you?

[patche81]

any ETA regarding this ?

[scorpionmit01]

Okay, there should be a release 3.8.6 tomorrow (March 1). It was indicated
that the fix for the 404 on CSS and JS should be in this? At that point,
how compatible should it be?

Is the risk:
Cloudflare might serve outdated content if you don't flush the cache?
or
Cloudflare might get 404s on updated content.

The former is a more manageable risk (update important content, flush the
cache, fix a typo, don't worry about it). Just trying to understand where
this appears to be with the 3.8.6 release. I can take responsibility for
having outdated content on my site if I dont flush the cache. It's far more
of a concern to have unpredictable 404s.

On Tue, Feb 23, 2021 at 9:03 AM patche81 [e-mail] wrote:

any ETA regarding this ?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wp-media/wp-rocket/issues/3180#issuecomment-784224189,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABYN4VPHVFV7AFDL6ZQO4ILTAOYRVANCNFSM4SCREAKQ
.

--
Book a meeting with me
https://www.cloudhq.net/meeting/[email protected]

[GeekPress]

any ETA regarding this ?

@patche81 We don't have any ETA to provide you yet until a milestone will be set on this issue.

@scorpionmit01 The risk will be > Cloudflare might serve outdated content if you don't flush the cache?

[ScottTravisHartley]

That is part of the issue the bigger issue is when WProcket's cache is cleared automatically and Cloudflare's is not. This can then cause a 404 error on minified CSS/JS files which can cause the layout to become out of sync.

[webtrainingwheels]

https://secure.helpscout.net/conversation/1443313354/244405?folderId=377611

[SimJoSt]

It would be great to have this big feature mentioned on the new roadmap for non-technical users as well: https://feedback.wp-rocket.me/roadmap

[dreamjobma]

I think they want to keep this for WP Rocket 4.0

[GeekPress]

@SimJoSt It hasn't been added to our new roadmap as we already know it's something we have to work on ;)

[SimJoSt]

@GeekPress that's true :)
But not every customer and interested party will read through the GitHub issues, so making it transparent to them, especially with an issue this big, would be very forthcoming 🙏

[GeekPress]

@SimJoSt You're right. Here we go: https://feedback.wp-rocket.me/16