Contao: Enabled Cache leads to "Invalid request token"

[bennyborn]

Affected version(s)
Contao 4.9.0

Description
When the Cache is enabled and the user tries to submit a form, Contao will throw an Invalid request token error even though the token is kept updated by /_contao/request_token_script.
This behaviour can be reproduced on demo.contao.org (enable cache in root page, add "Contact" form to "Home"-page, try to submit the form).

Also I had no success getting the cache to actually work at all with 4.9 (even on demo.contao.org).
Slow response times and response headers always state contao-cache: miss.

[Toflar]

Can you please explain how exactly I can reproduce this issue step by step? The /_contao/request_token_script is only needed when you enable the Always load from shared cache checkbox which was not part of your issue description. So I'm not sure if it is complete :)

[bennyborn]

By mentioning /_contao/request_token_script I wanted to say that I made 100% sure that the Request-Token in the form was indeed a valid one ;) The description is as simple as in my og post (reproduceable on demo.contao.org):

• Enable server side cache in root page
• Add "Contact" form to "Home"-page
• Try to submit the form

[asaage]

Can you please check if this is the same issue?
https://community.contao.org/de/showthread.php?77099-Erweiterung-umstellen-von-4-4-x-auf-4-8-x-Problem-mit-CSRF

I'm getting empty REQUEST_TOKEN and no csrf_https-contao_csrf_token cookie whenever it is an initial pageload (like for example a fresh private Tab) even without enabled cache in site-root.

[fritzmg]

• Enable server side cache in root page
• Add "Contact" form to "Home"-page
• Try to submit the form

I assume you mean the following configuration, when you say _"Enable server side cache in root page_":

• __Private cache timeout:__ 0
• __Shared cache timeout:__ 1 year
• __Always load from shared cache:__ disabled

I am unable to reproduce the problem this way. But mostly because for some reason Contao refuses to cache https://demo.contao.org/en/

I inserted the form on pages other then the index page, where the cache is actually fresh, and there were no problems whatsoever.

[bennyborn]

Could it have something to do with being logged into the backend?

I just tried the following settings on the page root

Of course added the "Contact" form to the page "Home".

Submitting this in the same browser session I was being logged into the backend with it did work without any problems. Trying to submit the form in an private session it instantly responded with "Invalid request token".

[fritzmg]

You need to test without being logged in. Having an active back end log in disables the shared cache.

I still cannot reproduce the problem. I now instead enabled the shared cache _and_ the private cache, inserted the contact form into https://demo.contao.org/en/content-elements.html, then made sure the page is cached and then submitted the form.

[bennyborn]

I'll try to do a screencast but unfortunately the demo seems to be offline currently.

Update
The behaviour is quite strange. In the last attempts I did not have to have Always load from shared cache checked to trigger the problem. Today I only got it triggered by doing so.

[Blog404DE]

The behaviour is quite strange. In the last attempts I did not have to have Always load from shared cache checked to trigger the problem. Today I only got it triggered by doing so.

the demo-page is running behind a CloudFlare reverse-proxy. I'm just guessing - but maybe that's the reason, why the demo-installation had a different behaviour than a "normal" local installation.

The described problem exists not only on 4.9.0 with Contao forms. The same happened if you have a Contao Extension/Module which uses the Request-Token {{REQUEST_TOKEN}}. As with Contao forms, the very first visit leads to empty request-tokens. The only way to avoid is adding the Contao module by insert-tag with the uncached-option {{insert_module::*|uncached}}. However, TL_JQUERY is ignored with this way, which can lead to malfunctions in some Contao modules.

[bennyborn]

We first noticed this problem in our own extension with 4.8. Back then the actual solution was to use the Insert-Tag instead of setting the request token using the PHP constant. What is puzzling me now is that in 4.9 I still get a new token on every page load since the cache does not work for me in 4.9 at all, but nonetheless I still receive Invalid request token ⁉️

[aschempp]

Please be aware that the mentioned behavior of not having a REQUEST_TOKEN is intended and correct.

1. if a user is a guest (no session cookie), a request token is not necessary because we do not need to prevent any cross-site requests (XSS attack)
2. by default, only if the user is a guest (as no session cookie), the file will be delivered from the cache. Therefore, a cache file never has no request token.
3. there is only one special case: Being logged in (having a cookie) and having the option _Always load from cache_ enabled. Because in this case, it is possible that the request token is checked (because there was a cookie when the page is submitted), but the page was delivered by the cache (because the checkbox was checked). In this case, the mentioned /_contao/request_token_script should take care of providing a REQUEST_TOKEN value after the page was fetched from the cache.

[bennyborn]

Please be aware that the mentioned behavior of not having a REQUEST_TOKEN is intended and correct.

I did not disagree on this one. There IS a REQUEST_TOKEN and it's still failing - that's the bug in there.

Edit: Nur für den Fall das wir hier ein Verständnisproblem haben oder mein Englisch nicht sauber war. Mit aktiviertem Cache (aktuell inklusive Always load from shared cache) bekommt das Formular einen REQUEST_TOKEN gesetzt, nach dem Abschicken gibt es aber die Meldung Invalid request token.

[ausi]

I was not able to reproduce this error on https://demo.contao.org/
Which browser are you using?

[bennyborn]

Which browser are you using?

@ausi I was using Chrome but the browser should really not matter...

[ausi]

@bennyborn Can you post a screencast on how to reproduce the error on https://demo.contao.org?

[asaage]

[ausi]

@asaage In your video the request token works as expected, see https://github.com/contao/contao/issues/1491#issuecomment-596981511

[asaage]

okay.
I have the same behaviour in a 4.4 install and the form-submission fails because neither cookie nor token are set. It might be a differnt problem though as caching is not involved at all.
I'll wait to the 4.9.2 release and see if that solves something.

[fritzmg]

@asaage what kind of problem are you experiencing? In your GIF there was none.

[asaage]

@fritzmg It's the infamous invalid-request-token errorscreen. But only if the user fills out a form on the first page he visits. On subsequent requests (or on reload) i have the token in the form and the csrf cookie present and everythink works as expected.

[Mynyx]

I can confirm this issue.

The /_contao/request_token_script starts a session for a guest, the guest also has a session cookie than. Clear all cookies and request https://demo.contao.org/_contao/request_token_script. The response contains the header set-cookie: PHPSESSID=[...]; path=/; secure; HttpOnly. Is that correct?

[bennyborn]

Still haven't found a good way to create a screencast but just recreated the issue again.

1. Enabled the caches

1. Added a form to the home page

1. Filled out and submitted the form in another browser session

@ausi Used Firefox this time ;)

[fritzmg]

This time I was able to reproduce the issue as well.

[Toflar]

I've debugged the issue and I know why this happens :)
The problem is that _contao/request_token_script is executed before the _contao/check_cookies image request. The check_cookies request will generate a new token and thus the one set by the request_token_script will be invalid now.
Not sure how we should fix this, I have to think about that.

[Mynyx]

@Toflar But another point is, if the session cookie would not be set by /_contao/request_token_script at least for guests the CSRF token would be ignored and especially also the cache would work correctly? Maybe there are two problems.

[Toflar]

Yeah there are actually more issues here, what I stated 8 minutes ago does not seem true.

[Toflar]

Okay, found the reason now. It's a service locator autowiring issue. The wrong token storage manager gets wired for the request token script endpoint. I'll see if I can provide a fix.