Yii2: CSRF fails if mbstring.encoding_translation set to On

[si3u]

What steps will reproduce the problem?

I config php.ini with information bellow:

[mbstring]
mbstring.language = Japanese
mbstring.internal_encoding = UTF-8
mbstring.encoding_translation = On

What is the expected result?

When I submit a form on website, I seen error

Error Bad Request (#400) Unable to verify your data submission

Additional info

| Q | A
| ---------------- | ---
| Yii version | 2.0.12
| PHP version | 7.0.20
| Operating system | Windows

[cebe]

looks like an error with CSRF instead of PHP encoding configuration. what exactly is the problem?

[yii-bot]

Thanks for posting in our issue tracker.
In order to properly assist you, we need additional information:

• When does the issue occur?
• What do you see?
• What was the expected result?
• Can you supply us with a stacktrace? (optional)
• Do you have exact code to reproduce it? Maybe a PHPUnit tests that fails? (optional)

Thanks!

_This is an automated comment, triggered by adding the label status:need more info._

[Ghiya]

I have the similar problem after updating to version 2.0.12.
In my case, as i find out by log traces, the issue is in CSRF cookie value validation.
Although the cookie is presented in $_COOKIES array as it has to be, its value fails validation in loadCookies method of yii\web\Request class.
And in that case, the required cookie wont be presented in $_cookies array property and new token will be generated in getCsrfToken method according to condition, in its turn method validateCsrfToken will get the different $trueToken value and validation will fail.

[Ghiya]

More specificly, hashed CSRF cookie data value $calculatedHash differs form its stored hash value $hash, so validation fails in yii\base\Security.
As i checked, in my case it happens in updated version, in the previous version CSRF cookie data value differs from the updated one and data validation works as it should.

In log traces it looks like:

yii\base\Security::validateData

Validating CSRF cookie : d4ae7ff74b0168747e431594a507418878be92ecdb3d8b1514c4bb92556085e9a:2:{i:0;s:13:"_csrf-backend";i:1;s:32:"F??R???֌?q???g8??`?????|yn?";}

yii\base\Security::validateData

Cookie `$hash` value:
d4ae7ff74b0168747e431594a507418878be92ecdb3d8b1514c4bb92556085e9

yii\base\Security::validateData

Cookie `$pureData` value:
a:2:{i:0;s:13:"_csrf-backend";i:1;s:32:"F??R???֌?q???g8??`?????|yn?";}

yii\base\Security::validateData

CSRF cookie `$hash` value differs from calculated `hash_hmac($pureData)` value
d4ae7ff74b0168747e431594a507418878be92ecdb3d8b1514c4bb92556085e9 : 91ff93ed84df7a2a56067cf085ac3c6d53aba1f60ad99914d6dd85ea0df860c4
returns : false

yii\web\Request::loadCookies

The value for the `_csrf-backend` is not validated.

[samdark]

@Ghiya do you have AJAX/redirects involved?

[si3u]

@samdark I use fresh source install via composer. Then, I submit to Register, Login form.

[samdark]

By fresh source you bean basic project template?

[si3u]

I use advanced project template.

[Ghiya]

@samdark It happens in any kind of request with CSRF validation and I have no AJAX or redirects involved.
For example, POST request from user authenthication form.
I'm using advanced project template too.
Seems like the issue is in cookie value control hash, cause data validation fails at the moment of its checking. However, my knowledge of the framework is not enough to draw certain conclusions.

[samdark]

Could you try basic template in the same environment to be sure it's not configuration specific?

[si3u]

@samdark Because I config environment in the php.ini file with information bellow

 [mbstring]
 mbstring.language = Japanese
 mbstring.internal_encoding = UTF-8
 mbstring.encoding_translation = On

Can you try it? I sure you will have to see this problem

[si3u]

I confirmed on basic template in the same environment, this problem appear.

[si3u]

$_COOKIE

| Name | Value
| ---------------- | ---
|_csrf |'83bc7498c79b4024059ab35ce118e844cbc5dc0c7b8bce92a705c1b67a216937a:2:{i:0;s:5:\"_csrf\";i:1;s:32:\"Q?*?Auj??h?(碻\0o? 9V????C\";}'

[samdark]

@Ghiya do you change mbstring settings?

[si3u]

@samdark I just try comment two lines

 [mbstring]
 //mbstring.language = Japanese
 //mbstring.internal_encoding = UTF-8
 mbstring.encoding_translation = On

I think because this config

mbstring.encoding_translation = On

[si3u]

[Ghiya]

@samdark In my case, setting mbstring.encoding_translation to the off value throws exception InvalidParamException for json encoding, with standard message Malformed UTF-8 characters, possibly incorrectly encoded.

Here is actual mbstring settings:

[samdark]

@Ghiya your case is different. Try turning off func_overload.

[samdark]

@si3u what's on the screenshot is OK. Value in the cookie is masked w/ HMAC.

[cebe]

I had one case where people were complaining about not being able to log cookie values because db target does not allow invalid UTF8 to be stored in pgsql table. maybe we should base64encode the value to avoid invalid chars?

[samdark]

Wouldn't hurt, I think.

[samdark]

mbstring.encoding_translation = on causes it.

[cebe]

should be fixed by #14590