Magento2: Upgrade to 2.2.1 from 2.1.2 fails with "Unable to serialize value" error

[sanjayjethva]

I have tried to upgrade to 2.2.1 from 2.1.2 version via composer upgrade.
Upgrade was successful but when i try to open website frontend or backend, its showing "Unable to serialize value" error.

Preconditions

1. Magento 2.2.1
2. PHP 7.0

Steps to reproduce

1. Upgrade 2.1.2 version to latest 2.2.1 via composer
2. run upgrade command and deploy command

Expected result

1. All good but when we try to open website it throwing error There has been an error processing your request

Actual result

{"0":"Unable to serialize value.","1":"#0 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/Translate.php(494): Magento\\Framework\\Serialize\\Serializer\\Json->serialize(Array)\n#1 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/Translate.php(190): Magento\\Framework\\Translate->_saveCache()\n#2 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/App\/Area.php(244): Magento\\Framework\\Translate->loadData(NULL, false)\n#3 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/App\/Area.php(215): Magento\\Framework\\App\\Area->_initTranslate()\n#4 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/App\/Area.php(142): Magento\\Framework\\App\\Area->_loadPart('translate')\n#5 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/View\/DesignLoader.php(55): Magento\\Framework\\App\\Area->load('translate')\n#6 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/App\/Action\/Plugin\/Design.php(48): Magento\\Framework\\View\\DesignLoader->load()\n#7 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/Interception\/Interceptor.php(121): Magento\\Framework\\App\\Action\\Plugin\\Design->beforeDispatch(Object(Magento\\Cms\\Controller\\Index\\Index\\Interceptor), Object(Magento\\Framework\\App\\Request\\Http))\n#8 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/Interception\/Interceptor.php(153): Magento\\Cms\\Controller\\Index\\Index\\Interceptor->Magento\\Framework\\Interception\\{closure}(Object(Magento\\Framework\\App\\Request\\Http))\n#9 \/var\/www\/vhosts\/demo.com\/eiselec\/generated\/code\/Magento\/Cms\/Controller\/Index\/Index\/Interceptor.php(39): Magento\\Cms\\Controller\\Index\\Index\\Interceptor->___callPlugins('dispatch', Array, Array)\n#10 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/App\/FrontController.php(55): Magento\\Cms\\Controller\\Index\\Index\\Interceptor->dispatch(Object(Magento\\Framework\\App\\Request\\Http))\n#11 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/Interception\/Interceptor.php(58): Magento\\Framework\\App\\FrontController->dispatch(Object(Magento\\Framework\\App\\Request\\Http))\n#12 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/Interception\/Interceptor.php(138): Magento\\Framework\\App\\FrontController\\Interceptor->___callParent('dispatch', Array)\n#13 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/module-store\/App\/FrontController\/Plugin\/RequestPreprocessor.php(94): Magento\\Framework\\App\\FrontController\\Interceptor->Magento\\Framework\\Interception\\{closure}(Object(Magento\\Framework\\App\\Request\\Http))\n#14 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/Interception\/Interceptor.php(135): Magento\\Store\\App\\FrontController\\Plugin\\RequestPreprocessor->aroundDispatch(Object(Magento\\Framework\\App\\FrontController\\Interceptor), Object(Closure), Object(Magento\\Framework\\App\\Request\\Http))\n#15 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/module-page-cache\/Model\/App\/FrontController\/BuiltinPlugin.php(73): Magento\\Framework\\App\\FrontController\\Interceptor->Magento\\Framework\\Interception\\{closure}(Object(Magento\\Framework\\App\\Request\\Http))\n#16 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/Interception\/Interceptor.php(135): Magento\\PageCache\\Model\\App\\FrontController\\BuiltinPlugin->aroundDispatch(Object(Magento\\Framework\\App\\FrontController\\Interceptor), Object(Closure), Object(Magento\\Framework\\App\\Request\\Http))\n#17 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/Interception\/Interceptor.php(153): Magento\\Framework\\App\\FrontController\\Interceptor->Magento\\Framework\\Interception\\{closure}(Object(Magento\\Framework\\App\\Request\\Http))\n#18 \/var\/www\/vhosts\/demo.com\/eiselec\/generated\/code\/Magento\/Framework\/App\/FrontController\/Interceptor.php(26): Magento\\Framework\\App\\FrontController\\Interceptor->___callPlugins('dispatch', Array, NULL)\n#19 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/App\/Http.php(135): Magento\\Framework\\App\\FrontController\\Interceptor->dispatch(Object(Magento\\Framework\\App\\Request\\Http))\n#20 \/var\/www\/vhosts\/demo.com\/eiselec\/vendor\/magento\/framework\/App\/Bootstrap.php(256): Magento\\Framework\\App\\Http->launch()\n#21 \/var\/www\/vhosts\/demo.com\/eiselec\/index.php(39): Magento\\Framework\\App\\Bootstrap->run(Object(Magento\\Framework\\App\\Http))\n#22 {main}","url":"\/","script_name":"\/index.php"}

[magento-engcom-team]

@sanjayjethva, thank you for your report.
We've created internal ticket(s) MAGETWO-84392 to track progress on the issue.

[fwolfst]

We have seen this, too. The internal serialization mechanism changed in 2.2 . Where before php serialization was used, now it favors JSON format.
People use custom scripts to convert stuff, but an ugly workaround can also do the job until its nicely fixed. This will involve guessing in what format values are "serialized". It probably comes with performance drawbacks, but sure it performs better than a broken shop, ...

See Edmunds answer on https://magento.stackexchange.com/questions/194010/magento-2-2-unable-to-unserialize-value or Siarhey Uchukhlebau s information on https://magento.stackexchange.com/questions/196216/magento-2-2-error-unable-to-unserialize-value .

[orlangur]

@magento-engcom-team could you clarify what was reproduced here? Vanilla Magento installation should upgrade just fine, third party extensions must be adopted according to release notes.

People use custom scripts to convert stuff, but an ugly workaround can also do the job until its nicely fixed.

@fwolfst there is nothing to "fix" here. You can see stuff properly converted in core modules and vendors of respective extensions should do it the same way.

There is no need to guess: you always know which serialization mechanism was used for a particular data.

[fwolfst]

@orlangur I can agree to that. In our case we saw some errors on imported orders, but its possible that we used old data-migration-tool config files (where the conversion isnt done) or these attributes came from an extension. Other cases where it failed were definately in extension-land. Lets not argue whether this (or actually: "what") has to be "fix"ed. The errors stem from a misinterpretation of how the data has to be (un-)serialized. Magento changed its default on this matter. If code parts disagree with magentos new defaults at least two users on stackoverflow found out how to get the error messages go away and have a working system, which is crucial for some users.

[orlangur]

Any 3rd party modules must be updated according to http://devdocs.magento.com/guides/v2.2/release-notes/backward-incompatible-changes.html#database-data-format-changes before using with Magento 2.2.

at least two users on stackoverflow found out how to get the error messages go away

I didn't say a word it is not possible technically. The problem here is that guessing serialized format is not a reliable solution and not acceptable in core.

"get the error messages go away" is not the right way to go, problems in custom code must be fixed instead.

[fwolfst]

The provided error message includes translation and cache. @sanjayjethva Did you try flushing the cache? :)

[sanjayjethva]

Hi @fwolfst , I have tried flush cache. but its not working.

[orlangur]

@sanjayjethva do you have any third party modules installed? It would be worth trying to disable them for the sake of upgrade and then enable back one-by-one to find a problematic module.

[sanjayjethva]

Hi All,
Issue is due to translation csv files.

[edcg]

@sanjayjethva I am having the same issue. How did you solve it?

[liquidia]

same issue while updating theme

[Sureshchikani]

I have same issue with my custom extension which is working fine with previous version but not working in 2.2.2. I am trying to find out the solution but still not get any solution.

If any one know about the issue, let me know the solution.

[orlangur]

@Sureshchikani release notes containing backward-incompatible changes information is a good starting point. There is also quite detailed http://devdocs.magento.com/guides/v2.2/ext-best-practices/tutorials/serialized-to-json-data-upgrade.html tutorial.

[speedupmate]

@magento-engcom-team @orlangur

In core_config_data where serialized array values are allowed/used trough m2 own config classes and currently upgrade does not handle the conversion . It should cause it can't work otherwise as it can't read the configuration in admin fields.

all serialized config field values in core_config_data that are defined as

<backend_model>Magento\Config\Model\Config\Backend\Serialized\ArraySerialized</backend_model>
need to be converted on m2 upgrade and back on downgrade , right now it's not done and admin just fails with hard exception.

It's not realistic to expect that 3rd party extension (all of them) need to have a upgrade script that handles one time core_config_data field value conversion in case of upgrade and back on downgrade to either higher or lower magento version.

sure that 3rd party needs to cover the change in serialiser and convert in their own schema parts but not in core functionality.

[orlangur]

need to be converted on m2 upgrade and back on downgrade

@speedupmate are you sure downgrades are supported at all?

It's not realistic to expect that 3rd party extension (all of them) need to have a upgrade script that handles one time core_config_data field value conversion

Surely, core_config_data table needs to be fully converted in core. But I believe it's already implemented properly.

[speedupmate]

@orlangur

if downgrades are not supported you can't really deploy and roll back changes in live between those versions. Without converting those values on your own.

Surely, core_config_data table needs to be fully converted in core. But I believe it's already implemented properly.

no its not done , if it is can you reference a part of code that does this? Currently upgrades are failing when serialized arrays are present in core_config_data

[orlangur]

if downgrades are not supported you can't really deploy and roll back changes in live between those versions

You are not supposed to literally downgrade version. However, you can swap two databases and thus return previous state.

Surely, core_config_data table needs to be fully converted in core. But I believe it's already implemented properly.

@speedupmate, actually, I was wrong here. We cannot convert the whole table as some rows are serialized and some not. Each extension should care about its own serialized data, examples: \Magento\Braintree\Setup\UpgradeData::convertSerializedDataToJson, \Magento\CatalogInventory\Setup\UpgradeData::convertSerializedDataToJson.

[speedupmate]

@orlangur

thanks for pointing out examples.

My point being that if m2 offers a way to store serialized arrays out of the box

<backend_model>Magento\Config\Model\Config\Backend\Serialized\ArraySerialized</backend_model>
then it should also be responsible for converting the values it offered to save as serialized arrays when the backend implementation changes.

PS: upgrade scripts are pretty simple, i did not dig deep enough but first glance detection of what they are changing is completely missing eq should they convert or not or see if double conversion is not happening.

[orlangur]

@speedupmate they are not supposed to detect anything, that's why you need to assure you really convert serialized value at first place. It is guaranteed that upgrade script is executed only once thus double conversion won't happen.

offers a way to store serialized arrays out of the box
<backend_model>Magento\Config\Model\Config\Backend\Serialized\ArraySerialized</backend_model>
then it should also be responsible for converting the values

Technically it is possible to find all such fields from module XMLs and convert them, but such mechanism would be too complicated and fragile. Also, don't forget that Magento is highly customizable and there is no guarantee in particular store such model is not rewritten or there is no additional logic attached to stored data.

More specific, this is Case 6 of http://devdocs.magento.com/guides/v2.2/release-notes/backward-incompatible-changes.html#database-data-format-changes which is clearly documented and there is nothing to fix from core perspective.

[speedupmate]

@orlangur let me argue a bit

Technically it is possible to find all such fields from module XMLs and convert them,

this is a simple foreach loop to find them by backend, since this is a core backend class its also a expected thing to cover those

be too complicated and fragile

try and catch to decode with json see if you get a valid config array back , if it fails try and catch with serialized array if it fails you output a reasonable error message to indicate that the format is not valid for this backend. Additional validation can be introduced etc.

there is no guarantee in particular store such model is not rewritten

what if it is not overwritten :D wonder what the odds are here vs reality. Currently m2 fails with hard exception without outputting a error message that is easy to get and sort out. That certainly is a better thing than the chance of this class being overwritten. The set of functionality that causes the issue does not even exist in older version.

More specific, this is Case 6

I agree and already provided a upgrade script to our customers just to move on , however this case does not have to exist at all since considered functionality is covered with core functionality that all extension providers can relay on.

[m2-assistant[bot]]

Hi @engcom-Charlie. Thank you for working on this issue.
Looks like this issue is already verified and confirmed. But if you want to validate it one more time, please, go though the following instruction:

• [ ] 1. Add/Edit Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [x] 2. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!

• [ ] 3. If the issue is not relevant or is not reproducible any more, feel free to close it.

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[engcom-Charlie]

Hello @sanjayjethva

Thank you for contribution and collaboration!

We are not able to reproduce this issue on the lates 2.3-develop branch by provided steps.

We are closing this issue due to _branch 2.2-develop was closed and Magento no longer accepts pull requests and issues to the v2.2 release lines to focus all development efforts on v2.3._
See Accepted pull requests and ported code for more details

If you still faced this issue on 2.3 please create a new Issue with all required details according to Issue reporting guidelines