Cms: Craft migration throws "Changes to the project config are not possible while in read-only mode" error

[khalwat]

Description

If you have:

        'allowAdminChanges' => false,

...in your general.php Craft will throw an error when you attempt to run pending migrations (whether via the AdminCP or via the CLI).

It looks like this migration is the culprit:

https://github.com/craftcms/cms/blob/3.1/src/migrations/m190112_124737_fix_user_settings.php#L27

Not Supported: Changes to the project config are not possible while in read-only mode.

Migration: craft\migrations\m190112_124737_fix_user_settings

Output:

Exception: Changes to the project config are not possible while in read-only mode. (/home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/services/ProjectConfig.php:301)
#0 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/migrations/m190112_124737_fix_user_settings.php(27): craft\services\ProjectConfig->set('users', Array)
#1 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/db/Migration.php(56): craft\migrations\m190112_124737_fix_user_settings->safeUp()
#2 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/db/MigrationManager.php(243): craft\db\Migration->up(true)
#3 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/db/MigrationManager.php(163): craft\db\MigrationManager->migrateUp(Object(craft\migrations\m190112_124737_fix_user_settings))
#4 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/services/Updates.php(215): craft\db\MigrationManager->up()
#5 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/controllers/BaseUpdaterController.php(445): craft\services\Updates->runMigrations(Array)
#6 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/controllers/UpdaterController.php(208): craft\controllers\BaseUpdaterController->runMigrations(Array, 'restore-db')
#7 [internal function]: craft\controllers\UpdaterController->actionMigrate()
#8 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#9 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/yiisoft/yii2/base/Controller.php(157): yii\base\InlineAction->runWithParams(Array)
#10 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/web/Controller.php(109): yii\base\Controller->runAction('migrate', Array)
#11 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/yiisoft/yii2/base/Module.php(528): craft\web\Controller->runAction('migrate', Array)
#12 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/web/Application.php(297): yii\base\Module->runAction('updater/migrate', Array)
#13 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/web/Application.php(683): craft\web\Application->runAction('updater/migrate')
#14 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/craftcms/cms/src/web/Application.php(223): craft\web\Application->_processUpdateLogic(Object(craft\web\Request))
#15 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/vendor/yiisoft/yii2/base/Application.php(386): craft\web\Application->handleRequest(Object(craft\web\Request))
#16 /home/forge/staging.theseasidestyle.com/releases/6b32376d04fc1fc610410292dec1b4fcc8753aee/web/index.php(21): yii\base\Application->run()
#17 {main}

Steps to reproduce

1.
2.

Additional info

• Craft version:
• PHP version:
• Database driver & version:
• Plugins & versions:

[smcyr]

Same problem here.
I do the migration from Craft 3.0.X to Craft 3.1 in a local environment and the project.yaml is generated.
After that, I commit everything and Jenkins push the code to a server and run migrations in command line php craft migrate/all.
Then, because I have allowAdminChanges to false on that environment, it fails to apply the migrations because it tries to update the project.yaml.

[brandonkelly]

@khalwat That m190112_124737_fix_user_settings migration is behaving correctly; it has a check to ensure it is only making the change if the incoming project.yaml hasn’t already been updated to/past the 3.1.15 schema version (which is when the migration was added):

https://github.com/craftcms/cms/blob/c4fafa3d89f915f6dc86ce51050ca30107264166/src/migrations/m190112_124737_fix_user_settings.php#L18-L23

So in theory the only way you’d get that exception is if you run this migration for the first time on an environment that has disableAdminChanges set to false (e.g. your dev environment). Which would be expected behavior; you shouldn’t set that setting to false on environments that will be modifying the project config.

@smcyr If you were getting this on an environment that already theoretically had the project config changes in place (and useProjectConfigFile is set to true), search for the error in your logs (storage/logs/) and see which migration you were seeing it on.

[smcyr]

@brandonkelly What would be the procedure to update my staging/production environments to Craft 3.1 ?

Here is the steps I did:

1. Update from Craft 3.0 to Craft 3.1 in local
2. Set useProjectConfigFile to true to generate the project.yaml file (which has the schemaVersion to 3.1.20)
3. Set allowAdminChanges to false for staging and production
4. Commit and push everything in GIT (including the project.yaml file)
5. Jenkins deploying the code on staging and running php craft migrate/all

But from the logs, maybe the problem lies with my code or Seomatic. Here are some parts of the log:

[17-Jan-2019 15:35:46 UTC] An Error occurred while handling another error:
craft\errors\SiteNotFoundException: No primary site exists in /var/www/website/vendor/craftcms/cms/src/services/Sites.php:496
Stack trace:
#0 /var/www/website/vendor/craftcms/cms/src/services/Sites.php(443): craft\services\Sites->getPrimarySite()
#1 /var/www/website/vendor/craftcms/cms/src/web/View.php(690): craft\services\Sites->getCurrentSite()
#2 /var/www/website/vendor/craftcms/cms/src/web/View.php(592): craft\web\View->resolveTemplate('_errors/500')
...
Previous exception:
craft\errors\SiteNotFoundException: No primary site exists in /var/www/website/vendor/craftcms/cms/src/services/Sites.php:496
Stack trace:
#0 /var/www/website/vendor/craftcms/cms/src/services/Sites.php(443): craft\services\Sites->getPrimarySite()
#1 /var/www/website/modules/website/WebsiteModule.php(134): craft\services\Sites->getCurrentSite()
#2 /var/www/website/modules/website/WebsiteModule.php(50): modules\website\WebsiteModule->redirectToLanguage()
...
[17-Jan-2019 15:36:08 UTC] An Error occurred while handling another error:
TypeError: Return value of craft\models\Info::getName() must be of the type string, null returned in /var/www/website/vendor/craftcms/cms/src/models/Info.php:110
Stack trace:
#0 /var/www/website/vendor/yiisoft/yii2/base/Component.php(139): craft\models\Info->getName()
#1 /var/www/website/vendor/nystudio107/craft-seomatic/src/models/MetaSiteVars.php(148): yii\base\Component->__get('name')
#2 /var/www/website/vendor/yiisoft/yii2/base/BaseObject.php(109): nystudio107\seomatic\models\MetaSiteVars->init()
...
Previous exception:
PDOException: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'dateDeleted' in 'where clause' in /var/www/website/vendor/yiisoft/yii2/db/Command.php:1258
Stack trace:
#0 /var/www/website/vendor/yiisoft/yii2/db/Command.php(1258): PDOStatement->execute()
#1 /var/www/website/vendor/yiisoft/yii2/db/Command.php(1148): yii\db\Command->internalExecute('SELECT `id`, `n...')
#2 /var/www/website/vendor/yiisoft/yii2/db/Command.php(399): yii\db\Command->queryInternal('fetchAll', NULL)
...
Next yii\db\Exception: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'dateDeleted' in 'where clause'
The SQL being executed was: SELECT `id`, `name`, `uid`
FROM `craft_sitegroups`
WHERE `dateDeleted` IS NULL
ORDER BY `name` in /var/www/website/vendor/yiisoft/yii2/db/Schema.php:664
Stack trace:
#0 /var/www/website/vendor/yiisoft/yii2/db/Command.php(1263): yii\db\Schema->convertException(Object(PDOException), 'SELECT `id`, `n...')
#1 /var/www/website/vendor/yiisoft/yii2/db/Command.php(1148): yii\db\Command->internalExecute('SELECT `id`, `n...')
#2 /var/www/website/vendor/yiisoft/yii2/db/Command.php(399): yii\db\Command->queryInternal('fetchAll', NULL)

Thanks for the help!

[smcyr]

I tried to do the upgrade in local from Craft 3.0 to 3.1 with the project.yaml previously generated and the allowAdminChanges to false and I had this error:

> add column config mediumtext NULL DEFAULT NULL AFTER `maintenance` to table {{%info}} ... done (time: 0.569s)
    > add column configMap mediumtext NULL DEFAULT NULL AFTER `config` to table {{%info}} ... done (time: 1.438s)
    > renaming project.yaml to project.yaml.NwOCsU7stz ... Exception: Changes to the project config are not possible while in read-only mode. (D:\projects\website\website-craft\vendor\craftcms\cms\src\services\ProjectConfig.php:301)
#0 D:\projects\website\website-craft\vendor\craftcms\cms\src\migrations\m180521_173000_initial_yml_and_snapshot.php(44): craft\services\ProjectConfig->set('dateModified', 1547826257)
#1 D:\projects\website\website-craft\vendor\craftcms\cms\src\db\Migration.php(56): craft\migrations\m180521_173000_initial_yml_and_snapshot->safeUp()
#2 D:\projects\website\website-craft\vendor\craftcms\cms\src\db\MigrationManager.php(243): craft\db\Migration->up(true)
#3 D:\projects\website\website-craft\vendor\craftcms\cms\src\db\MigrationManager.php(163): craft\db\MigrationManager->migrateUp(Object(craft\migrations\m180521_173000_initial_yml_and_snapshot))
#4 D:\projects\website\website-craft\vendor\craftcms\cms\src\services\Updates.php(215): craft\db\MigrationManager->up()
#5 D:\projects\website\website-craft\vendor\craftcms\cms\src\console\controllers\MigrateController.php(243): craft\services\Updates->runMigrations(Array)
#6 [internal function]: craft\console\controllers\MigrateController->actionAll()
#7 D:\projects\website\website-craft\vendor\yiisoft\yii2\base\InlineAction.php(57): call_user_func_array(Array, Array)
#8 D:\projects\website\website-craft\vendor\yiisoft\yii2\base\Controller.php(157): yii\base\InlineAction->runWithParams(Array)
#9 D:\projects\website\website-craft\vendor\yiisoft\yii2\console\Controller.php(148): yii\base\Controller->runAction('all', Array)
#10 D:\projects\website\website-craft\vendor\yiisoft\yii2\base\Module.php(528): yii\console\Controller->runAction('all', Array)
#11 D:\projects\website\website-craft\vendor\yiisoft\yii2\console\Application.php(180): yii\base\Module->runAction('migrate/all', Array)
#12 D:\projects\website\website-craft\vendor\craftcms\cms\src\console\Application.php(99): yii\console\Application->runAction('migrate/all', Array)
#13 D:\projects\website\website-craft\vendor\yiisoft\yii2\console\Application.php(147): craft\console\Application->runAction('migrate/all', Array)
#14 D:\projects\website\website-craft\vendor\yiisoft\yii2\base\Application.php(386): yii\console\Application->handleRequest(Object(craft\console\Request))
#15 D:\projects\website\website-craft\craft(22): yii\base\Application->run()
#16 {main}
Exception 'craft\errors\MigrateException' with message 'An error occurred while migrating Craft CMS.'

in D:\projects\website\website-craft\vendor\craftcms\cms\src\services\Updates.php:231

Stack trace:
#0 D:\projects\website\website-craft\vendor\craftcms\cms\src\console\controllers\MigrateController.php(243): craft\services\Updates->runMigrations(Array)
#1 [internal function]: craft\console\controllers\MigrateController->actionAll()
#2 D:\projects\website\website-craft\vendor\yiisoft\yii2\base\InlineAction.php(57): call_user_func_array(Array, Array)
#3 D:\projects\website\website-craft\vendor\yiisoft\yii2\base\Controller.php(157): yii\base\InlineAction->runWithParams(Array)
#4 D:\projects\website\website-craft\vendor\yiisoft\yii2\console\Controller.php(148): yii\base\Controller->runAction('all', Array)
#5 D:\projects\website\website-craft\vendor\yiisoft\yii2\base\Module.php(528): yii\console\Controller->runAction('all', Array)
#6 D:\projects\website\website-craft\vendor\yiisoft\yii2\console\Application.php(180): yii\base\Module->runAction('migrate/all', Array)
#7 D:\projects\website\website-craft\vendor\craftcms\cms\src\console\Application.php(99): yii\console\Application->runAction('migrate/all', Array)
#8 D:\projects\website\website-craft\vendor\yiisoft\yii2\console\Application.php(147): craft\console\Application->runAction('migrate/all', Array)
#9 D:\projects\website\website-craft\vendor\yiisoft\yii2\base\Application.php(386): yii\console\Application->handleRequest(Object(craft\console\Request))
#10 D:\projects\website\website-craft\craft(22): yii\base\Application->run()
#11 {main}

So, the culprit is this migration m180521_173000_initial_yml_and_snapshot.php. Maybe it shouldn't do anything if the project.yaml exist or if allowAdminChanges is set to false.

[smcyr]

If I comment the code that changes the project.yaml in m180521_173000_initial_yml_and_snapshot.php, I have then an error in m181130_143040_fix_schema_version.php which also modify the project.yaml.

Should I set useProjectConfigFile to false, run the migrations on my staging/production environments and after that, set useProjectConfigFile to true ? Or should I set allowAdminChanges to true for staging/production, after run the migrations staging/production and finally change allowAdminChanges back to false?

[brandonkelly]

The initial_yml_and_snapshot migration wasn’t written with the allowAdminChanges config setting in mind. I would recommend that you don’t set that to false on any environments until _after_ they’ve been updated to Craft 3.1.

[khalwat]

So in theory the only way you’d get that exception is if you run this migration for the first time on an environment that has disableAdminChanges set to false (e.g. your dev environment). Which would be expected behavior; you shouldn’t set that setting to false on environments that will be modifying the project config.

In our case, we run migrations on staging/production when we deploy, and in those environments, allowAdminChanges is set to false.

It sounds like we're potentially running into scenarios where we have environments where migrations really should be run, but we _also_ want allowAdminChanges set to false because while database changes are fine in those environments, we don't want project.yaml changed in those environments, because it is checked into git.

I feel like I'm missing something? It seems a pretty common scenario where we have a live environment where we don't want admin changes, and we don't want project.yaml changed, but we still want db migrations to happen?

Are you saying if we had run said migrations locally, it would have updated project.yaml in such a way that the migrations then would run successfully on staging/production?

[brandonkelly]

@khalwat Yeah that is the expected workflow. It’s just that one initial_yml_and_snapshot migration (the one that creates your project.yaml file initially) that isn’t expecting allowAdminChanges to be disabled yet. (And as of 3.1.2.1, it also suffered from a bug if a project.yaml file already existed, which it also doesn’t support yet.)

[brandonkelly]

Just added a warning to the allowAdminChanges config setting docs to clear this up.

[smcyr]

Just wanted to let you know that I migrated all my environments to 3.1 successfully with allowAdminChanges to true and changed it back to false after :) Thanks!

[darylknight]

I'm still getting Changes to the project config are not possible while in read-only mode on projects which have already been upgraded to 3.1.

Shouldn't it basically stop you making admin changes in the CMS, but allow updates that were committed and deployed? It's kind of weird/annoying that it won't let you update the file even though the changes came from the local machine.

[brandonkelly]

@darylknight Are you pushing Matrix changes? If so you might be getting bit by #3695.

If it’s something else, please post a new issue that includes the stack trace from the logs leading up to the error.

[darylknight]

Yes; several / most of them are Matrix changes. Hopefully that release will solve the problem - thanks Brandon!

[bymayo]

I just had this issue still, and hit a problem where I couldn't resync the project.yaml changes.

To resolve I changed allowAdminChanges to true and then cleared cache in the Utilities and refreshed the CP and it asked me to sync again.

[brandonkelly]

@bymayo Can you try comparing the server’s config/project.yaml file with a local one using a diffing tool like FileMerge or Kaleidescope? What changed (if anything)?

[aj-adl]

Also running into this issue... to me it seems like this flies in the face of what people wanted project.yaml for??

Ie: make changes in a dev environment where allowAdminChanges will almost definitely be true, then deploy those changes to staging and prod environments where allowAdminChanges is false, and prevent changes to project.yaml in staging and production because it's managed in git?

[andris-sevcenko]

Pretty sure this has been fixed for a while. Or, at least, the various things that could have caused is.

If you're experiencing this, can you send your pre-update DB, composer.lock and composer.json files to [email protected] and reference this thread so we can look into this for you?

[BenParizek]

We're running into this now as well and after scanning the above I'm not sure I understand what the appropriate solution is yet.

Here's an open ticket we have with this issue on Sprout Forms:
https://github.com/barrelstrength/craft-sprout-forms/issues/312

Here's the migration causing the issue:
https://github.com/barrelstrength/craft-sprout-forms/blob/v3/src/migrations/m190410_000000_add_payload_forwarding_to_integration.php

As I'm understanding this, our migration is trying to do to things.

1) Add new tables to the database
2) Update the plugin settings to set defaults, which requires we update the Project Config

I believe the recommended Project Config workflow is to disable allowAdminChanges in all environments except locally. In which case, we'd say everything is working as expected and the user should just run migrations locally and push updates to the Project Config.

However, the changes to the Project Config are meaningless until we've added the new database tables to the database. Those new db tables don't have any relationship to the Project Config and the migration to add them would need to be run in all environments that have allowAdminChanges disabled. So it would seem, in this scenario, we'd have to recommend to disable the Project Config to run the migrations and then re-enable it.

Is there a workflow that I'm missing here where we don't have to give the user multi-step instructions to get things upgraded properly?

[andris-sevcenko]

@BenParizek pretty sure @putyourlightson has already covered that in https://github.com/putyourlightson/craft-how-project-config-works#plugin-migrations

Your migration should always run the DB schema changes, but check the schema version for project config changes. The logic there being that the same deploy that pushes the migration will also push a project.yaml file with the changes (that the migration would make) already performed, when that migration was run in the development environment.

So you should update your migration to check for that. Here's a simple example for a migration that performs changes in DB and checks the schema version in project.yaml file.

[brandonkelly]

We also explain it here: https://docs.craftcms.com/v3/extend/project-config.html#project-config-migrations

[BenParizek]

Thanks @andris-sevcenko @brandonkelly. It's making sense now.