Magento2: On Magento 2.3.2, setup:upgrade command removes tables of disabled modules

[avgust1111]

Preconditions (*)

1. Magento 2.3.2 & 2.3-develop, 2.3.2-p1 (CE or EE- doesn't matter)
2. At least one installed module, which uses declarative schema

Steps to reproduce (*)

1. Disable module, which uses declarative schema, using console command (module:disable). For example, native Magento_ReleaseNotification extension
2. Run setup:upgrade command
3. Check in the database, tables of the disabled module from the first step (for Magento_ReleaseNotification - that's the only table 'release_notification_viewer_log')

Expected result (*)

1. Tables of the disabled module have been already removed

Actual result (*)

1. The database still contains tables of the disabled module, with all data.
2. See https://github.com/magento/magento2/issues/24458#issuecomment-528324463

[m2-assistant[bot]]

Hi @avgust1111. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• [x] Summary of the issue
• [ ] Information on your environment
• [ ] Steps to reproduce
• [ ] Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.3-develop instance - upcoming 2.3.x release

For more details, please, review the Magento Contributor Assistant documentation.

@avgust1111 do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• [ ] yes
• [ ] no

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[avgust1111]

@magento give me 2.3-develop instance

[magento-engcom-team]

Hi @avgust1111. Thank you for your request. I'm working on Magento 2.3-develop instance for you

[magento-engcom-team]

Hi @avgust1111, here is your Magento instance.
Admin access: https://i-24926-2-3-develop.instances.magento-community.engineering/admin
Login: admin Password: 123123q
Instance will be terminated in up to 3 hours.

[avgust1111]

do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

Yes, I've reproduced the bug on my local environment. Unfortunately, I can't reproduce the issue on the test Magento instance, specified above, because I need SSH credentials to run console commands, specified above

It would be very nice to sort that issue out as soon as possible

[m2-assistant[bot]]

Hi @milindsingh. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• [ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• [ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 4. Verify that the issue is reproducible on 2.3-develop branch
  Details

  - Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[webkul-deepak]

After disabling module, please enable safe mode, while executing "setup:upgrade" or "setup:install", as this will creates a data dump during the installation or upgrade process. After this please restore the database while enabling the module.

Please follow the commands for same :

"setup:upgrade --safe-mode=1" : run after disabling the mod, this will create database backup.

"setup:upgrade --data-restore=1" : to restore database again after enabling the module.

Please refer Magento doc. https://devdocs.magento.com/guides/v2.3/extension-dev-guide/declarative-schema/migration-commands.html

[avgust1111]

Hello, @webkul-deepak
Is that a final solution, or just a temporary one, until the bug won't be fixed?
Anyway, is that solution suitable for the situation, when module is enabled again after some period of time? Will database restoring affect all changes in all tables (both data and schema), or only tables, related to the enabled module?

[m2-assistant[bot]]

Hi @webkul-deepak. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• [ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• [ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 4. Verify that the issue is reproducible on 2.3-develop branch
  Details

  - Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[m2-assistant[bot]]

Hi @engcom-Charlie. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• [ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• [ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!

• [ ] 5. Add label Issue: Confirmed once verification is complete.

• [ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

[webkul-deepak]

Hi @avgust1111 the provided solution is not the temporary, it's the final solution. As it's the feature of Declarative Schema.
Yes, it will work same for modules which you've enabled after a long time.
Also "--data-restore=1" restores both data and schema, after enabling the module again.

[avgust1111]

Hello, @webkul-deepak.
Thank you for the update.
One moment is still unclear enough.

Will database restoring affect all changes in all tables (both data and schema), or only tables, related to the enabled module?

"--data-restore=1" restores both data and schema,

Will this action affect the whole database, or only tables, related to the module, which is enabled again?

[magento-engcom-team]

:white_check_mark: Confirmed by @engcom-Charlie
Thank you for verifying the issue. Based on the provided information internal tickets MC-22908 were created

Issue Available: @engcom-Charlie, _You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself._

[twosg]

Personally I would prefer letting the "disabled" table(s) and data untouched while running setup:upgrade after disabling modules. If you really would want to delete the table and its (possibly important) data, you could run setup:upgrade --cleanup_mode=1 or something similar. Because then you know that you do not want the data.

[danemacmillan]

I get that this is the point behind Declarative Schema, but there should still be an option to prevent data from getting purged at all, like @twosg commented. This is a really nuclear default; it should require more deliberate input from users instead.

Removing and re-adding the data using the --safe-mode and --data-restore options just seem like a more error-prone, timely, and generally worse option than a simple --leave-my-shit-in-place option.

By the way, I found this discussion from the documentation.

[kanevbg]

Maybe better solution is that module:uninstall Module_Name is to remove the data, and not setup:upgrade?
EDIT: Eventually deferred DB data removal mechanism can be added, eg. when module is uninstalled, it can be 'queued' for DB data removal when running setup:upgrade. The queue in this case should be easy to manage/edit.

[gabrieldagama]

Disabling a module should not lead to the deletion of the database table or its contents, as described by some contributors above.

Closing this issue since the current behavior is the expected one.

Thanks.

[avgust1111]

@gabrieldagama Please, note: earlier, documentation has been updated: https://github.com/magento/devdocs/issues/5629. I suppose if the behaviour was changed, it's necessary to update the documentation, what do you think?

[gabrieldagama]

Hi @avgust1111, thanks for highlighting. Did some investigation on this and the current behavior is the expected behavior on this issue. I guess the documentation is still correct. I will check internally, but I still think that shouldn't be the behavior, maybe there is a reason for that.

Thanks again for your feedback.

[LiamKarlMitchell]

Can you not delete tables, its rather annoying.
Delete the table if the module is completely deleted or add a warning on every setup:upgrade that the tables still exist...