Magento2: Can't create foreignKey with Declarative Schema on table not created through DS

[peterjaap]

Preconditions (*)

1. Magento 2.3.0

Steps to reproduce (*)

1. Create a module that uses Declarative Schema by creating a db_schema.xml
2. Try to create a foreignKey with as referenceTable a table that isn't created/controlled through Declarative Schema (in my case a 3rd party extension's table)
3. Run php bin/magento setup:upgrade

Expected result (*)

1. Table is created with a foreign key to referenced table

Actual result (*)

1. Error Notice: Undefined index: referenced_table_name_here in vendor/magento/framework/Setup/Declaration/Schema/Declaration/SchemaBuilder.php on line 352

When I edit vendor/magento/framework/Setup/Declaration/Schema/SchemaConfig.php:77 and add print_r(array_keys($data['table']));exit; there, I get a list of all tables created by extensions that use Declarative Schema. The referenced table isn't one of them and thus setup:upgrade throws an error.

[magento-engcom-team]

Hi @peterjaap. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• [x] Summary of the issue
• [x] Information on your environment
• [x] Steps to reproduce
• [x] Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento-engcom-team give me 2.3-develop instance - upcoming 2.3.x release

For more details, please, review the Magento Contributor Assistant documentation.

@peterjaap do you confirm that you was able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• [x] yes
• [ ] no

[magento-engcom-team]

Hi @engcom-backlog-nazar. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [x] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• [x] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• [x] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento-engcom-team give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!
  

• [ ] 5. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento-engcom-team give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• _Next steps are available in case you are a member of Community Maintainers._

• [ ] 6. Add label Issue: Confirmed once verification is complete.

• [ ] 7. Make sure that automatic system confirms that report has been added to the backlog.

[ghost]

Hi @peterjaap thank you for you report, It will be processed faster if you move this to https://github.com/magento/community-features

[peterjaap]

@engcom-backlog-nazar but it's not a feature? It's a bug!

[ghost]

@peterjaap according to devdocs, seems like no.

[ghost]

@peterjaap In the following example, the selected constraint node defines the characteristics of the FL_ALLOWED_SEVERITIES foreign key. https://magento-devdocs.github.io/devdocs-for-tests/guides/v2.3/extension-dev-guide/declarative-schema/db-schema.html

[peterjaap]

@engcom-backlog-nazar could you elaborate on that?

[ghost]

@peterjaap this mean that you can create a foreign key only with tables created by declarative schema, because all table data loaded now with checking db_schema.xml

[peterjaap]

Ok maybe it's an idea to add that in plain English?

And imho this is a serious impediment for the adoption of declarative schema. I've fallen back to writing InstallScripts again....

[ghost]

@peterjaap seems, like yes, but adding a check data table on install scripts, sounds like feature request. on my opinion

[peterjaap]

I meant adding it to the docs instead of that weird sentence about FL_ALLOWED_SEVERITIES.

[ghost]

@peterjaap devdoc always have not clearly documentation :) but you can create a PR to solve this here -> https://github.com/magento/devdocs

[Yonn-Trimoreau]

@engcom-backlog-nazar The sentence about FL_ALLOWED_SEVERITIES does not mention the fact that you cannot add a foreign key referencing a table not created through db_schema.xml... I read the sentence again and again.. Nope.

Seems like an issue.

Like @peterjaap said, this is a serious impediment for the adoption of declarative schema.

[ghost]

@Yonn-Trimoreau at this point, yes we must have ability to add foreign key to tables wich not created via db schema.xml but it's more a "feature" to my opinion, because db_schema.xml not cover this case.

[Yonn-Trimoreau]

@engcom-backlog-nazar If we read the documentation again and take what is _explicitly_ written about this issue:

Declarative setup is based on database structure declarations, and is used in projects such as Doctrine. Schema files declare what the database structure should be, and Magento determines the differences between the current table structure and what it should be. These differences can be represented with atomic SQL operations.

Two things:

• Magento determines the differences between the current table structure and what it should be. Sounds cristal clear to me.

• and is used in projects such as Doctrine: Doctrine does that. It can create foreign keys with external tables not described in any of its description formats. By using the @JoinColumn annotation. Because Doctrine applies its migration scripts based on the real database structure. Not on its own representation of the database structure.

[AnshuMishra17]

What is the status on this issue?
I am facing this issue on 2.3.1

[joshuaadickerson]

This is not a feature request. This is implementing the backwards compatibility that was promised.

[magento-engcom-team]

:white_check_mark: Confirmed by @sdzhepa
Thank you for verifying the issue. Based on the provided information internal tickets MC-18156 were created

Issue Available: @sdzhepa, _You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself._

[khaitchrif]

What is the status on this issue?
I am facing this issue on Magento CLI 2.4.1

[sivaschenko]

Closing this issue as there is no plans to implement external references for the db schema. The reference to the table added in a custom way should be added in a custom way as well.