Data-migration-tool: [PDOException] SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1-Migration_Default' for key 'EAV_ATTRIBUTE_SET_ENTITY_TYPE_ID_ATTRIBUTE_SET_NAME'

[VignaPranav]

Hi,
I am using this data-migration-tool, getting below error,
[PDOException]
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1-Migration_Default' for key 'EAV_ATTRIBUTE_SET_ENTITY_TYPE_ID_ATTRIBUTE_SET_NAME'

Note: I am migrating datas from magento1.13.1.0 to magento 2.1.0

[victor-v-rad]

Hi @VignaPranav

You probably migrate data for the second attempt without reverting DB of Magento 2 to its initial state. Also there are a lot of issues posted here which could contain a solution for your one. Like with #204 Search will help

[VignaPranav]

Hi @victor-v-rad

Thanks for the reply,
Every time reverting back the M2 DB to initial stage(Fresh DB installation) I am doing.
But difficulty is like, for product url migration it's taking nearly 4-5hrs, again reverting back to fresh M2 db every time is increasing my migration time. Is there any other alternate solution for this?

[victor-v-rad]

the official doc

The Data Migration Tool saves its current progress as it runs. If errors or user intervention stop it from running, the Tool resumes progress at the last known good state.

[victor-v-rad]

Does it help?

[VignaPranav]

Thanks for the document,
For all other intervention it's starting from the previous good state but for " duplicate entry " error along it's producing below two error's,

[Exception]
Notice: Undefined offset: 1 in /vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php on line 209
[PDOException]
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1-Migration_Default' for key 'EAV_ATTRIBUTE_SET_ENTITY_TYPE_ID_ATTRIBUTE_SET_NAME'

So for this I searched for solution they have suggested me to revert back the M2 DB to initial stage(Fresh DB installation)

[VignaPranav]

Hi @victor-v-rad ,
Could you suggest me some ideas regarding the above

[dfelton]

@VignaPranav

The Data Migration Tool supports running the steps in individual pieces if that helps. You could segment out your migration to run for example the "EAV Step" alone, verify it's success, if successful backup the database and have that be your new M2 restore point.

Just segment out the steps in your config.xml file into numerous XML files. I'd recommend following the order in which the tool has provided already for them.

[iphigenie]

@victor-v-rad I think I see what happens here

I am getting this message in alternance with Notice: Undefined offset: 1 in vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php on line 266

After I ruled out all possible issues with my data - as this is not the main project it took a couple months looking on and off I started looking at what was happening in the magento2 database before/after each run and error.

ON AN "ERROR" DURING THE EAV STAGE, WHEN THE SYSTEM SAYS "REVERTING" IT DOES NOT REVERT THE ATTRIBUTE_SET AND ATTRIBUTE_GROUP DATA FULLY, AND IT LEAVES THE DATA IN AN ERROR CAUSING STATE (The attribute sets and attribute groups are also not reverted properly if you use --reset)

1. m2's eav_attribute_set table ends up containing the "migration_" entries after the undefined offset error or any other error during a further substep of the eav process (hence causing the "duplicate" error next time)
2. m2's eav_attribute_set table ends up completely empty after this error (causing the "undefined offset")
3. on and on forever

For those struggling, I have managed to pass through by MANUALLY (in SQL) rebuilding the eav_attribute_set and eav_attribute_group tables in the database to have only the original entries, but with the IDs from after my first migration (taking out all the migration ones but leaving/putting back the missing default ones)

But if I then get another error in the EAV step (eg: class name reference from a module I am not planning to migrate) then this error is back on the next run so I have to do that manual clean up over and over.

And because especially on the EAV stage things are a trial and error process (what classes, fields etc does the system flag up? decide whether to migrate or ignore, rinse, repeat) this is very very frustrating.

It also means people are sidetracked by this error which has nothing to do with our data or site!

it says it rolls back

but the data is still there, which will cause a duplicate error next run

[victor-v-rad]

Hi @iphigenie

I suggest keeping entire DB dump of just installed Magento 2 and use it instead of manually removing migrated data.

[iphigenie]

Hi @victor-v-rad

updating this as the issue still happens in the newest release

The problem is that the tool creates entries, but if it fails, it leaves these entries in. And at the next run, these entries cause an error as "duplicates". This is entirely created by the tool, but the tool gives misleading error messages about it.

People then think their source installtion is the problem, when it is actually traces of the interrupted previous attempt at migration - so the tool itself should perhaps either clean after itself (it does on most other things) OR clean before it runs when run with -r OR it could at the very least give the user instruction, after a failed run, "please restore clean database".

PS: restoring the entire database after each failed run could be a bit slow especially for people who do not have command line access to their database, so a quick clean option would improve the experience a lot

[aruntechguy]

Unfortunately, this issue still happens even with data-migration-tool 2.1.7.

The reason is, new magento installation's eav_attribute_set table already contains some records with "Default" as value under attribute_set_name field.

Solution is to rename all values containing the name "Default" in "attribute_value_name" field of eav_attribute_set table to "Default_x" in the Magento 1 database and run the migration from scratch.

In my case, there were 9 records with name "Default". So I renamed them to Default_1 to Default_9 and it worked perfectly!

[mritu19]

Hi @aruntechguy.. yes even, this issue still happens even with data-migration-tool 2.1.9.
Thanks for your message. It worked for me. I was stuck from long time.
Anyway atleast I am free from '1_Migration_default" issue.

Further, i am getting the below Error during migration

[Exception]
Notice: Undefined offset: 1 in /home/cloudpanel/htdocs/mage2.loveweddingbands.com/vendor/magento/data-migration-tool/src/M
igration/Step/Eav/Data.php on line 291

I am looking into that.
Please suggest me if anyone has fixed this issue.

Thanks

[aruntechguy]

Every time data migration fails due to any reason, after fixing the issue, you will have to start over.

setup:uninstall, install again and run data migration again until you stumble upon next issue or till complete.

Good luck.

[jordanvector]

This is still an issue on magento 2.2. Whats funnier is its telling me that there is a duplicate entry of an attribute that is not in the destination db yet. Here is a full stack trace:

[I] jordan@lat-askc390 ~/S/m2ee> php bin/magento migrate:data /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/etc/commerce-to-commerce/1.13.1.0/config.xml --auto -vvv

100% [============================] Remaining Time: < 1 sec
100% [============================] Remaining Time: < 1 sec
100% [============================] Remaining Time: < 1 sec
100% [============================] Remaining Time: < 1 sec
25% [=======>--------------------] Remaining Time: < 1 sec

[PDOException]
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '4-Migration_Apparel' for key 'EAV_ATTRIBUTE_SET_ENTITY_TYPE_ID_ATTRIBUTE_SET_NAME'

Exception trace:
() at /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Adapter/Mysql.php:169
PDOStatement->execute() at /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Adapter/Mysql.php:169
Migration\ResourceModel\Adapter\Mysql->insertMultiple() at /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Adapter/Mysql.php:137
Migration\ResourceModel\Adapter\Mysql->insertRecords() at /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Destination.php:52
Migration\ResourceModel\Destination->saveRecords() at /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php:831
Migration\Step\Eav\Data->saveRecords() at /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php:311
Migration\Step\Eav\Data->migrateAttributeSetsAndGroups() at /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php:189
Migration\Step\Eav\Data->perform() at /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/src/Migration/Mode/AbstractMode.php:73
Migration\Mode\AbstractMode->runStage() at /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/src/Migration/Mode/Data.php:122
Migration\Mode\Data->runData() at /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/src/Migration/Mode/Data.php:69
Migration\Mode\Data->run() at /Users/jordan/Sites/m2ee/vendor/magento/data-migration-tool/src/Migration/Console/MigrateDataCommand.php:51
Migration\Console\MigrateDataCommand->execute() at /Users/jordan/Sites/m2ee/vendor/symfony/console/Command/Command.php:264
Symfony\Component\Console\Command\Command->run() at /Users/jordan/Sites/m2ee/vendor/symfony/console/Application.php:873
Symfony\Component\Console\Application->doRunCommand() at /Users/jordan/Sites/m2ee/vendor/symfony/console/Application.php:202
Symfony\Component\Console\Application->doRun() at /Users/jordan/Sites/m2ee/vendor/magento/framework/Console/Cli.php:104
Magento\Framework\Console\Cli->doRun() at /Users/jordan/Sites/m2ee/vendor/symfony/console/Application.php:120
Symfony\Component\Console\Application->run() at /Users/jordan/Sites/m2ee/bin/magento:23

migrate:data [-r|--reset] [-a|--auto] [--]

[numberend]

Hi
My problem SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'Magento_MsrpSampleData' SOLVED via run the sql
"delete setup_module where schema_version is null"

[ci2014]

The problem of this post still exists. This ticket shouldn't be closed. Obviously it's a problem in the data migration tool.

[ExtensionsStore]

Still happening in 2.2.1

[iphigenie]

Still happening in 2.2.2 - the system still leaves eav_attribute_set and eav_attribute_group tables in error causing stages after you have run it once. See my comment higher up in the thread.

Restoring the database and restarting migration from scratch is very slow since there's a lot of try-and-retry so I just rebuild the correct tables with SQL by a) Removing duplicate entries OR b) Recreating the missing default entries I outlined what I do in a comment in https://github.com/magento/data-migration-tool/issues/204

In short - every attribute_set_id that exists in eav_attribute_groups also needs to exist in eav_attribute_sets, and point to the correct eav_entity_type (otherwise: invalid offset... error). There should be no migrated entries in eav_attribute_sets (otherwise: duplicate... error)

[balkanmasturbator]

Created this account just to comment on this issue. Issue persists in 2.2.2 - absolute huge pain.

Why in the world is this ticket closed???? Open it back up and start having a conversation on this or fix it. There have been multiple credible individuals who have identified the problems.

To put this into perspective - we're upgrading to M2 just to see if it is the platform we want to stick with our clients or move to simple, easy to use CMS systems. This is not helping the case for us to stick with M2.

[jphat]

still a problem in Magento 2.2.5

[iphigenie]

At the very least all the documentation should be updated to tell people to take a clean snapshot of the empty database BEFORE running the first migration attempt. I didn't think I'd need this so I did not have to option to just reset the database, correct the settings, run again...

[gouridash]

I am getting the same issue with the 2.2.6 migration tool. I am stuck on the below issue.

[PDOException]
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1-Migration_Default' for key 'EAV_ATTRIBUTE_SET_ENTITY_TYPE_ID_ATTRIBUTE_SET_NAME
'

Can someone guide me how to overcome this?

[victor-v-rad]

empty your M2

[afoysal]

@victor-v-rad

Hi @VignaPranav

You probably migrate data for the second attempt without reverting DB of Magento 2 to its initial state. Also there are a lot of issues posted here which could contain a solution for your one. Like with #204 Search will help

How can I revert DB of Magento 2 to its initial state ?

[victor-v-rad]

Just install M2 from scratch and do db dump

[iphigenie]

Otherwise if you look at my entry above (March 2017) it suggests which 2 tables you need to revert, and how, if you didn't keep a backup, to figure out how to reset it.

I have found that reverting the database takes time - and when trying to get every attribute, class etc. mapped you could be going back 50 times! - and reverting just these tables is enough. The migration tool does a great job of reverting everything else.

@victor-v-rad the weird thing is the idea that the default M2 sets are "moved" in ID to make space for the migrated ones. I assume that's why it'd be hard to revert?

I'm planning a fresh test this weekend to see if that can be repeated on the last of 2.2.x and 2.3.0

[victor-v-rad]

@iphigenie sorry for the late reply. The error Class ? does not exist but mentioned in: ? is thrown in Volume check class which cannot revert EAV tables. But the issue can be fixed. MAGETWO-97205 to resolve it.

[victor-v-rad]

As for the error Duplicate entry '1-Migration_Default' for key .... It happens when the migration run for the second time with --reset option, when EAV step already migrated data. In this case it needs to reinstall M2 instance to clean it

[avra911]

If you have this issue, it means you are in a wrong place and you should do some steps in this particular order:

1. fix duplicate entry error, generated by an reference to an attribute group/set which does not exists anymore, which was not deleted on cascade. Consider cleanup something like this:

delete
from magento1.eav_entity_attribute
where attribute_group_id not in 
   (select attribute_group_id from magento1.eav_attribute_group);

1. run magento installer and take it back from beginning, otherwise you will enter in sort of a loop with this error. consider also a mysql tweak, while this is resource consuming:

set global max_allowed_packet=1000000000;
set global net_buffer_length=1000000;

[iphigenie]

confirming that the error handling in 2.3.1 seems to avoid leaving the attribute sets in an unhappy state. Excellent

[LiamKarlMitchell]

Still a problem in 2.3.2 when saving eav_attribute_group or maybe its eav_attribute_set...

eav_attribute_set has a unique constraint named EAV_ATTRIBUTE_SET_ENTITY_TYPE_ID_ATTRIBUTE_SET_NAME on the column entity_type_id, attribute_set_name

So my source data must have duplicates that violate this go figure.
Among other places....

  [PDOException (23000)]
  SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1-Migration_Default' for key 'EAV_ATTRIBUTE_SET_ENTITY_TYPE_ID_ATTRIBUTE_SET_NAME'

Stack

 () at /var/www/html/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Adapter/Mysql.php:173
 PDOStatement->execute() at /var/www/html/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Adapter/Mysql.php:173
 Migration\ResourceModel\Adapter\Mysql->insertMultiple() at /var/www/html/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Adapter/Mysql.php:139
 Migration\ResourceModel\Adapter\Mysql->insertRecords() at /var/www/html/vendor/magento/data-migration-tool/src/Migration/ResourceModel/Destination.php:52
 Migration\ResourceModel\Destination->saveRecords() at /var/www/html/vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php:838
 Migration\Step\Eav\Data->saveRecords() at /var/www/html/vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php:313
 Migration\Step\Eav\Data->migrateAttributeSetsAndGroups() at /var/www/html/vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php:190
 Migration\Step\Eav\Data->perform() at /var/www/html/vendor/magento/data-migration-tool/src/Migration/Mode/AbstractMode.php:81
 Migration\Mode\AbstractMode->runStage() at /var/www/html/vendor/magento/data-migration-tool/src/Migration/Mode/Data.php:124
 Migration\Mode\Data->runData() at /var/www/html/vendor/magento/data-migration-tool/src/Migration/Mode/Data.php:69
 Migration\Mode\Data->run() at /var/www/html/vendor/magento/data-migration-tool/src/Migration/Console/MigrateDataCommand.php:54
 Migration\Console\MigrateDataCommand->execute() at /var/www/html/vendor/symfony/console/Command/Command.php:255
 Symfony\Component\Console\Command\Command->run() at /var/www/html/vendor/magento/framework/Interception/Interceptor.php:58
 Migration\Console\MigrateDataCommand\Interceptor->___callParent() at /var/www/html/vendor/magento/framework/Interception/Interceptor.php:138
 Migration\Console\MigrateDataCommand\Interceptor->Magento\Framework\Interception\{closure}() at /var/www/html/vendor/magento/framework/Interception/Interceptor.php:153
 Migration\Console\MigrateDataCommand\Interceptor->___callPlugins() at /var/www/html/generated/code/Migration/Console/MigrateDataCommand/Interceptor.php:104
 Migration\Console\MigrateDataCommand\Interceptor->run() at /var/www/html/vendor/symfony/console/Application.php:893
 Symfony\Component\Console\Application->doRunCommand() at /var/www/html/vendor/symfony/console/Application.php:262
 Symfony\Component\Console\Application->doRun() at /var/www/html/vendor/magento/framework/Console/Cli.php:105
 Magento\Framework\Console\Cli->doRun() at /var/www/html/vendor/symfony/console/Application.php:145
 Symfony\Component\Console\Application->run() at /var/www/html/bin/magento:23

Then on re-run

  [Exception]
  Notice: Undefined offset: 1 in /var/www/html/vendor/magento/data-migration-tool/src/Migration/Step/Eav/Data.php on line 293

Thankfully it retains backup tables. I thought I could restore the original data in the Destination Magento 2 DB then fix up the problems, but later ran into issues.

Ran on Source M1

-- Fix SOURCE DB Orphaned things....
DELETE FROM eav_attribute_set WHERE entity_type_id NOT IN (SELECT entity_type_id FROM eav_entity_type);
DELETE FROM eav_entity_attribute WHERE attribute_group_id NOT IN (SELECT attribute_group_id FROM eav_attribute_group);
DELETE FROM catalog_eav_attribute WHERE attribute_id NOT IN (SELECT attribute_id FROM eav_attribute);

Ran on Destination M2

-- Fix DESTINATION DB partially migrated....
DELETE FROM eav_attribute_group;
DELETE FROM eav_attribute_set;
DELETE FROM eav_attribute;

INSERT INTO eav_attribute_set
SELECT * FROM migration_backup_eav_attribute_set;

INSERT INTO eav_attribute_group
SELECT * FROM migration_backup_eav_attribute_group;

INSERT INTO eav_attribute
SELECT * FROM migration_backup_eav_attribute;

Re-run

bin/magento migrate:data -ra vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.0.1/config.xml -vvv```



EAV_ENTITY_ATTRIBUTE_ATTRIBUTE_SET_ID_ATTRIBUTE_ID so thats eav_entity_attribute attribute_set_id, attribute_id unique constraint...

But my source db does not contain duplicates...
```sql
SELECT attribute_set_id, attribute_id FROM eav_entity_attribute GROUP BY attribute_set_id, attribute_id HAVING COUNT(*) > 1

Blahhhhh

[abkrim]

Same for me, in 2.3.3 form 1.9.3.1

SELECT attribute_set_id, attribute_id FROM tienda_ovh.st1_eav_entity_attribute GROUP BY attribute_set_id, attribute_id HAVING COUNT(*) > 1

0 results

ON originbl M1

mysql> SELECT * FROM eav_entity_attribute WHERE attribute_group_id NOT IN (SELECT attribute_group_id FROM eav_attribute_group);
Empty set (0.01 sec)

mysql> SELECT * FROM catalog_eav_attribute WHERE attribute_id NOT IN (SELECT attribute_id FROM eav_attribute);
Empty set (0.01 sec)

mysql> SELECT * FROM catalog_eav_attribute WHERE attribute_id NOT IN (SELECT attribute_id FROM eav_attribute);
Empty set (0.00 sec)

[pk1774]

Kindly check if you have correctly entered the database name in the config.xml

    <destination>
        <database host="localhost" name="xxxxxxx" user="xxxxx" password="xxxx" />
    </destination>

I am 100% sure this is the mistake you are making. Hope this helps.

[abdelkebir]

You need maybe (depends on your case) to drop the following tables:

drop table migration_backup_eav_attribute_group;

drop table migration_backup_eav_attribute_set;

drop table migration_backup_eav_entity_type;

Worked for me, but it depends on your case, I think every case is special.

[rafaelpatro]

Same issue in Migration Tool for Magento 2.4.0

SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '26-product-details' for key 'EAV_ATTRIBUTE_GROUP_ATTRIBUTE_SET_ID_ATTRIBUTE_GROUP_CODE'

Maybe it could be reopened. The attribute group code is created on the fly.
Script could check duplicated Group Names before creating Group Code values for insertion.

By the way, its a nice tool. Congrats!

[rjm87]

@rafaelpatro Did you fix this issue? I'm having the same issue when trying to migrate 1.9.4.5 > 2.4.1.

[rjm87]

I fixed the following error:
[PDOException (23000)] SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '75-Product details' for key 'EAV_ATTRIBUTE_GROUP_ATTRIBUTE_SET_ID_ATTRIBUTE_GROUP_NAME'

with

UPDATE magento.eav_attribute_group SET attribute_group_name = 'Product Details 2', attribute_group_code = 'product-details-2' WHERE attribute_group_id = '7';