Data-migration-tool: Mismatch of entities in the document: inventory_source_item Source: 2316 Destination: 1

[benhtn]

I'm on CentOS 7, PHP 7.2.13 using, nginx version: nginx/1.12.2, MySQL 5.7.23.

I've got a live CE 1.9.3.6 store and am importing settings and data using the data import tool into a CE 2.3.0 dev environment.

My import commands looked like this:

bin/magento migrate:settings -r -a /usr/share/nginx/html/magento2/vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.6/config.xml
bin/magento migrate:data -r -a /usr/share/nginx/html/magento2/vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.6/config.xml

During initial import I got these errors:

[2019-01-05 16:02:05][WARNING]: Source documents are not mapped: le_cartmigration_imports,le_cartmigration_recent,le_cartmigration_user,sagepayreporting_fraud,sagepaysuite_action,sagepaysuite_debug,sagepaysuite_fraud,sagepaysuite_paypaltransaction,sagepaysuite_session,sagepaysuite_tokencard,sagepaysuite_transaction,sagepaysuite_transaction_queue,searchanise_config,searchanise_queue 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_order. Fields: base_surcharge_amount,surcharge_amount 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_order_item. Fields: base_surcharge_amount,surcharge_amount 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_order_payment. Fields: repeat_code 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_quote_address. Fields: base_surcharge_amount,surcharge_amount 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_quote_address_item. Fields: base_surcharge_amount,surcharge_amount 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_quote_item. Fields: base_surcharge_amount,surcharge_amount 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_quote_payment. Fields: sagepay_token_cc_id,repeat_code

So I added this to vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.6/map.xml in the source > document_rules section:

<ignore><document>le_cartmigration_imports</document></ignore>
<ignore><document>le_cartmigration_recent</document></ignore>
<ignore><document>le_cartmigration_user</document></ignore>
<ignore><document>sagepayreporting_fraud</document></ignore>
<ignore><document>sagepaysuite_action</document></ignore>
<ignore><document>sagepaysuite_debug</document></ignore>
<ignore><document>sagepaysuite_fraud</document></ignore>
<ignore><document>sagepaysuite_paypaltransaction</document></ignore>
<ignore><document>sagepaysuite_session</document></ignore>
<ignore><document>sagepaysuite_tokencard</document></ignore>
<ignore><document>sagepaysuite_transaction</document></ignore>
<ignore><document>sagepaysuite_transaction_queue</document></ignore>
<ignore><document>searchanise_config</document></ignore>
<ignore><document>searchanise_queue</document></ignore>

and this to the source > field_rules section:

<ignore><field>sales_flat_order.base_surcharge_amount</field></ignore>
<ignore><field>sales_flat_order.surcharge_amount</field></ignore>
<ignore><field>sales_flat_order_item.base_surcharge_amount</field></ignore>
<ignore><field>sales_flat_order_item.surcharge_amount</field></ignore>
<ignore><field>sales_flat_order_payment.repeat_code</field></ignore>
<ignore><field>sales_flat_quote_address.base_surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_address.surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_address_item.base_surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_address_item.surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_item.base_surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_item.surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_payment.sagepay_token_cc_id</field></ignore>
<ignore><field>sales_flat_quote_payment.repeat_code</field></ignore>

And pointed config.xml to the edited map.xml.

Rerunning the migration tool, into a completely reset and clean destination 2.3.0 store/ db, I get the following warnings:

[2019-01-05 18:03:12][WARNING]: Mismatch of entities in the document: customer_visitor Source: 705985 Destination: 705982
[2019-01-05 18:03:12][WARNING]: Volume Check failed
...
[2019-01-05 18:03:15][WARNING]: Mismatch of entities in the document: inventory_source_item Source: 2316 Destination: 1
[2019-01-05 18:03:15][WARNING]: Volume Check failed

The troubleshooting guide doesn't have much to say on this, other than using the delta option. https://devdocs.magento.com/guides/v2.1/migration/migration-troubleshooting.html

Using the delta option I then get:

[2019-01-05 18:04:26][WARNING]: Mismatch of entities in the document: catalog_product_entity_varchar Source: 30938 Destination: 30130
[2019-01-05 18:04:26][WARNING]: Mismatch of entities in the document: session Source: 32371 Destination: 32327
[2019-01-05 18:04:26][WARNING]: Volume Check failed

[2019-01-05 18:04:28][WARNING]: Mismatch of entities in the document: inventory_source_item Source: 2316 Destination: 1
[2019-01-05 18:04:28][WARNING]: Volume Check failed

catalog_product_entity_varchar and inventory_source_item values don't change in source nor destination, sessions increase in the source, but not the detination.

When I visit the new site all the product categories say "We can't find products matching the selection." and in the admin backend I can see all the imported categories and see the "Products In Category" listed OK apparently, but when I try to go to Catalog > Products I get a 500 error and in the PHP error log I see:

2019/01/05 18:10:47 [error] 1165#0: *784 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Method Magento\Ui\TemplateEngine\Xhtml\Result::__toString() must not throw an exception, caught TypeError: Argument 1 passed to Magento\InventorySalesAdminUi\Model\GetSalableQuantityDataBySku::execute() must be of the type string, null given, called in /usr/share/nginx/html/magento2/vendor/magento/module-inventory-sales-admin-ui/Ui/Component/Listing/Column/SalableQuantity.php on line 70 in /usr/share/nginx/html/magento2/vendor/magento/module-ui/Component/Wrapper/UiComponent.php on line 0" while reading response header from upstream, client: 192.168.1.91, server: lws2.local, request: "GET /admin_mvea3p/catalog/product/index/key/40476ba4d69578fbcff70d90ac51cda930730b71e3db65430d327cdbe40a32ce/ HTTP/1.1", upstream: "fastcgi://unix:/run/php-fpm/php-fpm.sock:", host: "lws2.local", referrer: "http://lws2.local/admin_mvea3p/catalog/category/edit/key/c1e6582aa7733d99bf1d2a4b4883728be00e92ab4f33bf43d82bbfdc7d7d67f3/id/4/"

I am expecting to have all products in the destination database and to be able to view them on front and backends.

I see from this issue https://github.com/magento/data-migration-tool/issues/628 that perhaps SKUs are required and if I run this query

SELECT entity_id as product_id, sku FROM catalog_product_entity

on my source db I see SKU is NULL in all cases. Is having SKUs on all products a requirement of the tool?

[victor-v-rad]

SKU is required field when you create new product in Admin Panel. Maybe some of your extensions override it somehow. I tried to set null SKU in catalog_product_entity and got error when open product in admin panel or fronted.

[charmming]

please anybody help me...during migration i got this error and after migration opening all customer page shows the second error. thank you.

{"0":"Type Error occurred when creating object: MagentoEavModelEntityAttributeSourceConfig","1":"

1 MagentoFrameworkObjectManagerFactoryCompiled->create('MagentoEavMode...', array()) called at [vendor/magento/framework/ObjectManager/ObjectManager.php:56]n#2 MagentoFrameworkObjectManagerObjectManager->create('MagentoEavMode...', array()) called at [vendor/magento/framework/Validator/UniversalFactory.php:36]n#3 MagentoFrameworkValidatorUniversalFactory->create('MagentoEavMode...') called at [vendor/magento/module-eav/Model/Entity/Attribute/AbstractAttribute.php:641]n#4 MagentoEavModelEntityAttributeAbstractAttribute->getSource() called at [vendor/magento/module-customer/Model/AttributeMetadataConverter.php:68]n#5 MagentoCustomerModelAttributeMetadataConverter->createMetadataAttribute(&MagentoCustomerModelAttribute#0000000067aa633c000000002b7d916f#) called at [vendor/magento/module-customer/Model/Metadata/CustomerMetadata.php:87]n#6 MagentoCustomerModelMetadataCustomerMetadata->getAttributeMetadata('customerstatus') called at [vendor/magento/module-customer/Model/Metadata/CustomerMetadata.php:119]n#7 MagentoCustomerModelMetadataCustomerMetadata->getAllAttributesMetadata() called at [vendor/magento/module-customer/Model/Metadata/CachedMetadata.php:85]n#8 MagentoCustomerModelMetadataCachedMetadata->getAllAttributesMetadata() called at [vendor/magento/module-customer/Ui/Component/Listing/AttributeRepository.php:78]n#9 MagentoCustomerUiComponentListingAttributeRepository->getList() called at [vendor/magento/module-customer/Ui/Component/Listing/Columns.php:102]n#10 MagentoCustomerUiComponentListingColumns->prepare() called at [vendor/magento/framework/View/Layout/Generator/UiComponent.php:164]n#11 MagentoFrameworkViewLayoutGeneratorUiComponent->prepareComponent(&MagentoCustomerUiComponentListingColumns#0000000067aa67ff000000002b7d916f#) called at [vendor/magento/framework/View/Layout/Generator/UiComponent.php:161]n#12 MagentoFrameworkViewLayoutGeneratorUiComponent->prepareComponent(&MagentoUiComponentListing#0000000067aa67fe000000002b7d916f#) called at [vendor/magento/framework/View/Layout/Generator/UiComponent.php:142]n#13 MagentoFrameworkViewLayoutGeneratorUiComponent->generateComponent(&MagentoFrameworkViewLayoutDataStructure#0000000067aa6e7f000000002b7d916f#, 'customer_listing', array('attributes' => array('group' => '', 'component' => '', 'aclResource' => '', 'visibilityCondit...' => array())), &MagentoFrameworkViewLayoutInterceptor#0000000067aa6e86000000002b7d916f#) called at [vendor/magento/framework/View/Layout/Generator/UiComponent.php:102]n#14 MagentoFrameworkViewLayoutGeneratorUiComponent->process(&MagentoFrameworkViewLayoutReaderContext#0000000067aa6501000000002b7d916f#, &MagentoFrameworkViewLayoutGeneratorContext#0000000067aa673f000000002b7d916f#) called at [vendor/magento/framework/View/Layout/GeneratorPool.php:81]n#15 MagentoFrameworkViewLayoutGeneratorPool->process(&MagentoFrameworkViewLayoutReaderContext#0000000067aa6501000000002b7d916f#, &MagentoFrameworkViewLayoutGeneratorContext#0000000067aa673f000000002b7d916f#) called at [vendor/magento/framework/View/Layout.php:343]n#16 MagentoFrameworkViewLayout->generateElements() called at [vendor/magento/framework/View/Layout/Builder.php:129]n#17 MagentoFrameworkViewLayoutBuilder->generateLayoutBlocks() called at [vendor/magento/framework/View/Page/Builder.php:55]n#18 MagentoFrameworkViewPageBuilder->generateLayoutBlocks() called at [vendor/magento/framework/View/Layout/Builder.php:65]n#19 MagentoFrameworkViewLayoutBuilder->build() called at [vendor/magento/framework/View/Layout.php:253]n#20 MagentoFrameworkViewLayout->build() called at [vendor/magento/framework/View/Layout.php:875]n#21 MagentoFrameworkViewLayout->getBlock('menu') called at [vendor/magento/module-backend/Model/View/Result/Page.php:26]n#22 MagentoBackendModelViewResultPage->setActiveMenu('Magento_Customer...') called at [vendor/magento/module-customer/Controller/Adminhtml/Index/Index.php:28]n#23 MagentoCustomerControllerAdminhtmlIndexIndex->execute() called at [vendor/magento/framework/App/Action/Action.php:108]n#24 MagentoFrameworkAppActionAction->dispatch(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/module-backend/App/AbstractAction.php:231]n#25 MagentoBackendAppAbstractAction->dispatch(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/Interception/Interceptor.php:58]n#26 MagentoCustomerControllerAdminhtmlIndexIndexInterceptor->___callParent('dispatch', array(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#)) called at [vendor/magento/framework/Interception/Interceptor.php:138]n#27 MagentoCustomerControllerAdminhtmlIndexIndexInterceptor->MagentoFrameworkInterception{closure}(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/module-backend/App/Action/Plugin/Authentication.php:143]n#28 MagentoBackendAppActionPluginAuthentication->aroundDispatch(&MagentoCustomerControllerAdminhtmlIndexIndexInterceptor#0000000067aa61eb000000002b7d916f#, &Closure#0000000067aa61e5000000002b7d916f#, &MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/Interception/Interceptor.php:135]n#29 MagentoCustomerControllerAdminhtmlIndexIndexInterceptor->MagentoFrameworkInterception{closure}(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/Interception/Interceptor.php:153]n#30 MagentoCustomerControllerAdminhtmlIndexIndexInterceptor->___callPlugins('dispatch', array(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#), NULL) called at [generated/code/Magento/Customer/Controller/Adminhtml/Index/Index/Interceptor.php:26]n#31 MagentoCustomerControllerAdminhtmlIndexIndexInterceptor->dispatch(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/App/FrontController.php:159]n#32 MagentoFrameworkAppFrontController->processRequest(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#, &MagentoCustomerControllerAdminhtmlIndexIndexInterceptor#0000000067aa61eb000000002b7d916f#) called at [vendor/magento/framework/App/FrontController.php:99]n#33 MagentoFrameworkAppFrontController->dispatch(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/Interception/Interceptor.php:58]n#34 MagentoFrameworkAppFrontControllerInterceptor->___callParent('dispatch', array(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#)) called at [vendor/magento/framework/Interception/Interceptor.php:138]n#35 MagentoFrameworkAppFrontControllerInterceptor->MagentoFrameworkInterception{closure}(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/Interception/Interceptor.php:153]n#36 MagentoFrameworkAppFrontControllerInterceptor->__callPlugins('dispatch', array(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#), array(array('default_store_se...', 'page_cache_from...', 'storeCookieValid...', 'install', 'configHash'))) called at [generated/code/Magento/Framework/App/FrontController/Interceptor.php:26]n#37 MagentoFrameworkAppFrontControllerInterceptor->dispatch(&MagentoFrameworkAppRequestHttp#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/App/Http.php:136]n#38 MagentoFrameworkAppHttp->launch() called at [vendor/magento/framework/App/Bootstrap.php:258]n#39 MagentoFrameworkAppBootstrap->run(&MagentoFrameworkAppHttpInterceptor#0000000067aa6615000000002b7d916f#) called at [index.php:39]n","url":"/admin/customer/index/index/key/d1ee2aeb7f01d80b4dad604c2a131e393fd26935767a2c252777912e7c814828/","script_name":"/index.php"}root@mail:/var/www/html/magento2/var/report

[ninjacoder30]

@charmming even i am getting the same error while data-migration

[Ddcdom]

@victor-v-rad I know this is old, but its a bug in M1 which allows you to create a product without an SKU. If you open an existing product, click 'duplicate', then it creates a new product in the database without an SKU. So if you duplicate a product, then close the page without setting an SKU, you get an entry in catalog_product_entity with a NULL for SKU, which then causes the volume check to fail on upgrade.