Prestashop: Product Import does not clean Accessory and Attribute tables, if requested to clean Product data.

[k0lv]

Describe the bug
I was importing Products with Accessories, and in the results, have seen, that Products are linked incorrectly(not 100% as I defined). Some part were as I described in my .xlsx, but there were also some random relations. I thought, that the error is is import functionality, or in the source data, but finally found out two reasons:
1) I was importing data to PS several times, and every time used PrestaShop Cleaner v2.0.0 to clean the Product DB. I have found out, that 3 tables, are not cleaned by the Module: a) ps_accessory b) ps_attribute c) ps_tag_count // _(I was not able to find out how to register issue for the module, sorry.)_
2) Before importing, I was also setting parameter "Delete all products before import" to YES, but this also does not cleaned data.

Results (Issue):
As a result I receive new data-base of products, but with partially old Accessories. Here is my issue.

Additional observations:
As for ps_attribute and ps_tag_count, I do not know, if old data, that remains in tables can cause the issues for the future clean imports. I think it was worth to mention here. Maybe you can comment on that. Thank you.

Edit by Khouloud: steps to reproduce the issue

1. install PS1.7.6.2
2. Go to the Import page => import products
   Products_1.xlsx
   2.1 Delete all products before import
   2.2 Use product reference as a key

3. check the ps_accessory table => ok
   

4. Go to the Import page => import products
   Products_2.xlsx
   4.1 Delete all products before import
   4.2 Use product reference as a key

5. check the ps_accessory table => NOK
   

To Reproduce
Steps to reproduce the behavior:

1. Import 1st portion of Products with Accessories.
2. Use PS Cleaner to delete all Product Catalog.
3. Import 2nd portion of Products with Accessories with "Delete all products before import" set to YES. As a result you will get mixture of New and Old product relationships.

Additional information
PrestaShop version: 1.7.6.2
PHP version: 7.0.33

[khouloudbelguith]

Hi @k0lv,

I did not manage to reproduce the issue with PS1.7.6.2 & pscleaner v2.0.0.
I used this csv file
product.csv.zip
I attached a screen record
https://drive.google.com/file/d/1Cq1y1skIqeVt4T94T12u50htvC-ebgHv/view
Thanks to check & feedback.

[k0lv]

Hi @k0lv,

I did not manage to reproduce the issue with PS1.7.6.2 & pscleaner v2.0.0.
I used this csv file
product.csv.zip
I attached a screen record
https://drive.google.com/file/d/1Cq1y1skIqeVt4T94T12u50htvC-ebgHv/view
Thanks to check & feedback.

Hello, @khouloudbelguith. I think, that the main difference for yours and mine scenario is, that you are using Product ID, instead of Reference#. The trick is that ps_accessory table is working with ID's. So, if after cleaning, ps_attribute table has data for the old Product ID's (relationships), all the data will be corrupted, because in my case new portion of import will generate new ID's for the products (second import file, in my case had different sequence of the products, and as a results different ID's generated). You can easily have an evidence if you will check ps_accessory, ps_attribute and ps_tag_count tables in phpMyAdmin, after PS Cleaner, or Clean Product Import (with deletion) activity. Thank you for the fast response and Happy New Year! 🎄

1) Importing first portion to the fully clean DB:


RESULTS:

2) Delete the Catalogue:

RESULTS: Accessory table unchanged:

3) Import new product list:

RESULTS:

Issue: Old relationships applied to new product list.

P.S. I have found, that ps_attribute and ps_tag_count tables also retain historical data. Yet I do not know if that fact can cause any data integrity issues in the future.

[k0lv]

Workaround:
Found funny fact, that PS Cleaner "FUNCTIONAL INTEGRITY CONSTRAINTS" option, actually cleans the Accessory table. :) Wondering, why this is not done as a part of "CATALOG DELETION"...

[khouloudbelguith]

@k0lv, could you please provide me the two xlsx file used in your case to test it.
First, I used this file
product.csv.zip
using this configuration

Second, I used this file
product2.csv.zip
using this configuration

I checked the ps_accessory table => it is empty, but I checked in the BO & FO => related products well added.
https://drive.google.com/file/d/1qfHLLRoV6bQBHMUSX-z5wfJG8tj6VY6r/view

Thanks!

[k0lv]

Products_1.xlsx
Products_2.xlsx
@khouloudbelguith, adding both files, that I have used to replicate the issue.

1. I have used "Delete all products..." for all the imports.
2. Now I am very confused. How can you explain, that ps_accessory is blank, but products are related. This sounds like a magic for me, but I am not so familiar with the PS data structure, maybe I am wrong.

Update:
If you use Product Deletion function in Catalog, Accessory table is cleaned out:
A)

B)

[khouloudbelguith]

@k0lv, thanks for these clarifications.
I manage to reproduce the issue with PS1.7.6.2 & PS1.7.5.2 following these steps:

1. install PS1.7.6.2
2. Go to the Import page => import products
   Products_1.xlsx
   2.1 Delete all products before import
   2.2 Use product reference as a key
3. check the ps_accessory table => ok
   

1. Go to the Import page => import products
   Products_2.xlsx
   4.1 Delete all products before import
   4.2 Use product reference as a key
2. check the ps_accessory table => NOK
   

I’ll add this to the debug roadmap so that it’s fixed. If you have already fixed it on your end or if you think you can do it, please do send us a pull request!
Thanks!