Magento2: REST API catalogProductRepositoryV1 overwriting default values

[giano574]

Preconditions

1. Magento 2

Steps to reproduce

This demonstration is only with a single storeview, but can be reproduced with multiple storeviews or multiple websites like already mentioned by a bunch of other reporters above

1. Setup a clean Magento installation using the 2.2-develop branch (I used commit eb84c5393da)
2. Go into the backend
3. Make a note of the default storeview code, it's default (we'll need this later when we create an API request)

4. Create a new product:

   • name: "Base"
   • SKU: "base"
   • price: 100
   • description: "Base"
   • short description: "Base"

5. Look at the various catalog_product_entity_int, catalog_product_entity_text, catalog_product_entity_varchar tables in the database, they contain only values for store_id = 0, this is good so far!

6. Still in the backend, go to System > Integrations and add a new integration with full access
7. Activate the new integration and take a note of the 'access token' (example: abc)
8. Execute the following curl command, which only updates the "name" attribute, using the storeview code from 3, and the access token from 7:

curl -k -X PUT -H "Accept: application/json" -H "Content-Type: application/json" -H "Authorization: Bearer abc" -d "{ \"product\": { \"name\": \"New name on storeview level\" }}" https://domain/rest/default/V1/products/base

1. Go back in the backend and take a look at the product, but switch to the 'Default Store View' scope.

2. It is expected that the

3. Also take a look at the database again, to the catalog_product_entity_int, catalog_product_entity_text, catalog_product_entity_varchar tables, it is expected you'll only see one entry for store_id = 1, but there are like 10 entries, not good!

Expected result

Only attribute where the checkmark "Use Default Value" is unchecked is for the attribute "name"

Actual result

All the following attributes now also have a value on storeview level, which is not what is expected:
- status
- tax class
- visibility
- description
- short description
- url key
- meta title
- meta keywords
- meta description
- display product options in

[bartlubbersen]

@magento-team,

Is there any update on this issue, it still occurs in Magento 2.1.7 and makes it impossible to update product storeview values through API. Because if after an storeview API call the default value is changed, it won't work because the value is already unintentionally overwritten on storeview level.

[TomashKhamlai]

Hello! Thank you for submission!
I tried to reproduce this issue. Here is the full list of actions that I performed:
1) Install Magento using composer from repo.magento.com
2) Go to Admin and create Integration: System -> Extension -> Integrations
3) Press 'Add New Integration' button
4) Fill required fields
5) Switch to 'API' tab
6) Select 'All' in the dropdown with label 'Resource Access'
7) Press 'Save and Activate' and then 'Allow' button
8) Copy the value from Access Token Input and press Done button
9) Go to Stores -> Settings -> All Stores
10) Click on 'Create Website' button
11) Fill 'Danish' in the 'Name' input field and 'danish' in the 'Code' input field
12) Click on 'Save Website' button
13) Click on 'Create Store' button
14) Make sure that 'Web Site' drop-down has 'Danish' value selected.
15) Type 'Danish Store' in 'Name' input field
16) Select 'Default Category' as a 'Root Category'
17) Click on 'Save Website' button
18) Click on 'Create Store View' button
19) Make sure that 'Store' drop-down has 'Danish Store' value selected.
20) Type 'Store view (Danish)' in the 'Name' input field
21) Type 'danish' in the 'Code' input field
22) Select 'Enabled' from 'Status' drop-down
23) Click on 'Save Store View' button

As a result, I have got this:

Web Site | Store | Store View
-- | -- | --
Danish | Danish Store | Store view (Danish)
Main Website | Main Website Store | Default Store View

24) POST the following JSON to {domain.com}/rest/V1/products (catalogProductRepositoryV1) - product status set to 2 (disabled):

{
    "saveOptions": true,
    "product": {
        "attribute_set_id": 10,
        "visibility": 4,
        "price": 120,
        "type_id": "virtual",
        "name": "Rest product",
        "extension_attributes": {
            "stock_item": {
                "manage_stock": false,
                "is_in_stock": true,
                "use_config_manage_stock": false
            }
        },
        "sku": "TK207582890",
        "status": 2
    }
}

Result:

{
    "id": 1,
    "sku": "TK207582890",
    "name": "Rest product",
    "attribute_set_id": 4,
    "price": 120,
    "status": 2,
    "visibility": 4,
    "type_id": "virtual",
    "created_at": "2017-08-04 13:34:53",
    "updated_at": "2017-08-04 13:34:53",
    "extension_attributes": {
        "stock_item": {
            "item_id": 1,
            "product_id": 1,
            "stock_id": 1,
            "qty": null,
            "is_in_stock": true,
            "is_qty_decimal": false,
            "show_default_notification_message": false,
            "use_config_min_qty": true,
            "min_qty": 0,
            "use_config_min_sale_qty": 1,
            "min_sale_qty": 1,
            "use_config_max_sale_qty": true,
            "max_sale_qty": 10000,
            "use_config_backorders": true,
            "backorders": 0,
            "use_config_notify_stock_qty": true,
            "notify_stock_qty": 1,
            "use_config_qty_increments": true,
            "qty_increments": 0,
            "use_config_enable_qty_inc": true,
            "enable_qty_increments": false,
            "use_config_manage_stock": false,
            "manage_stock": false,
            "low_stock_date": null,
            "is_decimal_divided": false,
            "stock_status_changed_auto": 0
        }
    },
    "product_links": [],
    "options": [],
    "media_gallery_entries": [],
    "tier_prices": [],
    "custom_attributes": [
        {
            "attribute_code": "category_ids",
            "value": []
        },
        {
            "attribute_code": "options_container",
            "value": "container2"
        },
        {
            "attribute_code": "required_options",
            "value": "0"
        },
        {
            "attribute_code": "has_options",
            "value": "0"
        },
        {
            "attribute_code": "url_key",
            "value": "rest-product"
        },
        {
            "attribute_code": "tax_class_id",
            "value": "2"
        }
    ]
}

25) Inspect the database with the following two queries:

SELECT 
    attribute_id
FROM 
    eav_attribute
WHERE
    attribute_code = 'status' AND 
    entity_type_id = (SELECT 
                entity_type_id 
              FROM 
                eav_entity_type
              WHERE
                entity_type_code = 'catalog_product'
            )

Result:

+--------------+
| attribute_id |
+--------------+
|           97 |
+--------------+
1 row in set (0.00 sec)

26) Use value from step 2 (I got 97) in the next query:

SELECT * FROM 
    catalog_product_entity_int
WHERE 
    attribute_id = 97

Result:

+----------+--------------+----------+-----------+-------+
| value_id | attribute_id | store_id | entity_id | value |
+----------+--------------+----------+-----------+-------+
|        1 |           97 |        0 |         1 |     2 |
+----------+--------------+----------+-----------+-------+
1 row in set (0.00 sec)

Unfortunately, I am not able to reproduce this issue.

[magento-engcom-team]

@giano574 we are closing this issue due to inactivity. If you'd like to update it, please reopen the issue.

[koenner01]

I can confirm this is still a problem in 2.2

We have an existing product with data on global.

When trying to update the product through api:
{domain}/index.php/rest/{storecode}/V1/products/{sku}

By sending data with PUT method:

{
    "product": {
        "name": "Storeview test name",
    }
}

The database gets polluted with values on storeview level (description, meta_description, status, visibility, ...) for this product even though we only changed the name!

the '/V1/products' call with a POST causes the same behaviour.

This is a very serious bug and has been referenced multiple times across github, but each time the issues are being closed with 'cannot reproduce'

related: https://github.com/magento/magento2/issues/4952

[stephansteiner]

Why has this been closed? We have the same problem here.

[TomashKhamlai]

@stephansteiner, may I enquire some details about your problem?
POST or PUT method?
Any themes, modules or extensions after Magento was installed?
What is your Magento version?
Is it on the production server?
Did you try the steps comment-320252626?

I can retest and ask someone to reopen this issue if you can provide the steps for reproducing.

[koenner01]

snippet of the wall of text from our cloud support ticket:

1) Setup a clean 2.2.2 installation

2) Create a new (extra) storeview:
=> Store='Main Website Store'
=> Name='Some Random Storeview'
=> Code='random'
=> Status='Enabled'
=> Sort Order='0'

3) Make sure the shop is using flat tables
=> Stores > Configuration > Catalog > Catalog > Storefront > Use Flat Catalog Category = Yes
=> Stores > Configuration > Catalog > Catalog > Storefront > Use Flat Catalog Product = Yes

4) Create a simple product and save it
=> Fill in all required attributes with random stuff
=> Fill in all fields with storeview/website scope in all the closed tabs
=> Make sure to add 2 random images and set the 1th as base/small_image/thumbnail/swatch

5) Take note of the values in the database:

• catalog_product_entity_int
• catalog_product_entity_text
• catalog_product_entity_text

Break stuff through the backend:

6) Switch the storeview to our newly created 'random' storeview
=> Don't change any data AND DON'T OPEN ANY TABS

8) Save the product on storeview scope
=> If you look at the closed tabs after the product save reloaded the page, you will see that the 'use default value' checkboxes have all been unchecked

9) Take note of the values in:

• catalog_product_entity_int
• catalog_product_entity_text
• catalog_product_entity_text
  => All attributes from the closed tabs that have storeview/website scope have just had a value saved to the database

10) Switch back to the 'All store views' scope

11) Change all product attributes that have scope storeview or website with new values and save the product
=> Make sure to select the 2th image as base/small_image/thumbnail/swatch!
=> Make sure to change the meta_title, meta_keywords and meta_description

12) Take note of the values in:

• catalog_product_entity_int
• catalog_product_entity_text
• catalog_product_entity_text
  => The values with store_id 0 will contain the correct value
  => values saved to the database for our new storeview during step 7 are incorrect

13) Look at the frontend where the product is using the incorrect data on our 'random' storeview
=> the image used as base/small_image/thumbail/swatch is wrong
=> the meta data in the header is wrong

The problem can be split into multiple origins:

• Part of the problem lies with the fact that closed tabs will uncheck all 'use default value' checkboxes. This causes discrepancies between values on storeviews because (unwanted) values are being saved to the database.
  You cannot assume that a user will open all tabs when he is editting a product, just to make sure the 'use default value' checkboxes get saved properly

• Another part of the problem is the fact that for images there is no 'use default value' option. This will cause problems on the frontend when the user assumes the value of default is being used, while actually an incorrectly saved value is being used.

Break stuff through the api:

6) Update the product's name through the api:
=> PUT '../rest/{OurRandomStoreviewCode}/V1/products/{createdProductSku}
=> request body: {"product":{"name":"Test API"}}

7) Take note of the values in:

• catalog_product_entity_int
• catalog_product_entity_text
• catalog_product_entity_text
  => Besides the product's name being updated on the storeview, the database tables got filled with unwanted values for attributes that have scope storeview/website

I can't even wrap my head around this problem.
We are updating 1 attribute but the database is being filled with a lot of other (unwanted) values. If we now check the product through the adminhtml the same behaviour as the closed tabs was reproduced. All attributes in the closed tabs have had their 'use default value' checkbox unchecked...

[stephansteiner]

Please reopen as it should NOT possible to have attributes like product 'status' to be different in storeviews

[bansavage]

Still a blatant issue in Mage 2.1.7

[magento2dev]

Still an issue on Magento 2.2.3

[magento2dev]

Hi @TomashKhamlai,
I follow all your steps but I'm still getting default values overwritten, you should make something different than us ? which Magento version do you use ?

Thanks a lot !

[hostep]

@magento2dev: it's still an issue indeed, let's re-open this, so this issue is more visible.
This issue is about products, there is a very similar issue for categories: https://github.com/magento/magento2/issues/17186

We've got a support ticket with the Magento Cloud guys for 7 months around this issue, latest reply on that issue from a few days ago didn't sound very promising, is probably going to take multiple more months until this finally gets fixed unfortunately :(

(just noticed you've changed your github username, haha, so you were already kind of in the loop)

[magento2dev]

Hi @hostep, thanks for reopened this ticket !
I'm hoping someone or magento team @magento-engcom-team will finally find a solution quickly it's a very important issue... Magento Cloud guys should be able to find a solution I don't think it's very hard the solve when you know the core files, I looked into some files but not able to find a good solution yet.

(yes sorry I updated my username few days ago ;)

[magento2dev]

Does anyone developed a fix for this ?
Thanks !

[ghost]

We were not able to reproduce this issue by provided steps. If you'd like to update it, please reopen the issue.

[hostep]

@engcom-backlog-pb : what version did you test this against?

I tested this a couple of weeks ago against Magento 2.2-develop and it was definitely still happening.
If I find some time, I'll try to find exact steps to reproduce. Since this is a very high priority issue which should be solved rather sooner then later.

[hostep]

Re-opening for the second time, because test wasn't done properly, multiple reporters above already added steps to reproduce, but let's do this yet again, hopefully you'll be finally able to reproduce and see that there is a very big problem here.

Steps to reproduce

This demonstration is only with a single storeview, but can be reproduced with multiple storeviews or multiple websites like already mentioned by a bunch of other reporters above

1. Setup a clean Magento installation using the 2.2-develop branch (I used commit eb84c5393da)
2. Go into the backend
3. Make a note of the default storeview code, it's default (we'll need this later when we create an API request)

4. Create a new product:

   • name: "Base"
   • SKU: "base"
   • price: 100
   • description: "Base"
   • short description: "Base"

5. Look at the various catalog_product_entity_int, catalog_product_entity_text, catalog_product_entity_varchar tables in the database, they contain only values for store_id = 0, this is good so far!

6. Still in the backend, go to System > Integrations and add a new integration with full access
7. Activate the new integration and take a note of the 'access token' (example: abc)
8. Execute the following curl command, which only updates the "name" attribute, using the storeview code from 3, and the access token from 7:

curl -k -X PUT -H "Accept: application/json" -H "Content-Type: application/json" -H "Authorization: Bearer abc" -d "{ \"product\": { \"name\": \"New name on storeview level\" }}" https://domain/rest/default/V1/products/base

1. Go back in the backend and take a look at the product, but switch to the 'Default Store View' scope.

2. It is expected that the only attribute where the checkmark "Use Default Value" is unchecked is for the attribute "name", because that's what we've updated using the API, but note that all the following attributes now also have a value on storeview level, which is not what is expected:

   • status
   • tax class
   • visibility
   • description
   • short description
   • url key
   • meta title
   • meta keywords
   • meta description
   • display product options in

3. Also take a look at the database again, to the catalog_product_entity_int, catalog_product_entity_text, catalog_product_entity_varchar tables, it is expected you'll only see one entry for store_id = 1, but there are like 10 entries, not good!

Extra discussion

The case where this becomes really annoying, is when you want to update a single attribute using the API from an external system, but still want people to manage the other product attributes manually using the Magento backend.
In that case, they are very confused as to why when they update a value on the 'All Store Views' level, it doesn't come through on the frontend. And then you start looking in the database and discovering that certain attribute values are being overwritten on the storeview level which is very unexpected, since nobody did this on purpose.
This is a very big problem in the API currently.

Fixing this would be tremendously appreciated!

[ghost]

@hostep, thank you for your update.
We've acknowledged the issue and added to our backlog.

[magento-engcom-team]

Hi @ishakhsuvarov. Thank you for working on this issue.
Looks like this issue is already verified and confirmed. But if your want to validate it one more time, please, go though the following instruction:

• [ ] 1. Add/Edit Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 2. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento-engcom-team give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!
  

• [ ] 3. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento-engcom-team give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• [ ] 4. If the issue is not relevant or is not reproducible any more, feel free to close it.

[magento-engcom-team]

@ishakhsuvarov Thank you for verifying the issue. Based on the provided information internal tickets MAGETWO-95586, MAGETWO-95587 were created

[hostep]

Just to add some more details after a year-long (and very frustrating) discussion with Magento Enterprise support about this bug. They finally admitted this is a bug and there might be plans to fix it, but it will take some time since the internal ticket still needs to be triaged, they are "guessing" that it might take 6 more months until we finally see a solution for this bug.

And then to add some more info, let me quote one of the developer's responses, maybe the community then has a better understanding of the problem and can maybe help in finding a solution.

ProductRepository belongs to the persistent layer and simply stores an entity in the database.
The decision whether to save a particular attribute or not save according to store scopes it's not a responsibility of ProductRepository.

Let's see how it does via Magento admin interface:
Information about using attribute value from the parent scope comes via POST param use_default.
https://github.com/magento/magento2/blob/2.2-develop/app/code/Magento/Catalog/Controller/Adminhtml/Product/Initialization/Helper.php#L202
Then product attributes data are filtered and updated according to use_default param.
https://github.com/magento/magento2/blob/2.2-develop/app/code/Magento/Catalog/Controller/Adminhtml/Product/Initialization/Helper/AttributeFilter.php#L28
If attribute value should be preserved from the parent scope it is unset for the current scope.

In case of saving product via Magento admin, we get use_default from the request.
To retrieve this data in the merchant script MagentoCatalogModelAttributeScopeOverriddenValue::containsValue should be used.
https://github.com/magento/magento2/blob/2.2-develop/app/code/Magento/Catalog/Model/Attribute/ScopeOverriddenValue.php#L79

How Magento set use_default checkbox values for product EAV attributes you can see here:
https://github.com/magento/magento2/blob/2.2-develop/app/code/Magento/Catalog/Ui/DataProvider/Product/Form/Modifier/Eav.php#L716

The responsibility of ProductRepository is to return product entity with data from a particular scope and to save current entity state in a particular scope.
So, in general, a merchant should process product attributes values before saving in a way similar to how it implemented for Magento admin interface.
Only after that ProductRepository can be used for storing product in DB.

[RandyCillekens]

@hostep - Did you find a solutions or a work around for this? We are having the same issue to and it is a real pain in the **s for not being able to use the API where it is meant for.

[hostep]

@RandyCillekens: no, sorry, no solution, the client just learned to live with the bug and changed how they dealt with catalog data as far as I remember (I'm no longer working on this project though).

[dwaynebartels]

We have dozens of clients who have this problem.

We often update only name as store-view specific (language). Because of this bug all store-view data becomes outdated very quickly.
We explain the client that this is a Magento side bug, send a link to the bug and then propose that their Magento developers solve the issue.
So then 3 parties are involved looking at this issue, which consumes allot of time. It would be nice if this bug gets more attention.

[francescorm]

Is dangerous to delete single tuple of the database of the specific store view?
Can i delete tuple inside 'catalog_product_entity_int' without any problem?

[ThijsVanDenWollenberg]

Any update on this?
Can someone confirm this is still an issue in 2.3.1 ?

[francescorm]

Yes, it's still an issue. No way to update products using API if you need a multistore.
I'm studying some plugin but i'm always waiting information about my question on deleting tuple from 'catalo_product_entity_int'

[stephansteiner]

I do regular cleanup in the database by running some queries and doing some reindex afterwards.
Example: delete all global/website attribute values (like product status) from store-views:

delete from catalog_product_entity_int where attribute_id in (select ea.attribute_id from eav_attribute ea, catalog_eav_attribute cea where ea.attribute_id = cea.attribute_id and cea.is_global in (1,2)) and store_id<>0;

You can always change the scope of some default attributes in Magento to global like "status", images. But it depends on your shop requirements you need to implement.

[m2-assistant[bot]]

Hi @AfreenScarlet. Thank you for working on this issue.
Looks like this issue is already verified and confirmed. But if you want to validate it one more time, please, go though the following instruction:

• [ ] 1. Add/Edit Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• [ ] 2. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento-engcom-team give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and _stop verification process here_!
  

• [ ] 3. Verify that the issue is reproducible on 2.2-develop branch.

  Details

  - Add the comment @magento-engcom-team give me 2.2-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x

• [ ] 4. If the issue is not relevant or is not reproducible any more, feel free to close it.

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[mattlucian]

I discovered that if you pass a "null" value in the PUT request for the default store, it will override the current value to be "use default value"

PUT - {domain}/rest/default/V1/products/{sku}

{
    "product": {
        "name": "Storeview test name",
        "visibility" : null
    }
}

After this request - regardless of what visibility was prior on the product - it should now be grayed out and have "Use Default Value" checked for visibility.

This doesn't really solve the underlying issue and/or all use-cases behind this. But posting it here in case it helps others.

[erikrijn]

This issue has been open for 2+ years. It's causing major issues.
Will this ever be fixed?

[LaurensGoedel]

Issue still existing in 2.3.2. It is causing major issues for us too since 50% of our clients feed the products from other sources and using the API for importing them into Magento.

[Zyles]

This needs to be fixed.

Re-produced on 2.4-develop and has cause MASSIVE problems for us.

The rest API is unusable. We can't update anything because it messes up attributes on every single product.

If we disable one product we have to disable it on multiple storefronts.

If we change one attribute value we have to change it on multiple storefronts.

Makes the scoping and sharing of attribute data completely useless.

This issue alone is making us move away from Magento to another platform.

[chris-pook]

This also applies to the product image data. When saving a product via API the image data such as position and hidden status in catalog_product_entity_media_gallery_value is also duplicated to the storeview level, one additional row per image. Admin users now need to make any future image changes in store view context and the "all store views" position/hidden status become redundant. Very frustrating.

[Zyles]

Why is this not confirmed and reproduced on 2.4-develop?

It is reproduced on 2.2.x and work has not begun yet to fix it?!

[leeroybrun]

So disappointing to see that after THREE WHOLE YEARS this major bug is still there...

Has anyone found a good solution/workaround?

After many hours of wandering the web about this, I stumbled upon this : https://github.com/Vendic/EAVCleaner

It adds an eav:attributes:restore-use-default-value command to bin/magento which should

Check if product attribute admin value and storeview value are the same, so "use default" doesn't work anymore. Delete the storeview values.

I have not tried it yet, but it seems quite promising as a workaround if it is run periodically by cron.

[gabrieldagama]

Duplicate of https://github.com/magento/magento2/issues/26484