Pim-community-dev: API: Update of multiselect product value silently fails when data is not array

[pardahlman]

Hello, team 👋

I'm running Akeneo PIM 1.7.2 CE, here's my system data:
akeneo-pim-system-info_2017-04-23_17-21.txt

I have an multi-select attribute (pim_catalog_multiselect) that I wanted to add to an existing product through the create/update a product API endpoint. I didn't find any documentation on how to structure the values for this attribute type, so I tried different approaches. I noticed that if the value of the data property isn't an array the API returning a 200 OK without updating the product.

✔️ Correct data format, accepted by API

{
    "multiselect_attribute": [{
        "data": ["first", "second", "thrid"],
        "locale": null,
        "scope": null
    }]
}

❌ Wrong data format, silently fails in API

{
    "multiselect_attribute": [{
        "data": "first",
        "locale": null,
        "scope": null
    }]
}

In fact, the value of data can be any string (like non-valid) and the API accepts it.

This behavior only occurs for the update endpoint. If I try to do the same when creating a product the API responds with 422 and the message: _Property "multiselect_attribute" expects an array as data, "string" given. Check the standard format documentation."_

I would expect a similar response from the update endpoint as well.

_PS. I wouldn't mind a better documentation on how to send in attribute values, perhaps one example per attribute type?_

[ahocquard]

Hello,

I could not reproduce the problem. It always send a 422 error (by creating or updating the product) :

• Request done on the icecat catalog:

PATCH /api/rest/v1/products/17727158

 {
  "identifier": "17727158",
  "values": {
        "tshirt_style": [
        {
          "locale": null,
          "scope": null,
          "data": "toto"
        }
    ]
  }
}

• Response :

{
  "code": 422,
  "message": "Property \"tshirt_style\" expects an array as data, \"string\" given. Check the standard format documentation.",
  "_links": {
    "documentation": {
      "href": "http://api.akeneo.com/api-reference.html#patch_products__code_"
    }
  }
}

I think that the error code 204 No content is returned when the problem occurs, not `200 OK, isn't it ?

Could you try to reproduce the problem on the catalog _icecat_ and send me the request that you have done ?

Thanks !

[pardahlman]

Hello @ahocquard - thanks for getting back to me 👍

I realized that I was running 1.7.2 and that the format of the json in the issue was a bit of. I've updated my original post accordingly. I don't have a running _icecat_ instance running, but if you give me a few days I will give it a go.

[NolwennP]

Hi @pardahlman!

Examples of attribute values format depending on their types can be found here: https://api.akeneo.com/documentation/resources.html#product-values

Hope this help!

[pardahlman]

@NolwennP - thanks, that is _very_ helpful!

[pardahlman]

@ahocquard I haven't gotten around to test this properly, but I just noticed that I get a 200 OK when I try to add a media file with malformed file property.

This is what I'm seeing in the logs:

[2017-04-26 08:22:47] request.ERROR: Uncaught PHP Exception 
Symfony\Component\HttpKernel\Exception\BadRequestHttpException:
"Invalid json message received" at /akeneo/vendor/akeneo/pim-community
-dev/src/Pim/Bundle/ApiBundle/Controller/MediaFileController.php line 
356 {"exception":"[object] (Symfony\\Component\\HttpKernel\\Exception\\
BadRequestHttpException(code: 0): Invalid json message received at /akeneo/
vendor/akeneo/pim-community-dev/src/Pim/Bundle/ApiBundle/Controller/
MediaFileController.php:356)"} []

[ahocquard]

@pardahlman as far as I understand, you can't reproduce the initial problem and you have another problem with media?

If yes,could you create another issue to properly resolve the problem and be able to reference it in a ticket if needed?

What I don't understand in your message is that it's the property file that is malformed, but actually it's the product content that is not a valid json. And btw, what do you mean by "malformed" for a file?

Could you provide (in another issue) the request that you did (with curl for example, postman can generate it for you)?

I cannot reproduce it and I don't understand why symfony does not handle your exception (it should at least a 500).

Thanks!

[pardahlman]

Hello @ahocquard - first of, yes you are right - it was the product part of the request that was malformed (it surprised me that the product property was supposed to contain a json formed string which is why I assumed that it had to do with the file since I haven't found any documentation that verifies its format.).

The reason I didn't open a separate issue is that it felt related to the original problem report, just on a different end point. From issue description

I noticed that if the value of the data property isn't an array the API returning a 200 OK

From latest comment

I just noticed that I get a 200 OK when I try to add a media file with malformed file property

The conclusion I drew was that the problem might be connected with unhanded exceptions in the API, as I saw the following patterh

1. request with malformed payload
2. logs report un handled exception
3. api returns 200 OK

I'd be happy to open a separate issue and provide the information if you still think it's two different issues.

[ahocquard]

Hello @pardahlman,

Ok you're right, I think it's a problem coming from your environment/installation. Let's keep it in this issue.

Could you :

• tell me if you have any customisation in the PIM project
• send me the curl request (with the -v option to know exactly the request that is really sent) + response please. If you can reproduce it with the icecat catalog, it would be a bonus :)

I can't reproduce, I get an error code 400 if the product doesn't contain a valid JSON.

Thanks.

[pardahlman]

Hello @ahocquard 🙂

I'm running a very vanilla installation of Akeneo on a newly created docker container. The image as well as the docker-compose file I use are open sources, so you should be able to setup an identical environment by using this docker-compose file.

In the pursuit of fixing https://github.com/akeneo/pim-docs/issues/565 I also install exif and imagemagick, but I verified that the problem exists on a newly created docker container too.

I've captured the traffic, but currently can't export the cURL with a body 😞 Until I figure out what the problem is, you can have a look at the raw traffic.

Using the same attribute named multiselect_attribute with three valid options (first, second, third).

Trying to create new product

Creating a simple product with the following structure

{
    "identifier": "product_001",
    "enabled": false,
    "values": {
        "multiselect_attribute": [{
            "data": "foo",
            "locale": null,
            "scope": null
        }]
    }
}

Here's the request

POST http://127.0.0.1:8080/api/rest/v1/products HTTP/1.1
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Accept-Encoding: gzip, deflate
Authorization: Bearer ZDc1MzIyM2IxMjZhMTZiMmRmNTZlMGU0NTA4YjIxMTcxN2Q1MmJiYzJlMTEzM2RjYjc0YTE4YzUxMjRlZjU0NA
Cookie: BAPID=60ba5bf4cab7a4339c43bf1f676f94ba
Content-Length: 123
Host: 127.0.0.1:8080

{"identifier":"product_001","enabled":false,"values":{"multiselect_attribute":[{"data":"foo","locale":null,"scope":null}]}}

And here's the response

HTTP/1.1 422 Unprocessable Entity
Date: Fri, 28 Apr 2017 08:38:35 GMT
Server: Apache/2.4.10 (Debian)
X-Powered-By: PHP/5.6.30
Cache-Control: no-cache
Content-Length: 240
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Content-Type: application/json

{"code":422,"message":"Property \"multiselect_attribute\" expects an array as data, \"string\" given. Check the standard format documentation.","_links":{"documentation":{"href":"http:\/\/api.akeneo.com\/api-reference.html#post_products"}}}

This works as expected.

Create and then update

I created the same product, but without any values at all

POST http://127.0.0.1:8080/api/rest/v1/products HTTP/1.1
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Accept-Encoding: gzip, deflate
Authorization: Bearer OTVhYTE3NWVhNDNiZTIwNzY3MTAzZjg0M2IyOWNmZDI3ODVjNjFjNzA0MTUyY2JiZTE4MmY0NGQ2ZGRlNWVlMg
Cookie: BAPID=be2a69dfc26626692b8ff238be670cfc
Content-Length: 44
Host: 127.0.0.1:8080

{"identifier":"product_001","enabled":false}

The API responds with a 201 Created (as expected)

HTTP/1.1 201 Created
Date: Fri, 28 Apr 2017 08:40:13 GMT
Server: Apache/2.4.10 (Debian)
X-Powered-By: PHP/5.6.30
Cache-Control: no-cache
Location: http://127.0.0.1:8080/api/rest/v1/products/product_001
Content-Length: 0
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Content-Type: application/json

Then calling create/update endpoint with same payload as first request

{
    "identifier": "product_001",
    "enabled": false,
    "values": {
        "multiselect_attribute": [{
            "data": "foo",
            "locale": null,
            "scope": null
        }]
    }
}

PATCH http://127.0.0.1:8080/api/rest/v1/products/product_001 HTTP/1.1
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Accept-Encoding: gzip, deflate
Authorization: Bearer OTVhYTE3NWVhNDNiZTIwNzY3MTAzZjg0M2IyOWNmZDI3ODVjNjFjNzA0MTUyY2JiZTE4MmY0NGQ2ZGRlNWVlMg
Cookie: BAPID=be2a69dfc26626692b8ff238be670cfc
Content-Length: 123
Host: 127.0.0.1:8080

{"identifier":"product_001","enabled":false,"values":{"multiselect_attribute":[{"data":"foo","locale":null,"scope":null}]}}

Returns a 200 OK with what looks like a binary body

HTTP/1.1 200 OK
Date: Fri, 28 Apr 2017 08:40:20 GMT
Server: Apache/2.4.10 (Debian)
X-Powered-By: PHP/5.6.30
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 327
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8

       U  j @
E  
1ݴ xh  1  hv  l+   
B  rBZ    ޣ e 맺]!  m @ ,/ @߉:ɐ ѓ H     \@ %8 p
  # V > J /     = l  n 

The logs are correct, though

[2017-04-28 10:40:21] request.ERROR: Uncaught PHP Exception Pim\Component\Api\Exception\DocumentedHttpException: "Property "multiselect_attribute" expects an array as data, "string" given. Check the standar
d format documentation." at /akeneo/vendor/akeneo/pim-community-dev/src/Pim/Bundle/ApiBundle/Controller/ProductController.php line 374 {"exception":"[object] (Pim\\Component\\Api\\Exception\\DocumentedHttpE
xception(code: 0): Property \"multiselect_attribute\" expects an array as data, \"string\" given. Check the standard format documentation. at /akeneo/vendor/akeneo/pim-community-dev/src/Pim/Bundle/ApiBundle
/Controller/ProductController.php:374, Akeneo\\Component\\StorageUtils\\Exception\\InvalidPropertyTypeException(code: 200): Property \"multiselect_attribute\" expects an array as data, \"string\" given. at
/akeneo/vendor/akeneo/pim-community-dev/src/Akeneo/Component/StorageUtils/Exception/InvalidPropertyTypeException.php:205)"} []

[pardahlman]

OK, I think you are right that it has to do with my environment. I just realized that the new container I created didn't have the output_buffering value set in its php.ini (as you poined out in #6060). I updated the ini file with that value and now the response reads like this

<br />
<b>Warning</b>:  sort() expects parameter 1 to be array, string given in <b>/akeneo/vendor/akeneo/pim-community-dev/src/Pim/Component/Catalog/Comparator/Attribute/OptionsComparator.php</b> on line <b>43</b><br />
{"code":422,"message":"Property \"multiselect_attribute\" expects an array as data, \"string\" given. Check the standard format documentation.","_links":{"documentation":{"href":"http:\/\/api.akeneo.com\/api-reference.html#patch_products__code_"}}}

This is my current php.ini file

date.timezone="Europe/Stockholm"
memory_limit=768M
short_open_tag=1
error_reporting = E_ALL | E_STRICT
error_log = /var/log/php_errors.log
output_buffering=4096

Are there any other values that should be set?

[ahocquard]

Ok, I can reproduce your problem!

2 things:

• when you don't activate output_buffering, it seems that the content encoding is gzip. I don't know why. But that's why you have encoded characters in the content of the response. You should have the same thing in the UI of Akeneo, but your browser should decode that automatically for you. Not the .NET client (I suppose).

• when you activate output_buffering, you can see the content. There is a php warning. That's why you have an error code 200. The warning is sent to your client, so it starts the response with the default http code 200. Then the exception is thrown but your header are already sent ( (you can't change the header once it has been sent).

This is due to the comparators that are called when we are updating a product (not creating a product). Those comparators don't check the format of the received data. It's done after :/

https://github.com/akeneo/pim-community-dev/blob/master/src/Pim/Component/Catalog/Comparator/Attribute/OptionsComparator.php#L43

By reading the comparators, you can delete the datakey and you will have another notice.

It's not a very big issue because the problem is not on the data itself, but on the format when it's wrong.

Now, I would be very interested to reproduce it with the media !

To answer to your last question, I think it's ok with those settings. You could even put ouput_buffering Off and discover why the content is encoded..

[pardahlman]

Thanks for through investigation! I should have noticed the Content-Encoding header, thanks for pointing it out.

I'm not sure that I can reproduce the media request, as I've been tweaking that code quite a few times lately. However, I believe that at the point I reported the problem the entire request body was formatted in json, but was accepted by the api since it expected application/json since the accept header was missing - so it not surprising that something exploded during the request handling 🙂 . (as a side note, with the json formatted body I actually got the API to recognize the product at some point 😛 )

[mickaelandrieu]

@pardahlman I've quickly reviewed the thread: on this specific issue, can we consider this is not a bug?

I think we can close, but there is no hurry here :)

[ahocquard]

@mickaelandrieu there are bugs, generating PHP notice. It's when the structure is not well formatted, so, it's not a big bug. But still an error ;) I will try to fix it in the v1.8. We can close it imho.

[mickaelandrieu]

Let's keep it open for the history until it's fixed, we don't want to lose this damn good feedbacks from @pardahlman ;)

[jmleroux]

This version is not supported anymore and i will close the ticket.
Do not hesitate to reopen one if you still have the problem on recent Akeneo PIM versions.

Regards