Flurl always sets Content-Type for multipart/formdata string fields, which is not standards-compliant.

Created on 21 Nov 2018  路  6Comments  路  Source: tmenier/Flurl

It is known that the multipart form uploads using the default algorithm set content-type header for the string fields too. However, this is in violation of the HTML 5.2 standard:

The parts of the generated url resource that correspond to non-file fields must not have a Content-Type header specified. Their names and values must be encoded using the character encoding selected above.

The only way to generate a form that adheres to this is to add the fields one way or another, then remove the Content-Type header of each item.

We've run into problems with server apps that (perhaps hastily) assume that a part that has Content-Type: text/plain; charset=utf-8 is not actually an uploaded form field and hence the named field is considered not submitted by the app, and needed to produce a workaround.

I.e. a POST of form

POST /some-url/ HTTP/1.1
Content-Type: multipart/form-data; boundary="185d818a-28ca-4af6-ae86-7e23975bfd01"
Host: 127.0.0.1:12345
Content-Length: 232589
Expect: 100-continue
Accept-Encoding: gzip, deflate

--185d818a-28ca-4af6-ae86-7e23975bfd01
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name=some-label

value
--185d818a-28ca-4af6-ae86-7e23975bfd01
....

led to the server complaining that the class parameter was missing, while it would work if the code was modified so that the Content-Type header was removed from the added part (for example)

var sc = new StringContent("value");
sc.Headers.Add("Content-Disposition", "form-data; name=\"some-label\"");
sc.Headers.Remove("Content-Type");

Since our code is supposed to mimic a browser form, then my opinion is that Flurl should have means for actually producing compliant multipart/form-data bodies.

bug

Most helpful comment

This change makes it no longer possible to add json to a multi-part form upload. The AddJson method no longer adds the appropriate Content-Type, which makes it impossible to know if the incoming content is encoded as JSON, rendering this method useless. I was able to specify the content type, but only through the Add method and manually serializing the JSON data. It's pretty counter intuitive.

According to RFC 7578

4.4. Content-Type Header Field for Each Part
Each part MAY have an (optional) "Content-Type" header field, which
defaults to "text/plain". If the contents of a file are to be sent,
the file data SHOULD be labeled with an appropriate media type, if
known, or "application/octet-stream".

All 6 comments

Interesting, I wasn't aware of that. Flurl's multipart implementation is based on many examples with HttpClient that I've seen on Stack Overflow and elsewhere, virtually all of which involve adding a StringContent object to a MultipartFormDataContent. It appears that StringContent is adding the header here.

Since this hasn't come up before with Flurl, and I couldn't find any mention of it in any of those Stack Overflow answers, my guess is most server apps are forgiving of it. That said, I don't want to be out of standards compliance, so I am willing to fix this. A proper fix would probably be to use a ByteArrayContent, which StringContent inherits from and doesn't appear to add the header. I imagine that will work, but if for some reason it doesn't, removing the header will be my fallback.

Thanks for reporting this.

It is true that most ignore it - for example Python cgi module will consider a part to be a file upload only when it has Content-Disposition with nonempty filename. And what I read in RFCs it is true that older RFCs specifically ask to provide a content-type for form fields.

Anyway, since I've perfected the bug report in the other software and it is now easily reproducible, the other software in question is Minio S3-compatible file storage; https://github.com/minio/minio/issues/6844 is the issue - there's one working curl command that can upload a file to their sandbox server... these bugs are annoying in that in essence flurl is not conservative enough and minio is not liberal enough and both are kinda wrong now :D

This change makes it no longer possible to add json to a multi-part form upload. The AddJson method no longer adds the appropriate Content-Type, which makes it impossible to know if the incoming content is encoded as JSON, rendering this method useless. I was able to specify the content type, but only through the Add method and manually serializing the JSON data. It's pretty counter intuitive.

According to RFC 7578

4.4. Content-Type Header Field for Each Part
Each part MAY have an (optional) "Content-Type" header field, which
defaults to "text/plain". If the contents of a file are to be sent,
the file data SHOULD be labeled with an appropriate media type, if
known, or "application/octet-stream".

Well, we seem to have conflicting standards. I'm inclined to agree with @ztittle on how Flurl should work. I'm moving this to a new issue: #452.

@ztane It appears the bug you referenced was fixed. Can you confirm whether providing Content-Type to non-file parts no longer causes errors with Minio? Also, just curious, are you using AddJson specifically?

@ztittle This will happen in 3.0, so it could be a little while before that's in stable release. What you're doing now is the right work-around, you could even encapsulate it in an extension method like AddJsonWithHeader or something if you use it a lot.

@ztane @ztittle It turned out to be pretty simple to satisfy both of your use cases I think. Have a look at my latest comment in #452 and let me know if you have thoughts.

452 is released in prerelease 5: https://www.nuget.org/packages/Flurl.Http/3.0.0-pre5

Was this page helpful?
0 / 5 - 0 ratings