Azure-cli: Improves the logic of determining blob content type in upload-batch.

[troydai]

In issue #1436

I'm trying to use the "upload-batch" command to upload the static content of my web application.

However, I have the problem of every file ending up with the content-type "application/octet-stream", and in some cases (CSS, for example) the files need to have a specific content-type for browsers to interpret them. It would be nice for "upload-batch" to use the correct content-type for well-known extensions, or at least provide a way to specify the mapping in the command line.

At this time the only solution I see is to issue a separate "upload-batch" command for each extension using the available flags --content-type "text/css" --pattern "*.css", but this is a bit tedious.

Is this issue related to the problem I described?

[bacongobbler]

We just migrated some of our assets from S3 to wabs using a combination of

s3cmd sync s3://mybucket $PWD
az storage blob upload-batch -c mycontainer -s $PWD

And as it tuned out, our PNG images from the old S3 bucket were uploaded to Azure with content-type application/octet-stream as well. s3cmd (or s3 itself?) would've automatically handled this and set more sane content types for files, so I'm definitely on board for better content type resolution dependent on the file extension.

Our fix to this issue was to update the metadata of the files after the fact using

az storage blob update -c mycontainer -n myimage.png --content-type image/png

Just posting for others as a workaround if they hit this as well. :)

[SteveALee]

AzCopy allow's the type to be left out and the copy operation figures out the MIME type from the file extension.

We really need that for the az blob upload commands as well!!! (the feature not the exact syntax)

[troydai]

Move this to sprint 21 for potential changes per popular support.

[SteveALee]

@troydai What I really want is a command to syncronise a container etc to a local directory tree while setting the contentType for all files. That means it should remove files that no longer appear in the local directory tree. I have found no such option or other portable CLI tool to do this. this is critical in applications including a static web site.

Does this exist and I missed it? Surely this must be the case? Or should I raise I new issue to request it?

[troydai]

@SteveALee can you please open a new issue and give a few examples of your idea? Thanks.

[SteveALee]

@troydai troydai #4274

[ShannonHickey]

I'm attempting to replicate our use of the AWS S3 cli with the Azure Storage CLI. I was very surprised that the Azure CLI doesn't guess content types like the AWS S3 cli. Like @edevil in the original issue, I want to upload a bunch of content for a static site and I'm currently forced to break it down into separate batch commands for each content type. Please add content type guessing to the Azure CLI.

[troydai]

Thanks for the advice @ShannonHickey.

[troydai]

Feature is added in https://github.com/Azure/azure-cli/pull/4342

[ShannonHickey]

@troydai this is excellent news. Is there any documentation on which types can be guessed (and whether it's filename-based and/or content-based)? Also, is it extensible? Could we, for example, add an extension->content-type mapping to the list?

[bacongobbler]

From the code, yes. It guesses the content type based on Python's mimetypes package which has an add_type function to add more file extensions to the know mimetypes. I'd open a PR for any extensions you'd like to add to the list that are not being detected.

[bacongobbler]

Or perhaps you could write a follow-up PR which can read extensions from ~/.azure/mime.types with mimetypes.read_mime_types(). That would make it more scalable in the future rather than writing a new PR every time a new extension comes up that isn't captured by the mimetypes package.

[troydai]

@bacongobbler thank you for the suggestion. @ShannonHickey, we are using the Python mimetypes module to guess content type based on file names. Currently, we use its default mapping, which should be able to recognize most of the command file types. I'm not sure about the range of the files types we may need to deal with or if there are additional types so I begin with the most conservative implementation. There are options for us to make it more extensible in the future. The module can either infer types based on settings in your OS or we can provide meanings for you to define your own mapping.

Please use the command and let us know your experience, and feel free to open new issues if you have additional requirements.

[SteveALee]

@troydai How should we track when this is released? Thanks

[troydai]

It is just released last week in azure-cli 2.0.16 (storage module 2.0.14). Please give it a try and let me know if there is anything we can improve.

[SteveALee]

Great thanks I will do.

I looked on the releases but they seem to be out of date given what you say above.

[SteveALee]

oops, I meant - https://docs.microsoft.com/en-us/cli/azure/release-notes-azure-cli?view=azure-cli-latest

[troydai]

Thank you, it may take a few days for the page to be refreshed. We did send our release note. Thanks for the reminding, I will contact the doc team.