Azure-cli: az cosmosdb collection create not same as in portal

[davejhahn]

When using az cosmos collection create and --partition-key "", the key created is not the same as when creating it through the portal, and results in errors when inserting records.

To Reproduce
az cosmosdb collection create --collection-name --partition-key-path "/_id'" --db-name --resource-group --key --url

az cosmosdb collection create --collection-name --db-name --resource-group --key --url

The shard key returned looks like:
"partitionKey": {
"kind": "Hash",
"paths": [
"/_id"
]
}

When inserting, using the mongoimport.exe using upsert mode, you will get "error inserting documents: query in command must target a single shard key" for each insert (example json below) or with insert you'll get "error inserting documents: document does not contain shard key"

Example json:
[
{
"_id" : "8cd01693-b6be-4bc6-afce-a9f2015523d5",
"SomeThing" : "Test",
"SomeOtherThing" : "This is what it does",
"Severity" : "high"
}
]

When using the portal to create the partition key, using "_id" and doing a az cosmosdb collection show, as above, you'll see it as:

"partitionKey": {
  "kind": "Hash",
  "paths": [
    "/'$v'/_id/'$v'"
  ]
}

And insert or upsert works fine.

You'll see in the portal, for both methods, it shows "_/id" as the shard key.

There are some posts about this same problem, with workarounds inserting the path with the $v in it, however in azure cli 2.0.55 (and higher) at least it doesn't accept it or pass it as maybe it once did.

https://stackoverflow.com/questions/47399720/azure-cli-cosmosdb-partition-key-path-doesnt-work-without-v-before-and-af
https://stackoverflow.com/questions/45587691/comosdb-mongoapi-document-does-not-contain-shard-key

As it stands, I can't use it to create a new collection, I have to manually create it through the portal. Not sure if there is some known way that the shard key should be passed to get it , but following the examples from the two posts above gives me:

Invalid Arg {"Errors":["The partition key component definition path '\/'\'_id\/'\'' could not be accepted, failed near position '9'. Partition key paths
must contain only valid characters and not contain a trailing slash or wildcard character."]}

Maybe this worked previously? Sounded like the posters got passed it, but that was in late 2017.

Thanks.

[pjeannet-dior]

Hello, we have the exact same issue and the workaround posts did work. It took us a lot of time to find the issue (as well as this post), thanks to the author and I hope it will be corected soon.
It is fairly easy to reproduce BTW.

[DanielDiBe]

I had to escape the key in the following way:
az cosmosdb collection create --collection-name --partition-key-path "/'\$v'/_id/'\$v'" --db-name --resource-group --key --url

I hope that helps.

[davejhahn]

@DanielDiBe thanks, I will give this a try.

[shurd]

@davejhahn thank you for opening this issue. Are you still seeing the same problem? I tried to replicate it, but the portal and CLI appeared to be consistent when creating partition key paths. In both cases, I provided "/_id" and I can see that as the partition key path in the portal and in the CLI when using the show command. I did not see '$v' anywhere in the response.

az cosmosdb collection create -g -n -d -c --partition-key-path "/_id"

[davejhahn]

@shurd I will try again. I did not have success previously--I tried encoding every which way I found people doing it and none of them worked. It sounds like you are saying it just works without encoding now.

Not sure if this was something addressed at the Azure level or Azure CLI, just to be sure, I'll make sure to try with the latest version of the Azure CLI just to cover it.

[shurd]

I looked more into this issue, and it looks like this is a problem for MongoDB accounts only right now. When setting the partition key while creating a collection for a MongoDB account, you need to include "/'$v'" at the beginning and end of the partition key as well as between property levels.

For example, if you have the document:

{
    "id" : "some_id",
    "company": {
        "id": "345"
    }
}

and want to create a shard/partition key based on company.id (/company/id), then you would need to supply the following: /'$v'/company/'$v'/id/'$v'.

We are currently working on updating our database and collection groups to use our resource provider instead of the data plane python package it currently uses (azure-cosmos). When this change is made, the partition key will no longer have to be supplied in this format. We are currently tracking this internally as work item 416566, and the target ETA is September 2019.

[shurd]

Hi, in version 2.0.71 of the CLI, we released a preview version of the new dataplane resource management. As a part of this change, there is a fix to this problem so that you can easily supply a shard key when creating a Mongo DB collection.

Previous Command:

az cosmosdb collection create --collection-name {collection_name} --partition-key-path "/_id" --db-name {database_name} -g {resource_group_name} -n {database_account_name}

New (working) command:

az cosmosdb mongodb collection create --account-name {database_account_name} -g {resource_group_name} --shard "_id" --database-name {database_name} -name {collection_name}

This new command automatically appends the required /'$v' in the shard key.

[davejhahn]

Worked beautifully, thanks!

Dave

On Mon, Aug 19, 2019 at 1:52 PM shurd [e-mail] wrote:

Hi, in version 2.0.71 of the CLI, we released a preview version of the new
dataplane resource management. As a part of this change, there is a fix to
this problem so that you can easily supply a shard key when creating a
Mongo DB collection.

Previous Command:

az cosmosdb collection create --collection-name {collection_name} --partition-key-path "/_id" --db-name {database_name} -g {resource_group_name} -n {database_account_name}

New (working) command:

az cosmosdb mongodb collection create --account-name {database_account_name} -g {resource_group_name} --shard "_id" --database-name {database_name} -name {collection_name}

This new command automatically appends the required /'$v' in the shard key.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Azure/azure-cli/issues/8633?email_source=notifications&email_token=AAE4LAIVSEQV6S5C2NL4WSTQFLTU7A5CNFSM4G2EMOA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4T6HJI#issuecomment-522707877,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE4LAKQDIOVRYIEFAEQLWTQFLTU7ANCNFSM4G2EMOAQ
.

--

Dave Hahn
m. 414-803-6713 | LinkedIn http://www.linkedin.com/in/davejhahn/