Terraform-provider-azurerm: Support for CosmosDB encryption with CMK

[jjl109]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

CosmosDB accounts now support encryption using customer-managed keys.
This is enabled as shown in the documentation here: https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-setup-cmk#create-a-new-azure-cosmos-account

It should be possible to do this with Terraform also.

New or Affected Resource(s)

• azurerm_cosmosdb_account

Potential Terraform Configuration

Probably this should be provided in a block like:

resource "azurerm_cosmosdb_account" "db" {
  ...
  encryption {
    key_uri = "https://<my-vault>.vault.azure.net/keys/<my-key>"
  }
  ...
}

References

• #0000

[ArcturusZhang]

Hi @jjl109 thanks for this issue!

I am working at this and had some investigation. It turns out you could create a cosmosdb account with CMK enabled, but you could not enable CMK on the existing cosmosdb account.

I have opened an issue in azure-rest-api-specs to track this.

This is required if we want to implement the CMK using the similar way with storage or kusto cluster

[jjl109]

Thank you @ArcturusZhang for looking into this.

It seems to me that the behaviour of the REST API is by design, because currently it's only possible to enable CMK on CosmosDB at account creation time even in the Portal. See the documentation:

Currently, customer-managed keys are available only for new Azure Cosmos accounts. You should configure them during account creation.

CosmosDB also authenticates differently with Key Vault, using some kind of central identity:

... Then, search for Azure Cosmos DB principal and select it (to make it easier to find, you can also search by principal ID: a232010e-820c-4083-83bb-3ace5fc29d0b for any Azure region except Azure Government regions where the principal ID is 57506a73-e302-42a9-b869-6f12d9ec29e9).

As there seems to be a more fundamental difference in the way CMK is implemented in CosmosDB versus other services, I would suggest to implement this in Terraform according to its current behaviour. That is, make it that adding or removing the encryption block/key_uri property forces the CosmosDB account to be recreated.

[ArcturusZhang]

Thanks for the doc page link! I did not notice this on my previous attempts.

I know the authentication on keyvault and had succeeded enabling CMK on a new created cosmosdb account (with inline CMK).
And for the record, the principal id hardcoded in the doc page is not always right - at least in my subscription, the principal id is neither of those mentioned - the best practice should be using a data source to get the service principal by the display name

[ArcturusZhang]

Hi @jjl109 I just got the reply from the service team and they are planning to enable the CMK feature on the existing accounts.

Therefore I suppose it would be better to wait for this functionality to land to avoid any potential breaking changes or inconsistencies. Sorry for the inconvenience.

[jjl109]

No worries, thanks for looking into it.
Is there any indication though that the functionality will work any differently from an API perspective once they allow it to be enabled on existing accounts? Presumably it will be the same, just with the additional bonus of being able to enable it on an existing account.

[jjl109]

I'd like to re-ignite this discussion @ArcturusZhang - I don't see why provider support for using CMK with Cosmos needs to wait for what is potentially a much more complex change.
IMO the provider should as much as possible try to keep up with the Azure service feature set as it exists now.
Unfortunately we've had to revert to using ARM templates due to this.

[fascynacja]

Thank you for the current activity on the issue.
I personally can afford recreating the db account to apply new changes. I prefer to keep my config in terraform than switching to ARM templates.

[tillig]

I just got the reply from the service team and they are planning to enable the CMK feature on the existing accounts.

Therefore I suppose it would be better to wait for this functionality to land to avoid any potential breaking changes or inconsistencies. Sorry for the inconvenience.

@ArcturusZhang was there any sort of timeline on that? I honestly don't mean to sound pushy, I'm just trying to decide whether it's worth me going down the ARM template route right now or whether this is one of those "imminently arriving" things it might be worth waiting a month (or whatever) to see.

[petems]

Awesome, thanks for the assist everyone who reviewed! 😄

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!