Azure-docs: Unable to use managementGroups scope for usageDetails API

[AndrewGailer]

When trying to use /providers/Microsoft.Management/managementGroups/MyManagementGroup as the scope to call the usageDetails API, the following error is returned:

{
    "error": {
        "code": "400",
        "message": "The Principal Id cannot be null/empty in the header. (Request ID: c3c482d7-97a4-49d2-8048-df92fbad976d)"
    }
}

I cannot find any documentation on Principal Id and what value it should be set to.

If I use the scope /subscriptions/{SubID} then it works as expected.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 77db9ac7-6b1c-496b-36a3-24119c897546
• Version Independent ID: 31fbb3ed-1eb8-ff0c-05d8-1e92ccc1519f
• Content: Azure consumption API overview
• Content Source: articles/billing/billing-consumption-api-overview.md
• Service: billing
• GitHub Login: @bandersmsft
• Microsoft Alias: banders

[KrishnaG-MSFT]

@AndrewGailer Thanks for reaching out to us! To better assist you, could you please share the Azure document link / URL for which this feedback is applicable to and also provide more details on your query / scenario ?

Perhaps I am missing something but with given information, it looks like you are referring to managementGroups scope in Usage Details - List REST API. If that is the case then in place of {managementGroupId} you would have to provide parent ID of the managementGroup but not the group ID which you see in Azure portal. One way to get the parent ID of a managementGroup is by using Management Groups - Get REST API.

[AndrewGailer]

@KrishnaG-MSFT You are correct, I am referring to the Usage Details - List REST API you linked.
Which ID are you referring to? Here is the output of the Management Groups - Get REST API:

{
    "id": "/providers/Microsoft.Management/managementGroups/c4e1b03d-f9e8-4ad7-8c9a-878371138385",
    "type": "/providers/Microsoft.Management/managementGroups",
    "name": "c4e1b03d-f9e8-4ad7-8c9a-878371138385",
    "properties": {
        "tenantId": "9881063e-d6bc-48e3-8689-4a64f227da22",
        "displayName": "dev",
        "details": {
            "version": 1,
            "updatedTime": "2019-09-09T23:07:44.2545506Z",
            "updatedBy": "37b5c986-f01e-4c88-8f5a-29d792943d65",
            "parent": {
                "id": "/providers/Microsoft.Management/managementGroups/9881063e-d6bc-48e3-8689-4a64f227da22",
                "name": "9881063e-d6bc-48e3-8689-4a64f227da22",
                "displayName": "Tenant Root Group"
            }
        }
    }
}

This is Uri I am using: https://management.azure.com/providers/Microsoft.Management/managementGroups/c4e1b03d-f9e8-4ad7-8c9a-878371138385/providers/Microsoft.Consumption/usageDetails?api-version=2019-05-01

[KrishnaG-MSFT]

@AndrewGailer I was referring to

"parent": {
"id": "/providers/Microsoft.Management/managementGroups/9881063e-d6bc-48e3-8689-4a64f227da22"

So I would suggest you to try below Uri

https://management.azure.com/providers/Microsoft.Management/managementGroups/9881063e-d6bc-48e3-8689-4a64f227da22/providers/Microsoft.Consumption/usageDetails?api-version=2019-05-01

[AndrewGailer]

@KrishnaG-MSFT That would return the usage data for the parent management group, not the current one. I tried with the parent Id and received the same error as my original post.

The issue appears to be that the API does not accept the management group scope. If the scope is invalid I get a 404 error, but when it is a valid management group scope, it get the 400 error saying The Principal Id cannot be null/empty in the header.

The headers I am passing just have the authorization bearer token. Do you need to provide additional parameters when the scope is a management group?

[KrishnaG-MSFT]

@AndrewGailer Thanks for the update.
Assigning @bandersmsft for visibility, help and to provide insights on this feedback.

[bandersmsft]

@andrewgailer @krishnag-msft
Investigating this with the enginnering team.

in-process

[AndrewGailer]

@bandersmsft @KrishnaG-MSFT
Are there any updates on this issue?

[bandersmsft]

@andrewgailer Still awaiting reponse from the engineering team. Will follow-up.

[bandersmsft]

@andrewgailer Per the enginnering team, the following headers are needed.

x-ms-client-tenant-id :
x-ms-client-object-id :
x-ms-client-principal-id:

Does that do the trick?

[ms-premp]

@bandersmsft @AndrewGailer clients cannot set these headers. When they make ARM call, these headers get set by system. The fact that error says that this header is missing, it means that the call is being made in context of an App (AAD app or service prinicipal). This is at present not supported for management group scope.

[AndrewGailer]

@ms-premp @bandersmsft
I am currently making the call with PowerShell (Invoke-RestMethod) in the context of my account, not a service principal.
Is the managementGroups scope the only one that requires these headers? The same call will work if I use a different scope, such as subscriptions or resourceGroups.

[AndrewGailer]

I have just tested again and now instead of an error, the following is returned when using managementGroups as the scope

{
    "value":  [

              ]
}

[ebrucucen]

A similar issue we are facing: https://github.com/Azure/azure-sdk-for-go/issues/5843, is there any update on this issue?

[rithinskaria]

Hello all,

I am getting the same on Budget API when called at Management Group level, however works for subscription level:

URI: https://management.azure.com/providers/Microsoft.Management/managementGroups/112/providers/Microsoft.Consumption/budgets?api-version=2019-05-01

Response:

{
"error": {
"code": "400",
"message": "The Principal Id cannot be null/empty in the header. (Request ID: 850c7cda-a4a5-4085-bb38-3d3c223d4aca)"
}
}

Please advise

[rithinskaria]

This issue doesn't come up when a user principal token is used for the API call. So I found a workaround to take my access token from Azure context and pass to the call.

Here is the code:

$subscriptionId='xxxxx-xxxxx-xxxxxx-xxxxxx'

$context = Get-AzContext

$tenantId = (Get-AzSubscription -SubscriptionId $subscriptionId).TenantId

$tokenCache = $context.TokenCache

$cachedTokens = $tokenCache.ReadItems() | where { $_.TenantId -eq $tenantId } | Sort-Object -Property ExpiresOn -Descending

$accessToken = $cachedTokens[0].AccessToken

$headers = @{Authorization = "Bearer $accessToken"}
$URI = "https://management.azure.com/providers/Microsoft.Management/managementGroups/112/providers/Microsoft.Consumption/budgets?api-version=2019-10-01"
$http=Invoke-WebRequest -Uri $URI -Method GET -Headers $headers

[seandilda]

I am also seeing this issue when querying against an enrollment scope

[chrisdoofer]

+1 - I am seeing the exact same "The Principal Id cannot be null/empty in the header" error message when attempting to connect using the Azure Cost Management connector in Power BI or using the InvokeRestMethod in powershell when querying against the enrollment scope.

[ms-premp]

We cannot use an AAD App or any non-user context for making these calls. It has to be an user account used to get these API call successful.

[rithinskaria]

+1 - I am seeing the exact same "The Principal Id cannot be null/empty in the header" error message when attempting to connect using the Azure Cost Management connector in Power BI or using the InvokeRestMethod in powershell when querying against the enrollment scope.

The AAD service principal will have no access over enrollment scope, so you cannot use the SPN credentials you have to use a auth token which is of the EA Admin.

If you are looking for Usage at EA level , you can leverage the EA API: https://consumption.azure.com/v3/enrollments/{{eaNumber}}/usagedetailsbycustomdate?startTime=2019-10-07&endTime=2019-10-07. You can get the API key from EA Portal.

If you would like to ACM query API at enrollment itself, you have to use the EA Admin user token instead of SP token.

[vs268153]

I am also getting the same error while using the API to get a response from Postman, using service principal for the API.
URL: https://docs.microsoft.com/en-us/rest/api/consumption/aggregatedcost/getbymanagementgroup

GET https://management.azure.com/providers/Microsoft.Management/managementGroups/{managementGroupId}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01

[rithinskaria]

I am also getting the same error while using the API to get a response from Postman, using service principal for the API.
URL: https://docs.microsoft.com/en-us/rest/api/consumption/aggregatedcost/getbymanagementgroup

GET https://management.azure.com/providers/Microsoft.Management/managementGroups/{managementGroupId}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01

May I know if you are using a service principal or a user principal token for fetching the data? You can only use a user principal token when the scope is management group or above.

[vs268153]

I am also getting the same error while using the API to get a response from Postman, using service principal for the API.
URL: https://docs.microsoft.com/en-us/rest/api/consumption/aggregatedcost/getbymanagementgroup
GET https://management.azure.com/providers/Microsoft.Management/managementGroups/{managementGroupId}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01

May I know if you are using a service principal or a user principal token for fetching the data? You can only use a user principal token when the scope is management group or above.

I am using service principal

[rithinskaria]

I am also getting the same error while using the API to get a response from Postman, using service principal for the API.

URL: https://docs.microsoft.com/en-us/rest/api/consumption/aggregatedcost/getbymanagementgroup

GET https://management.azure.com/providers/Microsoft.Management/managementGroups/{managementGroupId}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01

May I know if you are using a service principal or a user principal token for fetching the data? You can only use a user principal token when the scope is management group or above.

I am using service principal

Please use a user principal token in the Auth header and you will be able to retrieve the information.

[vs268153]

I am also getting the same error while using the API to get a response from Postman, using service principal for the API.

URL: https://docs.microsoft.com/en-us/rest/api/consumption/aggregatedcost/getbymanagementgroup

GET https://management.azure.com/providers/Microsoft.Management/managementGroups/{managementGroupId}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01

May I know if you are using a service principal or a user principal token for fetching the data? You can only use a user principal token when the scope is management group or above.

I am using service principal

Please use a user principal token in the Auth header and you will be able to retrieve the information.

I have used Bearer token for auth

[rithinskaria]

Yes, so the bearer token that is being used now is the one generated using the client id and credentials of the service principal, which will not work.

In order for this to work you have to use the token which is created in the user context. I have a added PS script in this thread by which you can use to get the token from the user context, which can be further used by the API.

[vs268153]

I am also getting the same error while using the API to get a response from Postman, using service principal for the API.

URL: https://docs.microsoft.com/en-us/rest/api/consumption/aggregatedcost/getbymanagementgroup

GET https://management.azure.com/providers/Microsoft.Management/managementGroups/{managementGroupId}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01

May I know if you are using a service principal or a user principal token for fetching the data? You can only use a user principal token when the scope is management group or above.

I am using service principal

Please use a user principal token in the Auth header and you will be able to retrieve the information.

I have used Bearer token for auth

I have also used other API, with the same bearer token and they are working fine but while trying to get the Aggregated cost of multiple subscriptions using the management group Id then this API is not working.

[rithinskaria]

I am also getting the same error while using the API to get a response from Postman, using service principal for the API.

URL: https://docs.microsoft.com/en-us/rest/api/consumption/aggregatedcost/getbymanagementgroup

GET https://management.azure.com/providers/Microsoft.Management/managementGroups/{managementGroupId}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01

May I know if you are using a service principal or a user principal token for fetching the data? You can only use a user principal token when the scope is management group or above.

I am using service principal

Please use a user principal token in the Auth header and you will be able to retrieve the information.

I have used Bearer token for auth

I have also used other API, with the same bearer token and they are working fine but while trying to get the Aggregated cost of multiple subscriptions using the management group Id then this API is not working.

Prem has already shared the answer for this: We cannot use an AAD App or any non-user context for making these calls. It has to be an user account used to get these API call successful.

Other APIs will work, for these APIs you cannot user the SP’s token.

Please let me know if you are able to fetch the data using the token generated using a user principal.

[vs268153]

I am also getting the same error while using the API to get a response from Postman, using service principal for the API.

>

>

URL: https://docs.microsoft.com/en-us/rest/api/consumption/aggregatedcost/getbymanagementgroup

>

>

GET https://management.azure.com/providers/Microsoft.Management/managementGroups/{managementGroupId}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01

>

>

>

>

>

May I know if you are using a service principal or a user principal token for fetching the data? You can only use a user principal token when the scope is management group or above.

>

>

I am using service principal

>

>

Please use a user principal token in the Auth header and you will be able to retrieve the information.

I have used Bearer token for auth

I have also used other API, with the same bearer token and they are working fine but while trying to get the Aggregated cost of multiple subscriptions using the management group Id then this API is not working.

Prem has already shared the answer for this: We cannot use an AAD App or any non-user context for making these calls. It has to be an user account used to get these API call successful.

Other APIs will work, for these APIs you cannot user the SP’s token.

Please let me know if you are able to fetch the data using the token generated using a user principal.

Hi,

I have used your script and replace the subscription ID and URI with my API URI "https://management.azure.com/providers/Microsoft.Management/managementGroups/{{managementGroupId}}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01"

I am getting below error
Invoke-WebRequest : {"message":"No HTTP resource was found that matches the request URI 'https://consumption.azure.com/providers/Microsoft.Management/managementGroups/{{place managementGroupId}}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01'.","messageDetail":"No route providing a controller name was found to match request URI
'https://consumption.azure.com/providers/Microsoft.Management/managementGroups/{{place managementGroupId}}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-1
0-01'"}
At line:15 char:7

• $http=Invoke-WebRequest -Uri $URI -Method GET -Headers $headers
• ~~~~~~~~~~~~~
  
  
  
  • CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
  
  
  • FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
  
  

[rithinskaria]

I am also getting the same error while using the API to get a response from Postman, using service principal for the API.

>

>

URL: https://docs.microsoft.com/en-us/rest/api/consumption/aggregatedcost/getbymanagementgroup

>

>

GET https://management.azure.com/providers/Microsoft.Management/managementGroups/{managementGroupId}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01

>

>

>

>

>

May I know if you are using a service principal or a user principal token for fetching the data? You can only use a user principal token when the scope is management group or above.

>

>

I am using service principal

>

>

Please use a user principal token in the Auth header and you will be able to retrieve the information.

I have used Bearer token for auth

I have also used other API, with the same bearer token and they are working fine but while trying to get the Aggregated cost of multiple subscriptions using the management group Id then this API is not working.

Prem has already shared the answer for this: We cannot use an AAD App or any non-user context for making these calls. It has to be an user account used to get these API call successful.
Other APIs will work, for these APIs you cannot user the SP’s token.
Please let me know if you are able to fetch the data using the token generated using a user principal.

Hi,

I have used your script and replace the subscription ID and URI with my API URI "https://management.azure.com/providers/Microsoft.Management/managementGroups/{{managementGroupId}}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01"

I am getting below error
Invoke-WebRequest : {"message":"No HTTP resource was found that matches the request URI 'https://consumption.azure.com/providers/Microsoft.Management/managementGroups/{{place managementGroupId}}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-10-01'.","messageDetail":"No route providing a controller name was found to match request URI
'https://consumption.azure.com/providers/Microsoft.Management/managementGroups/{{place managementGroupId}}/providers/Microsoft.Consumption/aggregatedcost?api-version=2019-1
0-01'"}
At line:15 char:7

• $http=Invoke-WebRequest -Uri $URI -Method GET -Headers $headers

• ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  

  • CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
  • FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

Seems like I am hitting the same error. Not sure why!

[shahanur]

Without using service-principal the automation really does not make any sense. Has anyone overcome this issue yet? Also, what MS suggested to use additional headers (x-ms-client-tenant-id :
x-ms-client-object-id :
x-ms-client-principal-id:) did not work either. Is there any better solution? Why MS always does things a half-way-through?

[rithinskaria]

Without using service-principal the automation really does not make any sense. Has anyone overcome this issue yet? Also, what MS suggested to use additional headers (x-ms-client-tenant-id :

x-ms-client-object-id :

x-ms-client-principal-id:) did not work either. Is there any better solution? Why MS always does things a half-way-through?

The service principal will work on the subscription scope and you can make a Logic to add the cost of the child items in the management group.

[bandersmsft]

please-close

[seanbelling]

Is there any documentation the states that service principal auth tokens are unable to be used above the subscription scope? This github issue is the only mention of it I've been able to find anywhere

[kolbyrax]

Also getting the same error when using a UP token (mine, I'm Global Admin, EA Admin, Billing Admin) trying to create a cost management export to the Gov API (portal is using old API version that doesn't support exports):

`$subscriptionId='XXXXXX-XXXX-XXXX-XXXXXXX'

$context = Get-AzContext

$tenantId = (Get-AzSubscription -SubscriptionId $subscriptionId).TenantId

$tokenCache = $context.TokenCache

$cachedTokens = $tokenCache.ReadItems() | where { $_.TenantId -eq $tenantId } | Sort-Object -Property ExpiresOn -Descending

$accessToken = $cachedTokens[0].AccessToken

$json = @'
{
"properties": {
"schedule": {
"status": "Active",
"recurrence": "Daily",
"recurrencePeriod": {
"from": "2020-09-22T00:00:00Z",
"to": "2020-09-25T00:00:00Z"
}
},
"format": "Csv",
"deliveryInfo": {
"destination": {
"resourceId": "/subscriptions/XXXXXX-XXXX-XXXX-XXXXXXX/resourceGroups/rg/providers/Microsoft.Storage/storageAccounts/storageaccount01",
"container": "daily",
"rootFolderPath": "test2020-06-01"
}
},
"definition": {
"type": "ActualCost",
"timeframe": "MonthToDate",
"dataset": {
"granularity": "Daily"
}
}
}
}
'@

$headers = @{Authorization = "Bearer $accessToken"}
$URI = "https://management.usgovcloudapi.net/providers/Microsoft.Billing/billingAccounts/48390468/providers/Microsoft.CostManagement/exports/GovAllSubscriptions?api-version=2020-06-01"
$http = Invoke-WebRequest -Uri $URI -Method Put -Headers $headers -Body $json`