Azure-rest-api-specs: Consumption API (2019-10-01) missing startDate and endDate query parameters and incorrect bool type

Created on 30 Nov 2020 · 13Comments · Source: Azure/azure-rest-api-specs

The consumption API definition has a couple of problems in the 2019-10-01 version:

The query parameters startDate and endDate are missing for UsageDetails_List - for non-Enterprise Agreement subscriptions (e.g. PAYG or MCA) these allow filtering by date, adding properties/usageStart and properties/usageEnd to $filter has no effect. Manually specifying these parameters on the URL has the desired effect.
The returned results from the API for the LegacyUsageDetailProperties and ModernUsageDetailProperties uses the string value "1" in the field IsAzureCreditEligible, which is defined in the spec as a boolean. At least in the Go Azure SDK, the string value "1" is not parseable as a boolean, so the parse fails.

Consumption Service Attention bug

Source

mrsheepuk

👍2

All 13 comments

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @ms-premp.

Issue Details

The consumption API definition has a couple of problems in the 2019-10-01 version:

The query parameters startDate and endDate are missing for UsageDetails_List - for non-Enterprise Agreement subscriptions (e.g. PAYG or MCA) these allow filtering by date, adding properties/usageStart and properties/usageEnd to $filter has no effect. Manually specifying these parameters on the URL has the desired effect.
The returned results from the API for the LegacyUsageDetailProperties and ModernUsageDetailProperties uses the string value "1" in the field IsAzureCreditEligible, which is defined in the spec as a boolean. At least in the Go Azure SDK, the string value "1" is not parseable as a boolean, so the parse fails.

Author:	mrsheepuk
Assignees:	akning-ms
Labels:	`Consumption`, `Service Attention`, `needs-triage`, `question`
Milestone:	-

msftbot[bot] on 1 Dec 2020

For

1 The documentation should be updated soon.

This requesturl should work for a modern subscription
https://management.azure.com/subscriptions/_{{yourSubscriptionId}}_/providers/Microsoft.Consumption/usageDetails?api-version=2019-11-01&startDate=2020-10-01&endDate=2020-10-03

for a legacy subscription
https://management.azure.com/subscriptions/_{{yourSubscriptionId}}_/providers/Microsoft.Consumption/usageDetails?$filter=properties/usageStart ge '2020-11-01' and properties/usageEnd le '2020-11-03'&api-version=2019-10-01

$filter is not supported for the modern subscription calls. legacy subscription calls works with both formats.

2

It will be great if you share couple of requestIds for legacy & modern subscriptionId calls you are making. I tested a legacy & modern subscription call & I do not see that the api is returning the "isAzureCreditEligible" as number but it does return it as boolean.
A small note is I used 2019-10-01 api-version. It could be the sdk only issue

tujamg on 9 Dec 2020

Hi @tujamg, thanks for looking into this - yes, the URL indeed works with those parameters, this issue is specifically that the API specs in this repository (and, thus, the auto-generated Azure SDK for Go) don't have them so I can't use them without manually editing the SDK code.

For the second issue, I was testing using this PR against the Azure SDK for Go which introduces the 2019-10-01 version of this API, and it was definitely hitting the 2019-10-01 API version in the REST call (as, once the bool issue was corrected and I added the start/end date parameters, the start/end dates did work and it returned data for an MCA, which does not work against the older API version).

The call I was making that returned the string "1" instead of the bool true was against an 'MCA' account, reporting at the invoice section scope, e.g.:

https://management.azure.com/providers/Microsoft.Billing/billingAccounts/{{billingAccountID}}/billingProfiles/{{billingProfileID}}/invoiceSections/{{invoiceSectionID}}/providers/Microsoft.Consumption/usageDetails?api-version=2019-10-01&startDate=2020-10-01&endDate=2020-10-03

I'll attempt to repro the issue now and report back with concrete examples if I find them.

mrsheepuk on 9 Dec 2020

Here's a concrete example, with the IDs removed for privacy:

{
    "nextLink": "https://management.azure.com/providers/Microsoft.Billing/billingAccounts/SNIPPED/billingProfiles/SNIPPED/invoicesections/SNIPPED/providers/Microsoft.Consumption/usageDetails?api-version=2019-10-01&$filter=properties/usageStart ge '2020-11-20'&startDate=2020-11-20&$top=10&sessiontoken=14:1122334455&$skiptoken=SNIPPED&skiptokenver=v1&id=SNIPPED",
    "value": [
        {
            "kind": "modern",
            "id": "/providers/Microsoft.Billing/billingAccounts/SNIPPED/billingProfiles/SNIPPED/invoicesections/SNIPPED/providers/Microsoft.Consumption/usageDetails/SNIPPED",
            "name": "SNIPPED",
            "type": "Microsoft.Consumption/usageDetails",
            "tags": null,
            "properties": {
                "additionalInfo": "",
                "billingAccountId": "SNIPPED",
                "billingAccountName": "MyOrg",
                "billingCurrencyCode": "GBP",
                "billingPeriodEndDate": "0001-01-01T00:00:00Z",
                "billingPeriodStartDate": "0001-01-01T00:00:00Z",
                "billingProfileId": "SNIPPED",
                "billingProfileName": "MyProfile",
                "chargeType": "Usage",
                "consumedService": "Microsoft.Network",
                "costInBillingCurrency": 0.00455639754389479,
                "costInPricingCurrency": 0.00609691555348562,
                "costCenter": "",
                "date": "2020-12-07T00:00:00Z",
                "exchangeRate": "0.7473283013227710933413048352141096",
                "exchangeRateDate": "2020-12-01T00:00:00Z",
                "instanceName": "/subscriptions/SNIPPED/resourceGroups/res-name-e2e-b8mtv-infra-uksouth/providers/Microsoft.Network/loadBalancers/kubernetes",
                "invoiceId": "",
                "invoiceSectionId": "SNIPPED",
                "invoiceSectionName": "MyInvSection",
                "isAzureCreditEligible": "1",
                "meterCategory": "Load Balancer",
                "meterId": "27827eb0-7f60-4928-940b-f5fe15e7a4cb",
                "meterName": "Included LB Rules and Outbound Rules",
                "meterRegion": "Global",
                "meterSubCategory": "Standard Load Balancer",
                "previousInvoiceId": "",
                "pricingCurrencyCode": "USD",
                "product": "Standard Load Balancer",
                "productIdentifier": "DZH318Z0BNXH0002",
                "productOrderId": "d787a694-f421-2ec5-47f4-e98d5ed4f904",
                "productOrderName": "Azure plan",
                "publisherName": "Microsoft",
                "publisherType": "Azure",
                "quantity": 0.25,
                "resourceGroup": "res-name-e2e-azure-ui-e2e-b8mtv-infra-uksouth",
                "resourceLocation": "UKSOUTH",
                "resourceLocationNormalized": "UK South",
                "serviceFamily": "Networking",
                "serviceInfo1": "",
                "serviceInfo2": "",
                "servicePeriodEndDate": "2021-01-01T00:00:00Z",
                "servicePeriodStartDate": "2020-12-01T00:00:00Z",
                "subscriptionGuid": "SNIPPED",
                "subscriptionName": "my-subscription-name",
                "unitOfMeasure": "1 Hour",
                "unitPrice": 0.025,
                "customerTenantId": "",
                "customerName": "MyCustName",
                "partnerTenantId": "",
                "partnerName": "",
                "resellerMpnId": "",
                "resellerName": "",
                "publisherId": "",
                "reservationId": "",
                "reservationName": "",
                "frequency": "UsageBased",
                "term": "",
                "payGPrice": 0.025,
                "costInUSD": 0.00609691555348562,
                "paygCostInBillingCurrency": 0.00467080188326732,
                "paygCostInUSD": 0.00625,
                "exchangeRatePricingToBilling": 0.747328301322771,
                "partnerEarnedCreditRate": 0,
                "partnerEarnedCreditApplied": "0"
            }
        },

mrsheepuk on 9 Dec 2020

Can you please share the requestId or clientrequestId for this request?

tujamg on 10 Dec 2020

@tujamg Here you go, sorry I didn't know where to find that - got it from the headers now:

x-ms-request-id: 242ad27d-cf34-4b58-b2e6-1d341b5a3338
x-ms-client-request-id: bb87fddc-e6c2-47fa-8ac9-50eb4eb6dbdc

mrsheepuk on 10 Dec 2020

@mrsheepuk
For some reason I do not see you ids in my logs. Did you get a successful response on the above ids?
Is it possible for you to just send over the requestUrl to my ms email on my profile?

tujamg on 10 Dec 2020

@tujamg I've sent this over to your microsoft email from your profile - let me know if you don't receive it.

mrsheepuk on 10 Dec 2020

👍1

I am able to repro the issue locally. Will fix it and share the update, only the deployment is going to start sometime first week of Jan.
About the requestId and clientRequestId not found seems the GoSDK generates it but does not pass it in headers or something else because I see records of incoming calls but these id columns were all empty.

tujamg on 11 Dec 2020

Ah great news, glad to hear that's being repro'd now. Will the spec in this repo be updated to add the startDate and endDate query parameters too, as part of that?

About the requestId and clientRequestId not found seems the GoSDK generates it but does not pass it in headers or something else because I see records of incoming calls but these id columns were all empty.

FYI I got those IDs from the response sent when testing using Postman - wasn't using the Go SDK for the test details I shared above. Looks like there might be an issue with your logging, perhaps? If you need, I can re-run the test and send you the IDs immediately so they're close in time.

mrsheepuk on 11 Dec 2020

Hi @tujamg - happy new year! Is there any update on when these fixes might be published?

mrsheepuk on 5 Jan 2021

👍1

The workitem in the backlog and will share more updates on the ETA once we have it in the sprint.

tujamg on 13 Jan 2021

Hi @tujamg . Are there any updates on this? We are also in need of specifying startDate and endDate, and the manual way of putting it directly in the URL is a bit clunky. Also using Azure SDK for go :)