Azure-sdk-for-go: SDK should match MSDN reference

[FrenchBen]

It seems there's a fair amount of discrepancy between what the MSDN Azure Reference says a call should return and what the Azure SDK for Go actually gives you.

Take for example the process of getting the list of NIC associated with a VM Scale Set:
https://msdn.microsoft.com/en-us/library/azure/mt712626.aspx

This should provide a response with an array of value, which includes things like Name, ID, Type, Tags and Properties.

The matching SDK call can be found here:
https://github.com/Azure/azure-sdk-for-go/blob/master/arm/compute/virtualmachinescalesets.go#L352 and should return a type of VirtualMachineScaleSet.

Looking at the model definition of said struct, it feels that you get a 1-1 match:
https://github.com/Azure/azure-sdk-for-go/blob/master/arm/compute/models.go#L839

// VirtualMachineScaleSet is describes a Virtual Machine Scale Set.
type VirtualMachineScaleSet struct {
    autorest.Response `json:"-"`
    ID                *string                           `json:"id,omitempty"`
    Name              *string                           `json:"name,omitempty"`
    Type              *string                           `json:"type,omitempty"`
    Location          *string                           `json:"location,omitempty"`
    Tags              *map[string]*string               `json:"tags,omitempty"`
    Sku               *Sku                              `json:"sku,omitempty"`
    Properties        *VirtualMachineScaleSetProperties `json:"properties,omitempty"`
}

You then dig a bit deeper, as you expect to get the same properties defined by the MSDN reference; but that's when you start to move hand towards your face:
https://github.com/Azure/azure-sdk-for-go/blob/master/arm/compute/models.go#L1014

// VirtualMachineScaleSetProperties is describes the properties of a Virtual
// Machine Scale Set.
type VirtualMachineScaleSetProperties struct {
    UpgradePolicy         *UpgradePolicy                   `json:"upgradePolicy,omitempty"`
    VirtualMachineProfile *VirtualMachineScaleSetVMProfile `json:"virtualMachineProfile,omitempty"`
    ProvisioningState     *string                          `json:"provisioningState,omitempty"`
    Overprovision         *bool                            `json:"overprovision,omitempty"`
}

Uhh.. ok, the ref said I'd get something like:

"provisioningState": "Succeeded",
"resourceGuid": "XXXXXXXX-XXXX-XXXX-XXXXXXXXXXXX",
"ipConfigurations": [  {
    "name":  "ipconfig1",
    "id": "/subscriptions/{sub-id}/resourceGroups/myrg1/providers/Microsoft.Compute/virtualMachineScaleSets/myscaleset1/virtualMachines/vm1/networkInterfaces/nicconfig1/ipConfigurations/ipconfig1",
    "etag": "W/\"00000000-0000-0000-0000-000000000000\"",
    "properties":  { 
        ...
    } 
}  ],
"macAddress": "00-0D-3A-A0-AE-61",
"enableIPForwarding": false,
"primary": true,
"virtualMachine": {
    ...
}

Although I'm sure there's a "Good" reason behind all of this, I fail to see it. Anyone feel like explaining what's going on?

ps:
I did add some debugging to my code via the overly complicated autotrest and managed to see that the URL for the request matched 💯 the get request as defined by the MSDN Azure ref:
https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.Compute/VirtualMachineScaleSets/{vm-scaleset-name}?api-version={api-version}

[colemickens]

The MSDN doc page here doesn't even make any sense. It's showing the GET for a VMSS, and then the response only shows NIC information. Let me see if I can get someone to chime in.

In the meantime, you're expected to keep going down the rabbit hole.

Unfortunately, it looks like at the bottom there is just a VirtualMachineScaleSetIPConfigurationProperties that doesn't even have the private ip field... again, let me see if I can get someone to take a look. Sorry for the confusion here.

[FrenchBen]

Thanks @colemickens for the quick response!

I'm guessing what I should be looking for networking details; something along the lines of:
https://github.com/Azure/azure-sdk-for-go/blob/bd73d950fa4440dae889bd9917bff7cef539f86e/arm/network/loadbalancers.go#L190

To get the loadBalancer properties, defined here:
https://github.com/Azure/azure-sdk-for-go/blob/bd73d950fa4440dae889bd9917bff7cef539f86e/arm/network/models.go#L1441

and finally get the actual IPs:
https://github.com/Azure/azure-sdk-for-go/blob/bd73d950fa4440dae889bd9917bff7cef539f86e/arm/network/models.go#L1472

It's a bit frustrating to have to dig through the code to figure out how all of this works :\

[colemickens]

For what it's worth, I'd never looked at those particular bits before either. I was just starting here and then following the rabbit hole: https://godoc.org/github.com/Azure/azure-sdk-for-go/arm/compute#VirtualMachineScaleSet

If you follow the right links, you'll eventually end up at the nested struct VirtualMachineScaleSetIPConfigurationProperties which I think is what you're after.

So there's two problems here that I see:

1. The MSDN docs are just wrong, or at the very least, are misleading.
2. The thing you're looking for is the VirtualMachineScaleSetIPConfigurationProperties which should contain the private ip for the vm instance in the VMSS, but for some reason they're just absent. Compare to the non-VMSS version, which has the relevant field. This likely means that there's an issue with the Azure/azure-rest-api-specs repository that is resulting in the SDK missing these fields. That or there is more funkyness in the VMSS API than I was previously aware of.

Hopefully we can get some more solid answers soon, I'll keep you updated.

[colemickens]

Actually, it gets more confusing... the network package itself has other functions that relate to VMSS: https://godoc.org/github.com/Azure/azure-sdk-for-go/arm/network#InterfacesClient.ListVirtualMachineScaleSetNetworkInterfaces

[colemickens]

I guess the logic here is: the compute/VirtualMachineScaleSet contains properties that are common to the VMSS Profile and apply homogenously across all of the member VMs.

The NIC's specific configuration, including the IPConfig, is unique per-VM in the VMSS, so you need to go through the network package to access the relevant private ip fields.

Let me know if that's enough extra info to get you unblocked.

[FrenchBen]

Thanks for chasing this all down for me, as it should allow me to get what I need.
This however doesn't make this SDK very friendly.

Going down the path you provided, I do get to VirtualMachineScaleSetIPConfigurationProperties which points to a SubResource, which seems to be just an ID for the LB Resource.

netConf := *result.Properties.VirtualMachineProfile.NetworkProfile.NetworkInterfaceConfigurations
firstConf := netConf[0]
ipConf := *firstConf.Properties.IPConfigurations
firstIP := ipConf[0]
subRes := *firstIP.Properties.LoadBalancerInboundNatPools
fmt.Printf("LB Inbound: %v", subRes[0])

^^^ a bit much, no? Especially since that doesn't get you anything but the LB you should ping to get what you're actually looking for.

I hate to compare things, but the EC2 instance SDK gives you all that you want all at once.
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html

[salameer]

@marstr to check on this

edit [by marstr]: This comment tagged me as a pseudo-assignment while permissions were figured out. Now that I'm able to assign myself (as seen below) this comment is irrelevant.

[marstr]

Sorry that this took so long to resolve. Reading through, this is a really interesting thread. Clearly, there is some unintuitive overlap between the compute and network packages. I'm not really sure that it is an actionable thing for us at the moment, but definitely good that you brought it to our attention. Really the way to chase that part down is to head over to the Azure/azure-rest-api-specs repository to start a conversation about what is present in the swaggers.

The other concern, about the verbosity and depth of the tree that you have to navigate in the SDK, is definitely valid. This is pain that each of us team members have felt while using our API. Again though, I'm afraid there isn't really much immediate action that we can take. As a general point, we can catalog this so that we take it into consideration during a future refactor.

I'm going to close out this issue. If there is something you feel is unaddressed, or you have further questions, please don't hesitate to re-open this or create a new issue.