Azure-rest-api-specs: VM customData not accepting valid Base64-encoded string

[jsmartt]

Hi, I'm using the Ruby SDK for Azure to create Azure (Linux) VMs, but I'm running into periodic issues with the API not accepting my Base64-encoded customData string. Sometimes it accepts my string, and sometimes it rejects it, but what is strange is that there is no room for user error in this case, as it's building that string with code.

Here is a string (before base64 encoding) that does work:

"#include\r\nhttp://example.com/api/instance-provisioning-templates/d259e9b8-1518-4547-bffc-534c8b1c670c/render?instance_id=7814&token=eyJhbGciOiJIUzI1NiJ9.eyJpbnN0YW5jZV9pZCI6NzgxNCwiZXhwIjoxNTcxODQ5MDE4fQ.cidvYNYiWBe-N3JzdokN3ldmhQgPDznhAvAy6LqOw88"

And here is a string (before base64 encoding) that does not work:

"#include\r\nhttp://example.com/0/api/instance-provisioning-templates/d259e9b8-1518-4547-bffc-534c8b1c670c/render?instance_id=7814&token=eyJhbGciOiJIUzI1NiJ9.eyJpbnN0YW5jZV9pZCI6NzgxNCwiZXhwIjoxNTcxODQ5MDE4fQ.cidvYNYiWBe-N3JzdokN3ldmhQgPDznhAvAy6LqOw88"

It may be hard to spot, but the only difference is in the path: /0/api vs just /api. All of the extra stuff on the end of the path may not be necessary to produce the error, but I didn't want to test a bunch of other strings and wait for it to create or fail. The point is: something is broken in the parsing or validation mechanism of the Base64 decoder.

Here is a snippet of the code I'm using in Ruby to set the customData:

# In reality, the URL is generated, but here is a hard-coded example that fails:
url = 'http://example.com/0/api/instance-provisioning-templates/d259e9b8-1518-4547-bffc-534c8b1c670c/render?instance_id=7814&token=eyJhbGciOiJIUzI1NiJ9.eyJpbnN0YW5jZV9pZCI6NzgxNCwiZXhwIjoxNTcxODQ5MDE4fQ.cidvYNYiWBe-N3JzdokN3ldmhQgPDznhAvAy6LqOw88'

vm_params = Azure::Compute::Mgmt::V2018_06_01::Models::VirtualMachine.new
vm_params.location = 'westus'
... # Many more attributes get set here
vm_params.os_profile.custom_data = Base64.urlsafe_encode64("#include\r\n#{url}")

azure_compute_client.virtual_machines.create_or_update(resource_group_name, vm_name, vm_params)

The response is an error with this in the body:

{
  "error": {
    "code": "InvalidParameter",
    "message": "Custom data in OSProfile must be in Base64 encoding and with a maximum length of 87380 characters.",
    "target": "customData"
  }
}

Request Details:

• base_uri: https://management.azure.com
• path_template: subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachineScaleSets/{vmScaleSetName}
• api-versions: 2018-06-01, 2019-03-01, & 2019-07-01

Troubleshooting Attempted (with no success):

• Using Base64.encode64 instead of Base64.urlsafe_encode64
• Use \n instead of \r\n in the data
• Try making a VM scale-set instead of a single VM instance
• Try the request directly to the API (via Postman) to confirm the Ruby SDK is not the issue

I also verified that the JSON body sent in the request contains the expected value for the customData. I was able to run Base64.urlsafe_decode64 on the actual string value sent in the request body, and it does match the encoded string I set, so the SDK isn't trying to re-encode it or modify the value in any way.

[jsmartt]

One pattern that I may have identified based on a small set of examples is that it seems that Base64-encoded values ending with = or == are accepted, and values ending in other characters are not accepted. Not sure if that's a rule, or just coincidence with the values I've tried, but from what I've found, trailing = symbols are not required, but are just padding for strings with character counts that are not multiples of 3 (source).

[MyronFanQiu]

It's more like a service error. I will add service attention and compute label for it.

[Drewm3]

@jsmartt, could you check to see if CR/LF are getting added to the base64 string? Line feeds are not supported in custom data. I believe you need to use strict_encode64() to ensure there are no line feeds added. Could you retry with strict_encode64() to see if that resolves the issue?

[jsmartt]

@Drewm3, thanks for the comment! Custom Data strings that were failing with encode64() & urlsafe_encode64() are now working with strict_encode64(). I haven't tested it exhaustively, but I think you can go ahead and close this

[Drewm3]

Thanks and I am sorry this took so long to resolve.

[codexponent]

@jsmartt is right. You need strict encoding. I will provide a python sample code to encode the data or scripts.

import base64
data = open("script.sh", "rb").read()
encoded = base64.b64encode(data)
print(encoded)
# data = base64.b64decode(encoded)
# print(data)