Packer: vsphere-clone builder not using ssh keys in metadata

[mooperd]

I am trying to use the vsphere-clone builder to make some changes to the official ubuntu images. Currently - the builder does not create and inject an ssh-key or password in to the metadata which means the provisioner can't do its thing.

So far - this is working quite well with terraform.

{
  "variables": {
    "vsphere_server": "{{env `GOVC_VSPHERE_HOST`}}",
    "vsphere_user": "{{env `GOVC_USERNAME`}}",
    "vsphere_password": "{{env `GOVC_PASSWORD`}}",
    "vsphere_datastore": "{{env `GOVC_DATASTORE`}}",
    "vsphere_datacenter":"{{env `GOVC_DATACENTER`}}",
    "vsphere_cluster": "scrat-cluster",
    "vsphere_network": "GAD-VMWARE",
    "resource_pool":"andrew-kubeone",
    "vsphere_template_name": "template-virtual-machine"
  },
  "builders": [{
        "type": "vsphere-clone",

        "vcenter_server": "{{user `vsphere_server`}}",
        "username": "{{user `vsphere_user`}}",
        "password": "{{user `vsphere_password`}}",
        "insecure_connection": "true",

        "datacenter": "{{user `vsphere_datacenter`}}",
        "cluster": "{{user `vsphere_cluster`}}",
        "resource_pool": "{{user `resource_pool`}}",
        "datastore": "{{user `vsphere_datastore`}}",
        "template": "{{user `vsphere_template_name`}}",
        "vm_name": "packer-clone-from-image",
    "ssh_username": "ubuntu",
    "temporary_key_pair_name": "testing"
  }],
  "provisioners": [
    {
      "type": "file",
      "source": "./packer_scripts/install_core_dependencies.sh",
      "destination": "/tmp/install_core_dependencies.sh"
    },
    {
      "type": "shell",
      "inline": [
        "sudo /bin/bash /tmp/install_core_dependencies.sh"
      ]
    }
  ]
}

[SwampDragons]

Hi, thanks for reaching out and sorry for the delay in responding.

It looks like this is a documentation issue; the temporary_key_pair_name isn't enabled for all builders, but our documentation shows it as an option for all of them. We should fix the docs. Currently, Packer doesn't generate a keypair for you in the VSphere builder; we expect you to supply the credentials you need to be able to set up your instance.

Sorry for the confusion.

[mooperd]

Hi, vSphere supports cloud-init key injection using 'vapp properties'. The standard ubuntu images are already set up with cloud-init. If packer generated a keypair and injected it - it would work.

[mooperd]

Shall I make a new issue for implementing this feature?

[SwampDragons]

No need to make a new issue; we can leave this one open until someone can support this. Thanks for the details -- I did a quick search yesterday and wasn't sure how we'd go about supporting this.

[mooperd]

But to be clear - its not a 'docs' thing right?

[SwampDragons]

Right now it's both. We need to fix the docs to make sure that while we work on implementing this functionality, the docs don't lie and say it's available. But it's also a feature request. Hence leaving both the "docs" and "enhancement" tags :)

Probably the "docs" part will get solved before the "enhancement" part, but we can leave the issue open until both are solved.

[sylviamoss]

cloud-init key injection using 'vapp properties'.

Hey @mooperd, when you said the above were you talking about the vApp properties guestinfo.userdata and guestinfo.metada? Correct me if they are wrong.
I'm learning how to set these properties in order to implement what you suggested and I'm trying to figure out the right property that has to be set.

[mooperd]

Hey, I dont have access to a vsphere system at the moment but I think this
readme will help. https://github.com/vmware/cloud-init-vmware-guestinfo

Seems like guestinfo.userdata should be a base64 encoded gzip containing
some which then gets consumed by cloud-init.

If you get stuck you can ask the people in the govc project. They are the
Oracle for this kinda stuff.

https://github.com/vmware/govmomi

Cheers,

Andrew

On Fri, 26 Jun 2020, 14:51 Sylvia Moss, [e-mail] wrote:

cloud-init key injection using 'vapp properties'.

Hey @mooperd https://github.com/mooperd, when you said the above were
you talking about the vApp properties guestinfo.userdata and
guestinfo.metada? Correct me if they are wrong.
I'm learning how to set these properties in order to implement what you
suggested and I'm trying to figure out the right property that has to be
set.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/packer/issues/8769#issuecomment-650161849,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAW55RMWMECWQOLTN2RTZATRYSKUZANCNFSM4KXZZO4Q
.

[luis-garza]

Glad to see this issue will be implemented! :)

@sylviamoss as @mooperd said, govc is a really useful tool, with it you can retrieve all vApp properties defined in an image

govc import.spec -verbose=true ./ubuntu-20.04-server-cloudimg-amd64.ova | jq  

I'm not a vSphere expert, but I had some lessons learnt working with cloud-init images on vSphere and Terraform.
Here you can see how I inject cloud-init userdata in Terraform templates. Bear in mind there are a couple of ways to achieve it depending cloud-init setup in the base image.

• userdata injection for official Ubuntu cloud images:

  vapp {
    properties = {
      "hostname"  = var.vm_hostname
      "user-data" = base64encode(data.template_file.userdata.rendered)
    }
  }

• userdata injection for Ubuntu cloud images with VMware Guestinfo datasource for cloud.init installed:

 extra_config = {
    "guestinfo.metadata"          = base64encode(data.template_file.metadata.rendered)
    "guestinfo.metadata.encoding" = "base64"
    "guestinfo.userdata"          = base64encode(data.template_file.userdata.rendered)
    "guestinfo.userdata.encoding" = "base64"
  }

I hope it will help you.

[sylviamoss]

Nice! thanks for all the information @mooperd and @luis-garza.

I have a question for you...
Would it be enough being able to set vapp properties like terraform does? With that, you'd inject the userdata with any ssh key you'd like to.

I'm currently thinking about a way to make packer write the guestinfo.userdata file with the temporary generated ssh key, but I'm afraid this can override other existing userdata config. What do you think?

[mooperd]

In my very limited experiance I wouldn't be worryied about overwriteing
vapp properties. I think any system that uses these properties such as
terraform will overwrite them anyway.

On Mon, 29 Jun 2020, 16:16 Sylvia Moss, [e-mail] wrote:

Nice! thanks for all the information @mooperd https://github.com/mooperd
and @luis-garza https://github.com/luis-garza.

I have a question for you...
Would it be enough being able to set vapp properties like terraform does?
With that, you'd inject the userdata with any ssh key you'd like to.

I'm currently thinking about a way to make packer write the
guestinfo.userdata file with the temporary generated ssh key, but I'm
afraid this can override other existing userdata config. What do you think?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/packer/issues/8769#issuecomment-651151244,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAW55ROOWEZ24FG4XNUR5HLRZCO5HANCNFSM4KXZZO4Q
.

[luis-garza]

I think it would be really nice use public-keys vapp property to inject random temporary keys, to hide cloud-init complexity to the end users. This won't crash to other cloud-init userdata.

In any case, it would be fine if it's still possible to have direct access to inject cloud-init userdata for both systems, user-data vapp property & guestinfo.userdata. This way, we could manage our own keys or even run light config or provisioning stuff.

[sylviamoss]

I like the idea of injecting into public-keys vapp property.
In my local branch, it is already possible to set other vapp properties like terraform does. 👏

One more question for @luis-garza regarding the extra_config.
According to the VMware Guestinfo datasource it is possible to set the a value for guestinfo.userdata via vapp property as well. So I'm assuming it is fine to have only the config for vapp properties, leaving out a possible extra_config field. What do you think?

[sylviamoss]

@luis-garza Do you know where I can find the type of the public-keys property? I'm struggling to find it 🤔

[luis-garza]

As I've mentioned above govc could help you:

$ govc import.spec -verbose=true ./ubuntu-20.04-server-cloudimg-amd64.ova | jq  
{
  "AllDiskProvisioningOptions": [
    "flat",
    "monolithicSparse",
    "monolithicFlat",
    "twoGbMaxExtentSparse",
    "twoGbMaxExtentFlat",
    "thin",
    "thick",
    "seSparse",
    "eagerZeroedThick",
    "sparse"
  ],
  "DiskProvisioning": "flat",
  "AllIPAllocationPolicyOptions": [
    "dhcpPolicy",
    "transientPolicy",
    "fixedPolicy",
    "fixedAllocatedPolicy"
  ],
  "IPAllocationPolicy": "dhcpPolicy",
  "AllIPProtocolOptions": [
    "IPv4",
    "IPv6"
  ],
  "IPProtocol": "IPv4",
  "PropertyMapping": [
    {
      "Key": "instance-id",
      "Value": "id-ovf",
      "Spec": {
        "Key": "instance-id",
        "Type": "string",
        "Qualifiers": null,
        "UserConfigurable": true,
        "Default": "id-ovf",
        "Password": null,
        "Label": "A Unique Instance ID for this instance",
        "Description": "Specifies the instance id.  This is required and used to determine if the machine should take \"first boot\" actions",
        "Values": null
      }
    },
    {
      "Key": "hostname",
      "Value": "ubuntuguest",
      "Spec": {
        "Key": "hostname",
        "Type": "string",
        "Qualifiers": null,
        "UserConfigurable": true,
        "Default": "ubuntuguest",
        "Password": null,
        "Label": null,
        "Description": "Specifies the hostname for the appliance",
        "Values": null
      }
    },
    {
      "Key": "seedfrom",
      "Value": "",
      "Spec": {
        "Key": "seedfrom",
        "Type": "string",
        "Qualifiers": null,
        "UserConfigurable": true,
        "Default": null,
        "Password": null,
        "Label": "Url to seed instance data from",
        "Description": "This field is optional, but indicates that the instance should 'seed' user-data and meta-data from the given url.  If set to 'http://tinyurl.com/sm-' is given, meta-data will be pulled from http://tinyurl.com/sm-meta-data and user-data from http://tinyurl.com/sm-user-data.  Leave this empty if you do not want to seed from a url.",
        "Values": null
      }
    },
    {
      "Key": "public-keys",
      "Value": "",
      "Spec": {
        "Key": "public-keys",
        "Type": "string",
        "Qualifiers": null,
        "UserConfigurable": true,
        "Default": "",
        "Password": null,
        "Label": "ssh public keys",
        "Description": "This field is optional, but indicates that the instance should populate the default user's 'authorized_keys' with this value",
        "Values": null
      }
    },
    {
      "Key": "user-data",
      "Value": "",
      "Spec": {
        "Key": "user-data",
        "Type": "string",
        "Qualifiers": null,
        "UserConfigurable": true,
        "Default": "",
        "Password": null,
        "Label": "Encoded user-data",
        "Description": "In order to fit into a xml attribute, this value is base64 encoded . It will be decoded, and then processed normally as user-data.",
        "Values": null
      }
    },
    {
      "Key": "password",
      "Value": "",
      "Spec": {
        "Key": "password",
        "Type": "string",
        "Qualifiers": null,
        "UserConfigurable": true,
        "Default": "",
        "Password": null,
        "Label": "Default User's password",
        "Description": "If set, the default user's password will be set to this value to allow password based login.  The password will be good for only a single login.  If set to the string 'RANDOM' then a random password will be generated, and written to the console.",
        "Values": null
      }
    }
  ],
  "NetworkMapping": [
    {
      "Name": "VM Network",
      "Network": ""
    }
  ],
  "MarkAsTemplate": false,
  "PowerOn": false,
  "InjectOvfEnv": false,
  "WaitForIP": false,
  "Name": null
}

[luis-garza]

According to the VMware Guestinfo datasource it is possible to set the a value for guestinfo.userdata via vapp property as well. So I'm assuming it is fine to have only the config for vapp properties, leaving out a possible extra_config field. What do you think?

It makes sense, let's see if I can try out a nightly build version.

[sylviamoss]

I've implemented what we've been discussing here but I'm having a hard time building an environment where this will work for setting the ssh public key. Currently, it will only set the public key if the public-keys vapp property is available.
Here the binaries: https://circleci.com/gh/hashicorp/packer/63181#artifacts/containers/0

The only way for the shh public key injection to work is by cloning from a template created from an OVA file, which contains the public-keys property.

I expect people to clone a template that was created from an ISO using Packer's vsphere-iso builder. I want to find another way to inject this ssh key without depending on which template is being used.

[mooperd]

I think you would have to convert isos' into vapps to give them vapp
properties. That might not be wise as the resultant images might not have
cloud-init ready to trigger. It could confuse users.

On Wed, 1 Jul 2020, 10:12 Sylvia Moss, [e-mail] wrote:

I've implemented what we've been discussing here but I'm having a hard
time building an environment where this will work for setting the ssh
public key. Currently, it will only set the public key if the public-keys
vapp property is available.
Here the binaries:
https://circleci.com/gh/hashicorp/packer/63181#artifacts/containers/0

The only way for the shh public key injection to work is by cloning from a
template created from an OVA file, which contains the public-keys
property.

I expect people to clone a template that was created from an ISO using
Packer's vsphere-iso builder. I want to find another way to inject this
ssh key without depending on which template is being used.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/packer/issues/8769#issuecomment-652265586,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAW55RPZJQPN5O7VHCDGFYDRZLVVLANCNFSM4KXZZO4Q
.

[sylviamoss]

Yes. That doesn't work. I created a VM with cloud-init and created the public-keys vapp property from scratch and it didn't work. I don't know if I'm doing something else wrong.
But anyways terraform doesn't let you create these properties from scratch, according to the docs

[mooperd]

I think it's fine to assume that the user is bringing a vapp?

On Wed, 1 Jul 2020, 10:20 Sylvia Moss, [e-mail] wrote:

Yes. That doesn't work. I created a VM with cloud-init and created the
public-keys vapp property from scratch and it didn't work. I don't know
if I'm doing something else wrong.
But anyways terraform doesn't let you create these properties from
scratch, according to the docs
https://www.terraform.io/docs/providers/vsphere/r/virtual_machine.html#using-vapp-properties-to-supply-ovf-ova-configuration

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/packer/issues/8769#issuecomment-652269976,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAW55RK43IMQVASZSEMY2YTRZLWVBANCNFSM4KXZZO4Q
.

[mooperd]

Are you using terraform under the hood to manipulate vsphere?

On Wed, 1 Jul 2020, 10:22 Andrew Holway, [e-mail] wrote:

I think it's fine to assume that the user is bringing a vapp?

On Wed, 1 Jul 2020, 10:20 Sylvia Moss, [e-mail] wrote:

Yes. That doesn't work. I created a VM with cloud-init and created the
public-keys vapp property from scratch and it didn't work. I don't know
if I'm doing something else wrong.
But anyways terraform doesn't let you create these properties from
scratch, according to the docs
https://www.terraform.io/docs/providers/vsphere/r/virtual_machine.html#using-vapp-properties-to-supply-ovf-ova-configuration

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/packer/issues/8769#issuecomment-652269976,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAW55RK43IMQVASZSEMY2YTRZLWVBANCNFSM4KXZZO4Q
.

[sylviamoss]

For me it is. But I want to study a bit more to figure out a way to inject public ssh keys that would work for every case if that is possible 🤔
Whenever you have time, could you test the binaries I shared? They set the property with the generated public key when the property is available.

[mooperd]

I'd love to but I don't have access to a vmware environment at the moment.
If you can give me access to one I will happily test for you.

On Wed, 1 Jul 2020 at 10:25, Sylvia Moss [e-mail] wrote:

For me it is. But I want to study a bit more to figure out a way to inject
public ssh keys that would work for every case if that is possible 🤔
Whenever you have time, could you test the binaries I shared? They set the
property with the generated public key when the property is available.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hashicorp/packer/issues/8769#issuecomment-652272556,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAW55RKKOSEQD7YOC24J5TTRZLXHVANCNFSM4KXZZO4Q
.

[luis-garza]

The only way for the shh public key injection to work is by cloning from a template created from an OVA file, which contains the public-keys property.

That's really good, a great step forward!
What about injecting a random key in a fully cloud-image (VMware Guestinfo datasource) as I've mentioned here, it will work as well?

[sylviamoss]

I think that would work too and it would be one more specific case to cover.
Since the VMware Guestinfo datasource option requires writing the user-data file, I decided to keep only the public-keys vapp property for now and see how that works for the users.
It is possible to set the guestinfo properties via configuration_parameters (extra config), and now also via vapp > properties block (like terraform). So I'm going to leave that as an option for the users to set these themselves if they need to.

[luis-garza]

Hi again @sylviamoss
I can confirm what I expected, the ephemeral SSH key works like a charm for vApp native images, like official Ubuntu image, meantime it doesn't work for fully cloud init images.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.