Terraform: Vsphere datastore cluster support

[hugoboos]

Hi,

It looks like the Vsphere provider does not have support for datastore clusters.
When I provide a datastore cluster name as datastore under the disk section, Terraform reports back that it can't find the datastore.

With govc from https://github.com/vmware/govmomi I can list the the datastores with the command govc ls.

Thanks!

[tkak]

Current version supports datastore cluster feature partly. If you don't mind, could you tell me your use cases more detail? (.tf file, datastore directory path, etc)

[hugoboos]

We have a VMware cluster with resources pools, storage DRS. The whole enterprise thing.

What we want is to deploy virtual machines based on cluster names, resource pool names, datastore cluster names, etc. like below in the main.tf file. We do not want to specify the directory path but only the name of the object.

Regarding storage, Storage DRS should decide on which datastore the virtual machine ends up.

# main.tf

provider "vsphere" {
  vcenter_server = "vcenter"
  user           = "DOMAIN\account"
  password       = "password"
}

resource "vsphere_virtual_machine" "webserver01" {
  name   = "webserver01"
  vcpu   = 4
  memory = 2048

  datacenter    = "datacenter name"
  cluster       = "cluster name"
  resource_pool = "resource pool name"

  time_zone = "Europe/Amsterdam"

  network_interface {
    label = "network name"
  }

  disk {
    datastore = "datastore cluster name"
    size      = 40
  }
}

[chrislovecnm]

@tkak what changes would need to take place?

@mijit thoughts?

[kc11381]

Hi,

For this we need support at govmomi library itself. https://github.com/vmware/govmomi/blob/master/find/finder.go look at line number 255.
I think we should have something
if ref.Type == "Datastore" || ref.Type == "StoragePod"{

I have done it for my use case and it works fine.

[chrislovecnm]

@kc11381 you mind filing an issue with Govmomi and referencing it here?

[kc11381]

Hi Chris,

Here is the ticket:
https://github.com/vmware/govmomi/issues/386
Thank you.

[hugoboos]

Cool.

If we need to test something, please let me know. I'm willing to help out.

[daryn-kerr]

anyone had any success with this? would be great to get it working.

[mixacha]

Support for datastore clusters was merged in https://github.com/vmware/govmomi/pull/428 I guess nothing stops us on terraform side? Waiting on this one as its a God sent tool :-)

[mixacha]

@chrislovecnm itd be great if someone looked into this one now that vmware fully supports it

[chrislovecnm]

@mixacha unfortunately I no longer have my vsphere test bed ;(

[mixacha]

@chrislovecnm unfortunately :(
Really the last piece of puzzle for to enable enterprise companies to use the tool.

[derng]

Same here. I need to demonstrate how powerful Terraform is and I'm from one of the biggest enterprise companies :(

[GSkouroupathis]

+1

[mixacha]

I've got a "small" lab that I can run tests against on any ESX version (currently 5.5 and 6 are installed in it).

[chrislovecnm]

@mixacha appreciated. You develop at all?

[kc11381]

I will probably look at it. I had customized this plugin for my use case and I too needed Datastore cluster support. At that time I used some hack (basically joining paths) to support it and that is working fine.
Now that proper support is available from govmomi api's, we should look at it.

[chrislovecnm]

@kc11381 that would be aaaawwwweeesssoommmme!!!!

[Shruti29]

+1 badly need this :)

[chrislovecnm]

@stack72 @phinze @jen20

1. please remove the bug tag
2. please add feature

Everyone on this thread I will help this become a P1 feature.

[chrislovecnm]

Pulling in couple more active devs and users:

@dkalleg @kristinn need some design ideas here. Govomi has added support for datastores, https://github.com/vmware/govmomi/pull/428, and now it needs to be implemented in TF. Would either of you be willing to take this on? I can but it may be a couple of weeks, and the scope is probably not huge.

@hugoboos @tommymalmqvist @xantheran as vsphere gurus can you give us silly developers use cases, off of the top of my head:

1. what happens without DRS
2. what happens in a cluster
3. what happens without a cluser
4. what happenes without datastores
5. how does TF know which use case

@phinze the natives are saying that this has about as much need as windows support 😄

[kristinn]

@chrislovecnm I would love to. Unfortunately I don't have access to a vmware instance with a datastore cluster configured. I've just used our small vmware setup at work so far while I'm creating PR for TF. 😞

[chrislovecnm]

@kristinn any comments about design would be awesome. Yah lots of bells and whistles ..

[mixacha]

@kristinn You can always run a hypervisor within a hypervisor + vCenter in trial mode for 60 days with highest license tier available and attach two VMDKs to form a datastore custer for testing purposes :-)

@chrislovecnm I'm not a developer but logic inclines that terraform plugin should be able to determine regular datastore vs DS cluster (I believe these are marked as StoragePod type), and should a DS cluster be detected for placement, collect recommendations about initial placement - same as PowerShell cmd-lets are doing this. (also I think govc implemented a similar fashion to detect DS clusters and place virtual machines on top of them).

[chrislovecnm]

This is a long time coming :) It will take 2+ PR's to get in.

The first one https://github.com/hashicorp/terraform/pull/6777

[chrislovecnm]

Still working on this ... But it is getting closer 😄

[chrislovecnm]

If anyone on here is able to help testing, please let me know!!! Code is here: https://github.com/hashicorp/terraform/pull/7031

Initial support for VMs. Disks, Files, and CDs are not yet supported.

[pfrayer]

Hello, any news on this feature, aka using datastore cluster name ?

Having a combo terraform + vmware for huge company is a must have. If stuff like datastore cluster doesn't work, is totaly unsable/useless. And then terraform just become a "tool used for POC or dirty testing", and not for an industrialized-enterprise-grade usage.

[gbolo]

+1

Would be nice to be able to deploy to a datastore cluster.

[silveraid]

This would be really nice to have

[renaudhager]

This feature is definitely needed.

[Defilan]

Our environment relies heavily on datastore clusters and we use Terraform daily. This feature would be amazing!

[paultravers]

This would be a killer feature. Currently trying to use a datastore cluster crashes vsphere.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.