Terraform: Chef Solo provisioner

[imduffy15]

Hi All,

Having a chef solo provider would be useful for provisioner examples without having to include chef-server infrastructure.

I believe it should be possible to copy most of the code from packer? https://github.com/mitchellh/packer/blob/master/provisioner/chef-solo/provisioner.go

[danielBingham]

A Chef Solo provisioner could be very useful for more than just provisioner examples. Chef Solo is fantastic for immutable infrastructure implementations.

When you're following immutable infrastructure practices, you don't really need a chef server. The chef server is there to keep existing infrastructure up to date and in sync with changes to your provisioning code. But in immutable infrastructure, you shouldn't really be changing nodes once they are through with initial provisioning. Immutable Infrastructure removes the entire need for a chef server. You just need the chef to run once, for initial provisioning, and then never again.

Chef Solo is perfect for that.

[mario21ic]

News ?

[apparentlymart]

Hi everyone,

Thanks for the great discussion here.

We (the Terraform team) agree that such a provisioner would be useful, but we have no plans to work on this in the short term since our attention is currently focused elsewhere. In order to keep the backlog manageable we're closing old, stale feature requests in order to be explicit about their status.

However, if someone in the community had the time and motivation to work on this we'd love to review a PR!

In the mean time, for the immutable infrastructure use-case we generally recommend doing such provisioning with a tool like Packer, baking the result into an image for Terraform to use, rather than having Terraform itself do the provisioning. This helps avoid drift that can result from re-running Chef Solo repeatedly over time, as upstream dependencies change, etc, and thus minimizes the risk of differences between instances in a cluster. Packer already has a chef-solo provisioner that is being successfully used in this way by several organizations.

[joestump]

Another option is cloud-init. Something like:

runcmd:
  - [ chef-solo, -j, /etc/chef/node.json ]

We have Packer build skeleton AMIs that don't have any environment (just stubbed data). We then use cloud-init and TF templates for user data:

data "template_file" "solo_rb" {
  template = "${file("${path.module}/../../../../cloud-init/solo.rb")}"

  vars {
    environment = "${var.environment}"
  }
}

data "template_cloudinit_config" "config" {
  gzip          = true
  base64_encode = true

  part {
    content_type = "text/chef-attributes"
    content      = "${var.json}"
  }

  part {
    content_type = "text/chef-solo"
    content      = "${data.template_file.solo_rb.rendered}"
  }

  part {
    content_type = "text/part-handler"
    content      = "${file("${path.module}/../../../../cloud-init/part-handler.py")}"
  }

  part {
    content_type = "text/cloud-config"
    content      = "${file("${path.module}/../../../../cloud-init/goldstar-${var.environment}/${var.service}/cloud-config.yml")}"
  }
}

This example uses the multi-part handler and whatnot. I wrote a custom part handler that recognizes Chef Solo JSON files as well. So when cloud-init runs the following happens:

• var.json is a TF template that's passed in here. It's the "DNA" passed to chef-solo above.
• solo.rb is a TF template as well; environment and whatnot are defined there from TF.
• The part handler writes everything out.
• The cloud-config.yml runs chef-solo on boot.

I also have a little bit of stuff to sync down roles, data_bags, secrets, data bags, etc. from S3. Hopefully this helps clarify the Chef Solo, Packer, and TF picture a bit.

[joestump]

@apparentlymart if you think these docs might be useful somewhere in TF let me know and I'll try and get something PR'd.

[Mwea]

Sorry to reopen the issue but I felt this could help.

For those who're looking for a way to use chef-solo on terraform, you can now use these plugin :
https://github.com/Mwea/terraform-provisioner-chef-solo

[akuzminsky]

@joestump I took your approach as it worked best for my use case. Few notes:

• The part handler must come before "unknown" types text/chef-attributes and text/chef-solo. Otherwise cloud-init fails to recognize the custom types.

data "template_cloudinit_config" "config" {
  gzip          = true
  base64_encode = true
  part {
    content_type = "text/part-handler"
    content      = file("${path.module}/part-handler.py")
  }
  part {
    content_type = "text/chef-attributes"
    content = ...
  }
  part {
    content_type = "text/chef-attributes"
    content = ...
  }
  part {
    content_type = "text/chef-solo"
    content      = "cookbook_path '/var/cache/cookbooks'"
  }
  part {
    content_type = "text/cloud-config"
    content      = file("${path.module}/cloud-config.yml")
  }
}

• The part handler should configure logging otherwise LOG.* outputs nothing

logging.basicConfig(level=logging.DEBUG)
LOG = logging.getLogger(__name__)

• Accepting chef licenses was a separate pain in the ass. Neither --accept-license not the environment variable did the job. Eventually I had to create empty marker files.

#cloud-config
write_files:
    -   content: |
            -----BEGIN PGP PUBLIC KEY BLOCK-----
...
            -----END PGP PUBLIC KEY BLOCK-----
        path: /etc/pki/rpm-gpg/RPM-GPG-KEY-CHEF

    - content: ""
      path: /etc/chef/accepted_licenses/chef_dk

    - content: ""
      path: /etc/chef/accepted_licenses/chef_infra_client

    - content: ""
      path: /etc/chef/accepted_licenses/inspec

yum_repos:
    chef-stable:
        baseurl: https://packages.chef.io/repos/yum/stable/el/7/\$basearch/
        name: chef-stable
        enabled: true
        gpgkey: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CHEF

packages:
    - chef
    - chefdk
    - git

runcmd:
    - 'export HOME=/var/cache/cookbooks'
    - 'mkdir -p /var/cache/cookbooks/'
    - 'git clone https://github.com/<org>/<repo>git /var/cache/cookbooks/<some cookbook>'
    - 'cd /var/cache/cookbooks/<some cookbook>; chef exec berks vendor ../'
    - 'chef-solo -j /etc/chef/node.json -c /etc/chef/solo.rb --logfile /var/log/chef-solo.log'

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.