Terraform: Jenkins Provider

[bivas]

It will be great to have terraform configure Jenkins CI
Maybe something like Jenkins Job Builder (in term of supported and required plugins)

Support for:

• Configuring basic system
• Configuring maven job
  
  
  
  • Using SVN/Git/Mercurial
  
  
  • Build triggers
  
  
  • Dependent jobs
  
  
  • etc.
  
  
• Configure builders
• Configure plugins?

[radeksimko]

I was thinking about this use case couple months ago and I agree it would be really nice to be able to define Jenkins configuration in DSL like HCL.

The first issue is that you need (or _should_ have) a CI system for planning and applying Terraform configs, so you would either end up having a separate Jenkins instance or a few hand-crafted Jenkins jobs or a different system for managing Jenkins configuration.

Secondly if you think about full IaC - having _everything_ codified _per project/repository_:

• dev env (Vagrantfile, docker-compose.yml, ...)
• build process (packer json, Jenkinsfile, travis.yml, circle.yml)
• deployment (terraform tf files)

then the problem is that codifying Jenkins configuration may not fit well into that model as the project-related jenkins job can't "create itself".

i.e. We have a 🐔 🍳 situation here 😉

[jen20]

@radeksimko There may be a 🐔 and 🍳 situation, but of course, Atlas provides the full English breakfast as far as running Terraform goes :-)

I'm certainly not opposed to a Jenkins provider in principle, but we don't use it internally so we are unlikely to build it in the near future. We'd be highly likely to accept a PR made under the contributing guidelines however!

[radeksimko]

@jen20 Agreed, but my second point still stands - the Jenkins job would need to be managed separately from the project codebase which makes it slightly less appealing for full IaC environments per project 😺

[jhoblitt]

Speaking as a maintainer of https://github.com/jenkinsci/puppet-jenkins, I can report that creating jobs via the API is relatively straight forward with the qualification that the input needs to be an xstream xml dump. Configuring core settings, plugin parameters, etc. often requires directly modifying on disk state files (also xstream xml dumps) or uploading groovy snippets that modify internal state. I caution not to under estimate the amount of effort required to implement this. That said, if someone does decide to tackle this I would be happy to share notes.

[apparentlymart]

At my employer we use an out-of-tree provider to configure Buildkite, which serves a similar purpose to Jenkins. As @radeksimko noted, there is a bit of a chicken-and-egg problem here so we have a special repository called "codebases" which contains the Terraform configs to create version control repos, teams, the build pipelines in Buildkite, and then various webhook glue to make them work together. In spite of it being in a separate repo, it's still been valuable to manage all of that stuff in a central place rather than having to hand-configure Buildkite for each codebase, give it the right git URL, etc, etc, etc...

We actually started down the road of building a Jenkins provider before we decided to switch to Buildkite. One sorta-unique challenge with Jenkins is that its ability to be extended with plugins means that the XML format representing jobs can look very different on different versions of Jenkins, so we'd designed it such that it would just take the raw XML and load it into Jenkins rather than trying to define jobs within the Terraform DSL.

Sadly it looks like we didn't keep that half-finished code, so I don't have anything more concrete than that to say about it.

[stack72]

Hi,

Thanks so much for the request for this new provider!

While we'd love to see something like this, we don't currently have any plans to implement this ourselves. Until then, this issue is unlikely to see any movement and remain stale. We're trying to prune the stale issues (that aren't going to be addressed anytime soon) by closing them. Note that we only do this for enhancement requests and not bugs.

We have future plans to enable community plugins to be available easily from the Terraform binary itself, allowing community members to "ship" plugins with Terraform much easier. There isn't a timeline for this yet but we hope that will allow much easier shipping of both new providers and updates to existing.

Thanks!

[tommywo]

FYI: someone started working on it -> https://github.com/dihedron/terraform-provider-jenkins

[MartinKosicky]

It would be very interesting, because we can create a terraform (first class citizen) that will deploy "CORE" infrastructure. We have then another jenkins jobs that are configured using folder properties on which CORE infrastructure we are deploying. It would be cool if we deployed new core infrastructure and along the way it would create jenkins jobs that are configured to deploy into the given infrastructure (everything using a single terraform job). Consider you want to start your cloud in a new region and don't want to make milion new jobs for that region, or add input text fields into jenkins to specify each time where you want to deploy. Although I must admit I am not very experienced in the devops how to deploy to multi regions, if it should be a single job or multiple jobs

[apparentlymart]

Hi @MartinKosicky, and anyone else still watching this!

Note that the discussion above largely happened before the Terraform architecture change that made providers be independent codebases from Terraform itself, with separate distribution channels. Therefore it's now possible to write such a provider outside of Terraform Core, and as @tommywo noted it seems like at least one person was working on one.

Currently it's quite hard to use providers outside of the official set because they must be manually installed, but work is in progress towards automatic installation of third-party providers, as described in today's block post _Announcing Providers in the New Terraform Registry_.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.