Atlantis: Non-filesystem backends

[ojacobson]

I'm looking at deploying Atlantis to Heroku, since the free tier seems like a good fit for running this service on a project that only sees occasional updates. However, Heroku only provides an ephemeral filesystem, which disappears any time the service shuts down - at least once every 24 hours, and on the free tier, also after 15 minutes without a request. Since Atlantis writes its locks to disk, this means that locks are volatile and disappear quickly.

I'd love to store locks in an external service, such as PostgreSQL or etcd, if possible, so that filesystem ephemerality isn't a blocker.

I had a look at the locking package, and the lock API looks general enough to implement on other systems - treat this issue as a combined question ("is this important to anyone else?") and suggestion ("wouldn't it be nice if").

[lkysow]

Hi Owen,
Being able to run Atlantis with remote storage is definitely important to me and other users. My current plan is to implement remote storage in a couple of releases after I make Atlantis configurable enough to be useable by more people. That would be at least a month from now. When I get to remote storage, it will be the start of making Atlantis a platform that understands all of your Terraform repos, so it will encompass more than just being able to run without permanent disk.

In the meantime, if you'd like to take a stab at getting the current functionality to work with remote storage, that would be awesome.

In addition to the locks, we also store plan files on disk. We also don't re-clone a repo between plan/apply (although that's not too hard to fix). I think PostgreSQL would be a good backend to implement.

[ojacobson]

That's great to hear!

I didn't realize that Atlantis also stores the plans to disk, but in retrospect it makes perfect sense that it would do so. Fortunately, plans are "just" JSON, making them relatively easy to store elsewhere.

[lkysow]

NOTE: some people would like to be able to use S3 for this.

[hden]

People have all kinds of backend preferences (in our case GCS). I’d suggest you to make it a plugin. See https://www.terraform.io/docs/backends/types/index.html for reference.

[lkysow]

Closing as won't do. See https://github.com/runatlantis/atlantis/issues/265#issuecomment-481730130 for more details.