Atlantis: Proposal: Write plans to S3

[sargun]

I propose we write plans to S3, or enable writing plans to S3. I think we do this by allowing the user to specify --s3-plan-dir with an S3 URL that we write to and retrieve plans from.

[sargun]

@lkysow @anubhavmishra Can I get your blessing before taking this approach / working on this?

[lkysow]

Hi @sargun, I'm going to reach out to you to schedule a meeting because this is a large architectural change. Check your email :)

[majormoses]

Is the idea to make it easier to setup HA solutions? I would think that we would want to have that be server side config and not a per repository atlantis.yaml. One option we might consider is rather than this being in the code we allow that use case to be handled by solutions such as https://github.com/s3fs-fuse/s3fs-fuse

[lkysow]

This would be a server-side config for sure. I don't think it would be per repo, I think Atlantis would either run in local mode or "HA mode" where it uses a db. I'll flesh this out further on my call with Sargun.

[sargun]

@majormoses Yeah, I disagree. FUSE is a tool of last resort, iff we cannot come to another solution. The other issue here is that Atlantis puts plans and the Bolt database in the same place. The Bolt database wont work on s3-fuse.

[majormoses]

@sargun I agree with you it's not the ideal situation I stated that it could be one option we pursue as there is limited bandwidth available and it might be something that could offer you a quick win while we overhaul the architecture.

[jolexa]

Subscribing to this issue. I filed a similar thing in #237 but that was marked as a duplicate of #58

[lkysow]

Yes I consider this one a dupe of that too. I'm going to create a proper set of tickets in coordination with Sargun.

[psalaberria002]

I guess if we are adding an S3 backend, we should make it easy to also integrate with GCS (and others).

[lkysow]

I'm not committed to adding an S3 backend. I'd actually prefer to use a single database.

[sargun]

My proposal for enabling S3 would be to adopt go-cloud. I've proposed the ability to initiate it from a URL (https://github.com/google/go-cloud/issues/423).

[lkysow]

This project is in alpha and is not yet suitable for production.

I don't think we'll be able to use it right away if this is the case. Atlantis needs to be production ready.

[sargun]

@lkysow Aye, but if we start with enabling it via URI, then when we want to adopt a layer like this it's easy.

[pratikmallya]

@lkysow was there a proposal that came out of these discussions? I think enabling either s3/gcs instead of boltdb would be extremely helpful

[pratikmallya]

@lkysow bump. Its somewhat unfortunate that all of discussions around storing remote state seem to offer very little details about what the future plans are. Both this and https://github.com/runatlantis/atlantis/issues/58 talk about similar issues.

• If there is a design and someone just needs to implement it, would be nice to let us know
• If there's no design... can we simply add different implementations of https://github.com/runatlantis/atlantis/tree/master/server/events/locking for different backends and not just boltdb?

If there is no plan I would like to take a stab at 2. Its really frustrating to contribute to this repo because you seem to be opinionated on how to solve issues, but the details aren't out here anywhere, and this is something we really really want (HA for atlantis).

[lkysow]

Hi Pratik, Sargun and I did talk however nothing came out of it, I'm guessing they didn't end up using Atlantis.

If there's no design... can we simply add different implementations of https://github.com/runatlantis/atlantis/tree/master/server/events/locking for different backends and not just boltdb?

That wouldn't be sufficient to actually support an "HA" mode. Atlantis needs to be re-written to support a remote db, remote plans and re-cloning in-between plan and applys in order for HA to actually be effective. This would require a re-architecting of a large portion of the Atlantis codebase. In addition, all future releases that made changes to the Atlantis DB schema would need to include migration scripts and detailed upgrade instructions.

I'm not in a position where I can support such a large change to Atlantis at this time so unfortunately I think this issue should be closed.

Its really frustrating to contribute to this repo because you seem to be opinionated on how to solve issues, but the details aren't out here anywhere.

I'm really sorry you've experienced this. What issues do you want to work on and how can I help?

Often when people open an issue, they're not actually in a position where they can implement the feature themselves and I can't implement every feature people ask for, so if I were to write up a detailed design for every open issue then that would just be a lot of wasted time.

Yes I am very opinionated on how to solve the issues because as the maintainer I'm responsible for maintaining any features forever.

[lkysow]

Closing as won't do because I can't see this getting done anytime soon (see above ☝️)