Sceptre: Add validate-env command

[JQUINONES82]

I would like to test that all stacks in an environment generate templates properly and return a status code of 0 (no error).

If Sceptre user data has stack resolvers, could we some how use STS to avoid MFA for every stack verification.

Could we possibly have generate-env-template?

[johnculkin]

This would be great for continuous testing :-)

[jamesroutley]

I'm not sure if generate-env-template would work - the main use case of generate-template is to generate a native CloudFormation template, not check whether it compiles. Generate template prints the template to stdout, where users can pipe it to a file if they like. Generating a whole env of templates would break this piping model. We could write directly to files, but it feels too side-effectey for me, and I think that practically the syntax would be very verbose (I imagine you'd have to supply a command line flag for each stack specifying the file to store the template in).

A better command may be validate-env-templates - you should check out validate-template if you haven't already. It compiles the template, so will check that, but also makes an API call to AWS to perform a more in-depth validation of the template.

If you don't want to validate the templates and continue generating them, you could write a bash or powershell loop which searches for the stacks and runs the command on each of them individually.

As for the MFA problem, we'll look into it. It's been flagged up as an issue in the past, but it's a bit out of our control. Sceptre is stateless, so it doesn't store any session information between invocations. An environment-level command may help, but users enable security for a reason and so I'd be unwilling to hack around it too much.

[johnculkin]

+1 validate-env-templates

Makes sense given that we don't actually want the output, we just want to verify that things are working.

[theseanything]

Yup agree validate templates should be the goto for testing. Doing so for a whole env might be freebie if we update the CLI - #62, #60. If it were an env command, it would minimise MFA just to one entry.

[jamesroutley]

@JQUINONES82 What do you think about a validate-env command instead?

[johnculkin]

He agrees :-)

[theseanything]

This seem something that would fit in nicely with CLI changes proposed with V2.0 - should we add to that milestone @jamesroutley ?

[cbosss]

I'm interested in working on this. Has anyone given any thought to what the output would look like? My current implementation is simply concatenating the output of the validate-template calls. This provides a lot of feedback to the developer, but as the number of stacks grows, it might be difficult to see if all the stacks were valid and which are invalid.

Maybe we could add a --summarize flag, which would only print the stack name, and whether it was valid or not?

[ngfgrant]

Closing - Sceptre v2 now allows this to happen. Won't be fixed by the core team for v1.