Sceptre: Jinja support in templates

[brian-provenzano]

I'm fairly new to sceptre and I was wondering what is supported in jinja templates? Can we do things like "includes", looping etc or is it just limited to referencing sceptre_user_data?

{{ sceptre_user_data.something }} this works OK

{{ include('user-data.sh')|indent(6) }} is this supported? I couldnt get this to work. I was trying to reference a script as an external resource similar to terraform

I also tried this syntax:
{% include 'userdata.sh' %}

This is the reference I am using
http://jinja.pocoo.org/docs/2.10/templates/

It could be I have a general misunderstanding of jinja and sceptre as well. If so, forgive my stupid question :)

Also, I'm not sure if this is the best place to ask questions like this - I don't want to pollute the issue tracker with general product function questions if I dont have to. Is there another methods? Slack channel etc?

[1oglop1]

@brian-provenzano Yes colleague of mine ran into same issue where we found out that many directives are not supported like include or extends etc..
As a workaround we ended up with custom resolver - which is not very practical solution.

@niallgrantcloudreach - I already discussed it on CR slack and Juan said it was for security reason, which I believe is not valid argument since sceptre is leveraging jinja it should support all functionality or clearly state why certain things do not work.

custom resolver usage:

template_path: external-templates/directoryservice/cognito_userpool.py
sceptre_user_data:
  user-pools:
    SuperPool{{ environment_config.account_name | title}}{{ environment_config.short_region | title}}CognitoUserPool:
      schema:
        !jinja_include files/cognito/user-pool/schema.j2.yaml

Resolver code:

# -*- coding: utf-8 -*-
import os

import yaml
from sceptre.resolvers import Resolver
from jinja2 import Environment, FileSystemLoader


class JinjaInclude(Resolver):
    """
    Resolver for getting the contents of a file which would work as a jinja include

    :param argument: Absolute path to file.
    :type argument: str
    """

    def __init__(self, *args, **kwargs):
        super(JinjaInclude, self).__init__(*args, **kwargs)

    def resolve(self):
        """
        Retrieves the contents of a file at a given absolute file path.

        :returns: Contents of file resolved by jinja using sceptre namespace
        :rtype: dict
        """
        try:
            path, filename = os.path.split(self.argument)
            render_result = Environment(
                loader=FileSystemLoader(path)
            ).get_template(filename).render({
                "environment_config": self.environment_config
            })
            return yaml.safe_load(render_result)
        except (EnvironmentError, TypeError) as e:
            raise e

[akdor1154]

I'm currently using {% include %} in stack config happily, haven't tried it directly in a template though.

(for it to be disabled for "security reasons" would be puzzling given the ability to directly execute python code as templates anyway...)

[1oglop1]

@akdor1154 Do you have any example of using it? for some reason it did not work in our environment
and instead doing anything it just printed file name from include

given stack config

sceptre_user_data:
  {% include 'somefile.yaml.j2' %}

when called sceptre generate-template env stack the result is:
somefile.yaml.j2
nothing else is executed

[nbjcn1]

Include is fully supported as we simply use jinja with no customization. @1oglop1's responses are wrong

{{ include('user-data.sh')|indent(6) }} is not native jinja functionality and must be implemented as a macro for more info see (https://stackoverflow.com/questions/10821539/jinja-keep-indentation-on-include-or-macro)

Due to the nature of jinja, you can not escape the directory the file being rendered is in, so if you are in config you can only include files in config or subdirectories, and the same goes for your templates directory.

so in config/my-first-stack.yaml

{% include 'tags.yaml' %}

will read config/tags.yaml

but config/my-first-stack.yaml

{% include 'userdata.yaml' %}

will fail

This does cause the limitation that anything you include in your templates must be under a root templates directory, but as a reward all the jinja documentation is 100% right

if you want to include userdata in your template your userdata.sh must exist along (or below) side the template, it cannot be above

If you want to decide which file to include base on userdata you need to pass in the name instead of the contents, this is probably what you should be doing anyway. e.g

{% include sceptre_user_data.userdata_filename %}

[brian-provenzano]

Thanks @nbjcn1 !

That does appear to work for me. I can work with that limitation. It would be great if this was explained in the docs as well. It seems like it is quite misunderstood.

Are the other jinja features supported as well (looping, conditionals, etc)?

Finally, I am really enjoying sceptre so far - thanks to all at cloudreach for open sourcing this tool!

@1oglop1

"I already discussed it on CR slack..."

There is a Slack channel available for sceptre? If so, is it available to join?

[JuanCanham]

Yes everything in Jinja is supported, in both templates and config files (honestly I'd only use it for including common things like tags).

Unfortunately we only have an internal slack channel, so either stackoverflow or here is the best place to ask questions. This is based on lack of percived demand Vs effort to moderate, so if there are enough people who would like a slack/similar let us know and we'll re-evaluate.

Finally, I am really enjoying sceptre so far - thanks to all at cloudreach for open sourcing this tool!

No problem, can't take credit for the work of others, but I'm also glad we made it and opensourced it as both those things make my life easier.