Salt: Support passing variable bindings when including state SLS files

[pniederw]

It would be very useful to have something like pillar include's defaults: attribute (or file.managed's context: attribute) for state includes. Essentially, this would give a way to write composite state functions with SLS files alone.

[pniederw]

The same could be useful for Salt formulas. Instead of configuring them via pillar, they could be configured at the inclusion point, which is more straightforward and at the same time more flexible (different inclusion points can use different bindings, even on the same minion).

[basepi]

I'm probably misunderstanding, but does extend serve the purpose you're looking for? http://docs.saltstack.com/ref/states/extend.html

[mingfang]

I'm trying to do the same thing but extend does not work(well) for my use case.
I needed something similar to jinja's tempting feature http://wsgiarea.pocoo.org/jinja/docs/parts.html
I tried using jinja include file but kept getting a TemplateNotFound error.

[mingfang]

Ok I figured it our using Jinja templating.
This is an example of the template
https://github.com/mingfang/docker-salt/blob/master/srv/salt/docker/docker-template.sls
and this is how it is used
https://github.com/mingfang/docker-salt/blob/master/srv/salt/kibana/kibana-docker.sls

It works but it's very Jinja-ish and not Salt-ish at all.
It would be nicer for Salt to support SLS templating natively.

[basepi]

Ah, interesting. What might native SLS templating look like? Curious what "Salt-ish" templating would look like.

[mingfang]

hmm, maybe something that looks like the current extend.
so for my kibana example

include:
  - docker.docker-template

template:
  - docker:
    - name: kibana
    - ports: ['1234']

[regilero]

I do it with jinja macros also.

[scjudd]

I like @mingfang's example syntax. Between this, and not being able to extend Pillar data (#3991), I'm having a hard time finding ways to implement a template pattern.

---

I just stumbled across the stateconf renderer and am wondering if it may solve this problem. Thoughts?

[whiteinge]

Jinja macros are often abused in SLS files as a substitute for function calls (where calling an actual function via a custom execution module would be much cleaner) and we often discourage their use. However, this use-case is _exactly_ what they are designed to do: small inline templates take parameters.

The docker-template.sls file linked above could be wrapped with {% macro docker_template container_name, container_ports %}...{% endmacro %} and otherwise stay completely untouched. It would be called like this:

{{ docker_template('kibana', ['49082:80', '49021:49021']) }}

[dvcrn]

I am wondering whether this jinja trick is also possible through raw python. I have my users module written in python and would like to install and set up files in the directory of each user, for example zsh.

I wrote a zsh module which reads {{ homedir }} for the installation. How can I set that template variable in python directly and include my module?

[puneetk]

https://github.com/saltstack/salt/issues/1419 and https://github.com/saltstack/salt/pull/3613 is relevant

[daicoden]

FWIW - Template support is necessary for managing a SOA, and the macro solution was not obvious from reading the docs, seemed like a section was missing. I would pitch a syntax which mirrored file.manage specification.

include:
  - docker.docker-template:
    - template: jinja
    - defaults:
        custom_var: "default value"
        other_var: 123
  - docker.docker-template:
    - template: jinja
    - defaults:
        custom_var: "another value"
        other_var: 456
{% if grains['os'] == 'Ubuntu' %}
    - context:
        custom_var: "override"
{% endif %}

[cystek]

The macro solution also does not appear to work across different environments. That is, if I define the macro in a base environment but need to use it in a separate environment on top of base it will not work, giving an error "Jinja variable 'macro_name' is undefined." This feature, or at least allowing macros to work across environments, would be very helpful.

In case it's useful, the test macro in the base environment is:

{% macro test_template(file_path, file_source, template=False, file_user='root', file_group='root', file_mode='644') %}
{{file_path}}:
  file.managed:
    - source: salt://{{file_source}}
    {% if template %}
    - template: jinja
    {% endif %}
    - user: {{file_user}}
    - group: {{file_group}}
    - mode: {{file_mode}}
{% endmacro %}

and its attempted usage is:

include:
  - base: templates.test-template

{{test_template('/tmp/test.txt', 'templatetest/files/test.txt.jinja', True)}}

[anitakrueger]

I'm wondering if this is on any radar at the moment? I just ran into this because I have 2 states which need to include a common state and the 2 states need to be able to be applied on their own as well as together in a highstate. When running highstate however, I get the Detected conflicting IDs, SLS IDs need to be globally unique. error.

If I was able to use includes with parameters, then the highstate should work.

Here is my current code:
At the moment I have this, which produces the conflicting IDs error when both states are applied in the highstate:
a.sls

{%- from 'common.sls' import common with context %}
{{ common('appA') }}

b.sls

{%- from 'common.sls' import common with context %}
{{ common('appB') }}

common.sls

{%- macro common(app) %}
/opt/software/{{ app }}:
  file.directory:
    - user: root
    - group: root
{%- endmacro %}

Here is how I could see this work if includes supported defaults:
a.sls

include:
  - common:
    - template: jinja
    - defaults:
        app: appA

b.sls

include:
  - common:
    - template: jinja
    - defaults:
        app: appB

common.sls

/opt/software/{{ app }}:
  file.directory:
    - user: root
    - group: root

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[max-arnold]

+1 for parametrizable includes. Jinja macro doesn't work over salt-ssh and needs an extra_filerefs setting.

[stale[bot]]

Thank you for updating this issue. It is no longer marked as stale.

[jtoops009]

I'm also interested in passing in variable bindings since I have some templates that are mostly common and I rather not duplicate them.

[cystek]

@anitakrueger FYI, here's a possible workaround for your particular case using Jinja2 variables:

common.sls

{{ app }}-file-in-opt-software:
  file.directory:
    - name: /opt/software/{{ app }}
    - user: root
    - group: root

Creating a name out of the variable then defining the path later helps avoid name/state collisions like you are experiencing.

a.sls

{% set app = 'appA' %}
{% include common.sls %}

b.sls

{% set app = 'appB' %}
{% include common.sls %}

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[johnnybubonic]

requesting keeping open for points re: salt-ssh as @max-arnold mentions and deduplication per @jtoops009

[stale[bot]]

Thank you for updating this issue. It is no longer marked as stale.