Salt: YAML.org recommends `.yaml` extension for pillar data

[noelmcloughlin]

Description of Issue/Question

By convention YAML markup files should use .yaml extension but salt refuses to read YAML file saved with this extension. This is non-intuitive, a barrier to adopting salt, and does not follow recommendation from https://yaml.org/faq.html

Is there an official extension for YAML files?
Please use ".yaml" when possible.

Please allow .yaml extension for pillar data.

Setup

salt error for pillar data in saved as /srv/pillar/template.yaml

    Data failed to compile:
----------
    Pillar failed to render with the following messages:
----------
    Specified SLS 'template' in environment 'base' is not available on the salt master

Salt enforces .sls extension for files in YAML format.

1. L82
2. L120
3. L148

Here's the code in salt/pillar/__init__.py, which calls salt/fileclient.py.

Steps to Reproduce Issue

Versions Report

All versions

[aboe76]

@noelmcloughlin but pillar.sls files aren't only yaml they could contain jinja or ggp secrets,
see for the complete list: https://docs.saltstack.com/en/latest/ref/renderers/

[dwoz]

@noelmcloughlin As @aboe76 points out, pillars are a mixture of jinja templating and yaml hence the sls file extension. Having said that I think it might make sense to be able to load static data from yaml files under the pillar roots.

@saltstack/team-core thoughts on this?

[waynew]

I'm personally disinclined given the fact that they're not actually YAML files, they just use YAML syntax. YAML files just happen to be a subset of Salt state and pillar files. I suspect there may be documentation that suggests otherwise, but it seems like allowing .yaml for pillar is misleading.

This is non-intuitive, a barrier to adopting salt

If we have some documentation that suggests a .yaml extension is OK, then we should absolutely fix that. Otherwise, can you help me understand why that would be confusing? As far as I'm aware, our documentation only says to use .sls extensions, so if someone is reading the documentation they shouldn't be thinking they can use .yaml extensions. What am I missing here?

[thatch45]

I understand where this is coming from but I would strongly discourage it. it would mandate that how we look up sls files needs to be changed and could potentially break any existing pillars if they have added .yaml files tot he tree for any other reason.

I also think it makes adoption harder, because now there is an arbitrary, confusing construct that pops up in random pillar trees and a need to explain why we have multiple loaded file types

[waynew]

If you're really interested in being able to load from .yaml files, you can create a pillar module and in your ext_pillar function load the .yaml file yourself.

Or, there's also the cmd_yaml pillar module that you could use, though I'm not entirely sure on the syntax there.

[noelmcloughlin]

@noelmcloughlin but pillar.sls files aren't only yaml they could contain jinja or ggp secrets,
see for the complete list: https://docs.saltstack.com/en/latest/ref/renderers/

This issue is to request support for files with extension .yaml because its a data exchange format. People and projects can release software with bunch of yamls and people can use salt, ansible, or programmatic stuff to consume those files - they can add jinja2 and gpg secrets too.

[noelmcloughlin]

The background to this request is discussion at https://github.com/saltstack-formulas/template-formula/pull/62

[dwoz]

@noelmcloughlin One thing to note about this is that sls files do not have to contain any yaml.

If you want to consume raw yaml files as pillar data I think the best approach is to use the external pillar system:

https://docs.saltstack.com/en/latest/topics/development/external_pillars.html

It should be a fairly straight forward exercise to write an ext_pillar that reads your yaml files.

[noelmcloughlin]

Hi @dwoz
That sounds interesting. If there is consensus to leave this alone regarding extension, that's fine with me.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.