Salt: How can a binary file, such as a license key, be distributed via Pillar?

[clearclaw]

There doesn't seem to be a way to provision binary data into pillar values in order to feed to file.contents_pillar.

[basepi]

Thanks for creating this new issue. I'm still not sure of the best way to go about this, but we definitely want to support this!

[foxx]

+1 for this.

[kaithar]

Is there a limit for how much can be put in a normal pillar key?

[tkwilliams]

YAML (1.1) already suppors a binary storage type -- e.g.
http://yaml.org/type/binary.html -- and since pillar is already just
"arbitrary data", adding it should be as simple as using the syntax
outlined there.

That said, just how useful this would be in a particular context would
depend on your willingness to decode it back from base64 -- fairly simple
in a mako situation, but admittedly trickier in jinja...

On Sun, Jan 19, 2014 at 3:20 PM, kaithar [email protected] wrote:

Is there a limit for how much can be put in a normal pillar key?

—
Reply to this email directly or view it on GitHubhttps://github.com/saltstack/salt/issues/9569#issuecomment-32725756
.

[kaithar]

Yeah, thinking on it further, it would be better to have a sort of pillar_file_root with a top.sls like pillar_root and file_root, and then the .sls files specify a mapping of virtual/file/path: real/file/path.
That would allow a state to specify it wants, say, apache/private_key.pem and the mapping could rewrite that to a different file for each server, simplifying the state.

Another alternative would be to provide a masking and/or mapping option within the normal file_root so that the files can be kept along side the rest of the state files. That does have the risk of information leak due to misconfiguration though, which makes me wary of it.

[clearclaw]

Among the reasons I'd like to use a pillar (or pillar-like system) are:

-- I don't want it in the state tree because I don't want it copied to every faintly related minion and the state tree repository is visible to more people than I want to have access to the secure data
-- I want to limit visibility of the value to just those relevant minions (by targeting) for similar reasons of limiting the exposure and audience

I'd really just like to be able to say get-this-file-from-the-pillar without caring whether it is 7bit ASCII or binary, much as I do with file.managed/name/source in state trees.

And no, I don't want to much about with binhex, uuencode, xxencode, base64 or like binary->ASCII->binary encoding transformations.

[kaithar]

@clearclaw that's why I don't really like the idea of using a masking solution laid on the normal state tree, even though it is technically cleaner w.r.t. keeping related files in the same place.

The more I think about it, the more I definitely like the notion of a separated explicit access file server as I described in the first paragraph above.

[techdragon]

I quite like the idea of a pillar_file_root with its own top.sls
It seems quite logical that given the pillar is clearly just a data tree that arbitrary blobs of binary don't belong. So perhaps instead of needing to add files into the pillar we need a new thing like the pillar that is designed just for handling files.

In my case i really dont want to be storing most importantly SSL keys and to a lesser extent SSH keys in a way that delivers them onto every server managed by the master.

[LeTink]

Same as techdragon; I have the need to deploy ssl key/cert pairs to five machines, not the whole fleet of minions, and I'd rather a) keep them separate in pillar and b) w/o shoe-horning a bunch of lengthy strings into a single config file.

[arnisoph]

+1

[foxx]

@basepi Any idea how much work it would take to implement this? I'd be happy to donate some time to get this into the core, but would need some guidance on implementation.

[basepi]

@foxx I honestly am unsure what it would take. Currently, pillar is implemented as a single dictionary that we send down to the minions after compiling it. How do we implement binary files in this model? We could potentially encoded the file using the base64 library, which is designed to encoded arbitrary data for easy ascii sending. That's probably our best bet, but we would have to find a way to mark the fact that it's binary data, or make some utils to utilize the data on the other end.

[basepi]

Gah, sent early. Editing original comment to finish it.

[mr337]

+1, has any progress been made with this?

[basepi]

@mr337 Not yet, as far as I know. It's on the list.

[mr337]

@basepi thx for the update!

[srkunze]

This might be a related issue https://github.com/saltstack/salt/issues/14980.

[charleshbaker]

I'm looking for the best way to distribute kerberos keytab files which are binary (see: http://www.gnu.org/software/shishi/manual/html_node/The-Keytab-Binary-File-Format.html). That seems to fall into the realm of this issue.

+1

[ghost]

maybe file_roots on a per-minion/match/nodegroup basis. $0.02

[whiteinge]

Not a replacement for a true, file-based Pillar transport but a base64-based workaround that just uses plain ol' state and execution functions is in #19935.

[gczuczy]

May I ask whether there's any progress on this? Distributing things like keytabs is very important in an infrastructure, and some facility to do that would be nice, and at some places it's a crucial elements.

Also, if I may add to it, specifics. The tricky bit about keytabs, it's not just binary files, it's sensitive to its generation. A keytab contains a couple of principles, it needs to match the expected lists. Checking whether the client has the proper keytab is the tricky part. Generating a keytab again on the server side, then comparing the resulting keytab as a binary file with the client is a wrong approach, because every time a keytab is generated on the KDC a so called filed "kvno" is incremented, which invalidates keytabs with differently valued kvno fields for its principals.

Therefore the distribution of keytabs should take care how it's being checked whether it's the proper keytab being present on the minion. Doing the check should look like the following:
1) Salt knows the list of principals the keytab should contain
2) On the minion they existing keytab is queries whether it has the right principals
3) If the list of principals on the minion's keytab differs from the defined list, then it's regenerated on the master by contacting the KDC
4) The freshly generated keytab is shipped to the minion and placed on the filesystem.

Would be nice if salt had support for such things, working with kerberos would be way easier. Also, support for kerberos operations directly would be awesome.

[ghost]

+1: Kerb. & keytabs are useful. You might want to create an issue for the specific enhancements.

1. anything you can do from a command line salt can do on lots of machines without any special knowledge about it.
2. if you can write a python script to automate it, you can make that script salt-aware and automate all the remote execution steps.
3. Then add some DevOps logic to that in the form of a Salt States module, also in python.

"Modules are easy to write": http://docs.saltstack.com/en/latest/ref/modules/index.html
"States are easy to write": http://docs.saltstack.com/en/latest/ref/states/writing.html

[kaithar]

@gczuczy That level of sophistication is very much outside the scope of what this feature is intended to provide. It would definitely need to be a separate piece of code.

[gczuczy]

@seanchannel @kaithar Note taken. #20370 :)

[dario23]

until this is properly fixed, for all those who also look for that feature and find this bugreport:
at our site we currently use something like this, storing the base64-encoded host keytab in pillar instead of the actual binary data:

{% if 'krb5_host_keytab' in pillar %}
/etc/krb5.keytab.base64:
  file.managed:
    - contents_pillar: krb5_host_keytab
    - mode: 600

/etc/krb5.keytab:
   cmd.wait:
     - name: base64 -d /etc/krb5.keytab.base64 > /etc/krb5.keytab
     - watch:
       - file: /etc/krb5.keytab.base64
   file.managed:
    - replace: False
    - mode: 600
{% endif %}

[igorwidlinski]

+1

[poswald]

For the record, as a user I expected I could say pillar:// just like I can say salt:// in a pillar sls file:

users:
  poswald:
    public-keys:
      - salt://users/ssh_keys/poswald/id_rsa.pub
    private-keys:
      - pillar://users/ssh_keys/poswald/id_rsa

This is mentioned by @whiteinge in issue #19935 as well.

[iggy]

There's also #18406 to support pillar:// uris. I did some work on this at one point. Got it mostly working in a day, but it would serve anything in the pillar:// to any minion (so it ended up being a copy of salt:// uris that looked in pillar data). So it was missing the "this minion should have access to this data" layer. I can't imagine someone versed in python and salt internals couldn't knock this out in a couple of days.

[sysadmin4j]

+1 for keytabs
thanks @dario23 for the workaround

[basepi]

I was looking at this again today -- Ideally we would tie this solution into the fileserver and allow for serving files to minions based on permissions on the master. That solution is hefty and will take some serious work and thought.

As a stop-gap solution, I'm thinking about just adding an external pillar that would convert files in a given directory to pillar keys based on the definitions in a topfile-like structure. (Might be able to just use the topfile code, don't quite have that figured in my head)

You would just set it up as an external pillar, give it a directory, it would search for a top.sls in that directory, and base64 encode those files for specified minions, and stick that in pillar. We would then finish @whiteinge's work for interacting with base64-encoded pillar data to deploy those files on the minion itself.

This is a fairly straightforward solution (theoretically), and would be quite useful in my opinion. The drawback is that we have the whole file being shipped every time pillar is refreshed for that minion. Thus it would not be suitable for large files. But SSL certs, ssh keys, etc, are all small enough that this should not be an issue.

Thoughts?

(Note the label changes above are not promises, just want to get this on our radar for the next feature release.)

[QuinnyPig]

Idiot question: Why not integrate Pillar with Hashicorp's Vault for more robust secrets management?

[iggy]

Orthogonal problems really. Plus not everybody wants to be tied to something like Vault for managing their pillars. Sometimes, a bunch of files are "good enough".

[whiteinge]

I got crazy-sidelined on this work. I did start work on an ext_pillar module as well. I prefer @basepi's idea of grabbing everything in a dir compared to where I was headed.

[tbaker57]

Can this be integrated with the file_tree external pillar http://docs.saltstack.com/en/latest/ref/pillar/all/salt.pillar.file_tree.html ? Seems that the automatic base64 encoding/decoding is the only missing thing...

[basepi]

Ah, yes, good idea. We should add it there, and just add base64 functionality for binary files.

[igorwidlinski]

http://docs.saltstack.com/en/latest/ref/pillar/all/salt.pillar.file_tree.html would assume part of your set up uses nodegroups. Some people do not, ie. we extensively use pillar entries to group/classify servers.

[basepi]

I have not looked closely at that external pillar module, I was mostly operating under a vague memory of when it was added. It may not fit the bill, when we get to this one we'll investigate and make sure we make it as generally useful as possible.

[babilen5]

I much prefer @basepi's approach in that it doesn't conflate targeting and data like the, therefore not really useful, file_tree ext_pillar. I guess what is missing is the ability to recursively add entire directories.

[whiteinge]

Reopened #26690 to at least get this ball rolling. I don't have time to devote to adding features right now but the base functionality works just fine -- with a little help from Jinja (see the docstrings for usage). The ext_pillar addition described in the last few comments will be a good addition but this pull req is functional. Additional contributions/fixes/tweaks are welcome.

[meggiebot]

Discuss implementation with Tom when you get to this.

[DanyC97]

+1

[terminalmage]

We may now have the necessary support for this, via the file_tree external pillar combined with the contents_pillar argument to the file.managed state.

I'll do some investigation and, if I'm right about this add an example use case to the documentation and link to it from our FAQ.

[terminalmage]

OK, technically it will work, but I'm making some changes to make it work smoother. For instance, right now by default it is necessary to set contents_newline to False in a file.managed state (default value is True) to deploy binary contents, otherwise a newline will be appended to the end of the file. It's easy enough to check if the contents are binary though, so I'm going to modify the file.managed state to ignore this option if the contents are binary.

I'm also making some optimizations to the file_tree external pillar that will make reading in larger files more efficient, and improving the documentation for this process overall.

[terminalmage]

30268 adds support for binary contents in contents/contents_pillar/contents_grains to the file.managed state, and adds an example of deploying binary contents to the FAQ in the docs.

[anlutro]

@terminalmage How feasible do you think it would be to combine file_tree with git_pillar somehow? That is, I'd like to configure a file_tree external pillar using a directory that's located in a git repository, without having to manage it with a git.latest state.