Salt: Accessing pillar data in custom grains module in Salt

[DimentR]

Description of Issue/Question

I'm trying to get access to the Pillar of my GRAINS module in _grains.
Tell me, is it possible?

Steps to Reproduce Issue

Scenario 1

def _get_pillar():
    import salt.modules.pillar
    test_pillar = salt.modules.pillar['pillar.get']('postfix:relayhost')
    return test_pillar


def _myfunc():

    my_grain = {'foo': 'bar', 'hello': _get_pillar()}
    return my_grain


def main():
    # initialize a grains dictionary
    grains = {}
    grains['mygrain'] = _myfunc()
    return grains

Result:

Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 634, in grains
ret = fun()
File "/var/cache/salt/minion/extmods/grains/autoroles.py", line 25, in main
grains['mygrain'] = _myfunc()
File "/var/cache/salt/minion/extmods/grains/autoroles.py", line 18, in _myfunc
my_grain = {'foo': 'bar', 'hello': _get_pillar()}
File "/var/cache/salt/minion/extmods/grains/autoroles.py", line 12, in _get_pillar
test_pillar = salt.modules.pillar'pillar.get'
TypeError: 'module' object has no attribute '__getitem__'

Scenario 2

def _get_pillar():

    from salt.client import Caller
    test_pillar = Caller().sminion.functions['pillar.get']('postfix:relayhost')
    return test_pillar


def _myfunc():

    my_grain = {'foo': 'bar', 'hello': _get_pillar()}
    return my_grain


def main():
    # initialize a grains dictionary
    grains = {}
    grains['mygrain'] = _myfunc()
    return grains

Result:

Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 634, in grains
ret = fun()
File "/var/cache/salt/minion/extmods/grains/autoroles.py", line 25, in main
grains['mygrain'] = _myfunc()
File "/var/cache/salt/minion/extmods/grains/autoroles.py", line 18, in _myfunc
my_grain = {'foo': 'bar', 'hello': _get_pillar()}
File "/var/cache/salt/minion/extmods/grains/autoroles.py", line 9, in _get_pillar
test_pillar = Caller().sminion.functions'pillar.get'
File "/usr/lib/python2.7/dist-packages/salt/client/__init__.py", line 1594, in init
self.sminion = salt.minion.SMinion(self.opts)
File "/usr/lib/python2.7/dist-packages/salt/minion.py", line 528, in init
self.io_loop.run_sync(lambda: self.eval_master(self.opts, failed=True))
File "/usr/lib/python2.7/dist-packages/tornado/ioloop.py", line 445, in run_sync
return future_cell[0].result()
File "/usr/lib/python2.7/dist-packages/tornado/concurrent.py", line 215, in result
raise_exc_info(self._exc_info)
File "/usr/lib/python2.7/dist-packages/tornado/gen.py", line 230, in wrapper
yielded = next(result)
File "/usr/lib/python2.7/dist-packages/salt/minion.py", line 504, in eval_master
pub_channel = salt.transport.client.AsyncPubChannel.factory(self.opts, *_factory_kwargs)
File "/usr/lib/python2.7/dist-packages/salt/transport/client.py", line 154, in factory
return salt.transport.zeromq.AsyncZeroMQPubChannel(opts, *_kwargs)
File "/usr/lib/python2.7/dist-packages/salt/transport/zeromq.py", line 254, in __init__
self.auth = salt.crypt.AsyncAuth(self.opts, io_loop=self.io_loop)
File "/usr/lib/python2.7/dist-packages/salt/crypt.py", line 333, in new
new_auth.singleton_init(opts, io_loop=io_loop)
File "/usr/lib/python2.7/dist-packages/salt/crypt.py", line 375, in singleton_init
salt.utils.reinit_crypto()
File "/usr/lib/python2.7/dist-packages/salt/utils/init.py", line 411, in reinit_crypto
Crypto.Random.atfork()
File "/usr/lib/python2.7/dist-packages/Crypto/Random/init.py", line 37, in atfork
_UserFriendlyRNG.reinit()
File "/usr/lib/python2.7/dist-packages/Crypto/Random/_UserFriendlyRNG.py", line 224, in reinit
_get_singleton().reinit()
File "/usr/lib/python2.7/dist-packages/Crypto/Random/_UserFriendlyRNG.py", line 215, in _get_singleton
_singleton = _LockingUserFriendlyRNG()
File "/usr/lib/python2.7/dist-packages/Crypto/Random/_UserFriendlyRNG.py", line 159, in __init__
_UserFriendlyRNG.__init__(self)
File "/usr/lib/python2.7/dist-packages/Crypto/Random/_UserFriendlyRNG.py", line 85, in __init__
self._fa = FortunaAccumulator.FortunaAccumulator()
File "/usr/lib/python2.7/dist-packages/Crypto/Random/Fortuna/FortunaAccumulator.py", line 126, in __init__
self.pools = [FortunaPool() for i in range(32)] # 32 pools
File "/usr/lib/python2.7/dist-packages/Crypto/Random/Fortuna/FortunaAccumulator.py", line 55, in init
self.reset()
File "/usr/lib/python2.7/dist-packages/Crypto/Random/Fortuna/FortunaAccumulator.py", line 71, in reset
self._h = SHAd256.new()
File "/usr/lib/python2.7/dist-packages/Crypto/Random/Fortuna/SHAd256.py", line 94, in new
sha = _SHAd256(_SHAd256._internal, SHA256.new(data))
File "/usr/lib/python2.7/dist-packages/Crypto/Hash/SHA256.py", line 88, in new
return SHA256Hash().new(data)
File "/usr/lib/python2.7/dist-packages/Crypto/Hash/SHA256.py", line 75, in new
return SHA256Hash(data)
File "/usr/lib/python2.7/dist-packages/Crypto/Hash/SHA256.py", line 72, in __init__
HashAlgo.init(self, hashFactory, data)
RuntimeError: maximum recursion depth exceeded

Versions Report

Master:
Salt Version:
Salt: 2015.8.10

Dependency Versions:
Jinja2: 2.7.2
M2Crypto: 0.21.1
Mako: 0.9.1
PyYAML: 3.10
PyZMQ: 14.0.1
Python: 2.7.6 (default, Mar 22 2014, 22:59:56)
RAET: Not Installed
Tornado: 4.2.1
ZMQ: 4.0.4
cffi: Not Installed
cherrypy: Not Installed
dateutil: 1.5
gitdb: 0.5.4
gitpython: 0.3.2 RC1
ioflo: Not Installed
libgit2: Not Installed
libnacl: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.3.0
mysql-python: 1.2.3
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
python-gnupg: Not Installed
smmap: 0.8.2
timelib: Not Installed

System Versions:
dist: Ubuntu 14.04 trusty
machine: x86_64
release: 3.16.0-30-generic
system: Ubuntu 14.04 trusty

Minion:
distinction:
smmap: Not Installed

Setup

Later I read that in Grains only have access to salt.cmd.run, but may have some way?

[Ch3LL]

@DimentR I've been trying to write up a possible scenario for this and this is what i have so far:

#!/usr/bin/env python
import logging
import salt.config
log = logging.getLogger(__name__)

__opts__ = salt.config.minion_config('/etc/salt/minion')
__salt__ = salt.loader.minion_mods(__opts__)

def yourfunction():
    grains = {}
    test = __salt__['pillar.get']('testpillar', 'test')
    log.debug('GRAIN: {0}'.format(test))
    grains['testgrain'] = test
    return grains

Unfortunately it seems this is not working and when debuging it shows pillar as empty. I will take a look at this more closely next week to see if it is possible to get pilalr data into grains.

[sebw]

Facing the same issue. Code above doesn't work for me either.
I will hard-code credentials in the custom grain for now, as the account is only read-only and the pillar is actually presented to all minions.

[jcmcken]

Just my 2 cents: There should be a dead simple API for accessing either pillars or grains from other grains. In about 2 minutes I can Google and find how to reference one fact from another in Puppet, and it involves no custom setup code. Facts are lazily loaded as needed and can be specified with dependencies.

[florianb]

I have the same problem, i need to generate states based upon the return of an api-call which makes use of encrypted credentials.

@Ch3LL's hack doesn't work for me, too.

[JeanChristopheMorinRodeoFX]

Hi, I am currently facing that situation where I would need to access the pillar data from my custom grains, but I can't find a solution. Is this simply not supported or maybe it's a bug?

Thanks!

[Ch3LL]

Can i get anyone from @saltstack/team-core to speak here. I'm not sure if this is possible. My workaround as stated above does not work either. I'm not sure if this is meant to work, but @saltstack/team-core might have some better ideas here.

[gtmanfred]

As far as I am aware this is not possible, because grains data is computed before the pillar data is received.

Which os one of the reasons that the metadata grain doesn't check pillar data for the option to be enabled to run it.

https://github.com/saltstack/salt/blob/2017.7/salt/grains/metadata.py#L34

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.