Salt: pillar data in orchestration sls files are not resolved

[flassman]

Description of Issue/Question

Pillar data are not resolved if used in orchestration sls files. I know from documentation that reactors do not consider pillar data, but salt-run state.orchestrate should?! Or maybe I overlooked some restriction? If that is the case, what is the best practice to include data in the orchestrate runner? (other than adding the pillar data via command line, which does not work for us since we want to keep all this info in version control / filesystem)

Setup

I'm using Salt 2018.3.2 with following pillar data in /srv/pillar/mypillar.sls:

#!yaml
mypillar:
  some_key: some_value

and try to use it in the following state file /srv/salt/orch/mypillar.sls

write-pillar-file:
  file.managed:
    - name: /tmp/mypillar.txt
    - contents_pillar: mypillar

It works fine if called as a state:

$ salt 'localhost' state.apply orch.mypillar

Steps to Reproduce Issue

Yet does not work if called as orchestrate runner:

$ salt-run state.orchestrate orch.mypillar
[INFO ] Loading fresh modules for state activity
[INFO ] Fetching file from saltenv 'base', * done * 'orch/mypillar.sls'
[INFO ] Running state [/tmp/mypillar.txt] at time 18:32:03.120348
[INFO ] Executing state file.managed for [/tmp/mypillar.txt]
[ERROR ] Pillar mypillar does not exist
[INFO ] Completed state [/tmp/mypillar.txt] at time 18:32:03.122809 (duration_in_ms=2.461)

Versions Report

Salt Version:
Salt: 2018.3.2

Dependency Versions:
cffi: 1.11.5
cherrypy: Not Installed
dateutil: 2.7.2
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.10
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: 0.28.2
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: 2.14
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.15 (default, May 16 2018, 17:50:09)
python-gnupg: 0.3.8
PyYAML: 3.12
PyZMQ: 17.0.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 5.0.2
ZMQ: 4.1.6

System Versions:
dist: fedora 28 Twenty Eight
locale: UTF-8
machine: x86_64
release: 4.17.18-200.fc28.x86_64
system: Linux
version: Fedora 28 Twenty Eight

[Poil]

Hi,

The only way I've find is to use jinja at the beginning of my orchestrate.sls, same for SDB.
If someone have a better solution I take it :)

{% set vcenter = salt['pillar.get']('vcenter') %}
{% set cur_ip_backend = salt['sdb.get']('sdb://phpipam-api/get-by-hostname?hostname=flxbkd-{}{}-{}'.format(env, num, 

And if needed, to debug

{# do salt.log.error("debug stuff") #}

[flassman]

Unfortunately querying the pillar with Jinja does not make a difference: it works if I run the state with _state.apply_ but returns "None" if called via _salt-run state.orchestrate orch.router-cert_
I tried withe the pillar mentioned in my original post plus this /srv/salt/orch/mypillar.sls but the "pillardata" renders to "None" instead of the expected "some_value":

{% set pillardata = salt['pillar.get']('mypillar:some_key') %}
write-pillar-file:
  file.managed:
    - name: /tmp/mypillar.txt
    - contents:
      - {{ pillardata }}

@Poil would you mind trying this? I'm really curious to see whether I'm just too dumb or something is wrong with my environment - or whether accessing pillar data from salt-run simply does not work.

[garethgreenaway]

@flassman The issue is that pillar data is targeted at specific minions, which isn't happening at the stage of orchestration where you're looking to use pillar data above. Here is a similar issue that might offer some work arounds: https://github.com/saltstack/salt/issues/46387

[flassman]

Thanks a lot for the hint. With this information, I was able to access my pillar data like this (with 'localhost' being the minion ID of my salt master):

{% set pillardata = salt.saltutil.runner('pillar.show_pillar', kwarg={'minion': 'localhost'}) %}
write-pillar-file:
  file.managed:
    - name: /tmp/mypillar.txt
    - contents:
      - {{ pillardata['mypillar']['some_key'] }}