Salt: can not call custom _modules with jinja from pillar

[wuxxin]

• working: salt-call --local
  
  
  
  • salt-call 2014.1.13-401-gaeda351
  
  
  • salt-call 2014.7.0 (Helium)
  
  
  • salt-call 2014.7.1-38-g4158b17 (Helium)
  
  
• not working: salt-call (with master)
  
  
  
  • same versions as tested with --local
  
  
• use case:
  
  
  
  • as a devop i want to make some ip address related calculations in pillar, to do so i include a custom python module for this calculations and want to use it from pillar
  
  
• reproduction:
  
  
  
  • paste setup_issue.sh into empty directory
  
  
  • look at setup_issue.sh and if ok, chmod +x setup_issue.sh and run it (./setup_issue.sh)
  
  
  • masterless check (should work)
  
  
  • ./sc-local.sh saltutil.sync_all
  
  
  • ./sc-local.sh pillar.item test
  
  
  • check with master:
  
  
  • ./sc-master.sh saltutil.sync_all
  
  
  • # do this once again, because i didnt came up with a working solution for accepting the minion key on the first run
  
  
  • ./sc-master.sh pillar.item test
  
  
  • check output in master/
    
    
    
    
    
    • function is not found
    
    
    
  
  

---

#!/bin/sh
prog=`readlink -f $0`
dir=`dirname $prog`
for f in local local_root master master_root minion minion_root; do mkdir -p $dir/$f; done
mkdir -p $dir/./salt
mkdir -p $dir/./salt/_modules
mkdir -p $dir/./pillar
for a in local/minion master/master minion/minion; do 
  stype=${a%%/*}
  cat > $dir/$a <<EOF
root_dir: $dir/${stype}_root
pidfile: salt-${stype}.pid
pki_dir: pki
cachedir: cache
sock_dir: run
log_file: ${stype}-salt.log
key_logfile: ${stype}-key.log
master_port: 45060
publish_port: 45050
ret_port: 45060
fileserver_backend:
  - roots
pillar_roots:
  base:
    - $dir/pillar
file_roots:
  base:
    - $dir/salt
EOF
done
echo "id: pillar_test" >> $dir/local/minion
echo "file_client: local" >> $dir/local/minion
echo "id: pillar_test" >> $dir/minion/minion
echo "master: localhost" >> $dir/minion/minion
cat > $dir/sc-local.sh <<"EOF"
#!/bin/sh
prog=`readlink -f $0`
dir=`dirname $prog`
sudo salt-call --local --config-dir=$dir/local $@
EOF
chmod +x $dir/sc-local.sh
cat > $dir/sc-master.sh <<"EOF"
#!/bin/sh
prog=`readlink -f $0`
dir=`dirname $prog`
sudo salt-master --config-dir=$dir/master -d
sudo salt-minion --config-dir=$dir/minion -d
sleep 3
sudo salt-key --config-dir=$dir/master -A -y
sudo salt-call --config-dir=$dir/minion $@
sleep 1
sudo kill `cat $dir/master_root/salt-master.pid`
sudo kill `cat $dir/minion_root/salt-minion.pid`
EOF
chmod +x $dir/sc-master.sh
cat > $dir/./salt/top.sls<<"EOF"
base:
  '*':
    - test_state
EOF
cat > $dir/./salt/_modules/extf.py<<"EOF"
# -*- coding: utf-8 -*-

def test_equal(first, second):

    if first == second:
        return True
    else:
        return False
EOF
cat > $dir/./salt/test_state.sls<<"EOF"
curl:
  pkg:
    - installed
EOF
cat > $dir/./pillar/top.sls<<"EOF"
base:
  '*':
    - included_pillar
EOF
cat > $dir/./pillar/included_pillar.sls<<"EOF"
test:
  data: {{ salt['extf.test_equal']('1','1') }}
EOF

[wuxxin]

it would be nice to have at least a workaround

[babilen5]

I seem to be running into a similar issue which I have detailed in #19899

[gladiatr72]

This isn't an actual bug. For master-targetted modules, use the extension_modules option in /etc/salt/master (I have it set to /srv/modules)

Within this directory you have your usual suspects, so:

/srv/modules/modules
/srv/modules/pillar
/srv/modules/renderers
etc

Any functional module in /srv/module/modules is then available for use when processing pillar.
External pillars go in /srv/modules/pillar, etc...

If you have a custom module that is also useful on the minion-side, put in in /srv/salt/_modules and symlink it into /srv/modules/modules.

[jfindlay]

Thank you all for your feedback.

[wuxxin]

@gladiatr72 thank you for the workaround, i was able to make it working using the extension_modules setting and some symlinks.

i think there should be a clarification in the docs to state that:
without setting extension_modules and some symlinks, jinja inside the pillar(using a saltmaster) will loose some functionality (eg.custom modules) in comparison with a masterless salt-call (which is unexpected in my opinion)

[gladiatr72]

Cool. Glad I could help! There does need to be some additional documentation written in regards to this issue. The thing is that the /srv/salt/_modules directory is specifically for distributing custom execution modules to the minion for use by the salt-minion. If you are running a salt-minion on the salt master, it would be able to access them.

The separation between master and minion (custom) modules needs to be maintained. The cool thing about the minion modules is that the salt-minion software doesn't have to be restarted to bring code changes online. This isn't the case with the master extension_modules.

-S

[DanyC97]

bump, hopefully it get's some attention from docs side..thanks !

[o-sleep]

bump as well, wouldn't it make sense for saltutil.sync_* to copy to the master as well? This way reactor states could use the same modules that regular states can use, as well as have the same non-reboot required behavior.

[jfindlay]

See also #30647.

[o-sleep]

Perfect, I'll try it out tomorrow. Thanks.

[campbellmc]

+1 for some docs on this. The topic "CALLING SALT MODULES FROM TEMPLATES" in the SaltStack documents lead me astray

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.