Salt: grains.filter_by grain value doesn't support list

Created on 28 Feb 2017  路  22Comments  路  Source: saltstack/salt

Description of Issue/Question

grains.filter_by grain value doesn't support a list
when use list ,return AttributeError: 'list' object has no attribute 'split'

Setup

map.jinja:

{% set fluentd = salt['grains.filter_by']({
    'Ubuntu-12.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'Ubuntu-14.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'Ubuntu-16.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'CentOS-6': {'pkg': 'td-agent_2.3.4-0.el6.amd64.deb'},
    'CentOS-7': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'Debian': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'RedHat': {'pkg': 'td-agent_2.3.4-0.elrh.amd64.deb'},
},grain=['osfinger','os_family'], default='RedHat') %}

init.sls:

{% from "install/fluentd/map.jinja" import fluentd with context %}
/tmp/salt.debug:
  file.managed:
    - contents: |
    {%- for k,v in fluentd.items() %}
        {{ k }} => {{ v }}
    {%- endfor %}

Steps to Reproduce Issue

salt -S 192.168.12.192 state.sls install.fluentd 

192.168.12.192:
    Data failed to compile:
----------
    Rendering SLS 'base:install.fluentd' failed: Jinja error: 'list' object has no attribute 'split'
/var/cache/salt/minion/files/base/install/fluentd/map.jinja(1):
---
{% set fluentd = salt['grains.filter_by']({    <======================
    'Ubuntu-12.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'Ubuntu-14.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'Ubuntu-16.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'CentOS-6': {'pkg': 'td-agent_2.3.4-0.el6.amd64.deb'},
    'CentOS-7': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
[...]
---
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/salt/utils/templates.py", line 368, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib64/python2.6/site-packages/jinja2/environment.py", line 669, in render
    return self.environment.handle_exception(exc_info, True)
  File "<template>", line 1, in top-level template code
  File "/usr/lib64/python2.6/site-packages/jinja2/environment.py", line 713, in make_module
    return TemplateModule(self, self.new_context(vars, shared, locals))
  File "/usr/lib64/python2.6/site-packages/jinja2/environment.py", line 769, in __init__
    self._body_stream = list(template.root_render_func(context))
  File "/var/cache/salt/minion/files/base/install/fluentd/map.jinja", line 1, in top-level template code
    {% set fluentd = salt['grains.filter_by']({
  File "/usr/lib/python2.6/site-packages/salt/modules/grains.py", line 555, in filter_by
    val = salt.utils.traverse_dict_and_list(__grains__, grain, [])
  File "/usr/lib/python2.6/site-packages/salt/utils/__init__.py", line 1547, in traverse_dict_and_list
    for each in key.split(delimiter):
AttributeError: 'list' object has no attribute 'split'

Versions Report

Salt Version:
           Salt: 2016.11.2

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
          gitdb: 0.5.4
      gitpython: 0.3.2 RC1
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 0.9.1
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: 1.2.3
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
   python-gnupg: Not Installed
         PyYAML: 3.10
          PyZMQ: 14.0.1
           RAET: Not Installed
          smmap: 0.8.2
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: Ubuntu 14.04 trusty
        machine: x86_64
        release: 3.19.0-25-generic
         system: Linux
        version: Ubuntu 14.04 trusty
Magnesium State Module fixed-pending-your-verification good first issue has-failing-test
Source
picture oudemen

Most helpful comment

Same problem at my side, doing grain='osfinger,os_family' does not work either, it will fall back to default then.

The grain value could be a list. The function will return the
lookup_dict value for a first found item in the list matching
one of the lookup_dict keys.

I cannot find evidence that this is possible because this value is undergone split if it is str or not. A traversal is applied to this string, but this will give you an empty list if you do 'osfinger:os_family' because there is no dict behind osfinger that holds the key os_family. And yes, the only delimiter you can apply here is ':'.

picture ThePsyjo on 12 May 2017
👍2

All 22 comments

What exactly are you trying to do here?

Have it check for osfinger, and if it is not there, then try os_family?

gtmanfred picture gtmanfred on 1 Mar 2017

Can you try making it a comma seperated list?

THis could just be a documentation issue

try grain='osfinger,os_family'

gtmanfred picture gtmanfred on 1 Mar 2017

Same problem at my side, doing grain='osfinger,os_family' does not work either, it will fall back to default then.

The grain value could be a list. The function will return the
lookup_dict value for a first found item in the list matching
one of the lookup_dict keys.

I cannot find evidence that this is possible because this value is undergone split if it is str or not. A traversal is applied to this string, but this will give you an empty list if you do 'osfinger:os_family' because there is no dict behind osfinger that holds the key os_family. And yes, the only delimiter you can apply here is ':'.

ThePsyjo picture ThePsyjo on 12 May 2017
👍2

I also have this issue, it doesn't seem to work according to the docs

timwsuqld picture timwsuqld on 25 Jul 2018

Here's a command line version that may be the same or related issue. My salt looks like:

Salt Version:
           Salt: 2018.3.3

Dependency Versions:
           cffi: 1.5.2
       cherrypy: Not Installed
       dateutil: 2.5.0
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.3
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Dec  4 2017, 14:50:18)
   python-gnupg: 0.3.8
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-138-generic
         system: Linux
        version: Ubuntu 16.04 xenial

I'm trying to filter the disks grains says I have:

$ sudo salt-call --local grains.get disks
local:
    - sdb
    - loop0
    - loop1
    - loop2
    - loop3
    - loop4
    - loop5
    - loop6
    - loop7

OK, lets filter on loop disks:

$ sudo salt-call --local grains.filter_by '{lo*: x}' disks
local:
    x

I'm expecting an "x" for each loop disk, I guess. Picking and choosing is no help either:

$ sudo salt-call --local grains.filter_by '{"loop1": y}' disks
local:
    y


$ sudo salt-call --local grains.filter_by '{"loop0": x, "loop1": y}' disks
local:
    x
ldav1s picture ldav1s on 6 Nov 2018

@saltstack/team-triage can yall take a look?

gtmanfred picture gtmanfred on 6 Nov 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

stale[bot] picture stale[bot] on 9 Jan 2020

Still an issue

timwsuqld picture timwsuqld on 9 Jan 2020

Thank you for updating this issue. It is no longer marked as stale.

stale[bot] picture stale[bot] on 9 Jan 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

stale[bot] picture stale[bot] on 8 Feb 2020

@SaltStackSupport Can we please have the stalebot not be so aggressive. 30 days is way too aggressive

timwsuqld picture timwsuqld on 10 Feb 2020
👍1

Thank you for updating this issue. It is no longer marked as stale.

stale[bot] picture stale[bot] on 10 Feb 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

stale[bot] picture stale[bot] on 11 Mar 2020

Not stale

timwsuqld picture timwsuqld on 11 Mar 2020

Thank you for updating this issue. It is no longer marked as stale.

stale[bot] picture stale[bot] on 11 Mar 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

stale[bot] picture stale[bot] on 10 Apr 2020

@timwsuqld is this an issue on newer versions besides 2018.3, as well?

sagetherage picture sagetherage on 10 Apr 2020

Still relevant. I can replicate it as follows:

test_grains.jinja

{% set test_grains = salt['grains.filter_by']({
    'Ubuntu-16.04': {
        'apacheuser': 'www-data',
    },
    'Ubuntu-18.04': {
        'apacheuser': 'www-data',
    },
    'RedHat': {
        'apacheuser': 'apache',
    },
}, grain=['os_finger', 'osfamily']) %}

test_grains.sls

{% from "test_grains.jinja" import test_grains with context %}
echo {{ test_grains.apacheuser }}:
  cmd.run

# salt-call --local --file-root=./states/ state.sls test_grains test=True

[ERROR   ] Rendering exception occurred
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 169, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 443, in render_jinja_tmpl
    trace=tracestr)
SaltRenderError: Jinja error: 'list' object has no attribute 'split'
/var/cache/salt/minion/files/base/test_grains.jinja(1):
---
{% set test_grains = salt['grains.filter_by']({    <======================
    'Ubuntu-16.04': {
        'apacheuser': 'www-data',
    },
    'Ubuntu-18.04': {
        'apacheuser': 'www-data',
[...]
---
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 394, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1073, in make_module
    return TemplateModule(self, self.new_context(vars, shared, locals))
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1152, in __init__
    body_stream = list(template.root_render_func(context))
  File "/var/cache/salt/minion/files/base/test_grains.jinja", line 1, in top-level template code
    {% set test_grains = salt['grains.filter_by']({
  File "/usr/lib/python2.7/dist-packages/salt/modules/grains.py", line 600, in filter_by
    base=base)
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 504, in filter_by
    val = traverse_dict_and_list(traverse, lookup, [])
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 578, in traverse_dict_and_list
    for each in key.split(delimiter):
AttributeError: 'list' object has no attribute 'split'


[CRITICAL] Rendering SLS 'base:test_grains' failed: Jinja error: 'list' object has no attribute 'split'
/var/cache/salt/minion/files/base/test_grains.jinja(1):
---
{% set test_grains = salt['grains.filter_by']({    <======================
    'Ubuntu-16.04': {
        'apacheuser': 'www-data',
    },
    'Ubuntu-18.04': {
        'apacheuser': 'www-data',
[...]
---
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 394, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1073, in make_module
    return TemplateModule(self, self.new_context(vars, shared, locals))
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1152, in __init__
    body_stream = list(template.root_render_func(context))
  File "/var/cache/salt/minion/files/base/test_grains.jinja", line 1, in top-level template code
    {% set test_grains = salt['grains.filter_by']({
  File "/usr/lib/python2.7/dist-packages/salt/modules/grains.py", line 600, in filter_by
    base=base)
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 504, in filter_by
    val = traverse_dict_and_list(traverse, lookup, [])
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 578, in traverse_dict_and_list
    for each in key.split(delimiter):
AttributeError: 'list' object has no attribute 'split'


local:
    Data failed to compile:
----------
    Rendering SLS 'base:test_grains' failed: Jinja error: 'list' object has no attribute 'split'
/var/cache/salt/minion/files/base/test_grains.jinja(1):
---
{% set test_grains = salt['grains.filter_by']({    <======================
    'Ubuntu-16.04': {
        'apacheuser': 'www-data',
    },
    'Ubuntu-18.04': {
        'apacheuser': 'www-data',
[...]
---
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 394, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1073, in make_module
    return TemplateModule(self, self.new_context(vars, shared, locals))
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1152, in __init__
    body_stream = list(template.root_render_func(context))
  File "/var/cache/salt/minion/files/base/test_grains.jinja", line 1, in top-level template code
    {% set test_grains = salt['grains.filter_by']({
  File "/usr/lib/python2.7/dist-packages/salt/modules/grains.py", line 600, in filter_by
    base=base)
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 504, in filter_by
    val = traverse_dict_and_list(traverse, lookup, [])
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 578, in traverse_dict_and_list
    for each in key.split(delimiter):
AttributeError: 'list' object has no attribute 'split'

The part in the docs that suggests this should be possible:
https://docs.saltstack.com/en/master/ref/modules/all/salt.modules.grains.html#salt.modules.grains.filter_by

grain --

The name of a grain to match with the current system's grains. For example, the value of the "os_family" grain for the current system could be used to pull values from the lookup_dict dictionary.

Changed in version 2016.11.0: The grain value could be a list. The function will return the lookup_dict value for a first found item in the list matching one of the lookup_dict keys.

Trying grain='os_family,osfinger' also does not work.

timwsuqld picture timwsuqld on 14 Apr 2020
👍1

For completeness, the current version I'm running all that on.

$ salt-call --versions
Salt Version:
           Salt: 3000.1

Dependency Versions:
           cffi: 1.12.3
       cherrypy: Not Installed
       dateutil: 2.4.0
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.17 (default, Nov  7 2019, 10:07:09)
   python-gnupg: 0.4.1
         PyYAML: 3.11
          PyZMQ: 16.0.2
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-96-generic
         system: Linux
        version: Ubuntu 18.04 bionic
timwsuqld picture timwsuqld on 14 Apr 2020
👍1

thank you @timwsuqld we have disabled stale bot so that won't be an issue here any longer - I am kicking this back to triage and then will follow up tomorrow.

sagetherage picture sagetherage on 14 Apr 2020

I guess I kicked back too many issues to "re-triage" since this and a handful still are not. I will follow up with the person on triage and review this again, tomorrow, Friday 2020-04-17

sagetherage picture sagetherage on 16 Apr 2020

thank you @Akm0d !!

sagetherage picture sagetherage on 18 Apr 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

sfozz picture sfozz  路  3Comments
qiushics picture qiushics  路  3Comments
zieba88 picture zieba88  路  3Comments
erwindon picture erwindon  路  3Comments
Arguros picture Arguros  路  3Comments