Salt: Rendering SLS 'base:XXXXX' failed: found unexpected ':'

[steoli]

Description of Issue/Question

When running "salt-call state.apply" and having a grains file ":" sign within grains options it fails with this message

[CRITICAL] Rendering SLS 'base:atlassian/jira' failed: found unexpected ':'
local:
    Data failed to compile:
----------
    Rendering SLS 'base:atlassian/jira' failed: found unexpected ':'

The grains file looks like this:

jira_java_opts:
- Dcom.sun.management.jmxremote
- Dcom.sun.management.jmxremote.port=9010
- Dcom.sun.management.jmxremote.rmi.port=9010
- Dcom.sun.management.jmxremote.local.only=true
- Dcom.sun.management.jmxremote.ssl=false
- Dcom.sun.management.jmxremote.authenticate=false
- Duser.timezone=Europe/Stockholm
- XX:+PrintGCDetails
- XX:+PrintGCDateStamps
- XX:+PrintGCTimeStamps
- XX:+PrintGCCause
- Xloggc:/var/atlassian/application-data/jira/log/atlassian-jira-gc-%t.log
- XX:+UseGCLogFileRotation
- XX:NumberOfGCLogFiles=5
- XX:GCLogFileSize=20M
- XX:ReservedCodeCacheSize=384m
- XX:+UseG1GC
- Datlassian.mail.senddisabled=true
- Datlassian.mail.fetchdisabled=true
- Datlassian.mail.popdisabled=true
jira_version: 8.1.0
jira_xms: 3g
jira_xmx: 3g
roles: jira

If I remove the : sign from all the options above it works ok OR if I downgrade to version 2015.8.8+ds-1 it also works ok (im now running 2019.2.0+ds-1)

What todo ?

Setup

(Please provide relevant configs and/or SLS files (Be sure to remove sensitive info).)

Steps to Reproduce Issue

(Include debug logs if possible and relevant.)

Versions Report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

[DmitryKuzmenko]

Unfortunately I'm unable to reproduce this.
Could you please provide your salt --versions-report? Probably it would help.

[steoli]

salt --versions-report

Salt Version:
Salt: 2019.2.0

Dependency Versions:
cffi: 1.11.5
cherrypy: 3.5.0
dateutil: 2.4.2
docker-py: Not Installed
gitdb: 0.6.4
gitpython: 1.0.1
ioflo: Not Installed
Jinja2: 2.8
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: 1.0.3
msgpack-pure: Not Installed
msgpack-python: 0.4.6
mysql-python: Not Installed
pycparser: 2.19
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.12 (default, Nov 12 2018, 14:36:49)
python-gnupg: 0.3.8
PyYAML: 3.11
PyZMQ: 15.2.0
RAET: Not Installed
smmap: 0.9.0
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4

System Versions:
dist: Ubuntu 16.04 xenial
locale: UTF-8
machine: x86_64
release: 4.4.0-1079-aws
system: Linux
version: Ubuntu 16.04 xenial

On the minions where I have problem im running (also Ubuntu 16.04)
salt-common/unknown,now 2019.2.0+ds-1 all [installed]
salt-minion/unknown,now 2019.2.0+ds-1 all [installed]

salt-call state.apply output fails with this:
...
[INFO ] Fetching file from saltenv 'base', * done * 'atlassian/jira/init.sls'
[CRITICAL] Rendering SLS 'base:atlassian/jira' failed: found unexpected ':'
[INFO ] Fetching file from saltenv 'base', * done * 'atlassian/common/init.sls'
local:

Data failed to compile:

Rendering SLS 'base:atlassian/jira' failed: found unexpected ':'

So from here I can solve it by either downgrade minion to version 2015.8.8+ds-1 or remove the : sign from the grains file (the atlassian/jira state is reading a bunch of options from grains file)

[steoli]

The salt state using the grains options looks like this

/opt/atlassian/jira/bin/setenv.sh:
    file.managed:
        - source: salt://atlassian/jira/files/bin/setenv.sh
        - template: jinja
        - context:
           jira_xms: {{ salt['grains.get']('jira_xms') }}
           jira_xmx: {{ salt['grains.get']('jira_xmx') }}
           jira_java_opts: {{ salt['grains.get']('jira_java_opts') }}

[mrichar1]

I'm hitting the same problem trying to use the saltstack ntp-formula after an upgrade to 2019.2.0 (was working fine before that).

In my case the issue occurs when using a pillar who's value is a list, where the list item contains a ::

ntp_conf:
  restrict: ['127.0.0.1', '::1']

I've mansed to reduce the problem down to this example config which seems to trigger it:

state:

test:
  file.managed:
    - name: /tmp/test
    - context:
      config: {{ pillar.get('test') }}

pillar:

test:
  - "abc:def"

Data failed to compile:

Rendering SLS 'base:profile' failed: found unexpected ':'

If the pillar data is changed to a simple string (test: "abc:def") the error doesn't occur.

(I'm running masterless if that is at all relevant).

[mrichar1]

@DmitryKuzmenko is this enough new info to get the labels/milestones removed?

[DmitryKuzmenko]

@steoli thank you for the information.
@mrichar1 thank you for the mention and the additional info.
Sorry for late response. By a quick test on my laptop I still can't reproduce it. I'm looking up the source code to understand the possible reason.

[DmitryKuzmenko]

@saltstack/team-core does anybody have any thoughts on this?

[waynew]

Does

/opt/atlassian/jira/bin/setenv.sh:
    file.managed:
        - source: salt://atlassian/jira/files/bin/setenv.sh
        - template: jinja
        - context:
           jira_xms: {{ salt['grains.get']('jira_xms')|json }}
           jira_xmx: {{ salt['grains.get']('jira_xmx')|json }}
           jira_java_opts: {{ salt['grains.get']('jira_java_opts')|json }}

Work for you? i.e. adding the |json filter to your grains.get?

[steoli]

Wow, easy fixed as it turned out .. It worked like a charm running it trough json as suggested :-)

Many thanks

[mrichar1]

In my case this worked fine before the upgrade, and now it is no longer working. Additionally, I'm using a saltstack provided formula, only providing the pillar data, so I don't have control over the state to use the json filter.

Is it really the case that this type of list usage is no longer allowed in context?

[N-Mi]

Just hit the same issue with salt-ssh 2019.2.0 when using salt-formula-telegraf

Here is the pillar content (the problem is the URL of the influx_db plugin)

telegraf:
  agent:
    enabled: true 
    interval: 10
    round_interval: true
    metric_batch_size: 1000
    metric_buffer_limit: 10000
    collection_jitter: 0
#    precision: None
    debug: false
    quiet: false
#    logfile: ""
#    hostname: ""
    omit_hostname: false

    output:
      influxdb:
        urls: "http://1.2.3.4:8086"
        database: mydatabase
        write_consistency: any
        timeout: 5

    input:
      cpu:
        totalcpu: totalcpu_value
  remote_agent:
    enabled: false

Here is a snippet of produced yaml :

output_influxdb_agent:
  file.managed:
    - name: /etc/telegraf/telegraf.d/output-influxdb.conf
    - source: salt://telegraf/files/output/influxdb.conf
    - user: root
    - group: root
    - mode: 644
    - template: jinja
    - require:
      - pkg: telegraf_packages_agent
      - file: config_d_dir_agent
    - require_in:
      - file: config_d_dir_agent_clean
    - watch_in:
      - service: telegraf_service_agent
    - defaults:
        name: influxdb
        values: {u'urls': u'http://1.2.3.4:8086', u'database': u'mydatabase', u'write_consistency': u'any', u'timeout': 5}

I can resolve the issue by modifying the formula and add |json everytime pillar data is called.

[N-Mi]

Reading @mrichar1 comment, I see we have in common to use Salt without using salt-master (I'm using salt-ssh). This may be the key to reproduce this bug.

[DmitryKuzmenko]

@mrichar1 SaltStack development team doesn't maintain salt formulas. If you'd like you can create a new issue in the specific formula repository.
Also we will be allowing a user to revert to the old behavior in 2019.2.1 release as documented here: https://docs.saltstack.com/en/latest/topics/releases/2019.2.1.html#change-to-yaml-renderer

[einsiedlerkrebs]

I can confirm the issue running salt 2019.2.0 ( 2nd archinux release) and in combination with salt-ssh.
```
Salt Version:
Salt: 2019.2.0

Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.10.1
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: 0.30.1
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.6.1
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 3.8.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.16 (default, Mar 11 2019, 18:59:25)
python-gnupg: Not Installed
PyYAML: 3.13
PyZMQ: 18.0.1
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 5.1.1
ZMQ: 4.3.1

System Versions:
dist:
locale: UTF-8
machine: x86_64
release: 5.0.13-arch1-1-ARCH
system: Linux
version: Not Installed

```
I think the issue should be re-opened.

[gwaters]

I have this issue with the tomcat formula as well. Shouldnt the solution be, fix the yaml renderer so it functions like before ?

[waynew]

I don't know the specifics, but if I understand correctly there were some specific reasons behind changing it - either underlying libraries changed, or there were buggy behaviors to be fixed. From what I understand, there's no going back, so an issue should be raised in the formula repo.