Salt: netconfig.managed commits config on Cisco IOS in the presence of test=True

[dractyl]

Description of Issue

I have two netconfig.managed stanzas for a particular switch. With test=True on the command line, the first instance respects it and does not commit, while the second instance ignores test=True and commits the config.

Setup

I need to delete the switch config specific stuff, but I don't see that as relevant to this issue anyway.

SLS file:
cisco_basic_switch_config:
  netconfig.managed:
    - template_name: salt://{{ slspath }}/files/switch.jinja

cisco_individual_switch_config:
  netconfig.managed:
    - template_name: salt://{{ slspath }}/files/individual_config.jinja

When executed, cisco_basic_switch_config works properly and does not commit but then cisco_individual_switch_config runs and unexpectedly does.

This happened during a production maintenance window, so the flurry of unexpected log messages was something of a surprise.

We're moving to collapse these two configs into one, but neither the contents of the configs nor the fact that there are two should result in test=True being ignored.

FWIW, the configs are non-overlapping (one handles interfaces and the other, everything else)

Steps to Reproduce Issue

I have not attempted to reproduce as my maintenance window has passed, but here is the command and its output (with switch config specifics removed; I'm offering what I can even if it's an imperfect bug report):

salt 'switch-a' state.apply test=True

switch-a:
----------
          ID: cisco_basic_switch_config
    Function: netconfig.managed
      Result: None
     Comment: Configuration discarded.

              Configuration diff:

< A diff of what would have been done >

     Started: 05:41:18.578389
    Duration: 18581.07 ms
     Changes:   
              ----------
              diff:
 < The same diff of what would have been done, a bit verbosely perhaps >
---------
          ID: cisco_individual_switch_config
    Function: netconfig.managed
      Result: True
     Comment: Configuration changed!
     Started: 05:41:37.159805
    Duration: 14933.766 ms
     Changes:   
              ----------
              diff:

< A diff of actual changes that it actually did >

Summary for switch-a
------------
Succeeded: 2 (unchanged=1, changed=2)
Failed:    0
------------
Total states run:     2
Total run time:  33.515 s

I can give this another go in our maintenance window, two weeks from now.

Versions Report

On the master:

Salt Version:
           Salt: 2019.2.0

Dependency Versions:
           cffi: 1.12.3
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: 2.0.3
      gitpython: 2.1.8
          ioflo: Not Installed
         Jinja2: 2.10.1
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.7 (default, Oct 22 2018, 11:32:17)
   python-gnupg: 0.4.1
         PyYAML: 5.1.1
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: 2.0.3
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-36-generic
         system: Linux
        version: Ubuntu 18.04 bionic

On the Proxy:

Salt Version:
           Salt: 2019.2.0

Dependency Versions:
           cffi: 1.12.3
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10.1
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.7 (default, Oct 22 2018, 11:32:17)
   python-gnupg: 0.4.1
         PyYAML: 5.1.1
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-52-generic
         system: Linux
        version: Ubuntu 18.04 bionic

[raddessi]

I'm having this same issue right now in EOS, not even passing test: True to the state itself is being respected.

[sagetherage]

@dmurphy18

[raddessi]

https://github.com/saltstack/salt/pull/56299 may have resolved this, can someone check if behaviour has changed?

[sagetherage]

@raddessi I am seeing if there is someone who can test it this week, holiday this past Monday in US is making it a bit limited, but letting you know I am working on it!

56299 may have resolved this, can someone check if behaviour has changed?