Pyinfra: Add `success_exit_codes` global argument

[mkinney]

It looks like puppet.agent() reports an error when it should not.

Exit code 2 is a "success" (in puppet's view, see below)

Note: I used https://github.com/Fizzadar/pyinfra/pull/237/commits/18c8d431a45bb11628e6f6b9de91cb7e67c0b083

This is the output:

$ pyinfra @vagrant step3.py
--> Loading config...
--> Loading inventory...
    Getting Vagrant config...

--> Connecting to hosts...
    [@vagrant/master] Connected
    [@vagrant/agent] Connected

--> Preparing operations...
    Loading: step3.py
    [@vagrant/master]  Ready: step3.py
    [@vagrant/agent]  Ready: step3.py

--> Proposed changes:
    Groups: @vagrant
    [@vagrant/agent]    Operations: 1   Commands: 1
    [@vagrant/master]   Operations: 1   Commands: 1

--> Beginning operation run...
--> Starting operation: Create a puppet manifest
    [@vagrant/agent] Skipped
    [@vagrant/master] Success

--> Starting operation: Puppet/Agent
    [@vagrant/master] Skipped
    [@vagrant/agent] Info: Using configured environment 'production'
    [@vagrant/agent] Info: Retrieving pluginfacts
    [@vagrant/agent] Info: Retrieving plugin
    [@vagrant/agent] Info: Retrieving locales
    [@vagrant/agent] Info: Caching catalog for agent.example.com
    [@vagrant/agent] Info: Applying configuration version '1579617865'
    [@vagrant/agent] Notice: /Stage[main]/Main/Node[agent.example.com]/Package[httpd]/ensure: created
    [@vagrant/agent] Notice: /Stage[main]/Main/Node[agent.example.com]/Service[httpd]/ensure: ensure changed 'stopped' to 'running'
    [@vagrant/agent] Info: /Stage[main]/Main/Node[agent.example.com]/Service[httpd]: Unscheduling refresh on Service[httpd]
    [@vagrant/agent] Notice: Applied catalog in 9.94 seconds
    [@vagrant/agent] Error

--> Results:
    Groups: @vagrant
    [@vagrant/agent]    Successful: 0   Errors: 1   Commands: 0/1
    [@vagrant/master]   Successful: 1   Errors: 0   Commands: 1/1

From https://puppet.com/docs/puppet/latest/man/agent.html

Provide extra information about the run via exit codes; only works if '--test' or '--onetime' is also specified. If enabled, 'puppet agent' will use the following exit codes:

0: The run succeeded with no changes or failures; the system was already in the desired state.

1: The run failed, or wasn't attempted due to another run already in progress.

2: The run succeeded, and some resources were changed.

4: The run succeeded, and some resources failed.

6: The run succeeded, and included both changes and failures.

I stopped the service manually:
pyinfra @vagrant/agent init.systemd httpd sudo=True running=False

Then re-ran to show exit code:

(venv) mikekinneymbp:puppet mikekinney$ pyinfra -vv --debug @vagrant step2.py
    [pyinfra_cli.main] Checking potential directory:
    [pyinfra_cli.main] Checking potential directory:
--> Loading config...
--> Loading inventory...
    [pyinfra_cli.inventory] Creating fake inventory...
    [pyinfra.api.inventory] Expanding inventory connector: vagrant
    Getting Vagrant config...
localhost: >>> vagrant status --machine-readable
localhost: 1579618616,master,metadata,provider,vmware_desktop
localhost: 1579618616,agent,metadata,provider,vmware_desktop
localhost: 1579618617,master,provider-name,vmware_desktop
localhost: 1579618617,master,state,running
localhost: 1579618617,master,state-human-short,running
localhost: 1579618617,master,state-human-long,The VM is running. To stop this VM%!(VAGRANT_COMMA) you can run `vagrant halt` to\nshut it down%!(VAGRANT_COMMA) or you can run `vagrant suspend` to simply suspend\nthe virtual machine. In either case%!(VAGRANT_COMMA) to restart it again%!(VAGRANT_COMMA) run\n`vagrant up`.
localhost: 1579618617,agent,provider-name,vmware_desktop
localhost: 1579618617,agent,state,running
localhost: 1579618617,agent,state-human-short,running
localhost: 1579618617,agent,state-human-long,The VM is running. To stop this VM%!(VAGRANT_COMMA) you can run `vagrant halt` to\nshut it down%!(VAGRANT_COMMA) or you can run `vagrant suspend` to simply suspend\nthe virtual machine. In either case%!(VAGRANT_COMMA) to restart it again%!(VAGRANT_COMMA) run\n`vagrant up`.
localhost: 1579618618,,ui,info,Current machine states:\n\nmaster                    running (vmware_desktop)\nagent                     running (vmware_desktop)\n\nThis environment represents multiple VMs. The VMs are all listed\nabove with their current state. For more information about a specific\nVM%!(VAGRANT_COMMA) run `vagrant status NAME`.
    [pyinfra.api.connectors.vagrant] Loading SSH config for master
localhost: >>> vagrant ssh-config master
    [pyinfra.api.connectors.vagrant] Loading SSH config for agent
localhost: >>> vagrant ssh-config agent
localhost: Host master
localhost: HostName 127.0.0.1
localhost: User vagrant
localhost: Port 2200
localhost: UserKnownHostsFile /dev/null
localhost: StrictHostKeyChecking no
localhost: PasswordAuthentication no
localhost: IdentityFile /Users/mikekinney/Documents/python/mkinney_pyinfra/examples/puppet/.vagrant/machines/master/vmware_desktop/private_key
localhost: IdentitiesOnly yes
localhost: LogLevel FATAL
localhost:
localhost: Host agent
localhost: HostName 127.0.0.1
localhost: User vagrant
localhost: Port 2201
localhost: UserKnownHostsFile /dev/null
localhost: StrictHostKeyChecking no
localhost: PasswordAuthentication no
localhost: IdentityFile /Users/mikekinney/Documents/python/mkinney_pyinfra/examples/puppet/.vagrant/machines/agent/vmware_desktop/private_key
localhost: IdentitiesOnly yes
localhost: LogLevel FATAL
localhost:
    [pyinfra.api.connectors.vagrant] Got Vagrant SSH info:
['Host master', 'HostName 127.0.0.1', 'User vagrant', 'Port 2200', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /Users/mikekinney/Documents/python/mkinney_pyinfra/examples/puppet/.vagrant/machines/master/vmware_desktop/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '', 'Host agent', 'HostName 127.0.0.1', 'User vagrant', 'Port 2201', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /Users/mikekinney/Documents/python/mkinney_pyinfra/examples/puppet/.vagrant/machines/agent/vmware_desktop/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']
    [pyinfra.api.inventory] Expanding inventory connector: vagrant
    [pyinfra.api.connectors.vagrant] Got Vagrant SSH info:
['Host master', 'HostName 127.0.0.1', 'User vagrant', 'Port 2200', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /Users/mikekinney/Documents/python/mkinney_pyinfra/examples/puppet/.vagrant/machines/master/vmware_desktop/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '', 'Host agent', 'HostName 127.0.0.1', 'User vagrant', 'Port 2201', 'UserKnownHostsFile /dev/null', 'StrictHostKeyChecking no', 'PasswordAuthentication no', 'IdentityFile /Users/mikekinney/Documents/python/mkinney_pyinfra/examples/puppet/.vagrant/machines/agent/vmware_desktop/private_key', 'IdentitiesOnly yes', 'LogLevel FATAL', '']

--> Connecting to hosts...
    [pyinfra.api.connectors.ssh] Connecting to: @vagrant/master ({'allow_agent': False, 'look_for_keys': False, 'username': 'vagrant', 'port': 2200, 'timeout': 10, 'pkey': <paramiko.rsakey.RSAKey object at 0x1118ed950>})
    [pyinfra.api.connectors.ssh] Connecting to: @vagrant/agent ({'allow_agent': False, 'look_for_keys': False, 'username': 'vagrant', 'port': 2201, 'timeout': 10, 'pkey': <paramiko.rsakey.RSAKey object at 0x1119148d0>})
    [@vagrant/agent] Connected
    [@vagrant/master] Connected
    [pyinfra.api.state] Activating host: @vagrant/master
    [pyinfra.api.state] Activating host: @vagrant/agent

--> Preparing operations...
    Loading: step2.py
    [pyinfra.api.operation] Adding operation, {'Sign the agent, if needed'}, called @ step2.py:10, opLines=(0, 10), opHash=bca15b80109a88ea7a8e86f82272ecf855ed9396
    [pyinfra.api.facts] Getting fact: file (ensure_hosts: (@vagrant/master,))
    [pyinfra.api.util] Building command (shell_executable: sh sudo: True): ls -ld --time-style=long-iso /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || ls -ldT /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3
    [pyinfra.api.connectors.ssh] Running command on @vagrant/master: (pty=False) sudo -S -H -n sh -c 'ls -ld --time-style=long-iso /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || ls -ldT /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3'
[@vagrant/master] >>> sudo -S -H -n sh -c 'ls -ld --time-style=long-iso /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || ls -ldT /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3'
    [pyinfra.api.util] Building command (shell_executable: sh sudo: True): ls -ld --time-style=long-iso /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || ls -ldT /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3
    [pyinfra.api.connectors.ssh] Running command on @vagrant/agent: (pty=False) sudo -S -H -n sh -c 'ls -ld --time-style=long-iso /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || ls -ldT /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3'
[@vagrant/agent] >>> sudo -S -H -n sh -c 'ls -ld --time-style=long-iso /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || ls -ldT /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3'
[@vagrant/master] -rwxrwxr-x. 1 vagrant vagrant 220 2020-01-21 14:42 /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
[@vagrant/agent] ls: invalid tab size: ‘/tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3’
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 2
    Loaded fact file: ('/tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3',)
    [pyinfra.api.facts] Getting fact: sha1_file (ensure_hosts: (@vagrant/master,))
    [pyinfra.api.util] Building command (shell_executable: sh sudo: True): sha1sum /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || sha1 /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3
    [pyinfra.api.connectors.ssh] Running command on @vagrant/master: (pty=False) sudo -S -H -n sh -c 'sha1sum /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || sha1 /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3'
[@vagrant/master] >>> sudo -S -H -n sh -c 'sha1sum /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || sha1 /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3'
    [pyinfra.api.util] Building command (shell_executable: sh sudo: True): sha1sum /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || sha1 /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3
    [pyinfra.api.connectors.ssh] Running command on @vagrant/agent: (pty=False) sudo -S -H -n sh -c 'sha1sum /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || sha1 /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3'
[@vagrant/agent] >>> sudo -S -H -n sh -c 'sha1sum /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3 2> /dev/null || sha1 /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3'
[@vagrant/master] a5635aa4090c0abff3c1eca7fe18d6399403ad23  /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
[@vagrant/agent] sh: sha1: command not found
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 127
    Loaded fact sha1_file: ('/tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3',)
    [@vagrant/master]  Ready: step2.py
    [pyinfra.api.operation] Adding operation, {'Puppet/Agent'}, called @ step2.py:17, opLines=(0, 17), opHash=390f9f0b5aaba93454af019569e2e00e2bf88bf9
    [@vagrant/agent]  Ready: step2.py

--> Proposed changes:
    Groups: @vagrant
    [@vagrant/master]   Operations: 1   Commands: 2
    [@vagrant/agent]    Operations: 1   Commands: 1

--> Beginning operation run...
--> Starting operation: Sign the agent, if needed
    [pyinfra.api.operations] Starting operation Sign the agent, if needed on @vagrant/master
    [pyinfra.api.util] Building command (shell_executable: sh sudo: True): chmod +x /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3
    [pyinfra.api.connectors.ssh] Running command on @vagrant/master: (pty=None) sudo -S -H -n -i sh -c 'chmod +x /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3'
[@vagrant/master] >>> sudo -S -H -n -i sh -c 'chmod +x /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3'
    [@vagrant/agent] Skipped
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
    [pyinfra.api.util] Building command (shell_executable: sh sudo: True): /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3
    [pyinfra.api.connectors.ssh] Running command on @vagrant/master: (pty=None) sudo -S -H -n -i sh -c /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3
[@vagrant/master] >>> sudo -S -H -n -i sh -c /tmp/efa0845582d530a6acdf6ed1eea2420acb66e7b3
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 0
    [@vagrant/master] Success

--> Starting operation: Puppet/Agent
    [@vagrant/master] Skipped
    [pyinfra.api.operations] Starting operation Puppet/Agent on @vagrant/agent
    [pyinfra.api.util] Building command (shell_executable: sh sudo: True): puppet agent -t
    [pyinfra.api.connectors.ssh] Running command on @vagrant/agent: (pty=None) sudo -S -H -n -i sh -c 'puppet agent -t'
[@vagrant/agent] >>> sudo -S -H -n -i sh -c 'puppet agent -t'
[@vagrant/agent] Info: Using configured environment 'production'
[@vagrant/agent] Info: Retrieving pluginfacts
[@vagrant/agent] Info: Retrieving plugin
[@vagrant/agent] Info: Retrieving locales
[@vagrant/agent] Info: Caching catalog for agent.example.com
[@vagrant/agent] Info: Applying configuration version '1579618622'
[@vagrant/agent] Notice: /Stage[main]/Main/Node[agent.example.com]/Service[httpd]/ensure: ensure changed 'stopped' to 'running' (corrective)
[@vagrant/agent] Info: /Stage[main]/Main/Node[agent.example.com]/Service[httpd]: Unscheduling refresh on Service[httpd]
[@vagrant/agent] Notice: Applied catalog in 0.30 seconds
    [pyinfra.api.connectors.ssh] Waiting for exit status...
    [pyinfra.api.connectors.ssh] Command exit status: 2
    [@vagrant/agent] Error
    [pyinfra.api.state] Failing hosts: @vagrant/agent

--> Results:
    Groups: @vagrant
    [@vagrant/master]   Successful: 1   Errors: 0   Commands: 2/2
    [@vagrant/agent]    Successful: 0   Errors: 1   Commands: 0/1
(venv) mikekinneymbp:puppet mikekinney$

[Fizzadar]

Hmmm puppet should only exit with those codes if the --detailed-exit-code flag is applied? The original issue describes exit 0 being normal behaviour, and non-0 being error, unless this flag is provided.

[mkinney]

lol. You're probably correct, but that's what puppet does. It returns different return values.

There might be a setting somewhere (or it is default enabled now).

[Fizzadar]

V annoying! This is really a bug in puppet - but it brings up an interesting case where we want to ignore some exit codes (treat them as success). So I think another global argument success_exit_codes with [0] as the default would do nicely here. Then it'd work like:

puppet.agent(
    {'Run the puppet agent'},
    success_exit_codes=[0, 1, 2],
)

[mkinney]

Wow. That'd be a great addition as there are some other commands that do not always return 0.

[mkinney]

Tested and it works as expected. Ok to close.

[Fizzadar]

Excellent! This is also a requirement for the server.reboot op!