Packer: Allow non-zero exit codes for inspec provisioner

[nndegz]

Inspec has a non-zero code for tests with skipped controls that do not match or apply to the current system. Here https://github.com/inspec/inspec/issues/1825#issuecomment-382650911

0 run ok + all passed:
100 run okay + failures`
101 run okay + skipped only`

7385 adds exit codes for shell and windows provisioners and I was hoping the same could be done for inspec.

For example running the linux patch baseline profile on a fully patched machine will return a 101 exit code because no patches need to be applied
``` amazon-ebs: [2019-03-15T13:10:36-04:00] DEBUG: Registering rule verify-patches
amazon-ebs: [2019-03-15T13:10:36-04:00] DEBUG: Registering rule patches
amazon-ebs: [2019-03-15T13:10:36-04:00] DEBUG: Registering rule os-patches
amazon-ebs:
amazon-ebs: Profile: DevSec Linux Patch Benchmark (linux-patch-baseline)
amazon-ebs: Version: 0.4.0
amazon-ebs: Target: ssh://[email protected]:33148
amazon-ebs:
amazon-ebs: ✔ verify-patches: Operating system is up-to-date
amazon-ebs: ✔ Linux Update should be uptodate
amazon-ebs: ↺ patches: All operating system package updates are installed
amazon-ebs: ↺ Skipped control due to only_if condition.
amazon-ebs:
amazon-ebs:
amazon-ebs: Profile Summary: 1 successful control, 0 control failures, 1 control skipped
amazon-ebs: Test Summary: 1 successful, 0 failures, 1 skipped

and the resulting error

2019/03/15 13:11:07 [INFO] (telemetry) ending amazon-ebs
2019/03/15 13:11:07 ui error: Build 'amazon-ebs' errored: Error executing Inspec: Non-zero exit status: exit status 101
2019/03/15 13:11:07 Builds completed. Waiting on interrupt barrier...
2019/03/15 13:11:07 machine readable: error-count []string{"1"}
2019/03/15 13:11:07 ui error:
==> Some builds didn't complete successfully and had errors:
2019/03/15 13:11:07 machine readable: amazon-ebs,error []string{"Error executing Inspec: Non-zero exit status: exit status 101"}
2019/03/15 13:11:07 ui error: --> amazon-ebs: Error executing Inspec: Non-zero exit status: exit status 101
==> Builds finished but no artifacts were created.
2019/03/15 13:11:07 [INFO] (telemetry) Finalizing.
Build 'amazon-ebs' errored: Error executing Inspec: Non-zero exit status: exit status 101

==> Some builds didn't complete successfully and had errors:
--> amazon-ebs: Error executing Inspec: Non-zero exit status: exit status 101

==> Builds finished but no artifacts were created.
```

[nndegz]

I found a workaround, inspec has a --no-distinct-exit flag . It changes the exit code behaviour to use 0 and 1

Adds a --distinct-exit CLI option to exec, enabled by default. If disabled, changes behavior to exit code 1 on test or any other failure, exit 0 on normal or skip-only runs.

You can pass it with through packer using extra_arguments

"extra_arguments": [ "--no-distinct-exit" ]

[christianclarke]

Does anyone know of a way of trapping the error code from Inspec and returning 0?

Using the --no-distinct-exit argument will either return 0 or 1.

I have a use case where I want to run CIS Inspec tests and for Packer to create an AMI. Doing it this way, it makes it visible in our build pipeline the areas we need to fix but the AMI can still be used in Development. Effectively, it allows us to constantly iterate on the AMI's but still providing business value to the teams.

I did think of simply using shell-provisioner installing Inspec on EC2 then running, Inspec actually on the EC2 box. But I'd rather just use the InSpec provisioner built into Packer - but somehow trap the error.

Thanks appreciate any insight.

[kiwivogel]

Might be a good idea to put this in the documentation. I'll make some time. To add to the discussion. If you make a profile on top of some external profile with skips and overrides 101 should be allowed to pass. There should be a way to configure this.

[kclinden]

@christianclarke - Did you ever figure out a workaround for this? There are some hardening settings that I have disabled during packer build (such as changing the administrator username). I want Packer to accept the 1 return code from Inspec in this case.