Molecule: Consideration for adding another verifier: inspec

[mkinney]

Issue Type

• Feature request

Molecule and Ansible details

ansible 2.4.2.0
molecule, version 2.5.0

• Molecule installation method: pip
• Ansible installation method: pip

Desired Behaviour

Add the ability to use inspec as a verifier.

Actual Behaviour (Bug report only)

Only goss or testinfra are available. We have some experience/code with inspec that we would like to leverage in the verification step. If we do a PR that adds experimental capability, would you be open to it?

For example, this is what I have hacked up:

(venv) m-gwg8wl:ansible-windows [email protected]$ molecule verify
--> Test matrix

└── default
    └── verify

--> Scenario: 'default'
--> Action: 'verify'
--> Executing Inspec tests found in /Users/s003477/code/ansible-windows/molecule/default/tests/...
WARN: Unresolved specs during Gem::Specification.reset:
      net-ssh (< 5.0, >= 2.6.5, >= 2.9)
      multi_json (~> 1.10)
      winrm-fs (~> 1.0)
      docker-api (~> 1.26)
      thor (~> 0.19)
      rspec (~> 3)
      addressable (~> 2.4)
      diff-lcs (< 2.0, >= 1.2.0)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.

    Profile: tests from /Users/s003477/code/ansible-windows/molecule/default/tests/ansible_windows.rb (tests from .Users.s003477.code.ansible-windows.molecule.default.tests.ansible_windows.rb)
    Version: (not specified)
    Target:  winrm://vagrant@http://192.0.2.133:5985/wsman:3389


      File c:\windows\system.ini
         ✔  should exist

    Test Summary: 1 successful, 0 failures, 0 skipped
Verifier completed successfully.

$ cat /Users/s003477/code/ansible-windows/molecule/default/tests/ansible_windows.rb
describe file('c:windowssystem.ini') do
it { should exist }
end

[mkinney]

Forgot that this was the molecule.yml values:
```verifier:
name: inspec
options:
password: vagrant
target: winrm://[email protected]
env:
no_proxy: 192.0.2.133

[retr0h]

Hi @mkinney ! I have held off on inspec for few reasons.

• The way one must configure inspec to connect to the instances. It doesn't piggy back on the existing ansible_connection options that Molecule sets up, where Testinfra uses the ansible connection. This means testinfra can simply connect to the instances the same way Ansible does, and supports all the drivers Ansible does.

• Molecule v1 used to support server spec but was removed in v2. Molecule is somewhat biased, and supporting too many verifiers is tricky with limited time and knowledge of such verifiers.

• Ability to target host(s) for a particular test file or group of test files.

I would prefer to hold off on such a feature.

[retr0h]

Not going to support inspec at this time.

[jugatsu]

Example role with inspec support https://github.com/jugatsu/ansible-role-pritunl-server. A lit bit hacky)

[jugatsu]

Inspec support wil be big win btw.

[eshamow]

@retr0h if we were able to work around concerns 1 and 3, would 2 be a blocker?

We can put some time and energy into finding ways to integrate better, but don't want to do it if the intent is to stay away from inspec regardless.

[retr0h]

@eshamow why is it you wish to use Inspec vs Goss or Testinfra. What benefit does it provide?

[jugatsu]

@eshamow inspec is my favourite because of:

1. lots of resources https://www.inspec.io/docs/reference/resources/

2. active development under Chef, Inc. support

3. great for TDD and compliance-as-code

4. supports ssh,winrm,docker connections and doesn't required any dependencies on instance

5. clean syntax compared to testinfra

[jugatsu]

@eshamow any updates on this issue?

[eshamow]

@retr0h Sorry missed the first update on this.

I'd second @jugatsu's arguments, and add that I have a pretty large library of mocks and doubles that are developed in inspec that are non-trivial to port to another language/test suite.

We have used inspec to train our security teams and other groups to communicate requirements with us in terms of test feedback. As a result we have large groups writing tests against all sorts of integrations and doubles written against Inspec, and we use those in our molecule tests to verify compliance.

[jamesla]

@retr0h

This would be a great addition. I have been stuck on molecule version 1 because I run both linux and windows servers using the same boiler plate code to stick everything together. The reason I am stuck on version 1 is that once we lost server spec verifier support we lost Windows support (which works great in version 1 btw).

I agree with dropping server spec in the sense that it is old and not that well maintained and I think that adding an inspec verifier would be a fantastic solution to bring unofficial Windows support back to molecule and allow me to upgrade all of our box repositories to molecule v2!!!

[retr0h]

Hi @jamesla -

I have some questions about inspec. Currently Goss and testinfra are able to test all of the drivers Molecule supports b/c testinfra simply uses the Ansible connection backend. So anything Ansible can connect to, testinfra can as well. Super simple. Goss, is also able to test all drivers, simply b/c Ansible connects to the instances and installs the binary locally.

My question about inspec, is it's ability to test all drivers Molecule supports. I understand we can use ssh to connect to ec2, gce, azure, openstack, and vagrant. It appears inspec has docker support as well. Does inspec support lxc and lxd testing? Also, can I supply an ssh config file to inspec in the case where Molecule users are using the delegated driver, and simpy wish to provide the connection options on their own.

I do understand your concern, and appreciate your patience.

[retr0h]

@jamesla Also, how do you target a specific test file to a subset of hosts?

[jugatsu]

@retr0h https://github.com/jugatsu/ansible-role-pritunl-server/blob/master/test/test.sh

Just run inspec executable against target hosts.

[jugatsu]

Guys, i don't understand why we even debated about inspec support :) IMO inspec is must-have for every modern framework especially molecule :)

[retr0h]

@jugatsu I have already stated why.

[retr0h]

I have an idea.

[retr0h]

You guys win :)

[mkinney]

Doesn’t that mean that we all win? 😀

How can we help?

---

Mike Kinney

On Apr 18, 2018, at 7:00 PM, John Dewey [e-mail] wrote:

You guys win :)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[benpoliquin]

Thank you for supporting this!

[brucellino]

Reading this thread gave me feels... thanks @retr0h
However... I can't get inspec to be invoked. Is there an issue I should be watching?

[brucellino]

I'm an idiot - should have invoked the scenario with --verifier-name inspec

[retr0h]

Or change your molecule.yml's verifier.name to inspec.

[brucellino]

Yeah, I tried that, but I got

└── inspec
    └── verify

--> Scenario: 'inspec'
--> Action: 'verify'
Skipping, no tests found.

The problem was that no controls could be found (wrong dir).
Inspec goes out of its way to make compliance easy to reproduce across scenarios, distributing them as profiles in the chef supermarket or git repos even. I had naively wanted to have something like:

verifier:
  name: inspec
  profile:
    url: https://github.com/EGI-Foundation/ansible-style-guide
  lint:
    name: rubocop

or something like that. If I find some time, I'll try send a PR for this...

[retr0h]

Oh.... now that's cool. You can see my lack of inspec knowledge while implementing this feature.
Would a person ever test multiple profiles along with additional local test files?

Could we do something like:

verifier:
  name: inspec
  profiles:
    - url: https://github.com/EGI-Foundation/ansible-style-guide
    - url: https://...
    - url: https://...
  lint:
    name: rubocop

Also, what is the command inspec would use to execute a remote profile? Must the profile be downloaded before hand or will inspec take care of that?

[brucellino]

Yes absolutely the profiles are usually developed by Ops, or the InfoSec team, so they are the ones which are sending the devs the profiles that their applications have to pass. There is a dependency mechanism, just with Ansible roles.
See https://www.inspec.io/docs/reference/profiles/

I have taken a look at the code for generating the verify.yml as well. I have thoughts :face_with_head_bandage: Best open another issue on this to discuss this inspec integration there.

Let's be straight though : adding inspec to molecule is a huge frikkin win for everyone. I don't want to give you the impression that not enough has been done by you the maintainer! On the contrary... I really hope to find some free time soon to send a PR for how I think this should work.

Thanks for the work so far!