Molecule v2 is a huge rewrite and change. I am requesting review and feedback.
ansible-tests, test-ansible...Please provide feedback so we can better understand how you use Molecule. As an example, we currently do not support Serverspec in v2, your feedback is helpful.
I'll try and take a look at this tonight.
Thanks for posting this publicly, @retr0h! Happy to take a look at what you've been cooking up.
I don't know if I get the idea behind scenarios. Is this a concept for testing a role under different conditions e.g. a single node setup or a clustered setup of a database running on n nodes? I'd love to see that :+1:.
Was the serverspec support removed on purpose or will it come back?
Another name suggestion: testible (testable + ansible)
I don't know if I get the idea behind scenarios. Is this a concept for testing a role under different conditions e.g. a single node setup or a clustered setup of a database running on n nodes? I'd love to see that ๐.
Yeah, exactly...
Was the serverspec support removed on purpose or will it come back?
It was originally removed to keep scope down. We probably could and possibly should add it back. I'm not entirely sure how many people use serverspec.
Another name suggestion: testible (testable + ansible)
๐
Name change from molecule to ansible-test, which isn't getting very far.
Are we collecting new names for molecule v2? Is it possible ansible imports molecule as a core part and shipped together, namely ansible-test (just like ansible-galaxy is shipped with ansible)?
Ansible community has a pure-ansible style test stratedgy, may be we should convince them molecule is a more powerful and modern test framework for ansible developing roles and playbooks, and connects ansible community and IaC tendency.
Ansible community has a pure-ansible style test stratedgy, may be we should convince them molecule is a more powerful and modern test framework for ansible developing roles and playbooks, and connects ansible community and IaC tendency.
I am open to it, but haven't had much success. Honestly, I have spent over a year trying to get many PRs into Ansible. I'm not really interested in trying to teach them why Molecule is better ;)
I am open to it, but haven't had much success. Honestly, I have spent over a year trying to get many PRs into Ansible. I'm not really interested in trying to teach them why Molecule is better ;)
seems that Redhat is not very willing to collaborate with community, but prefers selling his Tower service.
As far as I understand it, molevule v2 still operates on the "role" level, right? Meaning one role, one molecule.yml etc.
If I want to test multiple roles in interaction (e.g. a VPN server and a VPN client role or several web servers + a load balancer in front) I might need some extra work to create a scenario? Maybe this could be a good example for the documentation.
hi @MarkusTeufelberger
I succesfully tested several role at once initializing molecule at the same level of the roles folder instead inside a specific role and setting the roles path via environment variable before invoking molecule test.
Maybe this could be a good example for the documentation.
This is documented through our tests. I updated the documentation pointing people to this location.
https://github.com/metacloud/molecule/commit/1ca1a71016ba7327e570125c2ec07dd94d4d30b6
Release candidate 2 is now available on pypi.
pip install molecule --pre
Seems like molecules inventory files aren't working.
root@molecule-dev:~/ansible-playbooks/roles/servers/wiki# molecule init scenario --role-name wiki --scenario-name default
--> Initializing new scenario default...
Initialized scenario in /root/ansible-playbooks/roles/servers/wiki/molecule/default successfully.
root@molecule-dev:~/ansible-playbooks/roles/servers/wiki# molecult test
molecult: command not found
root@molecule-dev:~/ansible-playbooks/roles/servers/wiki# molecule test
--> Scenario: [default]
--> Provisioner: [ansible]
--> Playbook: [destroy.yml]
ERROR! ERROR! /root/ansible-playbooks/roles/servers/wiki/molecule/default/.molecule/ansible_inventory.yml:6: Expected key=value host variable assignment, got: &id001
Installed via pip.
Unrelated: does this project have an IRC or something similar for support?
Seems like molecules inventory files aren't working.
@cwre you're version of ansible isn't supported. Only supports 2.2 and 2.3. See changelog.
Unrelated: does this project have an IRC or something similar for support?
There are people there, but your mileage may vary :) Check out the docs, there is good stuff in there.
There are people there, but your mileage may vary :) Check out the docs, there is good stuff in there.
I've tried joining #molecule on Freenode, but it's invite only.
@retr0h My apologies... I installed via apt on accident. Thanks for the patience!
I've tried joining #molecule on Freenode, but it's invite only.
Will be promoting v2 branch to master on June 5th, unless a major issue is found. However, Molecule v1 will remain the default version on pypi. For those of you building off master, this is an important detail.
The Vagrant driver is quite light on documentation, it would be nice to know for example how to set the back-end Vagrant will actually use (VirtualBox etc) and how to manage some settings there (e.g. increase RAM to 1024 MiB).
I know that other drivers could be nicer to use (since they could be managed by Ansible itself), however in one of my use cases for example I'm trying to test a role that starts and provisions LXD containers and I much prefer to have this stuff just run in a full virtual machine to avoid potentially complex setup requirements on the machine running molecule.
The Vagrant driver is quite light on documentation, it would be nice to know for example how to set the back-end Vagrant will actually use (VirtualBox etc) and how to manage some settings there (e.g. increase RAM to 1024 MiB).
Added the examples to sphinx.
@retr0h Is this "Molecule Vagrant Module" snippet from Sphinx just the duplicated (or alternative?) contents of create.yml and destroy.yml scenario files? If I'm using a different provider this "molecule_vagrant" stuff is ommited?
A few comments in the yml files or under the "Molecule Vagrant Module" in the docs would help to clear out my confusion.
@retr0h Is this "Molecule Vagrant Module" snippet from Sphinx just the duplicated (or alternative?) contents of create.yml and destroy.yml scenario files? If I'm using a different provider this "molecule_vagrant" stuff is ommited?
@butla The Sphinx docs for the vagrant driver is the example section of the ansible module.
I added more examples, which should be what you're looking for.
Will be promoting v2 branch to master on June 5th, unless a major issue is found. However, Molecule v1 will remain the default version on pypi. For those of you building off master, this is an important detail.
๐ 3
Promotion complete. This does not change the released version of molecule.
Hi,
I see in the docs that to get started, I need to do :
my question is about docker-py . it's been month now since the "docker-py" library has been renamed to "docker" it's available on pypi . this move has been part of a revamp they did for version 2.0 of the library.
I mostly run test on docker driver and our version of docker in prod is very recent 17.05-ce, so to have full use of the api of docker engine we would need to switch to "docker 2.0" version of the python library .
have you considered that ? or are you waiting on ansible to make the switch too... my understanding is that they are indeed, for some module, developing on the docker 2.0 python library .
FYI the last version of "docker-py is 1.10 "
The new version of the library is "docker 2.0 "
And by the way for the new naming, how about calling it :
or even how about :
Yes I know the "Spec" is everywhere ;) ...may The Spec be with you
my question is about docker-py . it's been month now since the "docker-py" library has been renamed to "docker" it's available on pypi . this move has been part of a revamp they did for version 2.0 of the library.
I mostly run test on docker driver and our version of docker in prod is very recent 17.05-ce, so to have full use of the api of docker engine we would need to switch to "docker 2.0" version of the python library .
have you considered that ? or are you waiting on ansible to make the switch too... my understanding is that they are indeed, for some module, developing on the docker 2.0 python library .
We use the Ansible libraries to manage Docker. Until Ansible switches to the new docker, we must continue to use docker-py.
Yes I know the "Spec" is everywhere ;) ...may The Spec be with you
I'm not a huge fan of the spec naming since spec implies a testing specification, such as given, when, then.
How does everyone feel about releasing molecule v2 as the main molecule package on pypi?
I've had some problems with migrating vagrant/virtualbox test runs, but I'm not entirely sure that it wasn't my fault. When do you want to make the decision? I'd like to know how long I have to check everything :-)
Also've had problems with the migration, but will keep at it. +1 for updating pipโI suspect that'll get you _more_ users. All the contexts I use Molecule in have versions pinned in the requirements.txt anyway, so I don't expect any surprises.
problems with the migration
@conorsch @butla which types of problems? Just getting stuff ported over to the new syntax?
@retr0h Yes, specifically migrating to the new syntax. Several times I've consulted the v2 docs and come up short with what I needed, but I see in this thread you've been knocking those out steadily, so I'll give it another go. (Will open issues if I can repro problems with the syntax migration vis-a-vis the docs.)
@retr0h My problem was that a centos7 Vagrant/virtualbox machine was reporting errors right after starting up, even before running Ansible. It might had something to do with virtualbox guest additions plugin I had installed for vagrant. I should try to reproduce it again, but I'm going away for the weekend. That's why I asked about when you want to make the decision :-) I could probably test it out again on Monday
I could probably test it out again on Monday
๐
Spent a lot of time porting to the v2 syntax. It's great! The custom playbooks approach for setup and teardown provide much more flexibility than v1, and the scenarios are a huge boon. Two issues stood out for me:
DockerfileโI ended up running docker build . -t foo and referencing the image "foo" in the molecule.yml config.That said, I'm happy with v2 being released on pypi and rolling with it.
This issue : https://github.com/metacloud/molecule/pull/305/files#diff-f2bcb04b4df2baf05d37ce3d8dd17efe
seems to have come back for me:
See:
--> Dependency: [galaxy]
- downloading role 'EPEL', owned by goozbach
- downloading role from https://github.com/goozbach-ansible/role-epel/archive/v0.2.1.tar.gz
- extracting goozbach.EPEL to (my-devdir)/redis/molecule/default/.molecule/roles/goozbach.EPEL
- goozbach.EPEL (v0.2.1) was installed successfully
Dependency completed successfully.
--> Scenario: [default]
--> Provisioner: [ansible]
--> Syntax Verification of Playbook: [playbook.yml]
ERROR! the role 'goozbach.EPEL' was not found in (my-devdir)/redis/molecule/default/roles:(my-devdir)/:(my-devdir)/redis/molecule/default
The error appears to have been in (my-devdir)/redis/molecule/default/playbook.yml': line 13, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
roles:
- role: goozbach.EPEL
^ here
ERROR:
RAN: ~/.pyenv/versions/python2/bin/ansible-playbook --syntax-check --inventory=(my-devdir)/redis/molecule/default/.molecule/ansible_inventory.yml (my-devdir)/redis/molecule/default/playbook.yml
Other info :
# ansible --version
ansible 2.3.1.0
config file =
configured module search path = Default w/o overrides
python version = 2.7.10 (default, Feb 7 2017, 00:08:15) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]
# molecule --version
molecule, version 2.0.0.0rc7
# python -V
Python 2.7.10
# ansible-galaxy --version
ansible-galaxy 2.3.1.0
config file =
configured module search path = Default w/o overrides
python version = 2.7.10 (default, Feb 7 2017, 00:08:15) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]
@alainchiasson I caught that the other day. Should have fixed that @ https://github.com/metacloud/molecule/commit/59678f34c35cba8950dbd6ebfaeed2de641f01f3, however I haven't have cut a new pre-release, but master that should work for you.
That fixed it. but other items poped up. I will dig more - maybe I missed another change.
I get :
--> Scenario: [default]
--> Provisioner: [ansible]
--> Playbook: [create.yml]
ERROR: Invalid lint named 'ansible-lint' configured.
For this config :
---
dependency:
name: galaxy
driver:
name: vagrant
lint:
name: ansible-lint
platforms:
- name: redis
box: centos/7
provisioner:
name: ansible
playbooks:
setup: ../default/create.yml
teardown: ../default/create.yml
converge: playbook.yml
scenario:
name: default
verifier:
name: testinfra
Oddly If I keep the playbooks redefinition molecule test will run the create first, when I remove it, it runs the destroy.
And if I also remove the lint section :
ยฑ |develop U:2 ?:2 โ| โ molecule test --scenario-name=default
--> Scenario: [default]
--> Provisioner: [ansible]
--> Playbook: [destroy.yml]
PLAY [localhost] ***************************************************************
TASK [Gathering Facts] *********************************************************
ok: [localhost]
TASK [Destroy molecule instance(s)] ********************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: IOError: [Errno 2] No such file or directory: u'redis'
fatal: [localhost]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""}
PLAY RECAP *********************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=1
lint:
name: ansible-lint
Should be
lint:
name: yamllint
Expect the RC versions of Molecule have some changes as we get closer to release.
Thanks it worked.
I think currently molecule still writes "ansible-lint" when executing molecule init...
@MarkusTeufelberger that shouldn't be the case. I have fairly significant functional tests which test this. Was just an old config on @alainchiasson's part and new functionality on mine.
molecule init scenario --scenario-name foo --role-name bar still writes ansible-lint for me with molecule rc9
Hi, I just recently started with Ansible and directly went with Molecule v2.
Testing works great so far, the only thing around which I couldn't wind my head around is the requirements.yml. For Ansible, this has to be at the toplevel of the playbook, for molecule to find it, it needs to be in the molecule/ folder. Anything I am missing? Couldn't find anything in the example playbooks. (think that's really a molecule-specific question and not the one of an Ansible newbie)
@retr0h @MarkusTeufelberger ๐
Could this be the confusion :
...
provisioner:
name: ansible
lint:
name: ansible-lint
...
The one that was failing for me was in the actual "Lint" section.
The ansible-lint is being run on the ansible files:
Idempotence completed successfully.
--> Scenario: [default]
--> Lint: [yamllint,flake8,ansible-lint]
--> Executing Yamllint on files found in /private/tmp/test-role/...
Lint completed successfully.
--> Executing Flake8 on files found in /private/tmp/test-role/molecule/default/tests/...
Lint completed successfully.
--> Executing Ansible Lint on /private/tmp/test-role/molecule/default/playbook.yml...
Lint completed successfully.
--> Scenario: [default]
--> Provisioner: [ansible]
This was generated from molecule init role --role-name test-role
molecule init scenario --scenario-name foo --role-name bar still writes ansible-lint for me with molecule rc9
@MarkusTeufelberger thanks. I must not be testing that use case. Thanks again for reporting it.
Testing works great so far, the only thing around which I couldn't wind my head around is the requirements.yml. For Ansible, this has to be at the toplevel of the playbook, for molecule to find it, it needs to be in the molecule/ folder. Anything I am missing? Couldn't find anything in the example playbooks. (think that's really a molecule-specific question and not the one of an Ansible newbie)
@StephenKing It goes in the scenario directory, so $role/molecule/default/requirements.yml.
@retr0h The problems that I had with v2 are gone (I tried rc11 now). I'm creating a pull request to switch to v2 in my project right now :)
The things I've struggled with:
@butla Thanks for the feedback. Yeah, convert.py needs some extra love. I'm mostly using it to convert our internal roles, unfortunately it doesn't do everything for everyone :( It's a ton of work.
@retr0h it'd be good to note that in the migration docs. Right now they're saying that it should work for Vagrant configurations, which isn't exactly true, as you see :-)
I am not too happy with the default configuration of yamllint (i'd like to disable the line length check for example) and it would also be nice to have ansible-lint back in the lint: section. Maybe this could be turned into a list to be able to set additional options to the various linters (even including flake8 or other linters added in the future)?
I am not too happy with the default configuration of yamllint (i'd like to disable the line length check for example) and it would also be nice to have ansible-lint back in the lint: section. Maybe this could be turned into a list to be able to set additional options to the various linters (even including flake8 or other linters added in the future)?
@MarkusTeufelberger that is already disabled. Molecule init ships with a .yamllint file which turns that off. You probably have molecule files prior to yamllint getting added.
You can drop in a .yamllint file or disable yamllint all together.
@MarkusTeufelberger that shouldn't be the case. I have fairly significant functional tests which test this. Was just an old config on @alainchiasson's part and new functionality on mine.
This was corrected @ eac0710623da541deef19189665d216d88c50267. Ensured functional test is testing this scenario in the future.
HI @retr0h which is the proper path/configuration for a custom yamllint file?
I've tried the documentation configuration (by the way on the doc site the official v2 path is still on ansible-lint instead of yamllint)
options:
config-file: foo/bar
But file seems to be ignored, is the path relative to molecule.yml file as usual?
I've spent 2 hours converting my code to match rules, but some of them (like quoting the boolean values) break ansible logic so it result in tons of warning before starting the converge.
I've also tried a fresh init with RC11 but no .yamlint file is created in the folder.
Am I missing anything?
thanks in advance.
Ah, I usually add molecule to existing roles and when running molecule init scenario ... the .yamllint file didn't get created.
Ah, I usually add molecule to existing roles and when running molecule init scenario ... the .yamllint file didn't get created.
@MarkusTeufelberger Yeah, .pyyaml is a project wide thing. It is only added when molecule init role is used.
I'm liking this a lot so far โค๏ธ ... I really appreciate the extensibility of using ansible playbooks for the spinning up and down of instances.
I agree with comments regarding the lack of docs but I found that I was able to piece together everything after running the --init command. So I would comment that you should throw more examples in that init scaffolding. I also strongly suggest you more prominently link to example roles that utilize molecule v2 in the docs. Less documentation for you to create and more real working examples that ops people like me enjoy ๐
Was the serverspec support removed on purpose or will it come back?
It was originally removed to keep scope down. We probably could and possibly should add it back. I'm not entirely sure how many people use serverspec.
It would be great to directly go with Inspec because it is compatible with serverspec, under steady development and it also replaced Serverspec.
It would be great to directly go with Inspec because it is compatible with serverspec, under steady development and it also replaced Serverspec.
At the moment I'm not really interested in adding additional verifiers. Ruby based verifiers are a pain to deal with dependency management for most mortals. Testinfra works very well, and for those that want something simpler, Goss.
@retr0h It seems that the only way to set no_log to False (e.g. in destroy.yml) is to hardcode it into the file.
Setting MOLECULE_DEBUG=True/true/TRUE doesn't help.
This is because the expression setting "no_log" returns a string, which is truthy as long, as it's not empty. You should probably use "{{ no_log }} | bool" when evaluating.
I'm using molecule 2.0.0.0rc12 and Ansible 2.3.1.0.
@butla I thought I was doing this. Mind pointing me to where you are having this issue? Maybe I missed a spot. Best I can tell I am doing exactly this. Could it be you had an old scenario before I implemented this?
[jodewey:~/git/molecule_2] [molecule2-env-2.7.12] master+ ยฑ grep -R MOLECULE_DEBUG molecule
molecule/config.py: 'MOLECULE_DEBUG': str(self.debug),
molecule/cookiecutter/scenario/driver/docker/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/create.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/docker/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/destroy.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/ec2/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/create.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/ec2/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/destroy.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/gce/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/create.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/gce/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/destroy.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/lxc/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/create.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/lxc/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/destroy.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/lxd/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/create.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/lxd/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/destroy.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/openstack/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/create.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/openstack/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/destroy.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/vagrant/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/create.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
molecule/cookiecutter/scenario/driver/vagrant/{{cookiecutter.molecule_directory}}/{{cookiecutter.scenario_name}}/destroy.yml: no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
@retr0h My assumption was wrong, my version of Ansible will evaluate "False" or "false" as False even though this string would be truthy in Python. But, I can't make it to log anything without directly modifying the yml. Maybe the environment variable isn't getting passed?
How do you run the tests with debug?
$ MOLECULE_DEBUG=True molecule test
doesn't work for me.
As for the no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}" expression - it's everywhere that I think it needs to be, that wasn't my problem.
Try molecule --debug
Yeah, you only get output when running --debug as @StephenKing suggested. Manually passing MOLECULE_DEBUG=True molecule test will not work since Molecule controls the setting of MOLECULE_DEBUG internally and resets whatever you may have passed.
All outstanding v2 bugs have been addressed with 2.0.0rc14.
Once the RCs were released, I started getting legitimate feedback, so my apologies for those of you that I broke. Once we release Molecule v2 we will not introduce breakage in minor/patch releases.
Hi there, I'd like to help with your testing, but I'm having some difficulty finding a simple workflow for testing roles with an AWS / EC2 provider. Does anyone have an example to share?
Hi there, I'd like to help with your testing, but I'm having some difficulty finding a simple workflow for testing roles with an AWS / EC2 provider. Does anyone have an example to share?
@b-long molecule init role -r foo -d ec2
Configure molecule/default/create.yml, molecule/default/destroy.yml, and molecule/default/molecule.yml to your liking.
Hi, I'm new to molecule and started using v2 for my roles. Here are some issues (rc14):
instance_raw_config_args is missing in create.yml.destroy removes all roles in .molecule/roles (empty dirs)docker terminal does not work properly:It looks that there is a way to avoid Ctrl-P behaviour in docker terminal. Create ~/.docker/config.json like this:
{
"detachKeys": "ctrl-q,ctrl-q"
}
@bitozoid please file issues with details.
It looks that there is a way to avoid Ctrl-P behaviour in docker terminal. Create ~/.docker/config.json like this:
The only true keybindings are vi keybindings ;p lol
@retr0h #917, #918, #919.
Have been away for bit - Nice to see a release 2. 0 !! ๐
v2 is out ๐
Thanks all for your feedback, issues, and help ๐
Most helpful comment
Will be promoting v2 branch to master on June 5th, unless a major issue is found. However, Molecule v1 will remain the default version on pypi. For those of you building off master, this is an important detail.