Community: Molecule Working Group Agenda

[gundalow]

Please leave a comment regarding any agenda item you wish to discuss.

If your IRC nick is different from your GitHub username, leave that as well.

Import by URL https://raw.githubusercontent.com/ansible/community/master/meetings/ical/molecule.ics into your calendar of choice

Once an item has been addressed it should get strike-though strike-though

When creating new agenda ensure molecule and meeting_agenda labels are set

Running IRC Meeting

Anyone can start the IRC meeting by doing

#startmeeting Ansible Molecule Working Group
#info Agenda https://github.com/ansible/community/labels/molecule
Hi everybody, who's around
#chair user1 user2 ...
#topic First Topic
#info Link to PR/Issue/more detail
#agreed [user1] to do X
#topic Second Topic
...
#topic Open Floor
...
Thank you everybody for attending
#endmeeting

[gundalow]

Possible items for agenda:

• Backlog review
• State of testing
• What needs doing for the next release https://github.com/ansible/molecule/milestone/1

[themr0c]

I would like to emphasize the point that for the community, the molecule docker image can be regarded as the most official docker image to get ansible. Until october, we had monthly releases of molecule. As a consequence, it was possible to use the latest docker image of molecule to get a docker image with the latest stable image of ansible. Now the latest molecule docker image is quite lagging behing, and it is becoming a problem. As a consequence, i believe it would be a good idea to keep the same rythm, and release as long the release won't introduce new major issues.

[Shaps]

Not sure if it's sensible at this stage, but having some sort of idea of a release schedule for molecule would be good

[decentral1se]

Seems main priority is what are the minimum steps necessary to get a next minor release.

Others, from my perspective:

• Access for community for labelling, milestoning, closing issues etc. (triage)
• Transparency on commit bits, who has them and when are they available? Will the community get them?
• Roadmap: are we just reacting to bugs and needs that arise from the current userbase or should we be aligning with roadmap of galaxy and ansible core (for example, collections is landing here quite fast with some unknown thinking behind that).

[dagwieers]

@themr0c I agree that the Ansible project should manage official Ansible docker images (and why not, also an image that works with VMware and VirtualBox).

[MarkusTeufelberger]

I'd be interested in some longer term goals:

• Should molecule become something like ansible-role-test or retain its name, brand and external repository?
• Cookiecutter templates for creating molecule enabled roles is nice, but feels very similar to ansible-galaxy init. Should this "create a sample role template with all best practices pre-filled" stuff migrate to galaxy or stay in molecule?
• Upstream the molecule Vagrant module (https://github.com/ansible/molecule/issues/1404)?
• Should molecule be more explicit in showing what happens (e.g. rendering out the built-in playbooks to create/destroy resources), or is that too much duplication and too limiting when these processes are updated?
• Currently testing roles with Docker is the easiest way to use molecule, but as soon as something like systemd units is involved, it quickly turns into a mess. What are other people's best practices to test roles that require a bit deeper access to system state locally and on a CI system? How can we add them to molecule/molecule's documentation?

Most of these are probably not good topics for a first meeting, but I'd like to have them discussed maybe after Ansible 2.8 drops. It would be great to be able to test roles locally using just the official package/binaries after installing Ansible 2.10.

[gundalow]

Import by URL https://raw.githubusercontent.com/ansible/community/master/meetings/ical/molecule.ics into your calendar of choice. See you Wednesday 1900 UTC in #ansible-molecule

[rgarrigue]

Since I mostly worked trying to get teams on the ansible + molecule way I'm interested in users' engagement & ease of use. So here are some ideas for later on

• State of the doc. Imho good, exhaustive technically but lacking in term of getting started. A traditional ops picking ansible for its learning curve might find molecule a difficult dev-ish looking stuff.

• Reports. Test is hard to sell to a (non technical) boss. Being able to produce a simple html or pdf is a big plus for some of us. I investigated this and ended up with a 1990's looking report through pytest. As a comparison, robot framework was chosen in my previous company for it's okay looking report.

• As Markus stated, molecule could be merged in ansible-galaxy init and completed, maybe optionally, with stuff like linters files (.flake8, .ansible-lint.yaml, ...) and CI/CD files (.travis.yaml, .gitlab-ci.yaml, ...)

[loomsen]

My thoughts, also mid- or longterm:

• molecule as a replacement for the now abandoned ansible-container as a means to build images (in the end, we actually built a docker image when we provision it with molecule, so why not just docker save and tag it)
• a kubernetes driver to run instances on a kubernetes cluster and provision those

[webknjaz]

@rgarrigue, AFAIK @acozine is going to be working on docs improvements soon'ish

[gundalow]

https://github.com/ansible/molecule/issues/1642 Proposal: Use prepare.yml instead of rendering Dockerfiles with the Docker driver from @MarkusTeufelberger

[fabianvf]

@loomsen wrt the kubernetes driver, for simple cases I've been able to bring up a Kubernetes cluster using a Kubernetes-in-docker image and test against it with just the docker driver (https://github.com/fabianvf/example-generated-operator/blob/master/molecule/test-local/molecule.yml)

things I'm interested in:

• Further exploration of testing against Kubernetes (we could look at the new kind tool as the basis for a driver)
• Ansible-based verifier - for the Kubernetes roles I'm working on Ansible is by far the easiest way to interact with the cluster. Currently I just end up importing an assertion playbook at the end of converge which looks and feels bad.

[robertdebock]

Agenda item:

• I'd like a command line option to not destroy instances on failure. Maybe --troubleshoot or --donotdestroy or something.

That is to debug and figure out what an instance looks like in order to improve playbooks.

Currently I spin up machines manually, run a playbook against them so I can login and troubleshoot.

[rgarrigue]

@robertdebock such an option already exists : --destroy=never. Or you can change the test matrix, specifying the same as the default one without the last destroy step.

[fabianvf]

@rgarrigue --destroy=never is fine, but I think --destroy=onfailure or something like that could still be useful (especially if you're running a series of scenarios that require teardown). It would also presumably not prevent the destroy step from being run at the beginning of the test sequence.

[samdoran]

• [x] https://github.com/ansible/molecule/pull/1659

[chris-short]

Molecule Verifiers: In working with @fabianvf on using Molecule to test Ansible Operators in OpenShift the need for Ansible as a verifier came up again and again.

This is also a concern of mine as the current bunch of Molecule verifiers fly in the face of The Ansible Way. testinfra for example is gnarly for beginners. InSpec is an option I'd like to support out of the box too.

Let's discuss!

[decentral1se]

These PRs have been lurking a while and are looking good to merge. Let's take a pass on them.

• https://github.com/ansible/molecule/pull/1674
• https://github.com/ansible/molecule/pull/1663
• https://github.com/ansible/molecule/pull/1653
• https://github.com/ansible/molecule/pull/1646
• https://github.com/ansible/molecule/pull/1535
• https://github.com/ansible/molecule/pull/1543

[decentral1se]

The CI PRs: https://github.com/ansible/molecule/pulls/themr0c + https://github.com/ansible/molecule/issues/1669. How to push forward this great work.

[vutkin]

Hi all, please look how we (lean_delivery org https://github.com/lean-delivery) solve problem with testing in docker with systemd support. We have created separate docker images with systemd support for tests in docker based on documentation from rh: https://hub.docker.com/r/leandelivery/docker-systemd/tags.

Used tags:

• centos7
• ubuntu-18.04
• ubuntu-16.04

examples for usage:
https://github.com/lean-delivery/ansible-development-kit/blob/develop/%7B%7Bcookiecutter.role_name%7D%7D/molecule/default/molecule.yml

Maybe it will be useful for community.

[MarkusTeufelberger]

@vutkin - Launching privileged containers seems a bit risky to me. https://developers.redhat.com/blog/2016/09/13/running-systemd-in-a-non-privileged-container/

[vutkin]

Hi @MarkusTeufelberger, Sure, but we accept this risk, it's only for testing purposes.
Besides I saw some strange behavior in tests when tested roles in docker without privileged permission (for Centos based image).

[decentral1se]

@MarkusTeufelberger @vutkin, please, let's keep this space for adding agenda items. That discussion can be had during the meeting. Thanks.

[ssbarnea]

Proposal for removal of DCO bot which makes contribution to molecule much harder and causes problems with rebases and even makes impossible to make small fixes using github web interface.

Even Ansible itself does not require signing commits. I propose to disable it and to look for a less intrusive replacement after, if any at all.

[singuliere]

To add to the next meeting agenda:

I nominate @lwm / decentral1se to be granted merge rights to the repository. Their outstanding contributions and daily work to improve molecule, triage issues etc. as well as his communication skills would benefit the project.

There may be a different process to actually do this but ... here it is anyways ;-)

[decentral1se]

I won't make it (probably) but I think we need to agree on the 2.20 release date. All the issues we had lined up on the project UI are done: https://github.com/ansible/molecule/projects/2 (72 items completed, well done everyone). We're running a risk of frustrating users (both new and old) if we let this drag on further.

[ssbarnea]

Proposal: adding a bot that auto-adds reviewers to open PRs, or at least that propose which ones to be added. This could speed reviewing especially for people from outside which may have no idea which people to tag (not having the power to effectvely add reviewers).

If you not already have another bot in mind, I could say I found the one used at https://github.com/kubernetes-sigs/kubespray/pull/4206 extreamly smart: suggesting people and actions for adding reviewers. Not sure if it can easily be reused but I am sure there are others doing almost the same thing.

[ssbarnea]

I would also want to propose having a look at https://github.com/ansible/molecule/issues/1744 and at least agreeing about the direction to be taken for enabling multi role testing. If we agree on expected behaviour we could gradually made the needed changes for reaching this goal.

[webknjaz]

@ssbarnea I've got (and still gaining) some expertise in writing bots. If I have a volunteer to do bot coding we can build something together. But currently, I don't have any bandwidth to do such thing on my own.

[themr0c]

on the topic for today: CI broken again on functional tests https://travis-ci.com/ansible/molecule

[Shaps]

@webknjaz I might have some bandwidth in the coming months to help you with bot stuff, that said, I do not have that much experience in bot-writing.

[MarkusTeufelberger]

https://github.com/ansible/ansibullbot exists, no need to reinvent the wheel. This is getting a bit off-topic though.

[gundalow]

2018-02-13 Meeting Summary

v2.20 Release

• There hasn't been a Molecule release since Red Hat became the maintainer, though we are fairly close Work remaining
• Last requirement is getting CI Green
• Original Molecule Release Procedure hasn't been updated since Red Hat became maintainer
• AGREED pre-release, publicise, get feedback, wait $period_of_time, then do v2.2.0 final. Need to ensure this includes the Docker image

Getting CI Green

• There has been a lot of great work by lots of people
• Cron should be ignoring devel (work around for https://github.com/ansible/molecule/issues/1727)
• smyers has kindly offered to fix the issues that ansible-lint v4.1.0 has detected, and a meta data issue
• @themr0c has kindly offerd to fix the remaining CI issues

Post v2.20

After the next release we can look at development process improvements:

• Who has commit
• How triage is done
• Use of GitHub Projects vs Milestones

Meeting logs

Summary
Ful log

[gundalow]

Help needed reviewing Adapt functional testing to recent lint updates #1749
to work out what's the right approach. This is one of the final blockers to getting CI green and therefore doing a Molecule release. So any help is really appreciated.

[webknjaz]

@MarkusTeufelberger ansibullbot is in a bad shape, it's very tightly bound to ansible/ansible repo and needs a rewrite. Oh, and it's poll-based which makes it hit API rate limits faster. So it's a no-go.

@Shaps typical bot is a small web-app which receives webhook events from the GitHub platform via HTTP.

To make a simplistic bot in Python, one can copy-paste something like https://github.com/python/miss-islington/blob/master/miss_islington/__main__.py and go from there. Except that one isn't GitHub Apps ready. From the coding perspective, it's not much to do: after doing basic infra setup all that's left is writing actual logic...

As a quick and dirty solution, we could fork https://probot.github.io/apps/auto-assign/ and modify it to post a comment instead of triggering assign. But it's in JS which is fine by me but could be uncomfortable for others.

OTOH I'm currently developing a framework for GitHub Apps based bots in Python so if we're not in a hurry we could wait until I have a first working PoC/alpha to depend on that which would simplify a lot of routine copy-paste...

P.S. Another problem would be hosting it.
P.P.S. There's also GitHub Actions but it's unclear whether non-push events work in public repos.

[gundalow]

Before we look at technical solutions for GitHub workflow I think it might be useful to define the use-case & workflows in https://etherpad.openstack.org/p/ansible-molecule (end of the file, heading GitHub workflow ).

[gundalow]

• Happy to announce that @lwm (decentral1se) and @themr0c have commit powers. Thank you both for everything.
• Topics for next meeting: CI progress, Triage

[themr0c]

• (Almost) all tasks for 2.20 done (https://github.com/ansible/molecule/projects/2 + https://github.com/ansible/molecule/milestones/v2.20) + CI green (https://travis-ci.com/ansible/molecule/branches) -> ready to ship 2.20 ?
  
  
  
  • https://github.com/ansible/molecule/issues/1609
  
  
  • https://github.com/ansible/molecule/pull/1691
  
  
  • https://github.com/ansible/molecule/issues/1727 -> move to 2.21?
  
  

Prepare next release 2.21 (start merging once 2.20 is released?) https://github.com/ansible/molecule/projects/4

• PR with consensus needing some help:
  
  
  
  • need rebase before merge https://github.com/ansible/molecule/pull/1740
  
  
  • some expert advice on https://github.com/ansible/molecule/pull/1754 ?
  
  
  • broken in ci https://github.com/ansible/molecule/pull/1526
  
  
• Review:
  
  
  
  • https://github.com/ansible/molecule/pull/1745
  
  
  • https://github.com/ansible/molecule/pull/1571
  
  
  • https://github.com/ansible/molecule/pull/1716
  
  
  • https://github.com/ansible/molecule/pull/1491
  
  
  • https://github.com/ansible/molecule/pull/1543
  
  
  • https://github.com/ansible/molecule/pull/1644
  
  
  • https://github.com/ansible/molecule/pull/1604
  
  
  • https://github.com/ansible/molecule/pull/1605
  
  
  • https://github.com/ansible/molecule/pull/1535
  
  
  • https://github.com/ansible/molecule/pull/1659
  
  
  • https://github.com/ansible/molecule/pull/1663
  
  
  • https://github.com/ansible/molecule/pull/1581
  
  
  • https://github.com/ansible/molecule/pull/1691
  
  
  • https://github.com/ansible/molecule/pull/1646
  
  

• We need a plan for the CI on cloud backends (azure, ec2, linode, probably soon digitalocean). Some ideas?

[gundalow]

2.20.0.0a1 has been created

This is a pre-release handle with care

• pip install molecule==2.20.0.0a1
• https://pypi.org/project/molecule/#history
• https://quay.io/repository/ansible/molecule?tab=tags
• Full changelog to be created

During the meeting we will review what's left and how to publicise this.

[gundalow]

Molecule as the basis for Galaxy Development Kit

[ssbarnea]

We should also discuss possible adoption of gilt for depencency installation.

• gilt is able to install ansible roles regardless how the repository hosting them is configured, something that cannot be done by galaxy or mazer
• galaxy will not get any fixes or features as it is in deprecated mode waiting for mazer to replace it
• gilt was created alongside molecule, same author and license which likely
• gilt can be used outside molecule, still not sure if anyone is using it outside
• gilt codebase is quite small
• does it make sense to make gilt a part of molecule to avoid extra maintenance burden? Also means dropping one external dependency to a project that seems unmaintained at this moment. For the moment, I created a gilt fork where I fixed most obvious problems and enabled travis ci.
• maybe checking with @retr0h could help us make a decision

If/When galaxy/mazer implement a similar feature as gilt, we can deprecate it.

[retr0h]

@ssbarnea gilt was created b/c galaxy didn't do what we wanted. Gilt is also used outside of Ansible for various other projects we have.

Unfortunately, Gilt isn't on the table to be given to RH and incorporated into Molecule.

[webknjaz]

@ssbarnea your information about ansible-galaxy CLI is deprecated. AFAIK I think it was decided that mazer will actually be merged back into it.

[seandst]

The docs say that "[Mazer] is not intended for use in production environments, nor is it currently intended to replace the ansible-galaxy command-line tool." I have not yet seen evidence to the contrary.

[themr0c]

For this week: quay tags https://github.com/ansible/molecule/issues/1818

[themr0c]

Minutes: https://meetbot.fedoraproject.org/ansible-molecule/2019-03-13/ansible_molecule_working_group.2019-03-13-19.00.html

[decentral1se]

• Prioritise session for 2.21 (kick off the rest to 2.22)
• Release schedule policy - what can we reasonably stick to

[webknjaz]

@decentral1se a wild idea (as an addition to your first point): you might try figuring out whether the current versioning scheme fits the project needs. I personally like semver, but maybe somebody has reasons not to have it. I think a strong versioning scheme makes it more clear what to expect from updates just by looking/comparing versions. For me, currently, it looks like minor (patch?) version update currently can mean anything (in the current implementation) ‒ bugfixes, features and breaking changes combined. What would bumping the major version mean in this case? (3.0)

[decentral1se]

Interesting point @webknjaz. I've thought about this a bit too ...

Another point for the WG - plan for the functional tests:

• We need a realistic plan for dealing with the functional tests. See https://github.com/ansible/molecule/pull/1871#issuecomment-476560666 for background that provoked this. Now that we're over the 2.20 hill (more or less), we have to take solving the functional tests seriously. I believe @webknjaz's original proposed solution was to setup a mock server for all of them. This is the robust solution and I am willing to put some time into it. I don't have much hope in 1) getting test credentials from the many many driver providers without paying lots of $$$ and waiting lots of time 2) Having them reliably run over the network on every CI run without breaking all the time and causing serious maintenance headaches. In short, I don't think functional testing is feasible without a mock server. Let's discuss!

[ssbarnea]

I do not think that mocking would help and it would be close to impossible to chaise all the api changes on the cloud apis. For this reasing I think mocking them would be a waste of time for us as it would be more work and less assurance than molecule works with current versions of foo cloud.

I would rather see making these providers standalone molecule plugins, hosted on their own repositories with their own CI. Those interestes in keeping them could take care of testing it and setting up CI.

[webknjaz]

@ssbarnea one way of solving the problem would be having parametrized tests. So each test would hit some API, but depending on param it'd be routed to the stub server or to the real API. Normal CI would only hit stubs, but some periodic (weekly?) build would hit both stub and real API to keep track of changes.

[retr0h]

As an FYI Molecule already has extensive integration tests for each provider it supports. These tests are already parameterized as well.

py.test test/functional -k <provider>

So to run all docker functional tests:

py.test test/functional -k docker

This thread has become very soapboxy. There is little benefit to stubbing out these functional tests. Run against real resources, and make sure it works.

[themr0c]

Meeting summary

• Agenda https://github.com/ansible/community/issues/427 (gundalow, 19:03:50)
• Travis-CI can now run 10 concurrent jobs, thanks to Red Hat for covering the cost (gundalow, 19:04:42)
• Please shout out on https://github.com/ansible/molecule/issues/1770 if CI still appears slow (gundalow, 19:05:34)

Open Floor (gundalow, 19:06:20)

• review welcome on https://github.com/ansible/molecule/pull/1714 Add Ansible verifier (gundalow, 19:12:27)

Minutes

• https://meetbot.fedoraproject.org/ansible-molecule/2019-03-27/ansible_molecule_working_group.2019-03-27-19.00.html

[decentral1se]

• Do we need a https://github.com/ansible/proposals for Molecule?

Some issues and requests are starting to feel like quite wide reaching issues which can bring a lot of improvement but will need to be agreed upon by a wider group of users of molecule. We might consider creating a more visible space for these discussions besides the issue tracker (too much noise). It would also be good to give some form / template to what needs to be taken into consideration (backwards compat etc.).

[gundalow]

Regarding proposals, I agree that they are a special case, on the he flip side ansible/proposals hasn't ever worked that well. How about if using the power of GitHub issue templates we can create a form for proposals, which will auto add label/proposals. So we still use ansible/molecule/issues though with a template and labels for searching? I can draft a PR for this shortly.

[gundalow]

Proposal template created in https://github.com/ansible/molecule/pull/1905. Discussion welcome in the PR.

[webknjaz]

Please discuss whether to proceed with the publishing of v2.20.post0 via https://github.com/ansible/molecule/issues/1922 / https://github.com/ansible/molecule/pull/1925

@gundalow suggested that we could just publish v2.21 as usual instead.

---

RESOLVED: It's been decided to publish v2.21.

[decentral1se]

I wish I had some more positive to bring to this agenda but I must say that we need to discuss the underlying causes of https://github.com/ansible/molecule/pull/1919. I would like to propose a single issue where we identify the remaining areas of the project code base where these "style debates" can happen and use tooling to implement policy. I find this to be quite urgent as it is IMHO becoming an issue between contributors and maintainers: especially where recent Github access constraints have been implemented bringing a new power dynamic where some must continue to argue their position and others must be "convinced".

Seems to be dealt with.

[tima]

Also can we please please please see about merging https://github.com/ansible/molecule/pull/1714? please?

[decentral1se]

Molecule can now speak Hetzner Cloud! Please help me by reviewing this PR!

https://github.com/ansible/molecule/pull/1811

Also, the CI is broken and we need to decide on how to fix that. I've started the discussion:

https://github.com/ansible/molecule/issues/2006

[gundalow]

Proposal Make Ansible default verifier now that https://github.com/ansible/molecule/pull/1714 is merged

[gundalow]

Add a CLI command for collections initialization https://github.com/ansible/molecule/pull/2014

[gundalow]

testinfra PR to use ansible-runner https://github.com/philpep/testinfra/pull/432 <-- needs testing with Molecule

[decentral1se]

• In order to push the new Ansible verifier out on a release, we need to supply documentation for it. This is detailed in https://github.com/ansible/molecule/issues/2013#issuecomment-488392662. Who can pick this up?
• How can we get back to matching releases with the Project boards (https://github.com/ansible/molecule/projects) and try to move towards having some transparency on when we make a release. Current status: 2.20.1 is out in the wild and we're more or less ad-hoc adding issues/PRs to the 2.21 project ...

[decentral1se]

Minutes for 2019-05-15 meeting

https://meetbot.fedoraproject.org/ansible-molecule/2019-05-15/ansible_molecule_working_group.2019-05-15-19.05.html

[decentral1se]

• What's left to make a next release / pre-release (/cc @jtanner)
  
  
  
  • PyPi access?
  
  
  • Changelog issues?
  
  
  • Any blocking bugs / issues / etc?
  
  

[ssbarnea]

@gundalow Apparently me and @decentral1se are not able to change channel topic. Found it today when I wanted to advertise the usage survey from https://github.com/pycontribs/pytest-molecule/issues/41

[decentral1se]

Current status of the project, or, Molecule needs YOU:

• Update wiki to reflect status of WG
• Tracking progress for Molecule 3.0 via a new project board over at https://github.com/ansible/molecule/projects/6
• Left this comment ;) If you have questions on how to get invovled, join us on IRC!

[MarkusTeufelberger]

Any updates on https://github.com/ansible/molecule/issues/1548 (which is "locked as resolved")?

[gundalow]

I've disabled DCO for Molecule, happy Automating!

[ssbarnea]

I would like to propose @sshnaidm as core to molecule. He wrote the podman drivers, uses molecule at work and has a good track of making good code reviews (some would argue as too detailed). This should help us avoid cases where no reviewers are available. Please user thumbs to vote.