Community: Changes impacting Committers/Contributors

[gundalow]

Every so often the Ansible Core Team change parts of the development process. Some of these changes directly impact how people with Commit powers should deal with Issues and PRs. We could improve how we communicate these changes, also given that the Core team is much larger than a few years ago, and across four continents we can't assume that everyone "just knows".

Some of these changes maybe of interest to people developing (though without commit powers)

Examples

• Changes/reminders about freeze dates so people know what can be merged
• Introduction and changes in changelogs
• Defined process for creating backlogs
• Changes to CI
• Changes to Ansibot workflow

Possible Implementation

1. Process/workflow $Foo is changed
2. PR raised to document $Foo
3. $Foo is mentioned in next IRC meeting, this also forms part of the docs review to ensure we've done a decent job explaining the what/why
4. Chronological list of changes is updated
5. Announce $foo via GitHub/Mail List/What?

Questions

1. Do people think this is needed?
2. How would people like to be notified of changes. Need to ensure these notifications don't get lost in the sea of notifications via GitHub and ansible-devel mailing list
3. What else has changed over the past year that's come as a surprise

Should be added to main docs, summary in CommitterPage

See other discussions in:

• https://github.com/ansible/community/issues/211#issuecomment-316879137

[gundalow]

@agaffney @amenonsen @caphrim007 @dagwieers @defionscode @emonty @jhawkesworth @mscherer @resmo @Shrews @trondhindenes @willthames @yungezz @zikalino Hello lovely humans. As people with commit powers I'd welcome your ideas and feedback on this, such as what communication issues have you faced?

Thanks in advance,
Gundalow

[Shrews]

This is quite timely. I was just lamenting the other day about how I can't keep up with changes to the workflow. For example, the changes to the changelog format caught me by surprise. Also, having not backported anything in a while, the change from direct cherrypicking to a stable branch to now using PRs was something I was unaware of (thankfully I asked first!).

I think a mailing list or one central document we can refer to is a great idea. I soooo often miss things in IRC because I just can't pay attention to all the channels I'm in sufficiently.

[agaffney]

It's definitely a good idea to improve the way these things are communicated. We should probably not rely on IRC for this, as even people like me that are always connected don't go back and read everything that happens, even in a relatively low traffic channel like #ansible-devel.

Some of these changes have been noted in emails to the ansible-devel mailing list, and I've found that's a good way (for me) to be informed about these changes. However, the fact that the ansible-devel ML is ~60% user questions that don't belong, ~30% actual development questions, and ~10% devel-related announcements makes it not exactly ideal.

[caphrim007]

I also am not aware of all the many changes. Which means I'm very likely doing something incorrectly these days.

like @agaffney I also don't hang out on IRC all day long, so some form of communication that can be received and read "when I am available" works for me. Email is my path of least resistence provided that there is something that I can filter it by in my client so it doesn't get lost in the ether.

My last "oh...ok" moment was when I recently went to deprecate a module and noticed that 3 or 4 new keys were now required and I had to go find (in the source) what they were as I wasn't aware of a docs article (whether there is one or not).

[gundalow]

How would you imagine this looks like, a history page with a heading per change, newest at top with links to other relevant documentation and discussion?

[caphrim007]

specifically when I was going about the module deprecation work, I started with google. "ansible deprecate module" did not raise any links that were relevant, so I started looking in github for pre-existing modules that started with an "_" and then a round of 5-10 pushes to a PR before shippable came back green.

While I've tried to reproduce what shippable does on my client side (so I can PR knowing it will be green) every time I've tried it, the tests locally (my box) always say things are cool and then shippable finds the "real" problems.

Historically I haven't needed a "what's new" feed. My day-to-day is usually quite "eyes-off" what happens in core, and it's not until something is not working as I expect that I begin to go mousing around for answers.

For a long time though I've been operating under the premise that "core generally doesn't break my work" and so I guess you could say I've gotten comfortable not needing to know.

But I admit, that's not a good excuse.

If there is a landing page made that is the chosen place to put this, I will certainly make use of it. I use RSS (in the form of Feedly) for updates, but I would be fine with email that is communicated out too.

[agaffney]

I think that a combination of what @gundalow suggests with a "history page" and the email announcements to ansible-devel for new items would probably work well.

[amenonsen]

I think that a combination of what @gundalow suggests with a "history
page" and the email announcements to ansible-devel for new items
would probably work well.

Email announcements and a history page would work for me too.

[resmo]

I have minimal concerns that a "history" page gets "cluttered" very quickly. And I see no benefits in seeing the "full change history" of the processes.

I would suggest to "mail to devel ML about what changed" and the doc page should only reflect current status/workflow (without history) for simplicity.

[dagwieers]

When I proposed a Newsletter I was looking for a means to create a larger contributor community that we currently lack. Something earlier or newer contributors can easily tap into. Easy to read with short articles that point to the relevant information we already have. Something people can skim over to select what they like. A Newsletter that would be send once every quarter or every 2 months, not more often than that.

Because we already have IRC, and a mailinglist for us. I don't think we need another technical solution for the existing active contributors, but we could use something that both celebrates the Ansible dev community, as well as informs them of new things that are coming related to developing modules, plugins, etc.

That's what https://github.com/ansible/community/issues/211#issuecomment-316879137 was about anyway.

[pilou-]

As one without commit powers:

Do people think this is needed?

Yes

How would people like to be notified of changes.

Newsletter - I like dagwieers proposal ! - and main doc.

[gundalow]

To hijack my own thread, can you please ensure you all have GitHub 2FA on otherwise "shortly" you will no longer have commit rights.

[resmo]

@gundalow I would even "enforce" it nowadays.

[gundalow]

@resmo once everyone does we will do. Problem is if we enable it now GitHub will kick some of you off the group.

[gundalow]

To move the discussions back on topic.

Thank you for your feedback so far. This confirms, as I thought, this is a real issue, and there are some examples that I hadn't considered.

I'll add this to the agenda for Contributors Summit and based on feedback I've got a few ideas that I believe meet everyone's concerns.

If you have any other thoughts please do add them here.

We can also use this issue to track items that need communicating, so we don't loose track of them. You may have noticed that I've referenced this issue from other issues that would impact developer and comitter workflow.

If you spot such issues please do mention them here, so we have a set of topics for our first update.

[gundalow]

Important commiter notice
Beta 1 of Ansible 2.7.0 has been released: https://groups.google.com/forum/#!topic/ansible-devel/gsOLo91Lr-Q

That also means that:

• stable-2.7 has been branched. Submit backport PRs to get things merged there. (You can use the cherry_picker script from pypi to aid in that if you want).
• There should be no more new features to stable-2.7

[gundalow]

Contributor Summit agenda (subject to change) has been created https://etherpad.openstack.org/p/ansible-summit-october-2018

In addition individual Etherpad pages have been created for each breakout session.

Please have a look and come armed with your ideas.

Comments welcome, especially if you can't attend in person, if you have other general feedback and would like more of a discussion please feel free to ping me on IRC

[gundalow]

Ansible to adopt molecule and ansible-lint projects, https://groups.google.com/forum/m/#!topic/ansible-project/ehrb6AEptzA

[gundalow]

We have two new Working Groups

Docs

• #ansible-docs
• https://github.com/ansible/community/wiki/Docs
  I think this would be of interest to everybody

Linode

• #ansible-Linode
• https://github.com/ansible/community/wiki/Linode

[gundalow]

DING DING DING. Week commencing 29th October we will move ansible-lint and molecule to their new home under github.com/ansible.

Please see https://goo.gl/KH3jEM for details of what will and will not change. Once again massive thanks to @willthames for ansible-lint and Cisco, specifically John Dewey, for their work on molecule.

[gundalow]

ansible-lint & Molecule

ansible-lint & Molecule have been moved to github.com/ansible. Using GitHub Milestones to track what's next for each . For Molecule this is adding Travis-CI. #ansible-molecule

Contributor Experience Meeting

What time works for you?

https://github.com/ansible/community/issues/391

[webknjaz]

Removed discourse and moved to https://github.com/ansible/community/issues/396 - Gundalow

[gundalow]

Draft Ansible 2.8 Roadmap and dates https://github.com/ansible/ansible/blob/devel/docs/docsite/rst/roadmap/ROADMAP_2_8.rst

[gundalow]

Details of the big PR review session taking place on Thursday 29th November, see https://github.com/ansible/community/issues/407 for details. Hope to see you there (add a comment if you plan to make it)

[gundalow]

CI UPDATE Modules' DOCUMENTATION.author have been checked and 250 modules updated to correctly state the author. CI now enforces the format author: Name (@handle), see https://github.com/ansible/community/issues/406 for background

[gundalow]

CI UPDATE Stricter checks for DOCUMENTATION.{option}.type is now stricter to be inline with argspec.

https://github.com/ansible/ansible/blob/devel/test/sanity/validate-modules/schema.py defines this list as:

'type': Any(None, 'bits', 'bool', 'bytes', 'dict', 'float', 'int', 'json', 'jsonarg', 'list', 'path', 'raw', 'str')

[caphrim007]

@gundalow thanks for mentioning this. it bit us yesterday

[dagwieers]

There are three recent changes to the module documentation worth mentioning here:

• Add a "seealso" section to the module docs -- https://github.com/ansible/ansible/pull/45949
  Starting with the devel branch (v2.8) your module documentation can include a seealso: section. This is a list of dictionaries with a very specific syntax to allow for references to similar or related Ansible modules, but also links to e.g. documentation sections, internal notes from the community wiki or upstream documentation See: Module format and documentation for details.

• Remove older version_added information from docs -- https://github.com/ansible/ansible/pull/49845
  The documentation indicates what modules and parameters were added recently, but the older this information becomes the less useful it is to our users. We recently changed the threshold for showing this information from 1.3 to 2.0. So anything related to versions older than v2.0 will no longer be reflected in the Ansible v2.8 documentation and be considered "has been available since ever".

• Show parameter types -- https://github.com/ansible/ansible/pull/49966
  Starting with the devel branch (v2.8) the module documentation will display the parameter types along the module's parameter listing, but only if this information is present from the DOCUMENTATION section of the module. If this information is missing you will find it replaced with a single dash. If you want a better experience for your users, please update your module's documentation with this information.

You can see what this new documentation looks like from these examples:

• https://docs.ansible.com/ansible/devel/modules/xml_module.html
• https://docs.ansible.com/ansible/devel/modules/win_copy_module.html
• https://docs.ansible.com/ansible/devel/modules/aci_tenant_module.html

We hope this offers a better user experience, especially for people ending up in module docs from search engines. Feedback and ideas are welcome at the Documentation WG.

[gundalow]

Module docs fragments have moved to lib/ansible/plugins/docs_fragments in Ansible 2.8, see https://github.com/ansible/ansible/pull/50172 for further info.

[gundalow]

missing_required_lib

Start to get modules to use the shared missing_required_lib error message. This error message builds a common error message in the case of an error when importing a required module. It includes information such as:

• The module name
• The host it is required on
• An optional reason if the module wishes to add this
• The Python executable that ran the module
  https://github.com/ansible/ansible/pull/51787

module.fail_json(msg=missing_required_lib("pexpect"),exception=PEXPECT_IMP_ERR)

Will be merged shortly. Docs forth comming as will converting more modules to use this structure

[gundalow]

2019-03-14 Alpha 1 Core freeze

No new features to support:core code. Includes no new options to existing Core modules

2019-03-21 Beta 1 Feature freeze

No new functionality (including modules/plugins) to any code

Other dates listed in https://github.com/ansible/ansible/blob/devel/docs/docsite/rst/roadmap/ROADMAP_2_8.rst

[gundalow]

Ansible 2.8 Core & Community freeze dates have been pushed back a week

The Final Release date hasn't changed. See https://github.com/ansible/ansible/pull/53832/files for diff. Please be pragmatic in what you do with this extra week :)

[gundalow]

_facts vs _info modules and ansible_facts

If a module is named <something>_facts, it should be because its main purpose is returning ansible_facts. Do not name modules that do not do this with _facts.
Modules that query/return general information (and not ansible_facts) should be named _info.

Docs will be updated in https://github.com/ansible/ansible/pull/54182

Please use https://github.com/ansible/ansible/issues/54280 for any discussion.

[zikalino]

@gundalow I wonder what should be our action regarding facts -> info change.
in fact we should rename all azure_rm_..._facts modules to azure_rm_..._info
so, there are several questions:

(1) we are still planning to merge a few missing azure_rm_*_facts modules before freeze. should we merge them as _facts or use new _info convention?

(2) there are several new azure_rm_..._facts modules in devel. should we just rename them before freeze, so we won't have to do that later?

(3) and finally if we decide to to (1) and (2), should we attempt to rename / create new _info modules for existing facts modules before freeze to keep everything consistent?

In fact, i really like the idea of introducing _info modules, it gives us opportunity to migrate all the facts modules into new info modules and make them match all the guidelines we worked out during last year or so. currently azure facts modules are quite inconsistent. some of them return ansible_facts and some don't, also they use very different format. so this is real opportunity to clean it up.

the only question is, should we try to fully migrate before freeze, and have everything in order already in 2.8, or just to start introducing _info modules after 2.8.
I would like to either do (1), (2) and (3) at once or just release things in old way and do all the changes after 2.8.

[Andersson007]

@gundalow , could you please give me a couple of examples of _info modules?
I couldn't have found any.
And is new postgresql_facts module actually _facts or _info type?
Seems it returns a lot of facts that we might use to change runtime playbook's behavior

[gundalow]

Ansible 2.8.0 delay

As some of you may have noticed, Ansible-2.8.0-alpha1 has been
delayed. It was scheduled for last week but we have one important
feature that we're still working on. To give it a chance to be
finished up, we've slipped Alpha by two weeks (now due out on
Thursday, next week) and slipped final by one week.

The new schedule there will be:

2019-03-28 Alpha 1 Core freeze
2019-04-04 Beta 1 Feature freeze
2019-04-18 Release Candidate 1
2019-04-25 Release Candidate 2 (if needed)
2019-04-02 Release Candidate 3 (if needed)
2019-05-09 Release

As always the /devel/ docs contain the latest ROADMAP

[gundalow]

@zikalino and @Andersson007 Great questions, thank you. I've created https://github.com/ansible/ansible/issues/54280 for _facts vs _info discussion and track what else we need to do.

Let's keep this issue (https://github.com/ansible/community/issues/346) to be just the announcements only to avoid too many emails and people possibly unsubscribing.

[gundalow]

Ansible 2.8.0 support:core features are frozen

All Ansible 2.8 Alpha 1 was released yesterday, this means No more features for Support:core parts of the codebase.

Community feature freeze is Thursday 11th April. After this point no new modules, plugins, or changes to those modules' options.

The new schedule there will be:

2019-03-28 Alpha 1 Core freeze
2019-04-04 Beta 1 Feature freeze
2019-04-18 Release Candidate 1
2019-04-25 Release Candidate 2 (if needed)
2019-04-02 Release Candidate 3 (if needed)
2019-05-09 Release

As always the /devel/ docs contain the latest ROADMAP

[gundalow]

stable-2.8 has been branched

Hi, happy to announce that stable-2.8 has been branched.

• devel will become Ansible 2.9 (release date unknown), and is open for new features
• Bug fixes will need to be backported into stable-2.8 as necessary
• Changelogs are suggested for all PRs and required for all backports (apart from updating tests or minor docs changes)

See Changelog & Backport process for more info

[gundalow]

Ansible Fest Atlanta

As I hope you've seen, we've announced Ansible Fest Atlanta!

• Contributors Summit is Monday 23 September and is FREE
• Ansible Fest Atlanta is happening Tue & Wed 24-25 September
• NEW Contributor Central happening on Tue & Wed
• If your company is interested in sponsoring Ansible Fest, have a look at the sponsorship prospectus
• Call for Speakers is open now
• Tickets will go on sale in May after Red Hat Summit, which will also allow you to book hotel (and possibly Contributor Summit tickets, more details to follow).

Contributor Summit

• Tell us what you'd like to see at Contributors Summit via the Atlanta Etherpad

Can't attend in person

As always, the event will be available via video conferencing, and IRC, so you can join for as much or little as you want..
Please import by URL atlanta-contributors-summit.ics (please avoid download and add, as you will not receive updates).

NEW Contributors Central

This year we want to do something different during the main days of Ansible Fest (Tue & Wed).
In particular, we want to:

• Answer questions, connect the community at all levels (newbies, users/customers, and contributors) to Ansible
• Recruit new contributors
• Build/solidify relationships with (and among) existing contributors
• Give people that don't use IRC (#ansible-devel) a place to talk

So we are launching Contributors Central and would love to get your thoughts on it.

• Does this sound like a good way to find and engage new contributors?
• What else could do to make it a success?
• If you are attending Ansible Fest would you be able to help run a session?

To help plan this we've created an Atlanta Planning Doc please add comments or join us in #ansible-community

Summary

• Save the date Monday 23rd -> Wednesday 25th September
• Let us know how we can make Contributors Summit even better, and what you want to get from it
• Tell us what you'd like to see at Contributors Summit via the Atlanta Etherpad
• Help us build our first Contributor Central event

[phips]

Hello all!

Sign up for Contributor Summit has now been rolled into the main AnsibleFest sign up page. So feel free to go ahead and get your name down to attend. If you'd already signed up to AnsibleFest, you'll be receiving a separate email detailing how to add Contributor Summit to your ticket.

CS is free to attend, but numbers are limited. Hopefully lots of you will be able to attend onsite again – but of course, if you're unable to join us in Atlanta, we will be popping up Bluejeans for remote attendance.

See you in September!

[gundalow]

Ansible Fest: Contributors Summit

Just a reminder that Contributors Summit is happening on Monday 23rd September and will be streamed online and you can make your voice heard via IRC.

When is this in my timezone?

To see what we plan to discuss see the top level Agenda
From there you can see the various breakout sessions.
A lot of the morning session will be around Collections

Call to Action!

Even if you can only make a small part of the day, or maybe can't attend at all, there are still ways take part:

• Add your thoughts, ideas and questions to the Breakout Agenda
• Vote on what you'd like to be discussed (ie add +yourname)

Background Reading

A large part of Monday Morning will be around "Ansible Collections", to get up to speed on what the current thoughts are, the following links will help you:

Blog: Thoughts on restructuring the Ansible Project
Blog: The future of Ansible Content Delivery
Webinar: Collections and the future of how Ansible Content is handled
Docs: Collections Tech Preview Guide

NEW: Community Central (drop in space)

At AnsibleFest 2019 we are excited to launch Community Central, a gathering place and information hub for Ansible contributors. This takes place 10-3pm each day at AnsibeFest.

Community Central builds on the success of Ask an Expert, with a focus on contributors rather than users.

Community Central will host informal talks and coding sessions on the main conference days. Members of the Ansible Engineering team and the various Ansible Working Groups will be there.

It's a great place to discover how you can contribute to Ansible, meet other contributors, or polish your latest Ansible pull request. If you already contribute to Ansible, or would like to find out how to get involved, this is the place for you!

Feel free to come along and say hello, or even bring your laptop and we can help you through any part of the contributor or development process. Keep an eye on the Agenda Builder to see what's happening.

Questions?

Any questions, feel free to ping me

Email: [email protected]
Freenode: #ansible-community
Matrix: ansible-community
GitHub: @gundalow
Twitter: @the_gundalow

[gundalow]

Ansible 2.9.0 dates have slipped, see the diff

2019-10-31 Release of 2.9.0

As always you can see the official dates at ROADMAPS

[gundalow]

Stricter checks for Module DOCUMENATION.return

Thanks to @felixfontein not use adding this test, though also for fixing over 100 modules.
See https://github.com/ansible/ansible/pull/63411 for more details

[ssbarnea]

Better to delay a major release than breaking stuff. Every time I rushed a release I regretted it.

In case anyone is interested molecule master code is running CI and pasing with Ansible 2.9rc4.

[felixfontein]

Module documentation: element type for lists

Module docs can be improved by adding elements: next to type: list for both option documentation (possible in devel and stable-2.9; see ansible/ansible#50335) and return value documentation (possible in devel and in post-2.9.0 stable-2.9; see ansible/ansible#62929).

To see this in action, search for contact or identifier here.

[gundalow]

Ansible 2.9.0 has been released

Porting Guide

Changelog

Future maintenance releases will occur approximately every 3 weeks. So expect the next one around 2019-11-21.

Amazing work by everybody, thank you!

[gundalow]

Next PR review day is Thursday 14th November from 1400UTC til late
What's this in my Timezone?

See you in #ansible-community on Freenode IRC
We #407 for more info

[felixfontein]

Please note that a couple of PRs have been merged which improve sanity checking for modules. Check the updated ignore.txt for the modules you maintain / are interested in, and watch out when merging something the next days :)

The improvements are:

• for type='list', elements is now required (ansible/ansible#66386) and also checked against elements in documentation (ansible/ansible#66385);
• the argument spec is validated with a schema (ansible/ansible#65747).

[felixfontein]

There are some more new sanity tests for modules. Make sure to check the current ignore.txt for the modules you maintain / care about!

• ansible/ansible#66898 make sure modules don't support state=list or state=info
• ansible/ansible#66961 basic checks for mutually_exclusive, required_if, required_by, required_together and required_one_of
• ansible/ansible#67243 some errors previously ignored because of special code for handling add_file_common_arguments=True are now reported

If your modules already live in a collection outside the Ansible main repo, make sure to also check them with devel's ansible-test, as it catches a lot more errors than the one coming with Ansible 2.9.

[gundalow]

Collections: Important changes to devel

Very soon we will start the mass migration of content (modules & plugins) out of ansible/ansible:devel into Collections. During this time the devel branch will be frozen for a while, followed by a mass DELETION (git rm) of files.

Please all take the time to read through https://github.com/ansible-collections/overview/blob/master/README.rst

Any feedback or further questions please ping me on IRC or email me [email protected]

Please also help spread the word by retweeting/upvoting/sharing the various bits of Communication we've already done

[gundalow]

Ansible Contributors Summit EUROPE

What

Ansible Contributor Summit is a one day event especially for community contributors to meet with each other, as well as Ansible development teams (Core, AWX/Tower, Galaxy) in person, to discuss important issues in the community and help shape the future of Ansible. For those unable to make it to Gothenburg, we welcome you to join virtually for part/all of the day. Online participation and streaming details to follow closer to the summit.

Where

Göteborg, Sweden

When

• Contributors Summit: Sunday, 29th March
• Ansible Hackathon: Monday & Tuesday 30 & 31st March
• FOSS-North: Monday & Tuesday 30 & 31st March

How

• If you plan to attend in person, please register on Eventbrite
• If you plan to attend the hackathon or any of FOSS-North please register
• If you plan to attend virtually details will be added to the Etherpad the day before

Agenda

• Please add suggested topics to the Etherpad
  For more info please see the announcement.

[gundalow]

All,
A reminder that our Virtual Contributors Summit is taking place on Sunday 29th March at 1100 UTC (What's this in my timezone)

Hope you can make it, please add your name to the bottom of the Etherpad

[gundalow]

Changes impacting Collections

If you a maintainer for a collection, please subscribe (button on right hand side) to https://github.com/ansible-collections/overview/issues/45

[gundalow]

Changelog process for collections

Feedback needed

We have a continuing need for changelogs (or release notes) to cover:

• ansible/ansible - currently has a working solution using changelog fragments
• collections - two options mentioned so far are reno and towncrier
• Ansible Community Distribution (ACD) - a bundled package that includes the trimmed down ansible/ansible and a set of curated collections that will mimic the current user experience of Ansible with all batteries included (2.9 level of content). ACD has no proposed solution yet, but whatever is used by collections has to provide fragments that can be drawn up into an ACD changelog/release note.

There are various proposals outlined https://github.com/ansible-collections/overview/issues/18 Please comment and vote for preferred solution.

[gundalow]

Dropping ANSIBLE_METADATA

In Ansible 2.9 and earlier we had the concept of ANSIBLE_METADATA, to express support level and deprecation. With the move to Collections, and support being defined if a Collection is in Galaxy vs Automation Hub, ANSIBLE_METADATA is no longer needed.See https://github.com/ansible-collections/overview/issues/57 for background

https://github.com/ansible/ansible/pull/69454 will cause ansible-test to error if ANSIBLE_METADATA is found in modules.

Call to action

• Please bulk remove the ANSIBLE_METADATA section from all modules in your collection

[gundalow]

Next Virtual Contributors Summit

We're going to set the date for our next Contributor Summit soon, please let us know what works best for you! https://doodle.com/poll/d8ttii3zpv5a2ic5

[gundalow]

Next Ansible PR review day (Collections)

The next PR day will be Wednesday 17th June 2020 from 1400UTC

See https://github.com/ansible/community/issues/407#issuecomment-640223103 for further info. Hope to see you there!

[felixfontein]

validate-modules change in devel

Until now validate-modules treated an unset default value for bool options exactly the same as default=False. This has been fixed in ansible/ansible#72699 and might cause some sanity tests to fail.