Awx: Smart inventories don't contain groups

[freehck]

ISSUE TYPE

• Feature Idea

COMPONENT NAME

• API
• UI

SUMMARY

Smart inventory contains now only hosts, but not their groups. This creates problems for roles that use group names.

ENVIRONMENT

• AWX version: 1.0.6.11
• AWX install method: docker on linux
• Ansible version: 2.5.3
• Operating System: Ubuntu 18.04 LTS

STEPS TO REPRODUCE

• Make an inventory Inv with some number of hosts: Ha, Hb and Hc
• Make a group Gtop
• Make some subgroups Gsub1, Gsub2, ... in Gtop
• Make new smart inventory Smrt with host filter by group Gtop
• Make a job template, and set Smrt as its inventory
• Run this job template

EXPECTED RESULTS

I expected to see groups Gtop, Gsub1, Gsub2 and so on: all the groups from Inv the filtered hosts in the Smrt are members of.

ACTUAL RESULTS

Only hosts are in the Smrt.

[wenottingham]

Smart inventories don't have groups by design; this would be a large change to how they're constructed on the backend.

[freehck]

@wenottingham so what are Smart Inventories need for?
I mean, I can't imagine how anyone uses inventory without groups.

[AlanCoding]

We are making some changes to the inventory script generation for job slicing, and I think this is a good opportunity to implement this request as well. The particular solution I have in mind I have discussed with @wwitzel3, and it comes down to a customized version of this query:

https://github.com/ansible/awx/blob/5910b8c5627d46ad52a41801c6998322973c650e/awx/main/models/inventory.py#L248

Specifically designed for smart inventories, so we don't filter on inventory, but instead host__in=self.hosts, or something like that.

This assumes that a particular implementation is being requested, and I am somewhat unclear on that. Our patch would be designed to returned host-group memberships from _source inventories_, it would not be allowing users to group hosts _inside of the smart inventory_. The issue wording seems to reflect this design, but I don't know that everyone agrees on it.

Also, consider that this patch will have a known limitation. In the case that 2 source inventories had a group of the same name, it would be treated as the same group for the purposes of the smart inventory (very important detail). However, annotating the group-group relationships would be borderline madness. You simply can't delineate _which group is which_, prevent cycles, etc. For these reasons, I think we should omit group-group (child-parent) relationships in this patch.

@wenottingham does this design & stated limitation sound good to you?

[wenottingham]

So it would be a flat hierarchy?

How performant would this be?

[AlanCoding]

Performance is the main technical issue to be addressed, we already know pretty well what the code change would be. There are some existing large inventory examples that we will need to test with this (and be sure that groups are involved).

[jaevans]

Any progress on this change? This is huge issue for us, causing us to need to maintain a number of different manually created inventories to get around the lack of groups in smart inventories. We'd be willing to take a reasonable performance hit to get the feature working.

[wenottingham]

It's not just performance, there are also questions of semantics.

Having a host/playbook behave differently in a smart inventory because it's now getting a different copy of group vars (as mentioned above) would likely be surprising, and there's no good workaround.

[R-0-Y]

It's not _just_ performance, there are also questions of semantics.

Having a host/playbook behave differently in a smart inventory because it's now getting a different copy of group vars (as mentioned above) would likely be surprising, and there's no good workaround.

Hi, and if instead of the group name, we use the group id ? With groups:N insteand of groups.name:group_name
In this case there wouldn't be any problems regarding group name.

Moreover, in our case, we only want a global inventory from our vmware infrastructure with tagged vm. And smart inventory were supposer to get all inventory needed for us. But without group_vars or group available, like some already said here, it's not really usable... :/

[wenottingham]

Hi, and if instead of the group name, we use the group id ? With groups:N insteand of groups.name:group_name
In this case there wouldn't be any problems regarding group name.

Doesn't help - an inventory is manifested as json, with groups containing hosts. You'll still end up with multiple 'groups' with the same name.

[sdktr]

Would be really nice to have group info concatenated into a smart inventory when merging regular inventories.

[vijeshkk]

Hi, and if instead of the group name, we use the group id ? With groups:N insteand of groups.name:group_name
In this case there wouldn't be any problems regarding group name.

Doesn't help - an inventory is manifested as json, with groups containing hosts. You'll still end up with multiple 'groups' with the same name.

I think this applies to hosts as well. bcoz if the host is present in multiple inventories, it still pulls that host to smart inventory, and still manages to avoid duplicates. Shouldn't we be able to apply similar logic for groups?

[vijeshkk]

I think, without having possibility to group hosts and use group_vars, smart inventory doesnt makes sense to most of our production scenarios.