Awx: Smart Inventory - Searching on host variables.

[wtcross]

ISSUE TYPE

• Feature Idea

COMPONENT NAME

• API
  
  
  
  • UI
  
  

SUMMARY

As an AWX user I should be able to create a smart inventory smart host filter based on host vars associated with a host. The behavior of this feature should be the same as how a user can create a smart host filter based on ansible_facts.

STEPS TO REPRODUCE

A dictionary called something like host_vars should be created that can be searched like this with a smart host filter:

host_vars.my_cool_hostvar:foo

ACTUAL RESULTS

I would expect this to only result in showing hosts that have a host variable my_cool_hostvar set to foo.

[wtcross]

The motivation of this feature request it to be able to create smart inventories based on variables created via dynamic inventory scripts. Think integration with Puppet DB or the ServiceNow CMDB.

[AlanCoding]

I don't see what the shortcoming is.

GET /api/v2/hosts/?host_filter=variables__icontains=foobar

When I do this, I see my host that I put the foobar variable into.

https://gist.github.com/AlanCoding/fc54f5557cc1ae19fdbf3582ac7d0de5

[chrismeyersfsu]

This would absolutely be an awesome additional; applying the json host_filter syntax to host vars.

Three things to consider.

1. We added a feature in tower like 2.2 to save comments in host vars. So we store them as strings basically. This is easily gotten around by continuing to store them the way we do currently and a new, shadow, json field.
2. Database indexing size considerations (see below).
3. Implicit host vars that come from groups and inventory.
   (3a.) differentiating from the want to search inclusively through the inv+group+host vars or exclusively, only the host vars.
   (3b.) Being able to support the inclusive search.

---

GIN index on 100,000 records w/ 2-4 KB average JSON size (typical ansible gather_facts payload).

Hosts | Table Size | Index Size
-- | -- | --
100,000 | 10 MB | 2.483 GB

[wtcross]

@AlanCoding The issue with your example is that it's doing a simple search on a single word. A user will want to search for a specific variable-value combination. A good example is something as simple as the ec2_state host variable created by the ec2 inventory source.

ec2_state: running

You'd have to search for both the key and value. On top of that I'm not sure how good of an idea it is to allow searching for variables using icontains when the value of one variable could contain the target text. A comment could even include the searched text.

[wtcross]

@chrismeyersfsu Those are three very good points. I like the idea of the json column. Whatever code is running the inventory script could simply strip out the comments and then store the json in the corresponding column for that host. This would make it host-specific and remove any overrides from group/inventory variables, but I think that might be ok. I see the primary use case as being used to search on variables returned by the inventory source itself.

Thoughts?

[AlanCoding]

@wtcross You are right. If the variables come from an inventory import, then we can guarantee they will be JSON (again, assuming a user did not modify them), and it appears that the host filter rejects any ", so there would be no way to even match a shallow key: value combination.

[rahst12]

A couple things -

1. In the Smart Inventory, the "Key" button says to use "facts" not "ansible_facts" to access Facts. That should be updated.
2. Variables can already be accessed, but only as the entire json. A primary use case, when using Open Stack, is the following:
   
   
   
   1. Create a Dynamic Inventory of All Hosts, named "All Hosts"
   
   
   2. Create a Smart Inventory, based on "All Hosts, filtering on only "Active" hosts, named "All Active Hosts".
      
      
      
      
      
      1. "variables.openstack__vm_state:active"
      
      
      
   
   

There are many instances in OpenStack where servers might be powered down, suspended, stuck in a rebooting cycle, etc. I need to be able to apply a template to only Active Servers, so I don't see a false-positive failure based upon an inability to SSH to the box.

Link to my StackOverflow Question about this: https://stackoverflow.com/questions/47100910/whats-the-syntax-for-using-facts-in-ansible-tower/47316839#47316839

[mruzicka-redhat]

This appears to be working in the API these days, but not in the UI due to the way that the search is processed. Any chance we might have this available in the UI in the future? We have a large Ansible Tower customer looking for this functionality.

[wenottingham]

The only way it's working in the API is by a free-text search of the variable blob, that will break if the variables are switched between JSON/YAML, and cannot be used to search complex variables (among other things). It's not productizable in its current state.

[rahst12]

This would be great if it were implemented. I'm also using Ansible Tower and this is one of a number of requested features.

[alexander-bauer]

This would be extremely useful for people using the Foreman dynamic inventory. Foreman parameters are exposed in the host variables, and those have a lot of potential for use in smart inventories.

[jasweet]

We've requested this feature as well. Would be nice to see it expedited.

[rahst12]

This appears to be working in the API these days, but not in the UI due to the way that the search is processed. Any chance we might have this available in the UI in the future? We have a large Ansible Tower customer looking for this functionality.

Have you gotten any traction on this. We're also a paying Ansible Tower user looking for this same feature... @mruzicka-redhat

[mruzicka-redhat]

@rahst12 this appears to be on the list, but there is no current ETA. If you have a TAM it would be good to raise this with them too. In either case, it might be a good idea to open a support case to request this addition, if you haven't already. It may help the team prioritize based on customer demand.

[rahst12]

@mruzicka-redhat What is a TAM?

[mruzicka-redhat]

@rahst12 Sorry, out of context and unrelated to this this thread. A TAM is a Technical Account Manager, which is an add on service through Red Hat. Not required, but TAMs (I am one BTW) can help advocate on feature requests (and plenty of other things).

Also an FYI, it seems my earlier comment about it working in the API is not really accurate. This isn't supported and may not return expected results.

[rahst12]

@mruzicka-redhat I have heard of a TAM then... Our sales rep said we need to work with him/her because we've got 7 GitHub issues that would be very very helpful to us if implemented:

Add IP/FQDN to Hosts Tables | https://github.com/ansible/awx/issues/662
UX improvement: Increase visibility of the ad hoc command feature. | https://github.com/ansible/awx/issues/612
Incorrect opacity on SMART INVENTORY and RUN COMMAND buttons in Inventories. | https://github.com/ansible/awx/issues/659
Playbook fails when ssh host key changes | https://github.com/ansible/awx/issues/452
Smart Inventory - Searching on host variables. | https://github.com/ansible/awx/issues/371
Gitlab (self-hosted) SSO | https://github.com/ansible/awx/issues/94
Documentation Updates for Smart Inventory -> Smart Host Filter -> Dynamic Host | https://github.com/ansible/awx/issues/663

I'm trying to get something setup to meet our TAM. A interesting question though - I'm not clear anymore how much of AWX/Tower is contributed to via the general community, and how much is driven by RedHat employees.

[mruzicka-redhat]

Bumping to see if there might be any additional movement on this request. Thank you.

[jagibson]

I'm surprised this doesn't work. It seems like basic functionality for smart inventories IMHO.

[dellarte]

Hello guy, since this feature isn't implemented yet, do you have any idea how can i filter only two hosts (choosed by human) amoung my WMWare dynamic inventory ?

[jasweet]

@dellarte
Your question, as worded, doesn’t seem relevant to this thread. You could use “limit” for what you’ve asked.

[dellarte]

@jasweet
I probably did not express myself well. I have created a Synchronized Vmware Inventory which lists my virtual machines in "hosts" very well.
I added custom variables directly on choosen host through the "details" page of an host in order to be able to filter this host in a smart inventory.
e.g. If I set a "upgradable: true" variable on 2 of my hosts (still available from vmware inventory), i would appreciate i can retrieve only this 2 hosts with a simple filter.
Since this is not implemented yet, i asked if anyone got a tips to do this

[jasweet]

You can build your inventory, using a dynamic script or otherwise, and then
create a playbook that caches the desired host variables as ansible facts.
If using a dynamic inventory script, and such a playbook, and you don't
want to run two separate jobs, you can create a workflow and incorporate
each into the workflow. Then you have a single call to kick it off.

On Tue, Jun 18, 2019, 6:54 AM gg [e-mail] wrote:

@jasweet https://github.com/jasweet
I probably did not express myself well. I have created a Synchronized
Vmware Inventory which lists my virtual machines in "hosts" very well.
I added custom variables directly on choosen host through the "details"
page of an host in order to be able to filter this host in a smart
inventory.
e.g. If I set a "upgradable: true" variable on 2 of my hosts (still
available from vmware inventory), i would appreciate i can retrieve only
this 2 hosts with a simple filter.
Since this is not implemented yet, i asked if anyone got a tips to do this

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ansible/awx/issues/371?email_source=notifications&email_token=AHK2NJW2V76NVBRLSUJJLXTP3DEH3A5CNFSM4D56MZ5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX6EFII#issuecomment-503071393,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHK2NJTLL43ZMEG5YQMD4WDP3DEH3ANCNFSM4D56MZ5A
.

[ableischwitz]

Bumping, is there still any activity for this RFE?

This would greatly improve the usability within bigger environments with lots of hosts reported by a smart-inventory.

[wenottingham]

If/when work starts, you'll see it here.

[yurtesen]

It would be a great feature to include this and also with feature to search inside lists or dictionaries even.
But I had some luck using regex also

variables.regex:"myvariable.*myvariablevalue"

Sure it can match some strange things also if one is not careful.
By the way, is there a way to use double quotes or square brackets in the regex?