Openshift-ansible: filter_plugin and lookup_plugin directories from the RPM install take precedence over the ones in the git checkout
TASK: [openshift_facts | Verify Ansible version is greater than or equal to 1.9.4 and less than 2.0] *
fatal: [ovs-10.mvdcdev44.us.alcatel-lucent.com] => Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ansible/runner/init.py", line 586, in _executor
exec_rc = self._executor_internal(host, new_stdin)
File "/usr/lib/python2.7/site-packages/ansible/runner/__init__.py", line 789, in _executor_internal
return self._executor_internal_inner(host, self.module_name, self.module_args, inject, port, complex_args=complex_args)
File "/usr/lib/python2.7/site-packages/ansible/runner/__init__.py", line 869, in _executor_internal_inner
if not utils.check_conditional(cond, self.basedir, inject, fail_on_undefined=self.error_on_undefined_vars):
File "/usr/lib/python2.7/site-packages/ansible/utils/__init__.py", line 269, in check_conditional
conditional = template.template(basedir, presented, inject)
File "/usr/lib/python2.7/site-packages/ansible/utils/template.py", line 124, in template
varname = template_from_string(basedir, varname, templatevars, fail_on_undefined)
File "/usr/lib/python2.7/site-packages/ansible/utils/template.py", line 382, in template_from_string
res = jinja2.utils.concat(rf)
File "", line 6, in root
File "/usr/lib/python2.7/site-packages/jinja2/runtime.py", line 153, in resolve
return self.parent[key]
File "/usr/lib/python2.7/site-packages/ansible/utils/template.py", line 205, in **getitem
return template(self.basedir, var, self.vars, fail_on_undefined=self.fail_on_undefined)
File "/usr/lib/python2.7/site-packages/ansible/utils/template.py", line 124, in template
varname = template_from_string(basedir, varname, templatevars, fail_on_undefined)
File "/usr/lib/python2.7/site-packages/ansible/utils/template.py", line 382, in template_from_string
res = jinja2.utils.concat(rf)
File "", line 10, in root
File "/usr/share/ansible_plugins/filter_plugins/oo_filters.py", line 592, in oo_persistent_volumes
kind = hostvars['openshift']['hosted'][component]['storage']['kind']
KeyError: 'storage'
FATAL: all hosts have already failed -- aborting
rushabh268
All 28 comments
Hey @rushabh268 could you post your inventory and could you confirm that you're using the latest code in the master branch? Some conditionals were recently added to the oo_persistent_volumes filter to ensure that the storage key is present prior to accessing but there may still be some issues with it.
abutcher
on 14 Apr 2016
@abutcher - Openshift version 3.1.1.6 Docker version 1.8.2, Ansible version 1.9.4, I was using 3.0.6-71 branch for the ansible code when I saw the issue. I switched to the master branch but still see the same issue. I was able to do the install till day before yesterday. Not sure what broke.
rushabh268
on 14 Apr 2016
@rushabh268 Are the openshift-ansible* rpm packages installed on the system where you're running ansible from? And are you running ansible out of a git checkout? Filters and other components in /usr/share/ansible will take precedence over a local checkout, so if the local checkout is out of sync with rpm managed files then it will create some problems.
abutcher
on 14 Apr 2016
@abutcher I had installed
openshift-ansible noarch 3.0.47-6.git.0.7e39163.el7aos @rhel-7-server-ose-3.1-rpms
openshift-ansible-filter-plugins noarch 3.0.47-6.git.0.7e39163.el7aos @rhel-7-server-ose-3.1-rpms
openshift-ansible-lookup-plugins noarch 3.0.47-6.git.0.7e39163.el7aos @rhel-7-server-ose-3.1-rpms
openshift-ansible-playbooks noarch 3.0.47-6.git.0.7e39163.el7aos @rhel-7-server-ose-3.1-rpms
openshift-ansible-roles noarch 3.0.47-6.git.0.7e39163.el7aos @rhel-7-server-ose-3.1-rpms
Since, there was a issue. I uninstalled these and installed the following:
pm -qa | grep ansible
openshift-ansible-lookup-plugins-3.0.47-1.git.0.4498ce3.el7aos.noarch
ansible-1.9.4-1.el7aos.noarch
openshift-ansible-filter-plugins-3.0.47-1.git.0.4498ce3.el7aos.noarch
openshift-ansible-roles-3.0.47-1.git.0.4498ce3.el7aos.noarch
openshift-ansible-3.0.47-1.git.0.4498ce3.el7aos.noarch
openshift-ansible-playbooks-3.0.47-1.git.0.4498ce3.el7aos.noarch
After that I did the install again but got the same error.
I am running ansible after a git clone of the ansible repo. But, I think after the re-install of the rpms, the local checkout would take preference.
rushabh268
on 14 Apr 2016
@rushabh268 You will need to completely uninstall the RPMs if you are using a local checkout, otherwise the local checkout will not be in sync with the filters being loaded from /usr/share/ansible.
abutcher
on 14 Apr 2016
@abutcher: Will the install work if I remove all the rpms? I think I would need ansible-1.9.4-1.el7aos.noarch if I am not wrong.
rushabh268
on 14 Apr 2016
@rushabh268 yes, only remove the openshift-ansible* rpms.
abutcher
on 14 Apr 2016
@abutcher So, I made it work but the other way. I swapped the playbooks and roles directory in the git checkout directory with the ones in /usr/share/ansible and then ran the install, and it worked like a charm. Not sure, if this is the correct way out! Do you want me to try after removing the rpms so as to narrow the issue?
rushabh268
on 14 Apr 2016
@rushabh268 You'll have to only use the RPM provided playbooks, roles, filters etc or only use the local checkout. If you plan on running from git, then uninstalling the RPMs is the way to go.
abutcher
on 14 Apr 2016
@abutcher So, I install atomic-openshift-utils because that's mentioned in the pre-requisites here and that basically installs all openshift-ansible-* rpms as dependencies.
But, my ansible playbooks need the upstream repo and that's why I clone it before I run the playbook and it used to work till yesterday.
I'll try without the rpms and see how it goes.
rushabh268
on 14 Apr 2016
@rushabh268 I think we'll want to mention the conflict in the documentation, I'll create an issue with openshift-docs linking back to this issue.
abutcher
on 14 Apr 2016
Thanks a lot for your help!
rushabh268
on 14 Apr 2016
Actually I think it makes more sense to document the conflict here in openshift-ansible or come up with some way to test/alert when a user is running locally and has openshift-ansible* packages installed.
abutcher
on 15 Apr 2016
And why should RPM and git playbooks interfere? It makes most sense to avoid that interference. One may want to just try the git code before new RPMs are available. Why mess with the system when running off the git tree?
akostadinov
on 15 Apr 2016
@akostadinov There is probably some configuration / environment variable the user could alter s.t. ansible does not look for filters/modules within /usr/share/ansible when they want to run off the git tree but still have rpms installed.
abutcher
on 15 Apr 2016
^ I believe it looks there by default without any configuration alterations.
abutcher
on 15 Apr 2016
hmm, I see:
-M DIRECTORY, --module-path=DIRECTORY
The DIRECTORY search path to load modules from. The default is
/usr/share/ansible. This can also be set with the ANSIBLE_LIBRARY
environment variable.
But I don't have a reproducer to try how exactly it should be set.
akostadinov
on 15 Apr 2016
Here we go, for each item in the ansible config, we need to add the openshift-ansible checkout
http://docs.ansible.com/ansible/intro_configuration.html#filter-plugins
abutcher
on 15 Apr 2016
Tested using openshift-ansible-filter-plugins-3.0.47-6 and current master. Prepending does not work... I had to completely replace the filter and lookup plugin paths to override.
[defaults]
lookup_plugins = ~/openshift-ansible/lookup_plugins
filter_plugins = ~/openshift-ansible/filter_plugins
We can ship a default ansible.cfg file pre-configured and tell the user to
run from the root of the checkout.
On Apr 15, 2016 3:41 PM, "Andrew Butcher" [email protected] wrote:
Tested using openshift-ansible-filter-plugins-3.0.47-6 and current master.
Prepending does not work... I had to completely replace the filter and
lookup plugin paths to override.[defaults]
lookup_plugins = ~/openshift-ansible/filter_plugins
filter_plugins = ~/openshift-ansible/filter_plugins—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/openshift/openshift-ansible/issues/1750#issuecomment-210611901
detiber
on 15 Apr 2016
@abutcher I have the setup so let me know if you need to verify something
rushabh268
on 15 Apr 2016
If I use ANSIBLE_CONFIG env variable, would that mean I need to override all global configuration options. Or can I only set proper lookup_plugins and filter_plugins options in this config and have all other settings be read from the global config?
It is too bad ansible is not looking for configuraiton file at same dir as the playbook. Why should it look at current dir instead of the playbook root dir? Can we file an feature request for that?
akostadinov
on 20 Apr 2016
@akostadinov As I understand it, you'll need an entirely separate config. I mentioned this to the ansible guys and they seemed to think that adjacent plugins should always be taking precedence. What we're encountering may be a bug but I don't think it'll be fixed for 1.9.
abutcher
on 20 Apr 2016
@akostadinov to be clear, I was suggesting that we could ship an ansible.cfg file in openshift-ansible that would handle this case appropriately.
detiber
on 20 Apr 2016
@detiber , sounds good, what should it contain? Does it accept relative path or user would need to change the above mentioned paths?
Another question (same as above), if one has ansibie.cfg in current dir, would that config invalidate any global config in /etc or will only particular values be overridden?
@abutcher , if the playbook finally starts to support ansible 2, would that fix the issue?
akostadinov
on 20 Apr 2016
@akostadinov the search path for the config is as follows:
- ansible.cfg in current directory
- ~/.ansible.cfg
/etc/ansible.cfg
The first file found in the search path is used exclusively.
The file does allow for the use of relative path, which is what we would use for setting the filter plugin path.
detiber
on 20 Apr 2016
FYI for anybody reading, another way to override settings is env variables:
ANSIBLE_LOOKUP_PLUGINS: openshift-ansible/lookup_plugins
ANSIBLE_FILTER_PLUGINS: openshift-ansible/filter_plugins
This issue has been inactive for quite some time. Please update and reopen this issue if this is still a priority you would like to see action on.
tbielawa
on 15 Nov 2016
Related issues
cgutshal
·
4Comments
dharmit
·
6Comments
outcoldman
·
6Comments
thebithead
·
5Comments
ceagan
·
6Comments
Most helpful comment
We can ship a default ansible.cfg file pre-configured and tell the user to
run from the root of the checkout.
On Apr 15, 2016 3:41 PM, "Andrew Butcher" [email protected] wrote: