Openshift-ansible: Install fails with "the output has been hidden due to the fact that 'no_log: true' was specified for this result"
Description
When I try to install a single-node biy version I get the following error
TASK [openshift_metrics : slurp] ***********************************************
ok: [master.some_domiain.ch -> localhost]
TASK [openshift_metrics : generate htpasswd file for hawkular metrics] *********
fatal: [master.some_domiain.ch -> localhost]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
Version
ansible 2.2.2.0 (also tried with 2.3.0.0-3.el7)
openshift-ansible-3.5.67-1-4-g5b874ad
Steps To Reproduce
- run ansible-playbook openshift-ansible/playbooks/byo/config.yml --inventory hosts
Expected Results
OpenShift to be installed correctly including metrics
ansible-playbook openshift-ansible/playbooks/byo/config.yml --inventory hosts
TASK [openshift_metrics : slurp] ***********************************************
ok: [master.some_domiain.ch -> localhost]
TASK [openshift_metrics : generate htpasswd file for hawkular metrics] *********
fatal: [master.some_domain.ch -> localhost]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
Additional Information
This is the inventory file I used
[OSEv3:children]
masters
nodes
etcd
[OSEv3:vars]
ansible_ssh_user=root
deployment_type=origin
openshift_node_kubelet_args={'max-pods': ['40'], 'image-gc-high-threshold': ['90'], 'image-gc-low-threshold': ['80']}
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/origin/master/htpasswd'}]
openshift_master_htpasswd_users={'chris': 'some_hash'}
openshift_master_default_subdomain=os.some_domain.ch
# Persistent Registry
openshift_hosted_registry_storage_kind=nfs
openshift_hosted_registry_storage_access_modes=['ReadWriteMany']
openshift_hosted_registry_storage_nfs_directory=/exports
openshift_hosted_registry_storage_nfs_options='*(rw,root_squash)'
openshift_hosted_registry_storage_volume_name=registry
openshift_hosted_registry_storage_volume_size=10Gi
# Metrics
openshift_hosted_metrics_deploy=true
openshift_hosted_metrics_storage_kind=nfs
openshift_hosted_metrics_storage_access_modes=['ReadWriteOnce']
openshift_hosted_metrics_storage_nfs_directory=/exports
openshift_hosted_metrics_storage_nfs_options='*(rw,root_squash)'
openshift_hosted_metrics_storage_volume_name=metrics
openshift_hosted_metrics_storage_volume_size=5Gi
openshift_hosted_metrics_storage_host=master.some_domain.ch
openshift_hosted_metrics_public_url=https://hawkular-metrics.os.some_domain.ch/hawkular/metrics
# Logging
#openshift_hosted_logging_deploy=true
#openshift_hosted_logging_storage_kind=nfs
#openshift_hosted_logging_storage_access_modes=['ReadWriteOnce']
#openshift_hosted_logging_storage_nfs_directory=/exports
#openshift_hosted_logging_storage_nfs_options='*(rw,root_squash)'
#openshift_hosted_logging_storage_volume_name=logging
#openshift_hosted_logging_storage_volume_size=1Gi
#openshift_hosted_logging_hostname=logging.os.some_domain.ch
#openshift_hosted_logging_elasticsearch_cluster_size=1
[masters]
master.some_domain.ch openshift_ip=10.135.25.190 openshift_hostname=master openshift_public_ip=46.xxx.xxx.xxx openshift_public_hostname=master.some_domain.ch openshift_schedulable=True openshift_node_labels="{'region': 'infra'}"
[nodes]
master.some_domain.ch openshift_ip=10.135.25.190 openshift_hostname=master openshift_public_ip=46.xxx.xxx.xxx openshift_public_hostname=master.some_domain.ch
[etcd]
master.some_domain.ch openshift_ip=10.135.25.190 openshift_hostname=master openshift_public_ip=46.xxx.xxx.xxx openshift_public_hostname=master.some_domain.ch
[nfs]
master.some_domain.ch openshift_ip=10.135.25.190 openshift_hostname=master openshift_public_ip=46.xxx.xxx.xxx openshift_public_hostname=master.some_domain.ch
sandrich
All 28 comments
Nevermind. Fixed by downgrading ansible to 2.2.1
sandrich
on 8 May 2017
re-opened as we need to fix anything that happens in 2.3 ASAP.
Can you verify though that you have python-passlib installed on your control host?
sdodson
on 8 May 2017
I only got the following installed
[root@master ~]# rpm -qa | grep passlib
python2-passlib-1.6.5-1.el7.noarch
@sdodson would it make sense to have the openshift_metrics role have an assert that the rpm exists on the control host before executing?
ewolinetz
on 9 May 2017
yeah, but it looks like @un1x86 has it installed already so not sure why it's failing.
sdodson
on 9 May 2017
@un1x86 just to verify, are you installing from your 'master' host, or is that your master node that you ran the rpm query from?
ewolinetz
on 9 May 2017
@sdodson @ewolinetz getting the same result with openshift-ansible-3.6.61-1 and have python2-passlib-1.7.0-4.el7.noarch installed. Any other requirements?
etsauer
on 10 May 2017
@ewolinetz I am installing it from the "master" host and I run the rpm query on the master host.
sandrich
on 10 May 2017
I'm not sure if its the difference between python2-passlib and python-passlib, but on my control host I have python-passlib-1.6.2-2.el7.noarch installed and I don't see this problem.
ewolinetz
on 10 May 2017
python2-passlib is automatically chosen when installing ansible from the openshift-origin repository.
sandrich
on 10 May 2017
@un1x86 can you update roles/openshift_metrics/tasks/generate_hawkular_certificates.yaml to remove this line [1] and rerun your play with a more verbose output?
ewolinetz
on 10 May 2017
@ewolinetz I'm now on the same version of python-passlib as you are, and it seems to have gotten me past this issue. It turns out that versions of that rpm are available in multiple channels. In my case, it was coming from openstack-newton. I ran a yum remove python2-passlib and then ran yum install python-passlib --disablerepo openstack-newton and i was able to move forward with the install.
@un1x86 can you check yum info python2-passlib and see where you are getting it from? This was mine.
[root@93b2824028ac ~]# yum info python-passlib
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
* base: mirror.beyondhosting.net
* epel: mirror.math.princeton.edu
* extras: distro.ibiblio.org
* updates: chicago.gaminghost.co
Available Packages
Name : python-passlib
Arch : noarch
Version : 1.6.2
Release : 2.el7
Size : 480 k
Repo : epel/x86_64
Summary : Comprehensive password hashing framework supporting over 20 schemes
URL : http://passlib.googlecode.com
License : BSD and Beerware and Copyright only
Description : Passlib is a password hashing library for Python 2 & 3, which provides
: cross-platform implementations of over 20 password hashing algorithms,
: as well as a framework for managing existing password hashes. It's
: designed to be useful for a wide range of tasks, from verifying a hash
: found in /etc/shadow, to providing full-strength password hashing for
: multi-user application.
@etsauer python-passlib is coming from epel while python2-passlib is coming from openshift-origin repository
Available Packages
Name : python-passlib
Arch : noarch
Version : 1.6.2
Release : 2.el7
Size : 480 k
Repo : epel/x86_64
Summary : Comprehensive password hashing framework supporting over 20 schemes
URL : http://passlib.googlecode.com
License : BSD and Beerware and Copyright only
Description : Passlib is a password hashing library for Python 2 & 3, which provides
: cross-platform implementations of over 20 password hashing algorithms,
: as well as a framework for managing existing password hashes. It's
: designed to be useful for a wide range of tasks, from verifying a hash
: found in /etc/shadow, to providing full-strength password hashing for
: multi-user application.
[root@master ~]# yum info python2-passlib
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror.imt-systems.com
* epel: mirror.imt-systems.com
* extras: mirror.imt-systems.com
* updates: mirror.imt-systems.com
Available Packages
Name : python2-passlib
Arch : noarch
Version : 1.6.5
Release : 1.el7
Size : 488 k
Repo : centos-openshift-origin
Summary : Comprehensive password hashing framework supporting over 20 schemes
URL : https://bitbucket.org/ecollins/passlib
License : BSD and Beerware and Copyright only
Description : Passlib is a password hashing library for Python 2 & 3, which provides
: cross-platform implementations of over 20 password hashing algorithms,
: as well as a framework for managing existing password hashes. It's
: designed to be useful for a wide range of tasks, from verifying a hash
: found in /etc/shadow, to providing full-strength password hashing for
: multi-user application.
@ewolinetz seems to run through like that. This is the output of the task.
TASK [openshift_metrics : generate htpasswd file for hawkular metrics] **************************************************************************************************************************************************************************************************
task path: /root/openshift-ansible/roles/openshift_metrics/tasks/generate_hawkular_certificates.yaml:45
[WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: not {{ openshift_metrics_heapster_standalone | bool }}
Using module file /usr/lib/python2.7/site-packages/ansible/modules/web_infrastructure/htpasswd.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1494481580.2-8145002959404 `" && echo ansible-tmp-1494481580.2-8145002959404="` echo /root/.ansible/tmp/ansible-tmp-1494481580.2-8145002959404 `" ) && sleep 0'
<localhost> PUT /tmp/tmp3I55uB TO /root/.ansible/tmp/ansible-tmp-1494481580.2-8145002959404/htpasswd.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1494481580.2-8145002959404/ /root/.ansible/tmp/ansible-tmp-1494481580.2-8145002959404/htpasswd.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python2 /root/.ansible/tmp/ansible-tmp-1494481580.2-8145002959404/htpasswd.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1494481580.2-8145002959404/" > /dev/null 2>&1 && sleep 0'
changed: [master.some_domain.ch -> localhost] => {
"changed": true,
"invocation": {
"module_args": {
"attributes": null,
"backup": null,
"content": null,
"create": true,
"crypt_scheme": "apr_md5_crypt",
"delimiter": null,
"directory_mode": null,
"follow": false,
"force": null,
"group": null,
"mode": null,
"name": "hawkular",
"owner": null,
"password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"path": "/tmp/tmp.TWwJaE8eod/hawkular-metrics.htpasswd",
"regexp": null,
"remote_src": null,
"selevel": null,
"serole": null,
"setype": null,
"seuser": null,
"src": null,
"state": "present",
"unsafe_writes": null
}
},
"msg": "Created /tmp/tmp.TWwJaE8eod/hawkular-metrics.htpasswd and added hawkular"
}
@sdodson it seems that this may be something we need to change in our repo, or change how we gen the htpasswd file. Thoughts?
ewolinetz
on 11 May 2017
Having the same problem. I've tried everything including downgrading ansible to 2.2.1 and installing both flavors of passlib:
rpm -qa|grep passlib
python-passlib-1.6.2-2.el7.noarch
python2-passlib-1.6.5-1.el7.noarch
TASK [openshift_metrics : generate htpasswd file for hawkular metrics] ***
fatal: [origin-master.c.xxx-dev.internal -> localhost]: FAILED! => {"changed": false, "failed": true, "msg": "This module requires the passlib Python library"}
tonyturino
on 25 May 2017
I'm still getting the message no matter if I've got python-passlib OR python2-passlib OR neither installed:
TASK [openshift_metrics : Check that python-passlib is available on the control host] ******
fatal: [origin-master.c.mcomops-dev.internal]: FAILED! => {
"assertion": "'not installed' not in passlib_result.stdout",
"changed": false,
"evaluated_to": false,
"failed": true
}
MSG:
python-passlib rpm must be installed on control host
tonyturino
on 26 May 2017
Same error. removed no_logs from step.
TASK [openshift_metrics : generate htpasswd file for hawkular metrics] *********************************************************************
[WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: not {{
openshift_metrics_heapster_standalone | bool }}
fatal: [852378-master1-dr.myriadpayments.com -> localhost]: FAILED! => {
"changed": false,
"failed": true
}
MSG:
invalid version number '1.7.0.post20170223134505'
passlib:
[ocpadmin@806783-jbox1 share]$ rpm -qa|grep passlib
python2-passlib-1.7.0-4.el7.noarch
trying to rollback libs for this.
@sdodson Any ideas?
mjudeikis
on 30 May 2017
downgrading passlib solved the issue.
[ocpadmin@806783-jbox1 ~]$ sudo yum search python2-passlib --showduplicates
Loaded plugins: product-id, search-disabled-repos, subscription-manager
======================================================= N/S matched: python2-passlib =======================================================
python2-passlib-1.6.5-1.el7.noarch : Comprehensive password hashing framework supporting over 20 schemes
python2-passlib-1.7.0-4.el7.noarch : Comprehensive password hashing framework supporting over 20 schemes
Name and summary matches only, use "search all" for everything.
[ocpadmin@806783-jbox1 ~]$ sudo yum downgrade python2-passlib-1.6.5
Loaded plugins: product-id, search-disabled-repos, subscription-manager
Resolving Dependencies
--> Running transaction check
---> Package python2-passlib.noarch 0:1.6.5-1.el7 will be a downgrade
---> Package python2-passlib.noarch 0:1.7.0-4.el7 will be erased
--> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================
Downgrading:
python2-passlib noarch 1.6.5-1.el7 rhel-7-server-ose-3.5-rpms 488 k
Transaction Summary
============================================================================================================================================
Downgrade 1 Package
Total download size: 488 k
Is this ok [y/d/N]: y
Downloading packages:
python2-passlib-1.6.5-1.el7.noarch.rpm | 488 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : python2-passlib-1.6.5-1.el7.noarch 1/2
Cleanup : python2-passlib-1.7.0-4.el7.noarch 2/2
Verifying : python2-passlib-1.6.5-1.el7.noarch 1/2
Verifying : python2-passlib-1.7.0-4.el7.noarch 2/2
Removed:
python2-passlib.noarch 0:1.7.0-4.el7
Installed:
python2-passlib.noarch 0:1.6.5-1.el7
Seen same issue with metrics and logging deplpoyments/updates for 3.5 when using newest rpms
mjudeikis
on 30 May 2017
Ansible pr to fix that https://github.com/ansible/ansible/pull/20202 i'll see if we can get that pushed through. Which repo included python2-passlib-1.7.0?
sdodson
on 30 May 2017
Looks like it wsa provided by Rackspace version of repos within spacewalk... raised with rackspace... :/
mjudeikis
on 30 May 2017
@sdodson and @mjudeikis I hit this https://github.com/openshift/openshift-ansible/issues/4111#issuecomment-304843311 on a 3.6 Origin install, when trying to add metrics.
~~~
cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)
yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
- base: repos-va.psychz.net
- epel: mirror.vcu.edu
- extras: mirror.us.leaseweb.net
- updates: mirror.vcu.edu
repo id repo name status
base/7/x86_64 CentOS-7 - Base 9,591
centos-openshift-origin36 CentOS OpenShift Origin 46+20
epel/x86_64 Extra Packages for Enterprise Linux 7 - x86_64 12,252
extras/7/x86_64 CentOS-7 - Extras 313+16
updates/7/x86_64 CentOS-7 - Updates 1,909
repolist: 24,111
sudo yum search python2-passlib --showduplicates
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
- base: repos-va.psychz.net
- epel: mirror.vcu.edu
- extras: mirror.us.leaseweb.net
- updates: mirror.vcu.edu
========================= N/S matched: python2-passlib =========================
python2-passlib-1.7.0-4.el7.noarch : Comprehensive password hashing framework
: supporting over 20 schemes
python2-passlib-1.7.0-4.el7.noarch : Comprehensive password hashing framework
: supporting over 20 schemes
~~~
As it does not look like I have an option to downgrade, the issue seems unresolved.
sferich888
on 28 Jan 2018
If I manually delete the package, and pull https://centos.pkgs.org/7/centos-extras-x86_64/python-passlib-1.6.5-2.el7.noarch.rpm.html and install just this package (not sure why yum can't see this), I also can work around this.
sferich888
on 28 Jan 2018
@sferich888 That's an ansible bug, are you running ansible 2.3.2 or 2.4.0 and newer?
sdodson
on 29 Jan 2018
I got this fatal error upon running ansible roles using vault info on ansible 2.2.1.
fatal: [x.x.x.x]: FAILED! => {"ansible_facts": {}, "changed": false, "failed": true, "message": "Decryption failed on creds.yml"}
Going thru this above notes, I did tried putting passlib & upgrading ansible to 2.2.3 and getting the same result :( Cant find any other possible fix to this error. I can edit the vault file with the file location saved under vault-password-file on commandline and it works without fail. And while running the roles, I tried both options putting the ini parameter vault_password_file = ~ /.vault_file.txt in ansible.cfg and on the command line as --vault-password-file=~ /.vault_file.txt and neither works :( Appreciate a quick reply to this query.
-bash-4.1$ rpm -qa |grep passlib
python-passlib-1.5.3-1.el6.noarch
-bash-4.1$ ansible --version
ansible 2.2.3.0
config file = ~/ansible.cfg
configured module search path = ['./library']
-bash-4.1$ python --version
Python 2.6.6
Note: Intentionally gave space after ~ as its marking the end points to strike off the line in this markdown format.
ivar1234
on 13 Feb 2018
@ivar1234 I don't think we've ever tried to run openshift-ansible on a RHEL 6 control host. Our most commonly used platform would be RHEL/Centos 7 and current Fedora releases. Is it possible to run ansible on one of those?
sdodson
on 20 Feb 2018
Seeing this issue when trying to install origin 3.9
PLAY [Determine openshift_version to configure on first master] ****************
TASK [Gathering Facts] *********************************************************
ok: [master1]
TASK [include_role] ************************************************************
TASK [openshift_version : Use openshift.common.version fact as version to configure if already installed] ***
ok: [master1]
TASK [openshift_version : include_tasks] ***************************************
included: /usr/share/ansible/openshift-ansible/roles/openshift_version/tasks/first_master_rpm_version.yml for master1
TASK [openshift_version : Set rpm version to configure if openshift_pkg_version specified] ***
skipping: [master1]
TASK [openshift_version : Set openshift_version for rpm installation] **********
included: /usr/share/ansible/openshift-ansible/roles/openshift_version/tasks/check_available_rpms.yml for master1
TASK [openshift_version : Get available origin version] ************************
ok: [master1]
TASK [openshift_version : fail] ************************************************
fatal: [master1]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
to retry, use: --limit @/usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.retry
[root@bastion ~]# ansible --version
ansible 2.4.2.0
config file = /root/.ansible.cfg
configured module search path = [u'/usr/share/ansible', u'/usr/share/ansible/openshift-ansible/library']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.5 (default, May 3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]
md5-917dfacf1a378616a9a7fc67504228e4
[root@master1 ~]# rpm -qa | grep passlib
python-passlib-1.6.5-2.el7.noarch
kargakis
on 15 Mar 2018
Seeing this error as well:
fatal: [10.243.30.117]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
paulcalabro
on 22 Oct 2018
how to solve when install ansible tower
An error occurred
fatal: [10.243.30.117]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}
wyswust
on 27 Mar 2019
Related issues
ceagan
路
6Comments
adamulacha
路
6Comments
outcoldman
路
6Comments
IronicBadger
路
7Comments
rharveyva
路
6Comments