Scylla: Scylla setup failed for `ImportError: No module named 'yaml'`
Installation details
Scylla version (or git commit hash): 666.development-0.20190319.7119440cb
Cluster size: 1
OS (RHEL/CentOS/Ubuntu/AWS AMI): ami-0bf1aadc2d4e2e86c
Instance type: i3.large i3.xlarge i3.2xlarge i3.4xlarge i3.8xlarge i3.16xlarge i3.metal
Test log: http://jenkins.cloudius-systems.com:8080/job/scylla-master/job/ami-tests/label=muninn/26/artifact/job-2019-03-31T13.25-624cddf/job.log
Description: Scylla setup failed for
ImportError: No module named 'yaml'
Running 'sudo /bin/journalctl --no-tail -u scylla-io-setup.service -u scylla-server.service -u scylla-ami-setup.service -u scylla-housekeeping-daily.service -u scylla-housekeeping-restart.service -u scylla-jmx.service '
[stdout] -- Logs begin at Sun 2019-03-31 10:26:11 UTC. --
[stdout] Mar 31 10:26:45 systemd[1]: Starting Scylla AMI Setup...
[stdout] Mar 31 10:26:48 scylla_ami_setup[5454]: Traceback (most recent call last):
[stdout] Mar 31 10:26:48 scylla_ami_setup[5454]: File "/usr/lib/scylla/scylla-ami/scylla_ami_setup", line 26, in <module>
[stdout] Mar 31 10:26:48 scylla_ami_setup[5454]: from scylla_util import *
[stdout] Mar 31 10:26:48 scylla_ami_setup[5454]: File "/usr/lib/scylla/scylla_util.py", line 31, in <module>
[stdout] Mar 31 10:26:48 scylla_ami_setup[5454]: import yaml
[stdout] Mar 31 10:26:48 scylla_ami_setup[5454]: ImportError: No module named 'yaml'
[stdout] Mar 31 10:26:48 systemd[1]: scylla-ami-setup.service: main process exited, code=exited, status=1/FAILURE
[stdout] Mar 31 10:26:48 systemd[1]: Failed to start Scylla AMI Setup.
[stdout] Mar 31 10:26:48 systemd[1]: Dependency failed for Scylla Server.
[stdout] Mar 31 10:26:48 systemd[1]: Dependency failed for Scylla JMX.
[stdout] Mar 31 10:26:48 systemd[1]: Job scylla-jmx.service/start failed with result 'dependency'.
[stdout] Mar 31 10:26:48 systemd[1]: Job scylla-server.service/start failed with result 'dependency'.
[stdout] Mar 31 10:26:48 systemd[1]: Unit scylla-ami-setup.service entered failed state.
[stdout] Mar 31 10:26:48 systemd[1]: scylla-ami-setup.service failed.
[centos@ip-172-10-15-81 ~]$ head -n 1 /usr/lib/scylla/scylla-ami/scylla_ami_setup
#!/usr/bin/python3
[centos@ip-172-10-15-81 ~]$ ls -l /usr/bin/python3
lrwxrwxrwx. 1 root root 9 Mar 31 10:09 /usr/bin/python3 -> python3.4
[centos@ip-172-10-15-81 ~]$ ls -l /usr/bin/python3.4
-rwxr-xr-x. 2 root root 11392 Feb 5 14:38 /usr/bin/python3.4
[centos@ip-172-10-15-81 ~]$ rpm -qa |grep -i yaml
libyaml-0.1.4-11.el7_0.x86_64
PyYAML-3.10-11.el7.x86_64
[centos@ip-172-10-15-81 ~]$
Traceback (most recent call last):
File "/usr/lib/scylla/scylla-ami/scylla_ami_login", line 26, in <module>
from scylla_util import *
File "/usr/lib/scylla/scylla_util.py", line 31, in <module>
import yaml
ImportError: No module named 'yaml'
/CC @hagitsegev @syuu1228 @roydahan @slivne @glommer
amoskong
All 11 comments
Recent related change:
- Commit https://github.com/scylladb/scylla/commit/b1283b23bbf16a8a54c08e983aa45b4fa2910bc8 corrected the PyYAML package name, made it fits with Fedora 29
Fedora 29:
[amos@fedora ~]$ sudo dnf search python\*pyyaml
Last metadata expiration check: 0:06:45 ago on Mon 01 Apr 2019 11:07:19 AM CST.
bat:latest:2920190307100850:7fdbb362-0.x86_64
================================================================================= Name Matched: python*pyyaml =================================================================================
python3-pyyaml.x86_64 : YAML parser and emitter for Python
python2-pyyaml.noarch : Dummy package depending on python-pyyaml
python2-pyyaml.x86_64 : YAML parser and emitter for Python
python3-pyyaml.x86_64 : YAML parser and emitter for Python
python34-PyYAML.x86_64 : YAML parser and emitter for Python 3.4
python36-PyYAML.x86_64 : YAML parser and emitter for Python 3.4
CentOS7
[centos@ip-172-10-15-81 ~]$ sudo yum search pyyaml
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: repos-va.psychz.net
* elrepo: iad.mirror.rackspace.com
* epel: d2lzkl7pfhq30w.cloudfront.net
* extras: repos-va.psychz.net
* updates: repos-va.psychz.net
===================================================================================== N/S matched: pyyaml =====================================================================================
python2-pyyaml.noarch : Dummy package depending on python-pyyaml
PyYAML.x86_64 : YAML parser and emitter for Python
python2-yamlordereddictloader.noarch : YAML loader for PyYAML that maintains key order
python34-PyYAML.x86_64 : YAML parser and emitter for Python 3.4
python34-yamlordereddictloader.noarch : YAML loader for PyYAML that maintains key order
python36-PyYAML.x86_64 : YAML parser and emitter for Python 3.4
It seems the scylla_setup and other scripts didn't use python3 provided by scylla-python3.
[centos@ip-172-10-15-81 ~]$ rpm -qa |grep scylla-python3
scylla-python3-3.6.6-1.el7.x86_64
[centos@ip-172-10-15-81 ~]$ rpm -ql scylla-python3|grep bin\/
/opt/scylladb/python3/bin/python3
/opt/scylladb/python3/bin/python3.6
[centos@ip-172-10-15-81 ~]$ ls -l /opt/scylladb/python3/bin/
total 4
lrwxrwxrwx. 1 root root 9 Mar 31 10:13 python3 -> python3.6
-rwxr-xr-x. 1 root root 273 Jan 1 1970 python3.6
[centos@ip-172-10-15-81 ~]$ /opt/scylladb/python3/bin/python3
Python 3.6.6 (default, Jul 19 2018, 16:29:00)
[GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import yaml
>>>
Hi Shlomi,
I saw you rebuild PASS 119, but it (125) still failed for this issue.
It's caused by epel repo update. related issue #4406
In the installation log of 125, we found both python34 and python36 were installed, then default /usr/bin/python3 will point to python3.6, but python36-PyYAML aren't installed. Only python34-PyYAML is scylla's dependency.
- Log : http://jenkins.cloudius-systems.com:8080/job/scylla-master/job/centos-rpm-test/label=muninn/125/artifact/latest/job.log
2019-04-04 11:53:12,778 process L0420 DEBUG| [stdout] Installed:
2019-04-04 11:53:12,778 process L0420 DEBUG| [stdout] scylla.x86_64 0:666.development-0.20190122.dbc1894bd.el7
2019-04-04 11:53:12,778 process L0420 DEBUG| [stdout]
2019-04-04 11:53:12,778 process L0420 DEBUG| [stdout] Dependency Installed:
...
2019-04-04 11:53:12,780 process L0420 DEBUG| [stdout] python3-pyudev.noarch 0:0.21.0-2.el7
2019-04-04 11:53:12,780 process L0420 DEBUG| [stdout] python34.x86_64 0:3.4.9-3.el7
2019-04-04 11:53:12,780 process L0420 DEBUG| [stdout] python34-PyYAML.x86_64 0:3.11-4.el7
2019-04-04 11:53:12,781 process L0420 DEBUG| [stdout] python34-libs.x86_64 0:3.4.9-3.el7
2019-04-04 11:53:12,781 process L0420 DEBUG| [stdout] python34-setuptools.noarch 0:39.2.0-3.el7
2019-04-04 11:53:12,781 process L0420 DEBUG| [stdout] python36.x86_64 0:3.6.6-5.el7
2019-04-04 11:53:12,781 process L0420 DEBUG| [stdout] python36-libs.x86_64 0:3.6.6-5.el7
2019-04-04 11:53:12,781 process L0420 DEBUG| [stdout] python36-six.noarch 0:1.11.0-3.el7
2019-04-04 11:53:12,781 process L0420 DEBUG| [stdout] scylla-conf.x86_64 0:666.development-0.20190122.dbc1894bd.el7
...
- job of of 119: no python3.6 packages are installed.
python34-libs
python34-PyYAML
python34
python34-setuptools
python3-pyudev
python34-six
amoskong
on 4 Apr 2019
We can fix this issue by moving to use scylla-python3 in running setup scripts.
But for old release, we need to force setup script to run by python34, or add python36 dependencies.
amoskong
on 4 Apr 2019
Some open source users just reported this. This is breaking everywhere in installed versions.
the problem is that we install python34-PyYAML, and python3 got updated recently in CentOS to python36.
And yes, python is so nice, that the python3.4 modules will not be visible by python3.6.
I am still working with @dyasny to figure out what is the best way to solve this.
glommer
on 4 Apr 2019
Looks like the EPEL update installs python3.6, replacing the python3 symlink to point at that. The update does not pick up PyYAML as a dependency needing an update though, so for now the workaround is to install python36-PyYAML manually
dyasny
on 4 Apr 2019
https://bugzilla.redhat.com/show_bug.cgi?id=1696324 submitted as per @avikivity request
dyasny
on 4 Apr 2019
@slivne can we close this?
Commits
d527ef19f7c94682d1010f48df05b2e4c35a9601
6e51a95668b16ac1c6fc41d8d6310c65e80bdefb
8568dc94f4e7289f22d7e05e7b538395bec6af00
switch to python36
tzach
on 16 Apr 2019
Yep
On Tue, Apr 16, 2019 at 11:31 AM Tzach Livyatan notifications@github.com
wrote:
@slivne https://github.com/slivne can we close this?
Commits
d527ef1
https://github.com/scylladb/scylla/commit/d527ef19f7c94682d1010f48df05b2e4c35a9601
6e51a95
https://github.com/scylladb/scylla/commit/6e51a95668b16ac1c6fc41d8d6310c65e80bdefb
8568dc9
https://github.com/scylladb/scylla/commit/8568dc94f4e7289f22d7e05e7b538395bec6af00
switch to python36—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/scylladb/scylla/issues/4379#issuecomment-483564212,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADThCEppqy7XqRAPdcE8eboUUGcnOviKks5vhYpEgaJpZM4cUwcQ
.
slivne
on 16 Apr 2019
for those struggling - here's the exact command for the workaround:
yum install python36-PyYAML
alekseys
on 2 May 2019
for those struggling - here's the exact command for the workaround:
yum install python36-PyYAML
Please feel free to ask questions in the mailing list or the Slack channel if you are struggling, we'll be happy to help
dyasny
on 2 May 2019
Related issues
tzach
·
7Comments
fgelcer
·
6Comments
tgrabiec
·
6Comments
Ritaja
·
3Comments
amoskong
·
6Comments