Patroni: Patroni Python Problem

Created on 17 Oct 2018 · 9Comments · Source: zalando/patroni

Hello all,

I installed patroni on CentOS and I'm getting of Python errors.

systemd: Started Runners to orchestrate a high-availability PostgreSQL.
systemd: Starting Runners to orchestrate a high-availability PostgreSQL...
patroni: Traceback (most recent call last):
patroni: File "/bin/patroni", line 6, in
patroni: from pkg_resources import load_entry_point
patroni: File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3112, in
patroni: @_call_aside
patroni: File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3096, in _call_aside
patroni: f(args, *kwargs)
patroni: File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3125, in _initialize_master_working_set
patroni: working_set = WorkingSet._build_master()
patroni: File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 580, in _build_master
patroni: return cls._build_from_requirements(__requires__)
patroni: File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 593, in _build_from_requirements
patroni: dists = ws.resolve(reqs, Environment())
patroni: File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
patroni: raise VersionConflict(dist, req).with_context(dependent_req)
patroni: pkg_resources.ContextualVersionConflict: (urllib3 1.24 (/usr/lib/python2.7/site-packages), Requirement.parse('urllib3<1.24,>=1.21.1'), set(['requests']))
systemd: patroni.service: main process exited, code=exited, status=1/FAILURE
systemd: Unit patroni.service entered failed state.
systemd: patroni.service failed.

I had them set up:

yum install -y gcc python-devel
yum install -y python2-pip
yum install -y etcd
yum install -y haproxy

pip install --upgrade setuptools
pip install patroni
pip install python-etcd
pip install psycopg2-binary

PostgreSQL was already installed and I have created a new etcd and a haproxy machines.

etcd:
tcp 0 0 10.0.0.1:2379 0.0.0.0:* LISTEN 1684/etcd
tcp 0 0 10.0.0.1:2380 0.0.0.0:* LISTEN 1684/etcd

haproxy:
tcp 0 0 10.0.0.2:5000 0.0.0.0:* LISTEN 1518/haproxy
tcp 0 0 10.0.0.2:7000 0.0.0.0:* LISTEN 1518/haproxy

firewall:

firewall-cmd --permanent --add-port=8008/tcp
firewall-cmd --permanent --add-port=2379/tcp
firewall-cmd --permanent --add-port=2380/tcp
firewall-cmd --permanent --add-port=5432/tcp
firewall-cmd --permanent --add-port=7000/tcp
firewall-cmd --permanent --add-port=5000/tcp

What is the problem? Thanks.

Source

rootqa

All 9 comments

The same problem as https://github.com/zalando/patroni/issues/693

pip install urllib3==1.23 should fix it

CyberDem0n on 17 Oct 2018

👍1

@CyberDem0n thanks for all help my friend but after doing this I still get a different errors:

patroni: File "/bin/patroni", line 11, in
patroni: load_entry_point('patroni==1.5.0', 'console_scripts', 'patroni')()
patroni: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 182, in main
patroni: return patroni_main()
patroni: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 149, in patroni_main
patroni: patroni.run()
patroni: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 114, in run
patroni: logger.info(self.ha.run_cycle())
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1236, in run_cycle
patroni: info = self._run_cycle()
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1144, in _run_cycle
patroni: return self.post_bootstrap()
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1041, in post_bootstrap
patroni: self.cancel_initialization()
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1036, in cancel_initialization
patroni: raise PatroniException('Failed to bootstrap cluster')

rootqa on 17 Oct 2018

I bet there must be more logs.
Also without looking on config file it's hard to guess what is wrong.

CyberDem0n on 17 Oct 2018

10.0.0.1 etcd
10.0.0.2 haproxy
10.0.0.3 psql1
10.0.0.4 psql2

install postgresql and patroni:

yum install postgresql-server postgresql-contrib (i think install version of 9.2)

pip install --upgrade setuptools

pip install patroni

pip install python-etcd

pip install psycopg2-binary

cat /etc/systemd/system/patroni.service

[Unit]
Description=Runners to orchestrate a high-availability PostgreSQL
After=syslog.target network.target
[Service]
Type=simple
User=postgres
Group=postgres
ExecStart=/bin/patroni /etc/patroni.yml
KillMode=process
TimeoutSec=30
Restart=no
[Install]
WantedBy=multi-user.targ

cat /etc/patroni.yml

scope: postgres
name: pg01

restapi:
listen: 10.0.3:8008
connect_address: 10.0.0.3:8008

etcd:
host: 10.0.0.1:2379

bootstrap:
dcs:
ttl: 30
loop_wait: 10
retry_timeout: 10
maximum_lag_on_failover: 1048576
postgresql:
use_pg_rewind: true

initdb:
- encoding: UTF8
- data-checksums

pg_hba:
- host replication replicator 127.0.0.1/32 md5
- host replication replicator 10.0.0.3/0 md5
- host replication replicator 10.0.0.4/0 md5
- host all all 0.0.0.0/0 md5

users:
    admin:
        password: admin
        options:
            - createrole
            - createdb

postgresql:
listen: 10.0.0.3:5432
connect_address: 10.0.0.3:5432
data_dir: /data/patroni
pgpass: /tmp/pgpass
authentication:
replication:
username: replicator
password: PASSWORD
superuser:
username: postgres
password: PASSWORD
parameters:
unix_socket_directories: '.'

tags:
nofailover: false
noloadbalance: false
clonefrom: false
nosync: false

cat /etc/etcd/etcd.conf

ETCD_LISTEN_PEER_URLS="http://10.0.0.1:2380"
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://10.0.0.1:2379"
ETCD_NAME="etcd0"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.0.0.1:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://10.0.0.1:2379"
ETCD_INITIAL_CLUSTER="etcd0=http://10.0.0.1:2380"
ETCD_INITIAL_CLUSTER_TOKEN="cluster1"
ETCD_INITIAL_CLUSTER_STATE="new"

cat /etc/haproxy/haproxy.cfg

global
maxconn 100
log 127.0.0.1 local2

defaults
log global
mode tcp
retries 2
timeout client 30m
timeout connect 4s
timeout server 30m
timeout check 5s

listen stats
mode http
bind *:7000
stats enable
stats uri /

listen postgres
bind *:5000
option httpchk
http-check expect status 200
default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
server postgresql_pg01_5432 10.0.0.3:5432 maxconn 100 check port 8008
server postgresql_pg02_5432 10.0.0.4:5432 maxconn 100 check port 8008

systemctl restart patroni

systemd: Started Runners to orchestrate a high-availability PostgreSQL.
systemd: Starting Runners to orchestrate a high-availability PostgreSQL...
patroni: 2018-10-17 09:29:08,127 INFO: Failed to import patroni.dcs.consul
patroni: 2018-10-17 09:29:08,143 INFO: Selected new etcd server http://172.31.1.132:2379
patroni: 2018-10-17 09:29:08,170 INFO: Lock owner: None; I am pg01
patroni: 2018-10-17 09:29:08,175 INFO: trying to bootstrap a new cluster
patroni: /usr/bin/initdb: unrecognized option '--data-checksums'
patroni: Try "initdb --help" for more information.
patroni: pg_ctl: database system initialization failed
patroni: 2018-10-17 09:29:08,188 INFO: removing initialize key after failed attempt to bootstrap the cluster
patroni: 2018-10-17 09:29:08,669 INFO: Lock owner: None; I am pg01
patroni: Traceback (most recent call last):
patroni: File "/bin/patroni", line 11, in
patroni: load_entry_point('patroni==1.5.0', 'console_scripts', 'patroni')()
patroni: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 182, in main
patroni: return patroni_main()
patroni: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 149, in patroni_main
patroni: patroni.run()
patroni: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 114, in run
patroni: logger.info(self.ha.run_cycle())
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1236, in run_cycle
patroni: info = self._run_cycle()
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1144, in _run_cycle
patroni: return self.post_bootstrap()
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1041, in post_bootstrap
patroni: self.cancel_initialization()
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1036, in cancel_initialization
patroni: raise PatroniException('Failed to bootstrap cluster')
patroni: patroni.exceptions.PatroniException: 'Failed to bootstrap cluster'
systemd: patroni.service: main process exited, code=exited, status=1/FAILURE
systemd: Unit patroni.service entered failed state.
systemd: patroni.service failed.

Dear @CyberDem0n are you want any other information? Thanks for all help.
Regards

rootqa on 17 Oct 2018

Postgres 9.2 is EOL for 1 year already.
Please take something fresh, for example 10

CyberDem0n on 17 Oct 2018

😄1

Dear @CyberDem0n thanks I will upgrade to PostgreSQL version 10 and will patroni automatically initial and startup PostgreSQL service or should I do initial config and statup?

rootqa on 17 Oct 2018

@CyberDem0n I upgraded to PostgreSQL 10 and I'm getting the error again:

systemd: Started Runners to orchestrate a high-availability PostgreSQL.
systemd: Starting Runners to orchestrate a high-availability PostgreSQL...
patroni: 2018-10-17 12:14:22,251 INFO: Failed to import patroni.dcs.consul
patroni: 2018-10-17 12:14:22,266 INFO: Selected new etcd server http://10.0.0.1:2379
patroni: 2018-10-17 12:14:27,548 INFO: Lock owner: None; I am pg01
patroni: 2018-10-17 12:14:27,565 INFO: trying to bootstrap a new cluster
patroni: 2018-10-17 12:14:27,586 ERROR: Exception during execution of long running task bootstrap
patroni: Traceback (most recent call last):
patroni: File "/usr/lib/python2.7/site-packages/patroni/async_executor.py", line 97, in run
patroni: wakeup = func(args) if args else func()
patroni: File "/usr/lib/python2.7/site-packages/patroni/postgresql.py", line 1641, in bootstrap
patroni: return do_initialize(config) and self.write_pg_hba(pg_hba) and self.save_configuration_files() \
patroni: File "/usr/lib/python2.7/site-packages/patroni/postgresql.py", line 567, in _initdb
patroni: ret = self.pg_ctl('initdb', *options)
patroni: File "/usr/lib/python2.7/site-packages/patroni/postgresql.py", line 290, in pg_ctl
patroni: return subprocess.call(pg_ctl + ['-D', self._data_dir] + list(args), *kwargs) == 0
patroni: File "/usr/lib64/python2.7/subprocess.py", line 524, in call
patroni: return Popen(popenargs, *kwargs).wait()
patroni: File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
patroni: errread, errwrite)
patroni: File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
patroni: raise child_exception
patroni: OSError: [Errno 2] No such file or directory
patroni: 2018-10-17 12:14:27,603 INFO: removing initialize key after failed attempt to bootstrap the cluster
patroni: 2018-10-17 12:14:28,109 INFO: Lock owner: None; I am pg01
patroni: Traceback (most recent call last):
patroni: File "/bin/patroni", line 11, in
patroni: load_entry_point('patroni==1.5.0', 'console_scripts', 'patroni')()
patroni: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 182, in main
patroni: return patroni_main()
patroni: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 149, in patroni_main
patroni: patroni.run()
patroni: File "/usr/lib/python2.7/site-packages/patroni/__init__.py", line 114, in run
patroni: logger.info(self.ha.run_cycle())
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1236, in run_cycle
patroni: info = self._run_cycle()
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1144, in _run_cycle
patroni: return self.post_bootstrap()
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1041, in post_bootstrap
patroni: self.cancel_initialization()
patroni: File "/usr/lib/python2.7/site-packages/patroni/ha.py", line 1036, in cancel_initialization
patroni: raise PatroniException('Failed to bootstrap cluster')
patroni: patroni.exceptions.PatroniException: 'Failed to bootstrap cluster'
systemd: patroni.service: main process exited, code=exited, status=1/FAILURE
systemd: Unit patroni.service entered failed state.
systemd: patroni.service failed.