Awx: AWX docker image builds failing for awx/devel branch

[geauxvirtual]

ISSUE TYPE

• Bug Report

SUMMARY

AWX docker image build failing off of awx/devel branch with error:
Error: pg_config executable not found.
pg_config is required to build psycopg2 from source

ENVIRONMENT

• AWX version: awx/devel HEAD

STEPS TO REPRODUCE

Build AWX docker image using the included image_build role under installer folder of project.

EXPECTED RESULTS

AWX docker images to build successfully

ACTUAL RESULTS

AWX docker image fails to build with error: Error: pg_config executable not found.

Looking at the container that failed and looking at the current changes with the recent Postgresql 10 support, the awx-web Dockerfile is installing the rh-postgresql10-postgresql-devel rpm. This RPM installs files under /opt/rh. pg_config can be found at /opt/rh/rh-postgresql10/root/bin/pg_config, however this isn't in the PATH for the container and the psycopg2 python module fails to install as it cannot find pg_config.

[bhundven]

I can confirm this (end of ansible failure):

', u'Collecting prometheus_client==0.6.0 (from -r /dev/stdin (line 75))
', u'  Downloading https://files.pythonhosted.org/packages/4c/bd/b42db3ec90ffc6be805aad09c1cea4bb13a620d0cd4b21aaa44d13541d71/prometheus_client-0.6.0.tar.gz
', u'Collecting psutil==5.4.3 (from -r /dev/stdin (line 76))
', u'  Using cached https://files.pythonhosted.org/packages/e2/e1/600326635f97fee89bf8426fef14c5c29f4849c79f68fd79f433d8c1bd96/psutil-5.4.3.tar.gz
', u'Collecting psycopg2==2.8.2 (from -r /dev/stdin (line 77))
', u'  Downloading https://files.pythonhosted.org/packages/23/7e/93c325482c328619870b6cd09370f6dbe1148283daca65115cd63642e60f/psycopg2-2.8.2.tar.gz (368kB)
', u'    Complete output from command python setup.py egg_info:
', u\"    running egg_info
    creating pip-egg-info/psycopg2.egg-info
    writing pip-egg-info/psycopg2.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
    writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'

    Error: pg_config executable not found.

    pg_config is required to build psycopg2 from source.  Please add the directory
    containing pg_config to the $PATH or specify the full executable path with the
    option:

        python setup.py build_ext --pg-config /path/to/pg_config build ...

    or with the pg_config option in 'setup.cfg'.

    If you prefer to avoid building psycopg2 from source, please install the PyPI
    'psycopg2-binary' package instead.

    For further information please check the 'doc/src/install.rst' file (also at
    <http://initd.org/psycopg/docs/install.html>).

[bhundven]

temporarily fixed it in my workspace with:

diff --git a/installer/roles/image_build/templates/Dockerfile.j2 b/installer/roles/image_build/templates/Dockerfile.j2
index d105d4b90..f5ab364c6 100644
--- a/installer/roles/image_build/templates/Dockerfile.j2
+++ b/installer/roles/image_build/templates/Dockerfile.j2
@@ -71,6 +71,7 @@ RUN chmod +x /tini
 RUN python3 -m ensurepip && pip3 install virtualenv
 RUN pip install supervisor

+RUN ln -sf /opt/rh/rh-postgresql10/root/bin/pg_config /usr/bin/pg_config
 RUN find / -name pg_config

 ADD Makefile /tmp/Makefile

[nakacya]

I didn't understand it well, but I had two problems.

psycopg2 / ?????. so can't find libpq.so.rh-postgresql10-5
pg_config is missing (rh-postgresql10-postgresql-devel is not installed)

diff --git a/roles/image_build/templates/Dockerfile.j2 b/roles/image_build/templates/Dockerfile.j2
index d105d4b90..ad68d986a 100644
--- a/roles/image_build/templates/Dockerfile.j2
+++ b/roles/image_build/templates/Dockerfile.j2
@@ -71,6 +71,9 @@ RUN chmod +x /tini
 RUN python3 -m ensurepip && pip3 install virtualenv
 RUN pip install supervisor

+RUN yum install -y rh-postgresql10-postgresql-devel
+RUN ln -sf /opt/rh/rh-postgresql10/root/bin/pg_config /usr/bin/pg_config
+RUN ln -sf /opt/rh/rh-postgresql10/root/usr/lib64/libpq.so.rh-postgresql10-5 /usr/lib64
 RUN find / -name pg_config

 ADD Makefile /tmp/Makefile

[bhundven]

The proper way would be to use centos alternatives:
https://blogs.igalia.com/dpino/2011/10/13/configuring-different-jdks-with-alternatives/

[bhundven]

also, @nakacya , rh-postgresql10-postgresql-devel is installed here already:
https://github.com/ansible/awx/blob/devel/installer/roles/image_build/templates/Dockerfile.j2#L43

[nakacya]

I confirmed that there is no problem without installing the second rh-postgresql10-postgresql-devel.

[refriedjello]

@nakacya Adding those first two lines resolved this issue for me. Thanks!

That resulted in the build succeeding, but the awx_web container wouldn't serve web pages.

Adding these two lines as described above resulted in a functioning container for me, that permits login to the UI.

RUN ln -sf /opt/rh/rh-postgresql10/root/bin/pg_config /usr/bin/pg_config
RUN ln -sf /opt/rh/rh-postgresql10/root/usr/lib64/libpq.so.rh-postgresql10-5 /usr/lib64

[svrraja]

Build was success but still unable to login, error as below
Internal Server Error

[rooftopcellist]

@nakacya @bhundven @svrraja @refriedjello @geauxvirtual This should be fixed now. Please pull the latest code and let me know if you are still seeing this issue. The issue was that psycopg2 could not find the pg_config because it was not in the path.

Fixed by: https://github.com/ansible/awx/pull/4824

[bhundven]

@rooftopcellist thanks!