Awx: Issues with the latest AWX install

[shen3lu4]

ISSUE TYPE

• Bug Report

COMPONENT NAME

• Installer

SUMMARY

Did a new install of AWX using the latest installation script and the images defaulted in the script on Kubernetes. The only changes made were the Kubernetes context and namespace in inventory file. After the installation, AWX homepage is showing "503 Service Temporarily Unavailable" error. And there are lot of errors in the postgresql container.

ENVIRONMENT

• AWX version: 2.1.2
• AWX install method: kubernetes
• Ansible version: 2.4.0.0
• Operating System: Oracle linux 7.4
• Web Browser: Chrome

STEPS TO REPRODUCE

Clone the AWX repo from https://github.com/ansible/awx.git. Go to installer directory and make changes to inventory file by simply setting kubernetes_context and kubernetes_namespace. Then run ansible-playbook -i inventory install.yml

EXPECTED RESULTS

AWX running without issues

ACTUAL RESULTS

The installation completed successfully with pods running. However, AWX web site is showing an 503 error and there are lot of errors in postgresql logs.

ADDITIONAL INFORMATION

postgresql errors:
ERROR: relation "conf_setting" does not exist at character 158
2019-01-18 21:32:20.376 GMT [264] ERROR: relation "main_instance" does not exist at character 24
2019-01-18 21:32:20.376 GMT [264] STATEMENT: SELECT (1) AS "a" FROM "main_instance" WHERE "main_instance"."hostname" = 'awx-0' LIMIT 1
2019-01-18 21:32:20.377 GMT [264] ERROR: current transaction is aborted, commands ignored until end of transaction block
2019-01-18 21:32:20.377 GMT [264] STATEMENT: SELECT pg_advisory_unlock(355684602)
2019-01-18 21:32:22.947 GMT [265] ERROR: relation "conf_setting" does not exist at character 158

[bmillemathias]

does doing these following commands in a container shell fixes the problem

awx-manage migrate --noinput
awx-manage provision_instance --hostname=$(hostname)
awx-manage register_queue --queuename=tower --instance_percent=100

[shen3lu4]

@bmillemathias Thanks for your reply. I ran the commands in awx-celery container and it showed the following output:
bash-4.2$ awx-manage migrate --noinput
Operations to perform:
Apply all migrations: auth, conf, contenttypes, main, oauth2_provider, sessions, sites, social_django, sso, taggit
Running migrations:
No migrations to apply.
bash-4.2$ awx-manage provision_instance --hostname=$(hostname)
Instance already registered awx-0
bash-4.2$ awx-manage register_queue --queuename=tower --instance_percent=100
Instance Group already registered tower

I redid the installation with a brand new pvc for database. The website was launched without issue but I see the following errors in the container logs:

--in awx-web:
Traceback (most recent call last):
File "/usr/bin/ansible", line 67, in
import ansible.constants as C
File "/usr/lib/python2.7/site-packages/ansible/constants.py", line 184, in
config = ConfigManager()
File "/usr/lib/python2.7/site-packages/ansible/config/manager.py", line 243, in __init__
self.update_config_data()
File "/usr/lib/python2.7/site-packages/ansible/config/manager.py", line 501, in update_config_data
raise AnsibleError("Invalid settings supplied for %s: %s\n%s" % (config, to_native(e), traceback.format_exc()))
ansible.errors.AnsibleError: Invalid settings supplied for DEFAULT_LOCAL_TMP: Unable to create local directories(/.ansible/tmp): [Errno 13] Permission denied: '/.ansible'
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ansible/config/manager.py", line 489, in update_config_data
value, origin = self.get_config_value_and_origin(config, configfile)
File "/usr/lib/python2.7/site-packages/ansible/config/manager.py", line 434, in get_config_value_and_origin
value = ensure_type(value, defs[config].get('type'), origin=origin)
File "/usr/lib/python2.7/site-packages/ansible/config/manager.py", line 91, in ensure_type
makedirs_safe(value, 0o700)
File "/usr/lib/python2.7/site-packages/ansible/utils/path.py", line 81, in makedirs_safe
raise AnsibleError("Unable to create local directories(%s): %s" % (to_native(rpath), to_native(e)))
AnsibleError: Unable to create local directories(/.ansible/tmp): [Errno 13] Permission denied: '/.ansible'

--in postgresql:
2019-01-22 16:18:49.979 GMT [302] ERROR: relation "conf_setting" does not exist at character 158
....
2019-01-22 16:22:41.873 GMT [658] LOG: incomplete startup packet
2019-01-22 16:22:43.364 GMT [665] LOG: incomplete startup packet
2019-01-22 16:22:55.714 GMT [680] ERROR: must be owner of database awx
2019-01-22 16:22:55.714 GMT [680] STATEMENT: ALTER DATABASE "awx" OWNER TO "awx"
2019-01-22 16:22:55.829 GMT [681] ERROR: must be owner of database awx
2019-01-22 16:22:55.829 GMT [681] STATEMENT: ALTER DATABASE "awx" OWNER TO "awx"
2019-01-22 16:23:01.767 GMT [688] ERROR: duplicate key value violates unique constraint "auth_user_username_key"
2019-01-22 16:23:01.767 GMT [688] DETAIL: Key (username)=(admin) already exists.
2019-01-22 16:23:01.767 GMT [688] STATEMENT: INSERT INTO "auth_user" ("password", "last_login", "is_superuser", "username", "first_name", "last_name", "email", "is_staff", "is_active", "date_joined") VALUES ('pbkdf2_sha256$36000$tc1uAOojcmbz$c+gB5MCs6WIF8lysyv41CH5J7B9keAyD5Hc8P+J4Xvg=', NULL, true, 'admin', '', '', 'root@localhost', true, true, '2019-01-22T16:23:01.708055+00:00'::timestamptz) RETURNING "auth_user"."id"

[bmillemathias]

I thought it could be an issue I ran into lately, sorry I can't help.

[karen-nguyen]

I had the same errors in my logs, I think there are a lot of unsynchronized changes for the upcoming 3.0.0 release (latest Docker tag for ansible/awx_task and awx_web). I reverted back to the installer version from a month ago and it works fine.

[shen3lu4]

Thanks. I went back to tag 2.1.2 and the installation now works. Usually
how do I find out which installer version I should use? Is there any
documentation indicating which awx image version should go with which
installer version?

On Wed, Jan 23, 2019 at 8:48 AM beanallergy [e-mail]
wrote:

I had the same errors in my logs, I think there are a lot of
unsynchronized changes for the upcoming 3.0.0 release (latest Docker tag
for ansible/awx_task and awx_web). I reverted back to the installer version
from a month ago and it works fine.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ansible/awx/issues/3032#issuecomment-456718259, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AjR6acIbSnR2x5ZSoC7qYqRK0utrPL61ks5vGCHBgaJpZM4aI7G0
.