Awx: The SECRET_KEY setting must not be empty error in awx installation

[AnuradhaGangu]

ISSUE TYPE

• Bug Report
  AWX installation causes SECRET_KEY issue

COMPONENT NAME

• Installer

SUMMARY

I am trying to install awx 4.0.0 on RHEL 7.6
I have used docker approach and all the images are pulled from hub while installing.
The playbook runs successfully and I see all 5 containers running.
But in the awx_task log file I see below error and when I hit the UI url .. it says internal server error.

ENVIRONMENT

• AWX version: 4.0.0
• AWX install method: docker on linux
• Ansible version: 2.7.5
• Operating System: RHEL 7.6
• Web Browser: Chrome
• Python: 2.7

STEPS TO REPRODUCE

Set the proxy credentials in inventory file and
Run install.yml playbook

EXPECTED RESULTS

All 5 containers should run and AWX UI should open

ACTUAL RESULTS

5 containers are running and as per install.md instructions I see below in logs file

Using /etc/ansible/ansible.cfg as config file
127.0.0.1 | SUCCESS => {
"changed": false,
"elapsed": 0,
"path": null,
"port": 5432,
"search_regex": null,
"state": "started"
}
Using /etc/ansible/ansible.cfg as config file
127.0.0.1 | SUCCESS => {
"changed": false,
"elapsed": 0,
"path": null,
"port": 11211,
"search_regex": null,
"state": "started"
}
Using /etc/ansible/ansible.cfg as config file
127.0.0.1 | SUCCESS => {
"changed": false,
"elapsed": 73,
"path": null,
"port": 5672,
"search_regex": null,
"state": "started"
}
Using /etc/ansible/ansible.cfg as config file
127.0.0.1 | SUCCESS => {
"changed": false,
"db": "awx"
}
After above lines, I see below error in awx_logs

    ImproperlyConfigured("The SECRET_KEY setting must not be empty.")

django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.

Am using python 2.7 and error says SECRET_KEY is improperly defined in python3.6 related files. (you can see in log trace). Complete log trace is below.

    ImproperlyConfigured("The SECRET_KEY setting must not be empty.")

django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
2019-03-29 12:09:51,123 INFO spawned: 'channels-worker' with pid 7757
2019-03-29 12:09:52,186 INFO success: channels-worker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-03-29 12:09:52,188 INFO success: dispatcher entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-03-29 12:09:52,341 INFO exited: callback-receiver (exit status 1; not expected)
2019-03-29 12:09:53,717 INFO spawned: 'callback-receiver' with pid 7758
2019-03-29 12:09:54,949 INFO success: callback-receiver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Traceback (most recent call last):
File "/usr/bin/awx-manage", line 11, in
load_entry_point('awx==4.0.0.0', 'console_scripts', 'awx-manage')()
File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/__init__.py", line 124, in manage
prepare_env()
File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/__init__.py", line 89, in prepare_env
if not settings.DEBUG: # pragma: no cover
File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/conf/__init__.py", line 56, in __getattr__
self._setup(name)
File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/conf/__init__.py", line 41, in _setup
self._wrapped = Settings(settings_module)
File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/conf/__init__.py", line 129, in __init__
raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.

Not sure what I am doing wrong . Request your help in fixing the issue

ADDITIONAL INFORMATION

[shanemcd]

Hi. The SECRET_KEY file should be created by the installer here.

I am curious what you see if you try to re-run the installation (with -v) and make note of what happens on the task Render SECRET_KEY file.

Once that file is created it is then bind-mounted into the containers in these locations.

Can you check your docker_compose_dir and let me know what you see? Thanks.

[AnuradhaGangu]

@ shanemcd, Thanks for your reply.

I checked on the info you mentioned.
I see below when I run playbook with -v option
I don't see Render SECRET_KEY filetask getting executed. I am pasting last tasks status from playbook run.

TASK [local_docker : Update CA trust in awx_web container] *************************************************************************************************************
changed: [localhost] => {"changed": true, "cmd": ["docker", "exec", "awx_web", "/usr/bin/update-ca-trust"], "delta": "0:00:18.318319", "end": "2019-04-01 12:39:37.554480", "rc": 0, "start": "2019-04-01 12:39:19.236161", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

TASK [local_docker : Activate AWX Task Container] **********************************************************************************************************************
changed: [localhost] => {"ansible_facts": {"docker_container": {-------
-----------
------------
------------
"Restarting": false, "Running": true, "StartedAt": "2019-04-01T07:09:12.806130001Z", "Status": "running"}}}, "changed": true}

TASK [local_docker : Update CA trust in awx_task container] ************************************************************************************************************
changed: [localhost] => {"changed": true, "cmd": ["docker", "exec", "awx_task", "/usr/bin/update-ca-trust"], "delta": "0:00:14.410477", "end": "2019-04-01 12:40:18.833052", "rc": 0, "start": "2019-04-01 12:40:04.422575", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

TASK [local_docker : Create /var/lib/awx directory] ********************************************************************************************************************
skipping: [localhost] => {"changed": false, "skip_reason": "Conditional result was False"}

TASK [local_docker : Create docker-compose.yml file] *******************************************************************************************************************
skipping: [localhost] => {"changed": false, "skip_reason": "Conditional result was False"}

TASK [local_docker : Start the containers] *****************************************************************************************************************************
skipping: [localhost] => {"changed": false, "skip_reason": "Conditional result was False"}

TASK [local_docker : Update CA trust in awx_web container] *************************************************************************************************************
skipping: [localhost] => {"changed": false, "skip_reason": "Conditional result was False"}

TASK [local_docker : Update CA trust in awx_task container] ************************************************************************************************************
skipping: [localhost] => {"changed": false, "skip_reason": "Conditional result was False"}

PLAY RECAP *************************************************************************************************************************************************************
localhost                  : ok=14   changed=7    unreachable=0    failed=0

I did check docker_decompose_dir and see below files / dirs

prw------- 1 awx  awx      0 Mar 28 16:50 awxfifo
-rw-r--r-- 1 awx  awx  16384 Jan 16 16:10 beat.db
drwxr-x--- 2 awx  awx    106 Apr  1 12:42 job_status
drwxr-x--- 2 awx  awx      6 Jan  8 14:11 projects
drwxr-xr-x 3 root awx     20 Jan  8 14:11 public
drwxr-xr-x 4 root root    32 Jan  8 14:04 venv

For the ca_trust proprty in inventory, I just have it uncommented as below :
ca_trust_dir=/etc/pki/ca-trust/source/anchors

Can you guide me on what am missing ?

[shanemcd]

Hi @AnuradhaGangu, that's really strange. I'm wondering if you are re-running an old installer that does not have this latest code? Can you please try to download the latest release (4.0.0), re-populate the new inventory file, and run the installer then? I'm fairly confident this will resolve your issue.

[AnuradhaGangu]

Thank you ...!!! I downloaded the latest one when I started trying awx. But I don't see the task Render SECRET_KEY file in compose.yml.
I will download the latest and let you know.

Thank you so much for the support.

[AnuradhaGangu]

@shanemcd, It worked finally...!!! I am able to see AWX login page. I just got latest awx and changed the inventory attributes and ran playbook.

Thanks a ton for the support. It helped me to close the task.

[zesk06]

I created the file:
/etc/tower/SECRET_KEY
in web and tasks containers, with the secret_key, and restarted my containers

[deeco]

I'm having same issue on latest docker containers that previously worked ... Internal Serve error on AWX Server

raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.
2019-05-01 22:59:54,236 INFO exited: daphne (exit status 1; not expected)

[pjstam]

Hello, deeco,

This is what I did to solve this SECRET_KEY issue. You are probally using an old installer/inventory file.

• Copy the original installer/inventory file that was used
• Downloaded the latest version from git
• Update the new inventory file with the changes made in the original one
• Run the installer

ansible-playbook -i inventory install.yml

There should be a line near the end

TASK [local_docker : Render SECRET_KEY file] ******************************************************************************************************************************************************************
changed: [localhost]

• This file was not in my docker_compose_dir before, now it is
• Change the directory to docker_compose_dir and update

docker-compose pull && docker-compose up --force-recreate

This solved the problem for me, hopefully it will work for you as well.

Gtz

[abdul-jabbar01]

I created the file:
/etc/tower/SECRET_KEY
in web and tasks containers, with the secret_key, and restarted my containers

It worked in my case. Thanks a lot 👍