Awx: Reload workers on code change (or improve documentation)

[cdvv7788]

ISSUE TYPE

• Documentation

COMPONENT NAME

• Installer

SUMMARY

I followed all the steps at https://github.com/ansible/awx/blob/devel/CONTRIBUTING.md and was able to set up the project locally. That much works.
However, when I started to modify python code, to see the changes I have to restart the whole process (tools_awx_1). I went to the irc (which is not very active) and couldn't find a way to enable the development server. Refreshing changes manually is very slow, and I don't think that every contributor does it that way.
I would like to have a little more information in the document about how to work and contribute to the project, because as it is, I would spend most of my time refreshing a server instead of making progress.
I noticed that there is something that should be reloading the code on changes (see additional information).
Maybe the code I am modifying is not running on uwsgi? (not familiar with the architecture yet, sorry). If the code needs to be refreshed on some async task runner (celery, maybe?), that is not happening.
Just fyi, I am trying to change notifications a little: awx/notifications/slack_backend.py

ENVIRONMENT

• AWX version: 4.0.0
• AWX install method: docker for mac
• Ansible version: 2.7.10
• Operating System: Macos
• Web Browser: Chrome

STEPS TO REPRODUCE

Follow steps at https://github.com/ansible/awx/blob/devel/CONTRIBUTING.md to setup server. Then change awx/notifications/slack_backend.py (with the dev server running)

EXPECTED RESULTS

The servers are reloaded, and the new code changes are reflected in the site.

ACTUAL RESULTS

It seems that uwsgi is reloaded, but code changes are still not reflected.

ADDITIONAL INFORMATION

[jbradberry]

@cdvv7788 The Slack notification backend does not delegate to task workers, so the reload of uwsgi should be sufficient. I am completely puzzled why the reloader would see your changes and reload, but you still have to restart your container to have those changes take effect.

Edit: though it may be that notifications are getting issued from within a task, so it might still be that the task worker reloader is not working correctly.

[cdvv7788]

@jbradberry I am running the test notification on the notification screen (the one with the bell icon).
I suspect of the async workers because of this:

uwsgi workers are logged like:

As I mentioned earlier, I am not very familiar with the project yet, but the dispatcher line (instead of uwsgi) makes me thing that it goes through a different worker.

[jbradberry]

Ok, you are probably correct. I'll dig into it a bit more tomorrow.

[jbradberry]

@cdvv7788 As a faster way of dealing with this before the fix gets in, try starting a shell session in your container and then running awx-manage run_dispatcher --reload.

[cdvv7788]

@jbradberry I am seeing the following:

However, code changes are still not making it through. I am using a logger.error line inside the slack file and it always prints the original line.
Other than the merge, is there something else required for the change to catch up?

[ryanpetrello]

I think this just plain doesn’t work because the parent dispatcher process _forks_ its children.
When code changes in the development environment, the parent dispatcher process isn't recreated _itself_, so its children still have the old code.

To really accomplish this we’d need to have some way to restart the entire dispatcher process tree. I don't think honcho has a way to restart processes it's managing though.

[ryanpetrello]

@cdvv7788 thanks for reporting this, it should be resolved when https://github.com/ansible/awx/pull/3855 merges (shortly).

[cdvv7788]

@ryanpetrello got it. Thanks a lot for your quick response.