Awx: 500 error occurs when Webhook is executed with SURVEY set in Template or Workflow

Created on 22 Oct 2019  路  9Comments  路  Source: ansible/awx
ISSUE TYPE
  • Bug Report
SUMMARY

If I set the URL of AWX Template or Workflow (SURVEY added) to GitHub Webhook and execute it, a 500 error will occur.

Error Response Body

{"detail":"A server error has occurred."}

If SURVEY is not added, no error will occur.

ENVIRONMENT
  • AWX version: 8.0.0
  • AWX install method: docker on linux
  • Ansible version: 2.8.6
  • Operating System: CentOS Linux release 7.7.1908 (Core)
  • Web Browser: Chrome
STEPS TO REPRODUCE
  1. Create of AWX Template(Workflow).
  2. Added SURVEY variable in Template(Workflow).
  3. Enable execution of Template(Workflow) via webhook.
  4. Set the template(Workflow) URL to the GitHub webhook.
  5. Push to GitHub repository.
EXPECTED RESULTS

I was expecting the Template to execute via webhook.

ACTUAL RESULTS

500 Internal Server Error

ADDITIONAL INFORMATION
api high bug
Source
picture sky-joker

All 9 comments

@sky-joker any chance you could share the contents of your awx logs (with a 500 error, you should see a verbose error, which should give us some clues about what's wrong).

ryanpetrello picture ryanpetrello on 22 Oct 2019

Hi, @ryanpetrello

The error log is as follows.


Oct 22 11:50:11 ip-192-168-0-37 journal:
Oct 22 11:50:11 ip-192-168-0-37 journal: 2019-10-22 11:50:11,989 DEBUG    awx.api.views.webhooks header signature: b'bf7e8590fc3e6dbf96d6fb1e4f04ca7c6468e5ba'
Oct 22 11:50:11 ip-192-168-0-37 journal: 2019-10-22 11:50:11,990 DEBUG    awx.api.views.webhooks calculated signature: b'bf7e8590fc3e6dbf96d6fb1e4f04ca7c6468e5ba'
Oct 22 11:50:12 ip-192-168-0-37 journal: 2019-10-22 11:50:11,998 ERROR    django.request Internal Server Error: /api/v2/workflow_job_templates/11/github/
Oct 22 11:50:12 ip-192-168-0-37 journal: Traceback (most recent call last):
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = get_response(request)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = self.process_exception_by_middleware(e, request)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = wrapped_callback(request, *callback_args, **callback_kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/usr/lib64/python3.6/contextlib.py", line 52, in inner
Oct 22 11:50:12 ip-192-168-0-37 journal:    return func(*args, **kwds)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return view_func(*args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/generic/base.py", line 71, in view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return self.dispatch(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/api/generics.py", line 297, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    return super(APIView, self).dispatch(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 495, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = self.handle_exception(exc)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 455, in handle_exception
Oct 22 11:50:12 ip-192-168-0-37 journal:    self.raise_uncaught_exception(exc)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 492, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = handler(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return view_func(*args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/api/views/webhooks.py", line 163, in post
Oct 22 11:50:12 ip-192-168-0-37 journal:    new_job = obj.create_unified_job(**kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/workflow.py", line 472, in create_unified_job
Oct 22 11:50:12 ip-192-168-0-37 journal:    workflow_job = super(WorkflowJobTemplate, self).create_unified_job(**kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/unified_jobs.py", line 341, in create_unified_job
Oct 22 11:50:12 ip-192-168-0-37 journal:    encrypt_dict(kwargs.get('extra_vars', {}), password_list)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/utils/encryption.py", line 140, in encrypt_dict
Oct 22 11:50:12 ip-192-168-0-37 journal:    encrypt_fields = set(data.keys()).intersection(fields)
Oct 22 11:50:12 ip-192-168-0-37 journal: AttributeError: 'str' object has no attribute 'keys'
Oct 22 11:50:12 ip-192-168-0-37 journal: 2019-10-22 11:50:11,998 ERROR    django.request Internal Server Error: /api/v2/workflow_job_templates/11/github/
Oct 22 11:50:12 ip-192-168-0-37 journal: Traceback (most recent call last):
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = get_response(request)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = self.process_exception_by_middleware(e, request)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = wrapped_callback(request, *callback_args, **callback_kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/usr/lib64/python3.6/contextlib.py", line 52, in inner
Oct 22 11:50:12 ip-192-168-0-37 journal:    return func(*args, **kwds)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return view_func(*args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/generic/base.py", line 71, in view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return self.dispatch(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/api/generics.py", line 297, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    return super(APIView, self).dispatch(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 495, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = self.handle_exception(exc)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 455, in handle_exception
Oct 22 11:50:12 ip-192-168-0-37 journal:    self.raise_uncaught_exception(exc)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/rest_framework/views.py", line 492, in dispatch
Oct 22 11:50:12 ip-192-168-0-37 journal:    response = handler(request, *args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
Oct 22 11:50:12 ip-192-168-0-37 journal:    return view_func(*args, **kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/api/views/webhooks.py", line 163, in post
Oct 22 11:50:12 ip-192-168-0-37 journal:    new_job = obj.create_unified_job(**kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/workflow.py", line 472, in create_unified_job
Oct 22 11:50:12 ip-192-168-0-37 journal:    workflow_job = super(WorkflowJobTemplate, self).create_unified_job(**kwargs)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/models/unified_jobs.py", line 341, in create_unified_job
Oct 22 11:50:12 ip-192-168-0-37 journal:    encrypt_dict(kwargs.get('extra_vars', {}), password_list)
Oct 22 11:50:12 ip-192-168-0-37 journal:  File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/utils/encryption.py", line 140, in encrypt_dict
Oct 22 11:50:12 ip-192-168-0-37 journal:    encrypt_fields = set(data.keys()).intersection(fields)
Oct 22 11:50:12 ip-192-168-0-37 journal: AttributeError: 'str' object has no attribute 'keys'
Oct 22 11:50:12 ip-192-168-0-37 journal: [pid: 163|app: 0|req: 12/82] 192.30.252.99 () {44 vars in 693 bytes} [Tue Oct 22 11:50:11 2019] POST /api/v2/workflow_job_templates/11/github/ => generated 41 bytes in 51 msecs (HTTP/1.1 500) 5 headers in 185 bytes (1 switches on core 0)
Oct 22 11:50:12 ip-192-168-0-37 journal: 192.30.252.99 - - [22/Oct/2019:11:50:12 +0000] "POST /api/v2/workflow_job_templates/11/github/ HTTP/1.1" 500 41 "-" "GitHub-Hookshot/795f661" "-"

thanks

sky-joker picture sky-joker on 22 Oct 2019

@sky-joker any chance you could share the output of /api/v2/job_templates/N/survey_spec/

...where N is the ID of the job template where you've defined the survey?

ryanpetrello picture ryanpetrello on 22 Oct 2019

Okay, I'm able to reproduce this by:

  1. Setting up a JobTemplate that has a survey with a "password" field (no default provided).
  2. Launching this JobTemplate using a GitHub webhook.
ryanpetrello picture ryanpetrello on 22 Oct 2019

@sky-joker,

Any chance this diff helps for you?

diff --git a/awx/api/views/webhooks.py b/awx/api/views/webhooks.py
index e3ed6e64c9..dbdd3916bc 100644
--- a/awx/api/views/webhooks.py
+++ b/awx/api/views/webhooks.py
@@ -151,13 +151,13 @@ class WebhookReceiverBase(APIView):
                 'webhook_credential': obj.webhook_credential,
                 'webhook_guid': event_guid,
             },
-            'extra_vars': json.dumps({
+            'extra_vars': {
                 'tower_webhook_event_type': event_type,
                 'tower_webhook_event_guid': event_guid,
                 'tower_webhook_event_ref': event_ref,
                 'tower_webhook_status_api': status_api,
                 'tower_webhook_payload': request.data,
-            })
+            }
         }

         new_job = obj.create_unified_job(**kwargs)
ryanpetrello picture ryanpetrello on 22 Oct 2019
👍1

Okay, I'm able to reproduce this by:

  1. Setting up a JobTemplate that has a survey with a "password" field (no default provided).
  2. Launching this JobTemplate using a GitHub webhook.

Yes :)
The SURVEY I set is as follows.

{
    "name": "",
    "description": "",
    "spec": [
        {
            "question_name": "GITHUB TOKEN",
            "question_description": "",
            "required": true,
            "type": "password",
            "variable": "GitHub_TOKEN",
            "min": 0,
            "max": 64,
            "default": "$encrypted$",
            "choices": "",
            "new_question": true
        }
    ]
}
sky-joker picture sky-joker on 22 Oct 2019
👍1

I think something like this should help:

https://github.com/ansible/awx/issues/5062#issuecomment-544946330

I can put together a PR shortly.

ryanpetrello picture ryanpetrello on 22 Oct 2019

Applying this patch(https://github.com/ansible/awx/issues/5062#issuecomment-544946330) resolved this issue :)
Thank you @ryanpetrello for quick and polite response!

sky-joker picture sky-joker on 22 Oct 2019

Verified this issue, the job is launched successfully now. Hence, closing the issue

appuk picture appuk on 25 Oct 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

IMOKURI picture IMOKURI  路  3Comments
gamuniz picture gamuniz  路  3Comments
agaffney picture agaffney  路  3Comments
darkaxl picture darkaxl  路  3Comments
beenje picture beenje  路  3Comments