Awx: tower_workflow_template does not accept a survey spec

[RylandDeGregory]

ISSUE TYPE

• Bug Report

SUMMARY

The tower_workflow_template module does not accept a survey specification (in JSON or YAML format).

ENVIRONMENT

• AWX version: 9.2.0
• AWX install method: Kubernetes 1.14.8
• Ansible version: 2.9.3
• Operating System: MacOS 10.15.1
• Web Browser: Chrome for MacOS 80.0.3987.100

STEPS TO REPRODUCE

• Create a Playbook for building a Workflow Template with a survey

---
- hosts: localhost
  gather_facts: false
  tasks:
  - name: Ensure Workflow Template exists
    tower_workflow_template:
      name: R Workflow
      description: A new workflow template
      schema: "{{ lookup('file', '{{ playbook_dir }}/surveys/workflow_schema.yml') }}"
      state: present
      survey_enabled: true
      survey: "{{ lookup('file', '{{ playbook_dir }}/surveys/survey_spec.json') }}"
      tower_username: admin
      tower_password: password
      tower_host: http://localhost:8052
      tower_verify_ssl: false

• Create Survey Specification file (survey_spec.json)

{
    "name": "",
    "description": "",
    "spec": [
        {
            "question_name": "Which AWS Account would you like to query?",
            "question_description": "Enter Account ID",
            "required": true,
            "type": "text",
            "variable": "account",
            "min": 0,
            "max": 20,
            "default": "",
            "choices": "",
            "new_question": true
        }
    ]
}

• Launch the Playbook to build Workflow Template from AWX.

EXPECTED RESULTS

• The job completes successfully without error.
• A Workflow Template is created with the provided survey specification, the same way tower_job_template does.

ACTUAL RESULTS

Playbook always fails with error:
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)\n",

Full error:

{
    "module_stdout": "",
    "module_stderr": "Traceback (most recent call last):\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1581716550.1960776-123130320192707/AnsiballZ_tower_workflow_template.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1581716550.1960776-123130320192707/AnsiballZ_tower_workflow_template.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1581716550.1960776-123130320192707/AnsiballZ_tower_workflow_template.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible.modules.web_infrastructure.ansible_tower.tower_workflow_template', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_tower_workflow_template_payload_8a_qpqdy/ansible_tower_workflow_template_payload.zip/ansible/modules/web_infrastructure/ansible_tower/tower_workflow_template.py\", line 203, in <module>\n  File \"/tmp/ansible_tower_workflow_template_payload_8a_qpqdy/ansible_tower_workflow_template_payload.zip/ansible/modules/web_infrastructure/ansible_tower/tower_workflow_template.py\", line 187, in main\n  File \"/usr/local/lib/python3.6/site-packages/tower_cli/models/base.py\", line 720, in modify\n    return self.write(pk, create_on_missing=create_on_missing, force_on_exists=True, **kwargs)\n  File \"/usr/local/lib/python3.6/site-packages/tower_cli/models/base.py\", line 1191, in write\n    survey_input = json.loads(survey_input.strip(' '))\n  File \"/usr/lib64/python3.6/json/__init__.py\", line 354, in loads\n    return _default_decoder.decode(s)\n  File \"/usr/lib64/python3.6/json/decoder.py\", line 339, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File \"/usr/lib64/python3.6/json/decoder.py\", line 355, in raw_decode\n    obj, end = self.scan_once(s, idx)\njson.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)\n",
    "exception": "Traceback (most recent call last):\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1581716550.1960776-123130320192707/AnsiballZ_tower_workflow_template.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1581716550.1960776-123130320192707/AnsiballZ_tower_workflow_template.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/var/lib/awx/.ansible/tmp/ansible-tmp-1581716550.1960776-123130320192707/AnsiballZ_tower_workflow_template.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible.modules.web_infrastructure.ansible_tower.tower_workflow_template', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_tower_workflow_template_payload_8a_qpqdy/ansible_tower_workflow_template_payload.zip/ansible/modules/web_infrastructure/ansible_tower/tower_workflow_template.py\", line 203, in <module>\n  File \"/tmp/ansible_tower_workflow_template_payload_8a_qpqdy/ansible_tower_workflow_template_payload.zip/ansible/modules/web_infrastructure/ansible_tower/tower_workflow_template.py\", line 187, in main\n  File \"/usr/local/lib/python3.6/site-packages/tower_cli/models/base.py\", line 720, in modify\n    return self.write(pk, create_on_missing=create_on_missing, force_on_exists=True, **kwargs)\n  File \"/usr/local/lib/python3.6/site-packages/tower_cli/models/base.py\", line 1191, in write\n    survey_input = json.loads(survey_input.strip(' '))\n  File \"/usr/lib64/python3.6/json/__init__.py\", line 354, in loads\n    return _default_decoder.decode(s)\n  File \"/usr/lib64/python3.6/json/decoder.py\", line 339, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File \"/usr/lib64/python3.6/json/decoder.py\", line 355, in raw_decode\n    obj, end = self.scan_once(s, idx)\njson.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)\n",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1,
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    },
    "_ansible_no_log": false,
    "changed": false
}

ADDITIONAL INFORMATION

• If the attributes survey and survey_enabled are not included in the module invocation, the Playbook creates the Workflow Template successfully.
• This module is not consistent with similar modules (namely tower_job_template). The survey specification attribute for tower_workflow_template is survey not survey_spec, and the module documentation for tower_workflow_template does not specify a variable type for this attribute (tower_job_template specifies dict).
• The same error occurs when providing either a YAML or JSON formatted survey specification. Though neither is specified in the module documentation, tower_job_template allows for either a JSON or YAML formatted survey spec.
  
  
  
  • Given the error, it seems as this module only expects JSON, and that should be noted in the documentation if true .
  
  
• The JSON code in survey_spec.json can be used with tower_job_template successfully using the same lookup plugin retrieval method.

[RylandDeGregory]

Update:

Tested on AWX 9.1.1 (Kubernetes) and AWX 7.0.1 (RPM on centOS 7). The same behavior was observed.

[wklo]

I found a workaround by patching the following file on your ansible host: PYTHON_LIB_DIR/site-packages/ansible/modules/web_infrastructure/ansible_tower/tower_workflow_template.py:
Change the following line contained in the argument_spec
from
survey=dict(required=False)
to
survey=dict(type='dict', required=False)

Apparently, without the type='dict' specification, the survey input will be treated as a string, instead of a dict, resulting in a JSONDecodeError as logged in this bug.

[ryanpetrello]

@wklo @beeankha sounds like an opportunity for a PR ^

[kedark3]

I am trying to work on creating survey for workflow as well, and I am not sure if I am using correct format, but it fails for me. Can provide more info if needed. I am using Yaml format for survey spec.

[beeankha]

@kedark3 Yes if you could please provide more info, that's always helpful!

The tower_workflow_template module will be getting converted in the upcoming weeks to not rely on tower-cli logic (since tower-cli is being deprecated), so we'll be looking into this issue while that conversion takes place.

[kedark3]

@beeankha thanks. Here's my code snippet of data I am passing on to the module:

name: Test workflow 2
    # https://github.com/ansible/awx/issues/5946
    survey_enabled: true
    survey:
      name: ''
      description: ''
      spec:
      - question_name: Enter a Unique name for your miq appliance
        question_description: Please choose a name that is different from any of existing
          VM names on rhv43
        required: true
        type: text
        variable: miq_vm_name
        min: 0
        max: 20
        default: ''
        choices: ''
    allow_simultaneous: yes
    schema:
      - job_template: Demo Job Template

Also do we have a timeline for changes you mentioned in your comment?

[AlanCoding]

This is very similar to https://github.com/ansible/awx/pull/6186

Change the following line contained in the argument_spec
from
survey=dict(required=False)
to
survey=dict(type='dict', required=False)

👍 to that.

We've been making this exact kind of change in a ton of different modules. This is exactly what we want. This is needed with or without the tower-cli conversion.

[RylandDeGregory]

Any ETA for this to be patched? Issue persists in the new collection and I've found another related bug today: https://github.com/ansible/awx/issues/6355

[AlanCoding]

I might just piggyback the fix onto https://github.com/ansible/awx/pull/6291, since that's establishing some basic test coverage in the neighborhood.

[ryanpetrello]

@AlanCoding did this get resolved in https://github.com/ansible/awx/pull/6291? If so, is it ready for needs_test? If not, what work is left?

[AlanCoding]

Yes, tower_workflow_template is deprecated, but did have the type changed, and we confirmed basic functionality.

tower_workflow_job_template is the replacement for that deprecated module, and also has survey_spec param, which is what we want people to use moving forward.

[appuk]

Verified that the tower_workflow_template_ module works with a survey spec using the playbook specified above
the playbook ran successfully

PLAY RECAP *************************************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

and a new WFJT is created with the survey

It is working as expected, hence closing the issue