Molecule: EC2 create.yml does not refer to the region in molecule.yml

[robertdebock]

Issue Type

• Bug report

Molecule and Ansible details

molecule --version
molecule, version 2.19.0

ansible --version
ansible 2.7.1
  config file = None
  configured module search path = [u'/home/robertdb/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.15 (default, Oct 15 2018, 15:26:09) [GCC 8.2.1 20180801 (Red Hat 8.2.1-2)]

Molecule installation method (one of):

• pip

Ansible installation method (one of):

• pip

Detail any linters or test runners used:

Desired Behavior

The region described in molecule.yml to be picked up in create.yml.

Sample of molecule.yml:

platforms:
  - name: instance
    image: ami-c86c3f23
    instance_type: t2.micro
    vpc_subnet_id: subnet-0e688067
    region: eu-central-1

Actual Behaviour

    TASK [Create molecule instance(s)] *********************************************
    task path: /home/robertdb/Documents/github.com/robertdebock/ansible-role-bootstrap/molecule/ec2/create.yml:61
    <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: robertdb
    <127.0.0.1> EXEC /bin/sh -c 'echo ~robertdb && sleep 0'
    <127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/robertdb/.ansible/tmp/ansible-tmp-1541617983.58-52416710308170 `" && echo ansible-tmp-1541617983.58-52416710308170="` echo /home/robertdb/.ansible/tmp/ansible-tmp-1541617983.58-52416710308170 `" ) && sleep 0'
    Using module file /usr/lib/python2.7/site-packages/ansible/modules/cloud/amazon/ec2.py
    <127.0.0.1> PUT /home/robertdb/.ansible/tmp/ansible-local-32028s7w6GE/tmpVNedoV TO /home/robertdb/.ansible/tmp/ansible-tmp-1541617983.58-52416710308170/AnsiballZ_ec2.py
    <127.0.0.1> PUT /home/robertdb/.ansible/tmp/ansible-local-32028s7w6GE/tmptebQEI TO /home/robertdb/.ansible/tmp/ansible-tmp-1541617983.58-52416710308170/async_wrapper.py
    <127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/robertdb/.ansible/tmp/ansible-tmp-1541617983.58-52416710308170/ /home/robertdb/.ansible/tmp/ansible-tmp-1541617983.58-52416710308170/AnsiballZ_ec2.py /home/robertdb/.ansible/tmp/ansible-tmp-1541617983.58-52416710308170/async_wrapper.py && sleep 0'
    <127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2 /home/robertdb/.ansible/tmp/ansible-tmp-1541617983.58-52416710308170/async_wrapper.py 818617875044 7200 /home/robertdb/.ansible/tmp/ansible-tmp-1541617983.58-52416710308170/AnsiballZ_ec2.py _ && sleep 0'
    changed: [localhost] => (item={'instance_type': u't2.micro', 'image': u'ami-a5b196c0', 'vpc_subnet_id': u'subnet-6456fd1f', 'region': u'eu-central-1', 'name': u'instance'}) => {
        "ansible_job_id": "818617875044.32137", 
        "changed": true, 
        "finished": 0, 
        "item": {
            "image": "ami-a5b196c0", 
            "instance_type": "t2.micro", 
            "name": "instance", 
            "region": "eu-central-1", 
            "vpc_subnet_id": "subnet-6456fd1f"
        }, 
        "results_file": "/home/robertdb/.ansible_async/818617875044.32137", 
        "started": 1
    }

    TASK [Wait for instance(s) creation to complete] *******************************
    task path: /home/robertdb/Documents/github.com/robertdebock/ansible-role-bootstrap/molecule/ec2/create.yml:80
    <127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: robertdb
    <127.0.0.1> EXEC /bin/sh -c 'echo ~robertdb && sleep 0'
    <127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/robertdb/.ansible/tmp/ansible-tmp-1541617984.89-185733746994798 `" && echo ansible-tmp-1541617984.89-185733746994798="` echo /home/robertdb/.ansible/tmp/ansible-tmp-1541617984.89-185733746994798 `" ) && sleep 0'
    Using module file /usr/lib/python2.7/site-packages/ansible/modules/utilities/logic/async_status.py
    <127.0.0.1> PUT /home/robertdb/.ansible/tmp/ansible-local-32028s7w6GE/tmp9kYCxO TO /home/robertdb/.ansible/tmp/ansible-tmp-1541617984.89-185733746994798/AnsiballZ_async_status.py
    <127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/robertdb/.ansible/tmp/ansible-tmp-1541617984.89-185733746994798/ /home/robertdb/.ansible/tmp/ansible-tmp-1541617984.89-185733746994798/AnsiballZ_async_status.py && sleep 0'
    <127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2 /home/robertdb/.ansible/tmp/ansible-tmp-1541617984.89-185733746994798/AnsiballZ_async_status.py && sleep 0'
    <127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/robertdb/.ansible/tmp/ansible-tmp-1541617984.89-185733746994798/ > /dev/null 2>&1 && sleep 0'
    failed: [localhost] (item={'_ansible_parsed': True, '_ansible_item_result': True, '_ansible_item_label': {'instance_type': u't2.micro', 'image': u'ami-a5b196c0', 'vpc_subnet_id': u'subnet-6456fd1f', 'region': u'eu-central-1', 'name': u'instance'}, u'ansible_job_id': u'818617875044.32137', 'failed': False, u'started': 1, 'changed': True, 'item': {'instance_type': u't2.micro', 'image': u'ami-a5b196c0', 'vpc_subnet_id': u'subnet-6456fd1f', 'region': u'eu-central-1', 'name': u'instance'}, u'finished': 0, u'results_file': u'/home/robertdb/.ansible_async/818617875044.32137', '_ansible_ignore_errors': None, '_ansible_no_log': False}) => {
        "ansible_job_id": "818617875044.32137", 
        "attempts": 1, 
        "changed": false, 
        "finished": 1, 
        "invocation": {
            "module_args": {
                "assign_public_ip": true, 
                "aws_access_key": null, 
                "aws_secret_key": null, 
                "count": 1, 
                "count_tag": "{'instance': 'instance'}", 
                "ebs_optimized": false, 
                "ec2_url": null, 
                "exact_count": 1, 
                "group": [
                    "molecule"
                ], 
                "group_id": null, 
                "id": null, 
                "image": "ami-a5b196c0", 
                "instance_ids": null, 
                "instance_initiated_shutdown_behavior": null, 
                "instance_profile_name": null, 
                "instance_tags": {
                    "instance": "instance"
                }, 
                "instance_type": "t2.micro", 
                "kernel": null, 
                "key_name": "molecule_key", 
                "monitoring": false, 
                "network_interfaces": null, 
                "placement_group": null, 
                "private_ip": null, 
                "profile": null, 
                "ramdisk": null, 
                "region": null, 
                "security_token": null, 
                "source_dest_check": null, 
                "spot_launch_group": null, 
                "spot_price": null, 
                "spot_type": "one-time", 
                "spot_wait_timeout": "600", 
                "state": "present", 
                "tenancy": "default", 
                "termination_protection": null, 
                "user_data": null, 
                "validate_certs": true, 
                "volumes": null, 
                "vpc_subnet_id": "subnet-6456fd1f", 
                "wait": true, 
                "wait_timeout": "300", 
                "zone": null
            }
        }, 
        "item": {
            "ansible_job_id": "818617875044.32137", 
            "changed": true, 
            "failed": false, 
            "finished": 0, 
            "item": {
                "image": "ami-a5b196c0", 
                "instance_type": "t2.micro", 
                "name": "instance", 
                "region": "eu-central-1", 
                "vpc_subnet_id": "subnet-6456fd1f"
            }, 
            "results_file": "/home/robertdb/.ansible_async/818617875044.32137", 
            "started": 1
        }, 
        "msg": "Either region or ec2_url must be specified"
    }

    PLAY RECAP *********************************************************************
    localhost                  : ok=6    changed=4    unreachable=0    failed=1   

I modified create.yml a bit to include that region:

    - name: Create molecule instance(s)
      ec2:
        key_name: "{{ keypair_name }}"
        image: "{{ item.image }}"
        region: "{{ item.region }}"
        instance_type: "{{ item.instance_type }}"
        vpc_subnet_id: "{{ item.vpc_subnet_id }}"
        group: "{{ security_group_name }}"
        instance_tags:
          instance: "{{ item.name }}"
        wait: true
        assign_public_ip: true
        exact_count: 1
        count_tag:
          instance: "{{ item.name }}"
      register: server
      with_items: "{{ molecule_yml.platforms }}"
      async: 7200
      poll: 0

Not sure if I'm using the molecule ec2 driver incorrectly, or that this is an actual bug.

[robertdebock]

I think I found the issue, PEBCAK: to solve this issue, set the region:

export EC2_REGION=eu-central-1

[gundalow]

@robertdebock hi! Is this something that needs making clearer in the docs somewhere?

[robertdebock]

Yes, to be honest, the documentation was not clear to me. I'll review and make a merge request where possible.

[gundalow]

@robertdebock That would be great, thanks!

[robertdebock]

Hm, I'm in doubt; the module documentation of ec2 mentions to set the variables.

So adding it to molecule's documentation would result in duplicate text.

I suggest to close the issue; even though I opened it. ;-)

[gundalow]

Thanks for confirming.

[jonashackt]

I think this issue persists. I just started using the EC2 infrastructure provider for Molecule in my example project jonashackt/molecule-ansible-docker-vagrant and got the same result. I think the Molecule EC2 docs leaves out some important things:

1. You need boto / boto3 as the AWS connectivity library installed via pip3 install boto boto3
2. One needs to install AWS CLI with pip3 install awscli and configure it via aws configure to use the correct credentials, which will be stored in ~/.aws/credentials and used by Molecule / Ansible / Boto from there
3. You need to edit the generated molecule.yml and add the region: "{{ item.region }}" configuration, as @robertdebock mentioned (or use the environment variable)

The latter is some kind of discrepancy - since all the other configuration parameters are generated correctly by a molecule init role --driver-name ec2 --role-name yourRoleName command - especially the credentials in ~/.aws/credentials are picked up correctly, but only the region setting from ~/.aws/config is not. I guess that's hard to guess for new users - and should be mentioned in the docs IMHO (I could file a PR, if I would know where to 😃).