Algo: DigitalOcean: Error creating tags
OS / Environment
MacOS
Ansible version
2.2.0.0
Version of components from requirements.txt
msrestazure: 0.4.17
setuptools: 38.2.3
ansible: 2.2.0.0
dopy: 0.3.5
boto: 2.48.0
boto3: 1.4.8
azure: 2.0.0rc5
msrest: 0.4.1
apache-libcloud: 2.2.1
six: 1.11.0
pyopenssl: 17.4.0
jinja2: 2.8
Summary of the problem
Digitalocean deployment fails when setting tags
Steps to reproduce the behavior
Run ./algo for digitalocean region #8, all [y/N] options set to yes.
The way of deployment (cloud or local)
local
Expected behavior
Ansible sets the tags for the droplet
Actual behavior
Get an error:
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Failed as requested from task"}
Full log
What provider would you like to use?
1. DigitalOcean
2. Amazon EC2
3. Microsoft Azure
4. Google Compute Engine
5. Install to existing Ubuntu 16.04 server
Enter the number of your desired provider
: 1
Enter your API token. The token must have read and write permissions (https://cloud.digitalocean.com/settings/api/tokens):
[pasted values will not be displayed]
:
Name the vpn server:
[algo.local]:
What region should the server be located in?
1. Amsterdam (Datacenter 2)
2. Amsterdam (Datacenter 3)
3. Frankfurt
4. London
5. New York (Datacenter 1)
6. New York (Datacenter 2)
7. New York (Datacenter 3)
8. San Francisco (Datacenter 1)
9. San Francisco (Datacenter 2)
10. Singapore
11. Toronto
12. Bangalore
Enter the number of your desired region:
Do you want macOS/iOS clients to enable "VPN On Demand" when connected to cellular networks?
Do you want macOS/iOS clients to enable "VPN On Demand" when connected to Wi-Fi?
List the names of trusted Wi-Fi networks (if any) that macOS/iOS clients exclude from using the VPN (e.g., your home network. Comma-separated value, e.g., HomeNet,OfficeWifi,AlgoWiFi)
:
Do you want to install a DNS resolver on this VPN server, to block ads while surfing?
Do you want each user to have their own account for SSH tunneling?
Do you want to apply operating system security enhancements on the server? (warning: replaces your sshd_config)
Do you want the VPN to support Windows 10 or Linux Desktop clients? (enables compatible ciphers and key exchange, less secure)
Do you want to retain the CA key? (required to add users in the future, but less secure)
PLAY [Configure the server] ****************
TASK [setup] *********************
ok: [localhost]
TASK [Generate the SSH private key] **************
ok: [localhost]
TASK [Generate the SSH public key] ***************
ok: [localhost]
TASK [Change mode for the SSH private key] ***********
ok: [localhost]
TASK [Ensure the dynamic inventory exists] ***********
ok: [localhost]
TASK [cloud-digitalocean : Set the DigitalOcean Access Token fact] ***
ok: [localhost]
TASK [cloud-digitalocean : Delete the existing Algo SSH keys] ******
FAILED - RETRYING: TASK: cloud-digitalocean : Delete the existing Algo SSH keys (10 retries left).
ok: [localhost]
TASK [cloud-digitalocean : Upload the SSH key] ***********
changed: [localhost]
TASK [cloud-digitalocean : Creating a droplet...] **********
ok: [localhost]
TASK [cloud-digitalocean : Add the droplet to an inventory group] ****
changed: [localhost]
TASK [cloud-digitalocean : set_fact] *************
ok: [localhost]
TASK [cloud-digitalocean : Tag the droplet] ************
failed: [localhost] (item=staging) => {"failed": true, "item": "staging", "msg": "error tagging resource '73204383': param is missing or the value is empty: resources"}
failed: [localhost] (item=dbserver) => {"failed": true, "item": "dbserver", "msg": "error tagging resource '73204383': param is missing or the value is empty: resources"}
Debugger invoked
(debug) c
TASK [cloud-digitalocean : debug] **************
ok: [localhost] => {
"fail_hint": [
"Sorry, but something went wrong!",
"Please check the troubleshooting guide.",
"https://trailofbits.github.io/algo/troubleshooting.html"
]
}
TASK [cloud-digitalocean : fail] ***************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Failed as requested from task"}
Debugger invoked
(debug) c
PLAY RECAP ***********************
localhost : ok=12 changed=2 unreachable=0 failed=1
hlummis
All 59 comments
Am also experience this issue.
mackwage
on 1 Dec 2017
I added the droplet's tag by hand. The tag is Environment:Algo and after that, the installation worked
thalessr
on 1 Dec 2017
same problem
Mottoweb
on 1 Dec 2017
@thalessr, how/where did you add the tag?
gtb104
on 1 Dec 2017
@gtb104, basically follow this tutorial https://www.digitalocean.com/community/tutorials/how-to-tag-digitalocean-droplets and as a the droplet's tag, add Environment:Algo because the installation's script will find the droplet by this tag.
A reminder in the command ansible-playbook deploy.yml -t digitalocean,vpn,cloud -e 'do_access_token=my_secret_token do_server_name=algo.local do_region=ams2' specifies algo.local as your droplet name and the tag must be add on this droplet.
thalessr
on 1 Dec 2017
@thalessr, after tagging, it there anything you need to do to "continue the installation", or was that the last step?
gtb104
on 1 Dec 2017
If there's a problem with the readme, please submit a PR!
dguido
on 1 Dec 2017
Looks like an issue has been opened against Ansible.
davidemyers
on 1 Dec 2017
@gtb104 yes, I forgot to mention it.. my bad =(. However, it is needed to rerun the installation commands with the same parameters. Eg: if in your first attempt it was used ansible-playbook deploy.yml -t digitalocean,vpn,cloud -e 'do_access_token=my_secret_token do_server_name=algo.local do_region=ams2' rerun it after adding the tags.
thalessr
on 1 Dec 2017
Until the patch has been merged and ansible gets updated, as a quick workaround you can make the little change from the commit in your local installation (see changed code in the link).
Ducki
on 2 Dec 2017
Here's a patch file you can use to get going again. Save the following patch to your algo directory in a file called, for example, do_tag.patch. Then run patch -b -p0 < do_tag.patch.
--- ./env/lib/python2.7/site-packages/ansible/modules/core/cloud/digital_ocean/digital_ocean_tag.py.orig 2017-12-02 09:42:56.044524930 -0500
+++ ./env/lib/python2.7/site-packages/ansible/modules/core/cloud/digital_ocean/digital_ocean_tag.py 2017-12-02 09:43:14.245074861 -0500
@@ -242,7 +242,7 @@
module = AnsibleModule(
argument_spec=dict(
name=dict(type='str', required=True),
- resource_id=dict(aliases=['droplet_id'], type='int'),
+ resource_id=dict(aliases=['droplet_id'], type='str'),
resource_type=dict(choices=['droplet'], default='droplet'),
state=dict(choices=['present', 'absent'], default='present'),
api_token=dict(aliases=['API_TOKEN'], no_log=True),
@thalessr Hmm, I added that tag (Environment:Algo) to my droplet in Digital Ocean, but re-running the deploy script yields the same error as before. @gtb104 Were you able to get it to work this way? Was there anything else you had to do to make it work?
tscizzle
on 3 Dec 2017
Thanks to @davidemyers, I created his patch file and ran the patch command before running ./algo. The Algo script completed successfully.
useradd-deploy
on 3 Dec 2017
Confirmed that worked for me. Thank you!
tscizzle
on 3 Dec 2017
@davidemyers works like a charm, thanks a lot!
Mottoweb
on 4 Dec 2017
The patch unfortunately does not work for me :(
TASK [cloud-digitalocean : Tag the droplet] ************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "error tagging resource '73812578': The resource you were accessing could not be found."}
Do you guys have any other workaround? Thanks!
ghost
on 5 Dec 2017
useradd-deploy
on 5 Dec 2017
Can't reproduce now. Looks like DO accepts both int and str now. Could anyone verify also?
jackivanov
on 8 Dec 2017
I agree, with the patch I posted above backed out I can create droplets again.
davidemyers
on 8 Dec 2017
@gunph1ld This has started to fail again without the patch, and resolves with the patch applied.
ElliotSanchez
on 9 Jan 2018
Still fails with the patch.
spartanhooah
on 10 Jan 2018
Confirmed failing with and without the patch.
TACIXAT
on 11 Jan 2018
Patch works, I had to actually use the patch (I was doing it by hand in a copied directory, something was failing in that workflow).
Confirmed with their API in CURL that integers are now not accepted. A quoted version did successfully tag my droplet.
$ curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $KEY" -d '{"resources":[{"resource_id":$RESOURCE,"resource_type":"droplet"}]}' "https://api.digitalocean.com/v2/tags/awesome/resources"
{"id":"parameter_missing","message":"param is missing or the value is empty: resources"}
The fix looks live in Ansible, maybe it hasn't reached pip yet. Until that hits, just patch manually.
TACIXAT
on 11 Jan 2018
Alternatively, to solve this temporarily, just upgrade the the devel branch of Ansible. Just run the following:
python -m pip --upgrade install git+https://github.com/ansible/ansible@devel
This will install the bleeding edge Ansible version 2.5.0.
It is generally a bad idea to do this, as certain features could potentially be deprecated in newer versions, but I can verify that I only get deprecation warnings, and that deploying works properly. I haven't thoroughly tested hosting providers other than DigitalOcean however.
But like @douggard said, it should be soon that the fix reaches pip anyways.
jmerizia
on 11 Jan 2018
I've tried both applying the patch and upgrading to the Ansible devel branch. I still see an error on the same step but the error has changed.
Before patching/upgrading:
TASK [cloud-digitalocean : Tag the droplet] ************************************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "error tagging resource '78855946': param is missing or the value is empty: resources"}
After patching/upgrading:
TASK [cloud-digitalocean : Tag the droplet] ***************************************************************************************************
fatal: [localhost]: FAILED! => {"attempts": 1, "changed": false, "msg": "error tagging resource '78855946': The resource you were accessing could not be found."}
It works if I manually tag the droplet as environment:algo and then try again.
GavinSchalliol
on 15 Jan 2018
Even manually tagging the droplet after patching didn't work for me. I get the first message @GavinSchalliol posted. I have verified that the patch has been applied to digital_ocean_tag.py
spartanhooah
on 16 Jan 2018
Same as @spartanhooah....tagging doesn't work and neither does patching.
cclayb
on 17 Jan 2018
Still doesn't work for me. I removed my algo directory and re-cloned the project. Here are the errors I'm seeing:
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "error tagging resource '79440844': The resource you were accessing could not be found."}
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Failed as requested from task"}
spartanhooah
on 20 Jan 2018
I'm having the same issue. Is it possible that the "resource could not be found" refers to the droplet, not the tag?
I tried with the default ansible (2.2?) and 2.4.3.0 installed with pip install ansible --upgrade. The result was the same, though the new version give some depreciation warnings.
dansherman
on 1 Feb 2018
Is anyone still looking into why this is happening for some of us?
spartanhooah
on 10 Feb 2018
I can't reproduce this. Can anyone run ansible with -vvvv and show me the output?
jackivanov
on 10 Feb 2018
Alright, here's the output after I gave my choices:
< PLAY [Configure the server] >
-----------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
a ______________
< TASK [setup] >
--------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
ok: [localhost]
_____________________________________
< TASK [Generate the SSH private key] >
-------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
changed: [localhost]
____________________________________
< TASK [Generate the SSH public key] >
------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
ok: [localhost]
____________________________________________
< TASK [Change mode for the SSH private key] >
--------------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
ok: [localhost]
____________________________________________
< TASK [Ensure the dynamic inventory exists] >
--------------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
changed: [localhost]
________________________________________________________
/ TASK [cloud-digitalocean : Set the DigitalOcean Access \
\ Token fact] /
--------------------------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
ok: [localhost]
_________________________________________________________
/ TASK [cloud-digitalocean : Delete the existing Algo SSH \
\ keys] /
---------------------------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
FAILED - RETRYING: TASK: cloud-digitalocean : Delete the existing Algo SSH keys (10 retries left).
ok: [localhost]
________________________________________________
< TASK [cloud-digitalocean : Upload the SSH key] >
------------------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
changed: [localhost]
___________________________________________________
< TASK [cloud-digitalocean : Creating a droplet...] >
---------------------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
changed: [localhost]
____________________________________________________________
/ TASK [cloud-digitalocean : Add the droplet to an inventory \
\ group] /
------------------------------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
changed: [localhost]
______________________________________
< TASK [cloud-digitalocean : set_fact] >
--------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
ok: [localhost]
_____________________________________________
< TASK [cloud-digitalocean : Tag the droplet] >
---------------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "error tagging resource '82000470': The resource you were accessing could not be found."}
___________________________________
< TASK [cloud-digitalocean : debug] >
-----------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
ok: [localhost] => {
"fail_hint": [
"Sorry, but something went wrong!",
"Please check the troubleshooting guide.",
"https://trailofbits.github.io/algo/troubleshooting.html"
]
}
__________________________________
< TASK [cloud-digitalocean : fail] >
----------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Failed as requested from task"}
____________
< PLAY RECAP >
------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
localhost : ok=12 changed=5 unreachable=0 failed=1
I DM'd an output with -vvvv directly to gunph1ld via Slack
jproz
on 12 Feb 2018
Any luck with our logs?
spartanhooah
on 17 Feb 2018
I can't reproduce on a clean environment, tagging works well. Could you clean up your environment and try from scratch?
jackivanov
on 17 Feb 2018
Sure, I can try. What all would I need to do to clean up the environment? Re-install Linux, or something less drastic?
spartanhooah
on 17 Feb 2018
No, don't need to reinstall the system. Just remove the algo directory and follow the steps
jackivanov
on 17 Feb 2018
Ok. Same thing (I removed the creatures to make the logs more compact):
```
< PLAY [Configure the server] >
< TASK [setup] >
ok: [localhost]
< TASK [Generate the SSH private key] >
ok: [localhost]
< TASK [Generate the SSH public key] >
ok: [localhost]
< TASK [Change mode for the SSH private key] >
ok: [localhost]
< TASK [Ensure the dynamic inventory exists] >
ok: [localhost]
/ TASK [cloud-digitalocean : Set the DigitalOcean Access \
\ Token fact] /
ok: [localhost]
/ TASK [cloud-digitalocean : Delete the existing Algo SSH \
\ keys] /
FAILED - RETRYING: TASK: cloud-digitalocean : Delete the existing Algo SSH keys (10 retries left).
ok: [localhost]
< TASK [cloud-digitalocean : Upload the SSH key] >
changed: [localhost]
< TASK [cloud-digitalocean : Creating a droplet...] >
ok: [localhost]
/ TASK [cloud-digitalocean : Add the droplet to an inventory \
\ group] /
changed: [localhost]
< TASK [cloud-digitalocean : set_fact] >
ok: [localhost]
< TASK [cloud-digitalocean : Tag the droplet] >
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "error tagging resource '82815151': The resource you were accessing could not be found."}
< TASK [cloud-digitalocean : debug] >
ok: [localhost] => {
"fail_hint": [
"Sorry, but something went wrong!",
"Please check the troubleshooting guide.",
"https://trailofbits.github.io/algo/troubleshooting.html"
]
}
< TASK [cloud-digitalocean : fail] >
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Failed as requested from task"}
< PLAY RECAP >
localhost : ok=12 changed=2 unreachable=0 failed=1
````
spartanhooah
on 17 Feb 2018
I'm having the same error output as spartanhooah, I'm using a brand new DO droplet running Ubuntu 16 as the client machine, I've tried the ansible2.4 branch and still no success. Maybe a workaround would be to remove the whole tagging process?
ghost
on 22 Feb 2018
I see the same problem as @beachfork and @spartanhooah. I downloaded a fresh version today. If I try to add a tag to the droplet and then run algo I get the following error:
TASK [Check the system] ******************
fatal: [???.65.225.???]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Warning: Permanently added '???.65.225.???' (ECDSA) to the list of known hosts.\r\nPermission denied (publickey).\r\n", "unreachable": true}
rapvelopment
on 25 Feb 2018
Is this issue still being investigated? Is there anything more the handful of us having the problem can do to help?
spartanhooah
on 3 Mar 2018
I second @spartanhooah's question.
rapvelopment
on 7 Mar 2018
I can't reproduce this issue either and I primarily use Algo on DigitalOcean. As Jack said, please delete and redownload Algo. You may also want to clean up your Digital Ocean account (ssh keys, etc).
dguido
on 7 Mar 2018
I'll try cleaning up the DO account, but I've already started from scratch on my local machine several times.
spartanhooah
on 8 Mar 2018
@spartanhooah I've finally figured out what was going on, it's not on Algo side, but it's something wrong on DO side. I've used doctl to manually clear my tags, using the command doctl compute tag del X and managed to remotely deploy algo normally! Hope it helps!
ghost
on 8 Mar 2018
@beachfork where did you put this new command? does the "X" represent a tag or is that part of the command? I'm not a DO expert.
jproz
on 8 Mar 2018
I also sent a -vvvv log to @dguido via slack
jproz
on 8 Mar 2018
@jproz Hey! Use doctl compute tag ls to see all the tags the you have, but you just need to clear the environment:algo one, so it would be doctl compute tag del enviroment:algo. Try to use the help with the doctl command when in doubt.
ghost
on 9 Mar 2018
@beachfork So is it a problem when there is already an existing tag? For instance, I already have an algo vpn server running and I am looking to build a new one before tearing down the old one. Thanks.
rapvelopment
on 10 Mar 2018
@dguido I tried @beachfork's suggestion; I deleted the tag on the existing vpn server; I put new ssh keys on DO. I continue to get the same error. What else can I try?
rapvelopment
on 10 Mar 2018
OK Guys - so I finally got around to doing what @beachfork suggested and everything worked (finally - after weeks and weeks). @dguido @jackivanov - It appears the issues is somewhere around tag reuse (specifically environment:algo) - I had to do a on doctl tag delete --force to get it to work.
jproz
on 12 Mar 2018
That鈥檚 excellent to hear. Jack and I are still unable to reproduce this issue. Can you submit a PR for the troubleshooting guide to address it? Thanks!
dguido
on 12 Mar 2018
@beachfork @kenydub how do I run that command? @beachfork mentioned snap in the gitter chat, but I've never heard of it.
spartanhooah
on 14 Mar 2018
@jproz @dguido It works!
@spartanhooah Here is what I did:
- Download doctl.
- Run
doctl auth init; it will ask you for your token which you can get (or generate) on the API tab at DigitalOcean. - Once you are authorized on DO, you can run
doctl compute tag listto see the list of tags; - Run
doctl compute tag delete enivronment:algo --forceto delete the environment:algo tag. - Finally run
doctl compute tag listto make sure that the tag has been deleted. - Run algo as directed.
Very strange error indeed. Truly a case of bit rot...
rapvelopment
on 15 Mar 2018
That did it, thank you!!
spartanhooah
on 16 Mar 2018
@dguido I don't know if this is a useful observation for you or not, but I noticed that the tag on DO is now Environment:Algo whereas it used to be environment:algo. Is it possible they changed the system to recognize case. If @jproz or @beachfork have not submitted a PR for the trouble shooting guide, I will do it.
rapvelopment
on 21 Mar 2018
@kenydub There's a spelling mistake in your proposed command 4.:
This is correct:
doctl compute tag delete environment:algo --force
(not enivronment)
This is a great fix btw :)
y0ast
on 30 Apr 2018
The fix for this would probably be something like - during the install if the user picks DO as the provider, the Algo scripts would use the user given API key and send a curl command with the following:
curl -X DELETE -H "Content-Type: application/json" -H "Authorization: Bearer USER_API_KEY" "https://api.digitalocean.com/v2/tags/environment:algo"
I'm sorry I can't be of any more help since I'm completely lost with Ansible, and I don't know if the maintainers would consider this a dirty fix, since trying to delete a tag that actually maybe not even exist. This problem seems to affect users that have been using Algo from a long time, before DO changed something with the tagging on their side.
pguizeline
on 9 May 2018
Related issues
xhdix
路
3Comments
jwebbstevens
路
4Comments
mikaelf
路
5Comments
Stanpol
路
4Comments
samkelleher
路
3Comments
Most helpful comment
Here's a patch file you can use to get going again. Save the following patch to your
algodirectory in a file called, for example,do_tag.patch. Then runpatch -b -p0 < do_tag.patch.