Ansible: Deprecate digital_ocean module

[ansibot]

From @kontrafiktion on 2016-07-20T12:08:25Z

ISSUE TYPE

• Feature Idea
  

  COMPONENT NAME

digital_ocean module

ANSIBLE VERSION

ansible 2.1.0.0
  config file =
  configured module search path = Default w/o overrides

SUMMARY

There is an overlap between "digital_ocean" and "digital_ocean_sshkey". The first allows to work with droplets _and_ ssh keys, the latter only handles ssh keys.
I propose to deprecate the existing digital_ocean module and create a new digital_ocean_droplet module.

DETAILS

1. The new digital_ocean_droplet module should not use dopy but simply ansible.module_utils.urls
2. It must be ensured that all ssh features of the digital_ocean module are covered by the digital_ocean_sshkey module

Copied from original issue: ansible/ansible-modules-core#4219

[ansibot]

From @bcoca on 2016-07-20T12:08:25Z

I think either idea works for us, now to bikeshed: do_droplet as a name?

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

I agree that dopy should be removed as it is not very well maintained. However, instead of using ansible.module_utils.urls, do you think we could use https://github.com/koalalorenzo/python-digitalocean instead since it is maintained very well?

[ansibot]

From @kontrafiktion on 2016-07-20T12:08:25Z

@harneksidhu I would prefer to remove _any_ extra dependencies
@bcoca I like do_droplet, but then we would have to rename all modules. Kills backward compatibility – or is it possible to have an alias for a module?

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

@kontrafiktion I would love to help with the development of this module if you need any assistance.

My main concern about only using ansible.module_utils.urls is that digital ocean appears to have a pipeline of features coming in (https://beta.digitalocean.com/). It would be hard to quickly add these new features onto Ansible on our own since there is already a large community of developers working on that initiative within python-digitalocean (they already started to work on block storage support as an example https://github.com/koalalorenzo/python-digitalocean/blob/master/digitalocean/Volume.py).

I am open to helping with any direction chosen of course!

[ansibot]

From @bcoca on 2016-07-20T12:08:25Z

@kontrafiktion to alias a module just create a symlink with leading underscore _alias that points to the new name (this was first feature i added as ansible employee!).

As for using the python library or not, this is a choice i'm leaving up to the community, I have barely looked at the DO API and don't know how complex/or not it is and how easy would it be to use directly or what lag the 'good library' has vs the features.

We normally prefer using an established library, but in this case that seems to be in flux right now. We have been 'bitten' by either the API or library being in flux (docker , azure ...) so I know it is not going to be a smooth ride either way.

[ansibot]

From @jctanner on 2016-07-20T12:08:25Z

@mgregson, @alukovenko ping

[ansibot]

From @alukovenko on 2016-07-20T12:08:25Z

DO API is not really complex, and I would also like to remove external dependencies. @harneksidhu concern about upcoming features is valid; however, those features will still need to be included in Ansible module regardless of internal implementation - at the end of the day, we still need to update the module itself.

One additional concern about using external modules is that it uses separate versioning / branching, which can (and will) lead to bugs that are hard to reproduce.

Given the fact that DO API is quite straightforward, I would prefer to:

1. Create our own implementation of API
2. Create new modules according to @kontrafiktion proposal
3. Deprecate existing modules, at the same time maintaining compatibility using _alias

We could use _do_droplet_ and _do_storage_ names for the new implementation

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

Should we consolidate this with #4200 and include block storage support as well?

[ansibot]

From @alukovenko on 2016-07-20T12:08:25Z

Let's merge #4200 to this issue

[ansibot]

From @alukovenko on 2016-07-20T12:08:25Z

@kontrafiktion , @harneksidhu - it would be great do collaborate on this to add up the effort; would you be willing to contribute to developing this new module(s)?

[ansibot]

From @kontrafiktion on 2016-07-20T12:08:25Z

I already started, I think I have a first version next week

[ansibot]

From @alukovenko on 2016-07-20T12:08:25Z

@kontrafiktion would you like to do it by your own? I would like to join.

[ansibot]

From @kontrafiktion on 2016-07-20T12:08:25Z

You are more than welcome. I have no idea yet, how to split up the work.
I just published my work in progress at https://github.com/kontrafiktion/ansible-modules-core/tree/4219_deprecate_digital_ocean_module

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

@kontrafiktion Thank's for taking a shot at this! Not sure how to split up the work either but am available if you want to send some tasks my way or need help with testing/reviewing code. I'm looking forward to getting rid of dopy.

[ansibot]

From @alukovenko on 2016-07-20T12:08:25Z

@kontrafiktion - would you be able to give us some access for your fork? then we can collaborate in a usual way with creating issues and committing directly...

[ansibot]

From @kontrafiktion on 2016-07-20T12:08:25Z

@alukovenko not sure, what you mean: I can add you as a collaborator. Anybody else as well?

[ansibot]

From @alukovenko on 2016-07-20T12:08:25Z

That's exactly what I meant. Thanks!

[ansibot]

From @alukovenko on 2016-07-20T12:08:25Z

@kontrafiktion: Just a followup: if you turn on Options -> Features -> Issues, then we can use it to split tasks

[ansibot]

From @kontrafiktion on 2016-07-20T12:08:25Z

done

[ansibot]

From @Fale on 2016-07-20T12:08:25Z

DO has deprecated v1 and therefore digital_ocean_sshkey is now broken. I hope the rewrite will soon be completed and merged in Ansible before the next release

[ansibot]

From @kontrafiktion on 2016-07-20T12:08:25Z

The digital_ocean module handles ssh keys as well

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

I should have a module submitted by Monday

[ansibot]

From @pmarques on 2016-07-20T12:08:25Z

Hi guys, I'm think I'm a bit late, I wrote a library inside my playbooks because I needed to manage the Floating IPs. I started it some time ago and I only use the internal ansible modules, ansible fetch_url.
I made it a separated "role" just to open to the community, https://github.com/pmarques/ansible-digitalocean. So, let me know our thoughts about it.

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

@pmarques Can you rename the module using a do_* prefix and submit a PR? An example would be do_floating_ip or something similar. We will be renaming the modules using the prefixes. We can review the code together once you submitted it the PR.

[ansibot]

From @pmarques on 2016-07-20T12:08:25Z

@harneksidhu let me know what do you think. ansible/ansible-modules-core#5221

There is one thing that I don't like, there is no clean way to test if a Floating IP is created DigitalOcean community questions.

[ansibot]

From @andrewsomething on 2016-07-20T12:08:25Z

Hi. I'm looking into porting the existing modules to not depend on dopy. (WIP here: https://github.com/andrewsomething/ansible-modules-core/tree/issues/4219)

The digital_ocean_tag module was already written using just ansible.module_utils.urls.fetch_url It contains two generic classes (Rest and Response) that would be helpful in this. What's the best practice to make them reusable? Create ansible.module_utils.digital_ocean? @pmarques is using a different approach in his Floating IPs branch. It seems like we should standardize on something across the various DigitalOcean modules.

[ansibot]

From @kontrafiktion on 2016-07-20T12:08:25Z

We discussed exactly that question in this thread: https://github.com/kontrafiktion/ansible-modules-core/issues/1
For now we decided to first implement everything without a common module (simply using copy/paste). Only when we really know what should belong in the common module, we want to move forward. And the question even is: does this belong into a 'digital ocean' utils module or should it rather be part of the 'urls' module?
And: ansible-modules-core is planned to be merged with ansible, so we wanted to wait fir that as well

[ansibot]

From @pmarques on 2016-07-20T12:08:25Z

@andrewsomething I had already implemented a while ago, I didn't had seen the digital_ocean_tag module and so I made a simplest version :/ I was also thinking about create a common module in module_utils but like @kontrafiktion I would like to have it working/implemented before moving into there.

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

@andrewsomething

Seems like we are already working on same thing.
Here's my work in progress.

https://github.com/harneksidhu/ansible-modules-core/blob/4219_do_rework/cloud/digital_ocean/do_droplet.py

Are you interested in doing the droplet module yourself or do you want me to complete it? I was planning on finishing it by Thursday. Let me know if i should continue

[ansibot]

From @andrewsomething on 2016-07-20T12:08:25Z

@harneksidhu Awesome stuff. Please continue, but let me know if you aren't able to complete it for some reason and I can pick up where you left off.

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

@andrewsomething

Thank you! I'll ping you for a PR review.

[ansibot]

From @pmarques on 2016-07-20T12:08:25Z

I've the updated to use the same approach of Request / Response ansible/ansible-modules-core#5221

Note that it is a work in progress and maybe needs a cleanup

[ansibot]

From @pmarques on 2016-07-20T12:08:25Z

I also have ported ssh keys to rid off _dopy_.... do_sshkeys.py.

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

@pmarques thanks for porting the ssh key module. I'll use it to test do_droplet that I'm still working on

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

@pmarques Can do_sshkeys use the following keys for the api_token:

api_token with an alias of API_TOKEN

Can it also accept the following environment variables:
DO_API_TOKEN or DO_API_KEY

It would be nice to have this so that it can be consistent with the other modules

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

@pmarques
Can I complete do_sshkeys.py on my branch? I am ready to submit a PR for do_droplet.py but I need do_sshkeys.py completed first in order for it to work since ssh key id is a requirement for the droplet module.

[ansibot]

From @pmarques on 2016-07-20T12:08:25Z

@harneksidhu I'm updating it submit a PR, implementing the alias and add the rename. You can use the Key fingerprint as key ID

[ansibot]

From @pmarques on 2016-07-20T12:08:25Z

@harneksidhu PR submitted, ansible/ansible-modules-core#5275 @andrewsomething if you could give some feedback on this one I appreciate :)

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

Thanks!! :

[ansibot]

From @harneksidhu on 2016-07-20T12:08:25Z

5278 Has been submitted. Will need #5275 to go in first though.

[ansibot]

From @ed11s8c on 2016-07-20T12:08:25Z

Any updates on the progress of removing the requirement for DOPY with digital ocean modules?

[ansibot]

cc @zbal
click here for bot help

[ansibot]

cc @BondAnthony
click here for bot help

[geerlingguy]

Subscribing. Would like to see this happen too :)

[BondAnthony]

resolved_by_pr #33984

[Gunni]

https://github.com/ansible/ansible/pull/33984 has not been merged, so it is not solved until that gets merged, right?

[BondAnthony]

@Gunni that would be correct, the resolved by pr just helps the bot link issues to PR. Reduces the number of times labels are added to the issue and random pings. The issue will still remain open until the PR is merged.

[kbreit]

Any update on this?

[kbreit]

I am developing a module to replace digital_ocean. I can't guarantee I'll ever finish it, but I've got a working prototype in place with a WIP PR created.

[gundalow]

I believe this will be addressed by https://github.com/ansible/ansible/pull/33984