Umap: [RFC] Feature: ParametricUMAP PyTorch implementation

[leriomaggio]

In the spirit of the PR #578 which allows dependencies not to interfere too much with the test suite, I was wondering whether you would consider adding an extra dependency 😅

More seriously, I started working to an alternative PyTorch-based implementation for ParametricUMAP (will soon sync the branch on my fork)

If you would generally appreciate the idea and the contribution, I will elaborate better here my UEP (_UMAP Enhancement Proposal_) to have the two implementations co-existing, with the less impact possible in the API for backward compatibility.

[lmcinnes]

I would like to get @timsainb 's opinion on this. Personally I can see a lot of value in supporting other deep learning frameworks, and since dependencies can be optional I don't see and extra potential dependency as a problem. I do know that other have asked about a PyTorch version, so there is certainly interest. I think it would be a very exciting addition. The key would be to ensure it syncs up with @timsainb 's implementation in TF and doesn't cause (too many) weird interactions as to what happens under the different backends. It sounds like you have thought about that, so please feel free to elaborate on that -- I would like to hear more.

[timsainb]

I think that would be a great addition.

[lmcinnes]

If Tim is on board, then I think we can call this a go! Really looking forward to seeing this in place.

[leriomaggio]

Fantastic!! 🙌
Thanks so much @timsainb and @lmcinnes for getting back to me about this!
Will submit my initial PR very soon so that we can start discussing design options there.

I will be sharing quick ideas here as well while I do work on it, so that we could potentially upvote/downvote options very quickly - will be starting soon in my next post!

Thank you so much :)

[leriomaggio]

Q: How do we allow multiple DL backend for Parametric UMAP (so far, tf and torch)

_iow_, what implementation will I import whenever I will do:

from umap.parametric_umap import ParametricUMAP

My idea would be in favour of a _Convention of Configuration_ based approach - so no configuration file a-la-keras to choose the backend

General Idea

• If torch is available but not tf --> go with torch
• if tf is available but not torch --> go with tf
• if tf and torch are both installed in the environment --> go by default with @timsainb / tf implementation (unless specified - as in _importing the default torch-based class/implementation_)

Similarly, whenever a custom encoder &| decoder parameter(s) are provided in the constructor, the choice of the backend will be directly derived from the type of the object.

Again, the goal is to be as less disruptive as possible with current APIs

What do you think? :)

[lmcinnes]

Sounds ideal to me. I really like the idea of having the least friction for users, and this would meet that goal. It may be worth providing a warning, or some kind of feedback to the user, in the fallback/default case, so that they know that a decision has been made without their input. That is, if both tf and torch are installed, and no custom encoder or decoder is provided, let the user know that tf will be used, and how they can force the torch option if they would prefer.

In practice I assume the main reasons to have a strong preference is that the user has one but not the other installed, or because they want to define their own network architecture (encoder/decoder) in their preferred framework. Are there other reasons to have a significant preference?

[cyrusmaher]

@lmcinnes I'm currently running into problems with tensorflow's protobuf size limits when I try to run on larger datasets. Maybe pytorch won't have this issue? I haven't been able to find an easy workaround so far.

[leriomaggio]

@cyrusmaher I have the impression that this may depend from many different factors: e.g. _available hardware architecture_, or _dataset size_ just to mention the two most obvious.

One thing for sure is that torch does not use protobuf so if you think your issue is related to that, it _might_ not have the same issue.
That being said, It is very hard to think of possible workaround or solution without having a clearer idea of the actual problem and errors.
So, my personal piece of advice here, would be to open a specific issue on that - if at all possible - specifying the environment used to replicate the issue.
In this way, it would be easier to understand _what_ and _where_ is the problem :)

HTH