Umap: Mixed-type datasets

[amir-ghasemi]

Thanks for sharing the great algorithm and library!

Wondering what would be the recommended way of feeding mixed-type data with some categorical features to UMAP? Binary encoding (possibly with appropriate distance metrics)?

[lmcinnes]

This is actually a problem I am working towards solving in general, but I do not yet have all the bits and pieces required in place in the code yet, so unfortunately there is no easy pluggable solution at this time. As a teaser of what is to come, if you have just one categorical data column you can use the 0.3dev branch and do a fit with X as the numerical data and y as the single categorical column (cast to 0-up integers, one for each category). This is the upcoming supervised (and semi-supervised) dimension reduction. Going a step further is not really well supported yet, but working off the current master on github you can theoretically do something like the following ... first split the data into numeric and categorical, then binarize the categorical data (pd.get_dummies or similar). The something along the lines of:

fit1 = umap.UMAP().fit(numeric_data)
fit2 = umap.UMAP(metric='dice').fit(categorical_data)
prod_graph = fit1.graph.multiply(fit2.graph)
new_graph = 0.99 * prod_graph + 0.01 * (fit1.graph + fit2.graph - prod_graph)
embedding = umap.umap_.simplicial_set_embedding(new_graph, fit1.n_components, fit1.initial_alpha, fit1.a, fit1.b, fit1.gamma, fit1.negative_sample_rate, 200, fit1.init, np.random, False)

More interesting things can be done with more mixed data, but it's really built off variations on this sort of approach.

[amir-ghasemi]

Thanks Leland! This makes sense. The example you provided is great. I will give it a try and report back.

[jay-reynolds]

Hi, I'm using umap 0.3.2 and trying the approach outlined above but running into problems in that the resulting embedding produces a single, centered globular distribution, whereas the separate embeddings of my two distinct feature types (interval vs categorical) exhibit interesting structure.

I've done the following, for instance:

fit1 = umap.UMAP(metric='braycurtis').fit(df_b.values)
fit2 = umap.UMAP(metric='jaccard').fit(df_dummies.values)
prod_graph = fit1.graph_.multiply(fit2.graph_)
new_graph = 0.99 * prod_graph + 0.01 * (fit1.graph_ + fit2.graph_ - prod_graph)
embedding = umap.umap_.simplicial_set_embedding(fit1._raw_data, new_graph, fit1.n_components, 
                                                fit1.initial_alpha, fit1._a, fit1._b, 
                                                fit1.repulsion_strength, fit1.negative_sample_rate, 
                                                200, fit1.init, np.random, fit1.metric, 
                                                fit1._metric_kwds, False)

It wasn't clear what I should have used for the data parameter in simplicial_set_embedding(). I tried both fit1._raw_data and fit2._raw_data, but neither alone seems appropriate here -- both produce similar results (a single diffuse blob).

Any advice would be greatly appreciated!

[lmcinnes]

The data you pass in to simplicial set embedding shouldn't matter too much unless you end up with lots of separate connected components. I admit that I can't say immediately what might be causing this -- it looks like you are doing something fairly reasonable. There is some slightly newer code that you could try, but I'm not sure it will help in your case. I'll have to look up what the right code is, because it would be a series of internal function calls, and I don't have time right now. I'll try to get back to you soon.

[lmcinnes]

Okay, so this is a little less than ideal because these are decidedly not public APIs, so it gets messy, but you can try:

fit1 = umap.UMAP(metric='braycurtis').fit(df_b.values)
fit2 = umap.UMAP(metric='jaccard').fit(df_dummies.values)
intersection = umap.umap_. general_simplicial_set_intersection(fit1.graph_, fit2.graph_, mix_weight=0.5)
intersection = umap.umap_.reset_local_connectivty(intersection)
embedding = umap.umap_.simplicial_set_embedding(fit1._raw_data, intersection, fit1.n_components, 
                                                fit1.initial_alpha, fit1._a, fit1._b, 
                                                fit1.repulsion_strength, fit1.negative_sample_rate, 
                                                200, 'random', np.random, fit1.metric, 
                                                fit1._metric_kwds, False)

which may work a little better. Note that we are passing 'random' instead of fit1.init as this will ensure the fit1._raw_data and fit1.metric won't come into it at all. This is not exactly ideal, but it might suffice to see if we can get a better result for you.

The other thing to note is the mix_ratio in the call to general_simplicial_set_intersection. This is the balance between fit1 and fit2. A value of 0.0 means essentially just use fit1 and a value of 1.0 means essentially just use fit2. You can try playing with values in between to see if that can help move you away from a pure blobby structure.

[jay-reynolds]

Yes, this works much better, thank you!
I'll wedge the range for the mix value and report my findings.

[lmcinnes]

Glad that its working better. A proper interface for general dataframe
handling (based on the new sklearn ColumnTransformer) based on this newer
code is among my plans for 0.4. It is good to know it works (at least
somewhat) in practice.

On Mon, Sep 10, 2018 at 6:49 PM Jay Reynolds [e-mail]
wrote:

Yes, this works much better, thank you!
I'll wedge the range for the mix value and report my findings.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/lmcinnes/umap/issues/58#issuecomment-420061450, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ALaKBSpjGCr-1MCqi61W49Mrh8Wz2rVaks5uZtSkgaJpZM4SuCFW
.

[jlmelville]

To extend this idea to multiple blocks of data, if we have say three graphs, graphA, graphB, graphC, is it sufficient to do (in pseudo-code):

intersectAB = general_simplicial_set_intersection(graphA, graphB)
intersectABC = general_simplicial_set_intersection(intersectAB, graphC)
intersectABC = reset_local_connectivity(intersectABC)

or does reset_local_connectivity need to be called after each pair of graphs are intersected?

[lmcinnes]

It is an interesting question -- theoretically you don't need to reset the local connectivity 'til the end, but implementation-wise I believe it would be more beneficial to do so at each step. I have been getting started on this and am still playing with the right implementation approximations to what theory says should be done.

[he-ritter]

How would new data then be processed?
I am clueless what to do after calling

test_ numerical_transform = fit1.transform(X_numerical_test)
test_categorical_transform = fit2.transform(X_categorical_test)

[lmcinnes]

Transform for mixed-type data is something that is certainly not available at this time. It isn't theoretically infeasible, but implementation-wise it would require a non-negligible amount of code refactoring to make it happen. Sorry @MeTooThanks .

[ekerazha]

@MeTooThanks Maybe you could try to train a Neural Network (or an Extreme Learning Machine) to learn the mapping between the numeric and categorical transformed data and the "intersected" transformed data. @lmcinnes Opinions?

[lmcinnes]

@ekerazha You can certainly try, but I suspect getting good parameters/architecture for the network and successfully training it without overfitting to the original input data will be a potentially large challenge.

[ivenzor]

@lmcinnes Hello, thanks for your awesome work in UMAP.
Quick question: I have read issues #58, #104 and #241, and I just wanted to confirm that in order to use categorical, ordinal and/or mixed datasets, at the moment best way to handle this data in UMAP is to split the numeric and categorical variables, then one-hot encode the categoricals (pd.get_dummies) with a dice metric, then merge the two splits and continue. Am I correct?

[lmcinnes]

@ivenzor Yes, that would be the right approach right now. One alternative would be to check out the 0.4dev branch which has a (very!) experimental class called DataframeUMAP that would take a pandas dataframe and a tuple of (column name, metric) lists (similar to the ColumnTransformer in sklearn) and does all the required manipulations.

[ivenzor]

Ok, I will also check the experimental DataFrameUMAP. Many thanks for your reply.

[gibsramen]

Hi, all.

Recently I worked on a small project looking at this very issue. I used the cluster package in R to calculate the Gower distance matrix on mixed-data and passed that to UMAP with metric="precomputed". Results turned out pretty well (seems better than one-hot encoding), so it's one option for anyone who would like to do some mixed-type analysis while this functionality is still in development.

To my knowledge Gower distance isn't implemented in any Python package (though I am working on remedying that right now...)