Umap: Bug: NaN result with projection to 1D latent space

[rmitsch]

Hi,

thanks for publishing and maintaining this cool project! I played around with UMAP a bit and encountered an oddity: I have a small (n=178) distance matrix that I want to project onto a 1D space. If I run UMAP in version 0.3.0 with the default parameters, sometimes the resulting low-dimensional data consists of NaNs exclusively while sometimes the result seems perfectly fine.

I narrowed it down to the random_state being used, which seems a bit fishy. To reproduce: I call

res = umap.UMAP(
    n_components=1,
    random_state=8, 
    metric='precomputed'
).fit_transform(high_dim_data)

print(numpy.count_nonzero(numpy.isnan(res)))

where high_dim_data is as defined in https://gist.github.com/rmitsch/cd99fcacf6a8f3c0d41af6f903735251.

I didn't observe this behaviour with n_components > 1, but haven't checked thoroughly yet. I just tried various values for random_state until I could reproduce the NaN result.

Is this a known problem? I couldn't find any related issues.

[lmcinnes]

It is not a known problem. My best guess is that the spectral
initialization is going badly for your data. As a workaround you can add
init='random' to your UMAP parameters and that may fix the issue for
now. I'll have to try and track down what is actually going wrong here.
Thanks for the data and a random seed -- that will help a great deal with
reproducing the problem and hopefully fixing it!

On Tue, Jul 17, 2018 at 9:16 AM Raphael Mitsch [e-mail]
wrote:

Hi,

thanks for publishing maintaining this cool project! I played around with
UMAP a bit and encountered an oddity: I have a small (n=178) distance
matrix that I want to project onto a 1D space. If I run UMAP in version 3.0
with the default parameters, sometimes the resulting low-dimensional data
consists of NaNs exclusively while sometimes the result seems perfectly
fine.

I narrowed it down to the random_state being used, which seems a bit
fishy. To reproduce: I call

res = umap.UMAP(
n_components=1,
random_state=8,
metric='precomputed'
).fit_transform(high_dim_data)

print(numpy.count_nonzero(numpy.isnan(res)))

where high_dim_data is as defined in
https://gist.github.com/rmitsch/cd99fcacf6a8f3c0d41af6f903735251.

I didn't observe this behaviour with n_components > 1, but haven't
checked thoroughly yet. I just tried various values for random_state
until I could reproduce the NaN result.

Is this a known problem? I couldn't find any related issues.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/lmcinnes/umap/issues/85, or mute the thread
https://github.com/notifications/unsubscribe-auth/ALaKBZSHieXSJvN53OnWRFuZlDgANuYkks5uHeOggaJpZM4VS04v
.

[rmitsch]

Thanks for taking a look into it.

[asanakoy]

I faced the same issue when trying to run UMAP with n_components=1 with precomputed metric.
It works from time to time, but in most of the cases it prodices NaN embeddings.
Dataset size 1000x1000.
Setting init='random didn't help.

UMAP was installed from git commit 91431b29049f74d66b575d1e632576a6642b9213

[lmcinnes]

Thanks for the report -- that sounds like something that is more likely to
be reproducible. I'm on vacation right now but I'll try to look into this
when I get back.

On Thu, Aug 2, 2018 at 10:00 AM Artsiom [e-mail] wrote:

I faced the same issue when trying to run UMAP with n_components=1 with
precomputed metric.
It works from time to time, but in most of the cases it prodices NaN
embeddings.
Dataset size 1000x1000.
Setting init='random didn't help.

UMAP was installed from git commit 91431b2
https://github.com/lmcinnes/umap/commit/91431b29049f74d66b575d1e632576a6642b9213

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/lmcinnes/umap/issues/85#issuecomment-409936459, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ALaKBZl8ro-aGVlYcOz_lpImvDA4ViR5ks5uMwYQgaJpZM4VS04v
.

[lmcinnes]

So the bad news is that I grabbed the data from the gist and ran with the supplied parameters but can't reproduce the error locally here. That, unfortunately, makes it hard to track down the actual issue. I'll keep a lookout, but I can't make any promises on getting this fixed too soon as it seems to be intermittent/hard to reproduce.

[asanakoy]

@lmcinnes have you tried to run several times with different random initializations? It is non-deterministic. Sometimes works, sometimes not.

[lmcinnes]

Fine suggestion. I set a loop running until I got a failure, which is now reproducible with that seed. I'll try to dig into this soon and see what the actual underlying issues is. Thanks!

[snakers4]

I faced the same issue when trying to run UMAP with n_components=1 with precomputed metric.
It works from time to time, but in most of the cases it prodices NaN embeddings.
Dataset size 1000x1000.
Setting init='random didn't help.

Faced a similar issue - but then doing something like 300k * 50 => 300k * 2
When doing 300k * 50 => 300k * 10 faced no errors whatsoever

[lmcinnes]

Got it! Well, at lest the one that I could reliably reproduce. If distinct points ended up having perfectly identical coordinates during the embedding optimization that could trigger problems. I believe I now have a catch in place for that. I would be happy if any of the reporters (especially with reproducible issues) could verify that the current master now resolves the problem for them.

[rmitsch]

Solves the problem for me. The version currently available on pip (umap-learn 0.3.2) does not contain the fix though, does it? I just copied the changes in 866c079 by hand for now.

Thanks for the fix!

[lmcinnes]

I wanted to verify this fixes the issue for others before releasing an
update to PyPI (which should be on version 0.3.2 by now anyway -- note
that you need umap-learn on PyPI). Will try to get an update rolled out
soon. Thanks for the feedback and the reproducer everytone -- it helped a
lot.

On Fri, Aug 17, 2018 at 8:43 AM Raphael Mitsch [e-mail]
wrote:

Fixes my problem. The version currently available on pip (0.1.1) does not
contain the fix though, does it? I just copied the changes in your commit
by hand for now.

Thanks!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lmcinnes/umap/issues/85#issuecomment-413853449, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ALaKBQyzIqFgIik7z46-pUO_3_NGxnmFks5uRrpVgaJpZM4VS04v
.

[asanakoy]

Fixed the problem for me. Installed from master 16e7ce624dc984cb4a626d7cdfccff4bd8e7ce2c