Umap: Segmentation fault

[MichaelMonashev]

umap-learn 0.4.2

Code:

        embedding = umap.UMAP(
            n_neighbors=80,
            min_dist=0.0,
            n_components=30,
            verbose=True,
        ).fit_transform(predicts)

predicts has shape (314496, 2048).

Error:

/home/xxx/.local/lib/python3.6/site-packages/sklearn/externals/joblib/__init__.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
  warnings.warn(msg, category=FutureWarning)
start UMAP...
UMAP(a=None, angular_rp_forest=False, b=None,
     force_approximation_algorithm=False, init='spectral', learning_rate=1.0,
     local_connectivity=1.0, low_memory=False, metric='euclidean',
     metric_kwds=None, min_dist=0.0, n_components=30, n_epochs=None,
     n_neighbors=80, negative_sample_rate=5, output_metric='euclidean',
     output_metric_kwds=None, random_state=None, repulsion_strength=1.0,
     set_op_mix_ratio=1.0, spread=1.0, target_metric='categorical',
     target_metric_kwds=None, target_n_neighbors=-1, target_weight=0.5,
     transform_queue_size=4.0, transform_seed=42, unique=False, verbose=True)
Construct fuzzy simplicial set
Wed May 13 06:38:30 2020 Finding Nearest Neighbors
Wed May 13 06:38:30 2020 Building RP forest with 33 trees
Wed May 13 06:39:54 2020 NN descent for 18 iterations
         0  /  18
         1  /  18
Segmentation fault

[lmcinnes]

One short term workaround may be to install pynndescent. It is newer code that will handle that part of the computation (and if it is installed umap will automatically make use of it) that may resolve the issue.

[MichaelMonashev]

Thank you so much!
I use n_neighbors=500 and it works without segfault. And result clustering much better!

[Rocketknight1]

Just a follow up that I also encountered this issue (1M samples, 512D data, euclidean distance, n_neighbors=50) and it was also fixed with pynndescent. It might be worth flagging this in the FAQ!