Umap: UMAP broken with numba==0.43

[sauln]

Hi @lmcinnes, I wanted to let you know about a new dependency issue that just popped up. It looks like the most recent Numba release and UMAP don't play nice together. There was a ton of updates included in the Numba release notes, so it's not clear what exactly the problem is.

With Python 3.7.1, umap-learn==0.3.7, and numba==0.43.0 (released March 13th), this simple UMAP test script produces the following error:

import umap
import numpy as np
r = np.random.random((100,3))
r2d = umap.UMAP().fit_transform(r)

Traceback (most recent call last):
  File "test.py", line 4, in <module>
    r2d = umap.UMAP().fit_transform(r)
  File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/umap/umap_.py", line 1566, in fit_transform
    self.fit(X, y)
  File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/umap/umap_.py", line 1398, in fit
    self.verbose,
  File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/numba/dispatcher.py", line 350, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/numba/dispatcher.py", line 317, in error_rewrite
    reraise(type(e), e, None)
  File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/numba/six.py", line 658, in reraise
    raise value.with_traceback(tb)
numba.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Unknown attribute 'shape' of type none

File "venv/lib/python3.7/site-packages/umap/umap_.py", line 88:
def smooth_knn_dist(distances, k, n_iter=64, local_connectivity=1.0, bandwidth=1.0):
    <source elided>
    target = np.log2(k) * bandwidth
    rho = np.zeros(distances.shape[0])
    ^

[1] During: typing of get attribute at /Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/umap/umap_.py (88)

File "venv/lib/python3.7/site-packages/umap/umap_.py", line 88:
def smooth_knn_dist(distances, k, n_iter=64, local_connectivity=1.0, bandwidth=1.0):
    <source elided>
    target = np.log2(k) * bandwidth
    rho = np.zeros(distances.shape[0])
    ^

This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.

To see Python/NumPy features supported by the latest release of Numba visit:
http://numba.pydata.org/numba-doc/dev/reference/pysupported.html
and
http://numba.pydata.org/numba-doc/dev/reference/numpysupported.html

For more information about typing errors and how to debug them visit:
http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#my-code-doesn-t-compile

If you think your code should work with Numba, please report the error message
and traceback, along with a minimal reproducer at:
https://github.com/numba/numba/issues/new

After downgrading to numba==0.42, everything ran perfectly fine, so this is a fine temporary solution.

[lmcinnes]

Well that's disconcerting. I'll have to see if I can track down what is
going wrong here -- it seems like distances is getting passed as none
somehow , but that should have broken things before, so it is rather
unclear what exactly is going astray. Thanks for the report -- very useful
to know that things are suddenly broken.

On Fri, Mar 15, 2019 at 3:17 PM Nathaniel Saul [e-mail]
wrote:

Hi @lmcinnes https://github.com/lmcinnes, I wanted to let you know
about a new dependency issue that just popped up. It looks like the most
recent Numba release and UMAP don't play nice together. There was a ton of
updates included in the Numba release notes
http://numba.pydata.org/numba-doc/latest/release-notes.html, so it's
not clear what exactly the problem is.

With Python 3.7.1, umap-learn==0.3.7, and numba==0.43.0 (released March
13th), this simple UMAP test script produces the following error:

import umap
import numpy as np
r = np.random.random((100,3))
r2d = umap.UMAP().fit_transform(r)

Traceback (most recent call last):
File "test.py", line 4, in
r2d = umap.UMAP().fit_transform(r)
File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/umap/umap_.py", line 1566, in fit_transform
self.fit(X, y)
File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/umap/umap_.py", line 1398, in fit
self.verbose,
File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/numba/dispatcher.py", line 350, in _compile_for_args
error_rewrite(e, 'typing')
File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/numba/dispatcher.py", line 317, in error_rewrite
reraise(type(e), e, None)
File "/Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/numba/six.py", line 658, in reraise
raise value.with_traceback(tb)
numba.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Unknown attribute 'shape' of type none

File "venv/lib/python3.7/site-packages/umap/umap_.py", line 88:
def smooth_knn_dist(distances, k, n_iter=64, local_connectivity=1.0, bandwidth=1.0):

target = np.log2(k) * bandwidth
rho = np.zeros(distances.shape[0])
^

[1] During: typing of get attribute at /Users/sauln/research/scratch/test_umap/venv/lib/python3.7/site-packages/umap/umap_.py (88)

File "venv/lib/python3.7/site-packages/umap/umap_.py", line 88:
def smooth_knn_dist(distances, k, n_iter=64, local_connectivity=1.0, bandwidth=1.0):

target = np.log2(k) * bandwidth
rho = np.zeros(distances.shape[0])
^

This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.

To see Python/NumPy features supported by the latest release of Numba visit:http://numba.pydata.org/numba-doc/dev/reference/pysupported.html
andhttp://numba.pydata.org/numba-doc/dev/reference/numpysupported.html

For more information about typing errors and how to debug them visit:http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#my-code-doesn-t-compile

If you think your code should work with Numba, please report the error message
and traceback, along with a minimal reproducer at:https://github.com/numba/numba/issues/new

After downgrading to numba==0.42, everything ran perfectly fine, so this
is a fine temporary solution.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lmcinnes/umap/issues/209, or mute the thread
https://github.com/notifications/unsubscribe-auth/ALaKBcwDoxhSEv8qFHf1JUY3IQXTxBW4ks5vW_GwgaJpZM4b3JBH
.

[kramred]

It's a bug in Numba 0.43: https://github.com/numba/numba/issues/3879

You can work around the bug by introducing the following change in umap_.py (although you probably shouldn't):

diff --git a/umap/umap_.py b/umap/umap_.py
index b37469a..f488dec 100644
--- a/umap/umap_.py
+++ b/umap/umap_.py
@@ -445 +445 @@ def fuzzy_simplicial_set(
-    if knn_indices is None or knn_dists is None:
+    if (not knn_indices) or (not knn_dists):

[stuartarchibald]

Apologies, this is indeed a Numba bug from the use of a new compiler pass, I've got a fix worked out locally and it should make it into the next release. The root cause is a long standing problem (for which I also have a fix), it just wasn't anticipated that this would be hit by the new compiler pass. The reason this code hit the problem is that it is using @jit which permits falling back to objectmode compilation if nopython mode fails, and some mutations to the IR from the failed nopython mode are erroneously carried over.

The new pass strips out dead code based on constant inference which means doing things like:

from numba import njit
@njit
def foo(a, b=None):
  if b is None:
    return a
  else:
    return a + b

foo(1, 1)
foo(1)

is now possible, whereas before type inference would not be able to deal with the else branch as it would "see" a + (might be)None. This reduces code by up to 50% in size and massively reduces complexity in Numba internal implementations, so it was kind of necessary to implement.

The reason the patch by @kramred works is because it is making it such that the compiler pass cannot work out if the branch is live or not as there's no constant comparison of a class handled at present.

[stuartarchibald]

Fix to the invalid mutation in the fall back compilation path is here: https://github.com/numba/numba/pull/3883

[lmcinnes]

Thanks to both @stuartarchibald and @kramred for this. I think at this stage I will just wait for the patch to numba to come out, and in the meantime people can either apply @kramred's fix, or rollback their numba installation to 0.42.

[stuartarchibald]

A 0.43.1 patch release candidate should be out in the next few days, any chance you could please test it when it lands @lmcinnes? Thanks.

[lmcinnes]

I'll do my best. Thanks @stuartarchibald !

[seibert]

We've posted a release candidate conda package for Numba 0.43.1 to the Numba channel:

conda install -c numba numba=0.43.1rc1

Stuart manually verified that this solves the UMAP issue. We will post the final packages and wheels on Wednesday.

[lmcinnes]

@seibert Thanks, this indeed seems to resolve the problem. I definitely appreciate the speedy work on this.

[stuartarchibald]

@lmcinnes great, thanks for confirming!

[kramred]

New Version installed from pip/pypi also seems to resolve the issue. Thanks for the quick fix!