Julia: LAPACK hangs when matrix elements are not finite

[atteson]

julia hangs on the following:

pinv([0.0 NaN; NaN Inf])

I don't expect it to return something meaningful but an uninterruptible hang doesn't seem right.

[andreasnoack]

versioninfo() please.

[atteson]

I've tried it on the 2 versions below:

Julia Version 0.5.0
Commit 3c9d75391c (2016-09-19 18:14 UTC)
Platform Info:
  System: Linux (x86_64-redhat-linux)
  CPU: Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, haswell)


Julia Version 0.6.0-pre.beta.0
Commit bd84fa1bad (2017-03-31 12:58 UTC)
Platform Info:
  OS: Linux (x86_64-redhat-linux)
  CPU: Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, haswell)

[andreasnoack]

LAPACK doesn't like NaNs. The SVD solvers hang. MKL seems to handle this but not netlib LAPACK so we might need to all(isfinite, A) || error() in most of the LAPACK calls.

[atteson]

Thanks. I had a feeling it was in LAPACK.

I did the same check in my code so I'm fine for now. I just wanted to inform you.

I recognize that for some calls, checking could be a bit of a performance hit but it seems better than the alternative to me.

[nalimilan]

This reminds me of this recent R thread (see in particular the excerpt from NEWS in the middle of the thread).

Also see this blog post.

[simonbyrne]

Wow, R apparently patch their own BLAS for this reason: http://icl.cs.utk.edu/lapack-forum/archives/lapack/msg01536.html

[simonbyrne]

What is LAPACK policy on NaNs? is this a bug?

[simonbyrne]

Seems like they do consider infinite loops due to NaNs a bug: http://www.netlib.org/lapack/bug_list.html#_strong_span_class_green_bug0101_span_strong_s_d_c_z_hsein_nan_in_h_can_cause_an_infinite_loop

[andreasnoack]

Reported in http://icl.cs.utk.edu/lapack-forum/viewtopic.php?f=13&t=5136

[langou]

Hello, thanks for the thread. I think I fixed the issue with https://github.com/Reference-LAPACK/lapack/commit/a1e853bd005a2107a0a43f92782a158a9fd6e56c. It would be great if you can confirm whether https://github.com/Reference-LAPACK/lapack/commit/a1e853bd005a2107a0a43f92782a158a9fd6e56c fixes the infinite loop observed in julia with pinv([0.0 NaN; NaN Inf]) or not. Happy to have feedback on the fix and work on the issue some more if needed, just let us know. Best wishes, Julien.

[andreasnoack]

@langou Great. Thanks for looking into this. I've just tested with the latest version of LAPACK. It fixes the issue here and the routine returns arrays of NaNs so it looks like we are all set.

[ViralBShah]

We are now using openblas 0.3 and lapack 3.8, and I don't see the issue.

julia> pinv([0.0 NaN; NaN Inf])
2×2 Array{Float64,2}:
 NaN  NaN
 NaN  NaN

[nalimilan]

Better add a test before closing though?