Homebrew-core: Tracking issue for migrating to OpenBLAS

[zbeekman]

Formulae using Accelerate

The following formula still appear to depend on Apple's Accelerate framework:

• [x] arpack (uses veclibfort)
• [x] dynare (uses veclibfort)
• [x] hypre (uses veclibfort)
• [x] octave (uses veclibfort)
• [x] qrupdate (uses veclibfort)
• [x] sundials (uses veclibfort)
• [x] suite-sparse (explicit -framework Accelerate)
• [x] superlu (uses veclibfort)

This list was compiled via brew uses veclibfort and also ag -w 'accelerate'

There are additional recursive dependencies, and shogun may need modifying because it pulls in the lapack formula under certain circumstances, but OpenBLAS provides lapack functionality. See also brewsci/homebrew-num which defaults to OpenBLAS for all formulae requiring BLAS: https://github.com/brewsci/homebrew-num/tree/master/Formula

Additional recursive dependencies:

veclibfort

• armadillo
• libbi
• shogun

suite-sparse

• ceres-solver
• petsc
• petsc-complex
• rawtoaces

[zbeekman]

CC: @dpo

[equal-l2]

Is there a discussion or something about this migration?
I want to know the motivation for the migration.

[zbeekman]

@equal-l2 FYI, this is not finalized/set in stone yet.

There is some discussion over on discourse: https://discourse.brew.sh/t/is-there-a-reason-math-formulas-use-veclibfort-instead-of-openblas/4658

The biggest motivation for doing this is many large scientific projects are starting to adopt OpenBLAS over Apples accelerate due to:

• Bugs, crashes, and strange behaviour
• Accelerate does not implement a recent LAPACK version

See also https://github.com/Homebrew/homebrew-core/issues/37898#issuecomment-472779968.

If you have questions, comments or concerns, now is the time to share them!

Thanks!

[equal-l2]

@zbeekman Thank you for the explanation. The motivations make sense for me.

[fxcoudert]

I think it's the right time to make the change. Many of the formulas listed above are dependencies of one another, so we probably want to switch them all together to avoid mixed linkage.

[zbeekman]

Yes, that was my thinking too.

[zbeekman]

@fxcoudert I'm going to wait until after the GCC 9 upgrade before I try tackling this.

[fxcoudert]

@zbeekman GCC 9 has landed in Homebrew

[zbeekman]

It appears that hypre ships with their own version of blas and lapack. As such, I don't think it makes sense to link against a system blas or lapack. This should simplify dependencies. I've emailed the hypre devs to double check.

[zbeekman]

I think shogun should use OpenBLAS for LAPACK and BLAS. I'm going to do some research then include an update for shogun

[zbeekman]

Hmmm.... ok, nm, I'm leaving shogun alone. No releases in over two years extreme patching happening with a frankenstein CMake build.

[fxcoudert]

@zbeekman regarding your commits: please leave the gcc dep comment unchanged. It's identical in all formulas that rely on GCC to get gfortran, which makes it easy to grep for when GCC is updated

[zbeekman]

Anyone know how to fix this java error when building octave:

build-aux/install-sh -c -d scripts/java/org/octave && \
        ( cd ./scripts/java; \
          "/Library/Java/JavaVirtualMachines/openjdk-12.0.1.jdk/Contents/Home/bin/javac" -source 1.6 -target 1.6 -Xlint:-options \
                     -d /private/tmp/octave-20190513-95317-1024fek/octave-5.1.0/scripts/java \
                     org/octave/ClassHelper.java )
Updating 'libgui/languages/zh_CN.qm'...
    Generated 856 translation(s) (842 finished and 14 unfinished)
    Ignored 55 untranslated source text(s)
build-aux/install-sh -c -d scripts/java/org/octave && \
        ( cd ./scripts/java; \
          "/Library/Java/JavaVirtualMachines/openjdk-12.0.1.jdk/Contents/Home/bin/javac" -source 1.6 -target 1.6 -Xlint:-options \
                     -d /private/tmp/octave-20190513-95317-1024fek/octave-5.1.0/scripts/java \
                     org/octave/Matrix.java )
build-aux/install-sh -c -d scripts/java/org/octave && \
        ( cd ./scripts/java; \
          "/Library/Java/JavaVirtualMachines/openjdk-12.0.1.jdk/Contents/Home/bin/javac" -source 1.6 -target 1.6 -Xlint:-options \
                     -d /private/tmp/octave-20190513-95317-1024fek/octave-5.1.0/scripts/java \
                     org/octave/OctClassLoader.java )
Picked up _JAVA_OPTIONS:  -Duser.home=/Users/ibeekman/Library/Caches/Homebrew/java_cache
Picked up _JAVA_OPTIONS:  -Duser.home=/Users/ibeekman/Library/Caches/Homebrew/java_cache
Picked up _JAVA_OPTIONS:  -Duser.home=/Users/ibeekman/Library/Caches/Homebrew/java_cache
error: Source option 6 is no longer supported. Use 7 or later.
error: Target option 6 is no longer supported. Use 7 or later.
error: Source option 6 is no longer supported. Use 7 or later.
error: Target option 6 is no longer supported. Use 7 or later.
make[2]: *** [scripts/java/org/octave/OctClassLoader.class] Error 2
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [scripts/java/org/octave/ClassHelper.class] Error 2
error: Source option 6 is no longer supported. Use 7 or later.
error: Target option 6 is no longer supported. Use 7 or later.
make[2]: *** [scripts/java/org/octave/Matrix.class] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

It looks like octave needs to bump source and target versions to 7...

[dpo]

@schoeps would know about Octave.

[schoeps]

The java issue should have been fixed ages ago in https://savannah.gnu.org/patch/?9439. It's the same issue, right? Is there somewhere an old Octave version interfering? If I recall correctly the ptch was easy, the version was hardcoded in the source code.

By the way: I enjoy this initiative. I basically stopped contributing to homebrew when homebrew-science was killed and core-maintainers insisted to use accelerate and not offering openblas as an option. Thanks!

[zbeekman]

By the way: I enjoy this initiative.

Glad to hear it! Yes, at this point Accelerate is getting crusty and there's no reason not to follow other pkg managers and coalesce around a solid, actively maintained linalg library providing BLAS and LAPACK interfaces.

I'd love to see more modernization and numerics/HPC focused contributions. Unfortunately, often pkg authors/maintainers (i.e., upstream) don't always follow build system best practices/modernization so it's not always easy. See, for example, metis, hypre, etc.

[zbeekman]

@schoeps: Homebrew transitioned to openjdk. Octave hasn't been re-built since that happened, IIRC. Building default octave from source is currently broken due to this problem. I filed a bug report at #39848. It's not clear to me exactly how to fix this, but I'll look at the patch you referenced.

[zbeekman]

Implemented/resolved in PR #39809