Julia: Reorganize what is loaded when we say `using LinearAlgebra`

[ViralBShah]

Currently, as we have moved things into stdlib, I feel that the usability of the linear algebra libraries has gone down. While arrays and matrices are in Base, LinearAlgebra mainly gives you access to BLAS, LAPACK and other dense linear algebra, SparseArrays to get some sparse matrix stuff, SuiteSparse to get sparse direct solvers, and IterativeEigenSolvers for the capabilities in ARPACK. That is mainly for the convenience of laying out the libraries and for maintenance, but not necessarily best for the user.

There are a couple of different options I can imagine, and there might be others.

1. Make all of the above available in LinearAlgebra.
2. Rename LinearAlgebra to DenseLinAlg (which gives you BLAS, LAPACK, and all the other matrix types and operations) and combine all the other sparse related packages into SparseLinAlg (SparseArrays, SuiteSparse, IterativeEigenSolvers).

Once the deprecations are deleted in 1.0, it will be quite difficult to figure out which function is in which package.

[JeffBezanson]

We could add a SparseLinearAlgebra that groups together SparseArrays, SuiteSparse, and IterativeEigensolvers.

Triage thinks LinearAlgebra is not just about dense linear algebra; it defines names used for sparse linalg as well.

[ViralBShah]

It's a bit odd for LinearAlgebra to define the names, as well as have a collection of different matrix types (many of them sparse in nature) and solvers, and then to have a different SparseLinAlg package and a collection of external packages.

The whole thing is a bit unsatisfying and largely an accident based on what was possible to implement in pure Julia at the time and what needed external libraries.

[ararslan]

See also https://github.com/JuliaLang/julia/issues/26210

[JeffBezanson]

It's a bit odd for LinearAlgebra to define the names, as well as have a collection of different matrix types and solvers.

I don't imagine that means you want to break it up further? The idea is to define the vocabulary, plus provide a default implementation (which we pick to be dense). It's a somewhat-opinionated notion of what "linear algebra" should include by default.

[ViralBShah]

Well, we don't just have dense. We have Diagonal, Bidiagonal, various tridiagonals, and their solvers, Bunch Kaufman factorizations, etc. I think just the names, and some basic dense linear algebra would have been ok. But the current list is a bit arbitrary.

[JeffBezanson]

Well, "basic dense linear algebra" is subject to interpretation. Do you want to break LinearAlgebra into StructuredMatrices, Eigensolvers, MatrixFactorizations, etc? I thought you wanted the opposite of that.

[JeffBezanson]

I would vote for leaving LinearAlgebra alone, and combining the sparse packages into SparseArrays or SparseLinearAlgebra.

[ViralBShah]

I suspect that leaving LinearAlgebra and combining the sparse stuff into SparseLinearAlgebra is strictly better for the time being than where we are - so let's do that.

[JeffBezanson]

Probably one principle here is that you shouldn't have to know the name of a library used in the implementation, e.g. using SuiteSparse.

[tkelman]

You would need to maintain that if you intend to ever make it possible for alternate implementations to be able to use the syntax. Solving a sparse linear system of equations isn't something that only suitesparse is capable of doing, it's an unfortunate accident that the syntax and main sparse types are claimed by it. Rather than a single implementation for say sparse backslash, the reality is there's a list of different implementations you might want to dispatch the same syntax (for the same input types) to based on what libraries are available and user preference.

[ViralBShah]

As @KristofferC also pointed out, you may even want to replace the sparse matvec. At the moment you can only do this compile-time, like swapping out a BLAS, or swapping out SuiteSparse and using something else (assuming there are alternate bindings in the library).

I suspect figuring these things out is mostly a post 1.0 thing, and for now, we just want to make what we have convenient. Moving things into stdlib is definitely taking us in that direction.

[ViralBShah]

We are going to leave this as is for now, but try and move SuiteSparse and Arpack out of stdlib.

cc @andreasnoack