Spack: `spack load` often interferes with system executables by setting LD_LIBRARY_PATH

[haampie]

I'm opening this issue since this popped up twice on Slack in a short time, and I've had the same issue too.

If you're on older OS, it is very likely that running spack load [any package] will break system executables like git, lua and others (or at least generate warnings). The problem is spack modifies LD_LIBRARY_PATH when running spack load ..., and system executables will now start using incompatible, new spack libraries (e.g. git needs openssl, lua needs curses).

Steps to reproduce the issue

In my case spack load [pkg] where [pkg] is any dependent of ncurses made my terminal completely unusable, because I'm using z.lua to jump around between directories, which is based on lua, which would fail and output warning message every time I would hit enter in the terminal.

Another example on CentOS 8, when you run

$ spack load cmake
$ git pull
/usr/libexec/git-core/git-remote-https: symbol lookup error: /lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b

The problem here is that before and after running spack load cmake different libs are loaded:

$ ldd /usr/libexec/git-core/git-remote-https
    ...
    libz.so.1 => /path/to/projects/spack/opt/spack/linux-centos8-thunderx2/gcc-10.2.0/zlib-1.2.11-3jsx5es2g3wrh7zw45a4yk7qgi2cioot/lib/libz.so.1 (0x0000ffffbe3f0000)
    ...
    libcrypto.so.1.1 => /path/to/projects/spack/opt/spack/linux-centos8-thunderx2/gcc-10.2.0/openssl-1.1.1g-xrglcntszao4zn6btswk2o6svrapsjeu/lib/libcrypto.so.1.1 (0x0000ffffbe0b0000)
    ...
    libssl.so.1.1 => /path/to/projects/spack/opt/spack/linux-centos8-thunderx2/gcc-10.2.0/openssl-1.1.1g-xrglcntszao4zn6btswk2o6svrapsjeu/lib/libssl.so.1.1 (0x0000ffffbdd10000)
    ...
$ ldd /usr/libexec/git-core/git-remote-https
    ...
    libz.so.1 => /lib64/libz.so.1 (0x0000ffff97510000)
    ...
    libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x0000ffff971e0000)
    ...
    libssl.so.1.1 => /lib64/libssl.so.1.1 (0x0000ffff96e40000)
    ...

Potential solution

I think it is best to never set LD_LIBRARY_PATH upon spack load by default, since
a. it causes too many annoying side effects, and
b. spack already sets up rpaths for its executables and libraries.

Maybe we can instead have LD_LIBRARY_PATH set _only_ when a certain flag is provided spack load --ld-library-path [pkg]?

Pinging @tgamblin, since I remember some potential troubles about openmpi and rpath / ld_library_path.

Additional information

• [ ] I have run spack debug report and reported the version of Spack/Python/Platform
• [x] I have searched the issues of this repo and believe this is not a duplicate
• [ ] I have run the failing commands in debug mode and reported the output

[adamjstewart]

I see where you're coming from, and we've had several users raise this issue in the past: #3955 (@hartzell)

At the same time, a lot of Python packages don't support RPATH, and can't be used unless their dependencies are in (DY)?LD_LIBRARY_PATH. I'm not sure what the best strategy is here, but let's move this conversation to #3955.

Closing as a duplicate.