Stack: "can't load .so/.DLL" error with Symbol not found while installing optparse-applicative-0.13.2.0 dependency on mac via Stack for Hakyll `stack install hakyll`

[archywillhe]

I run into this error while installing the optparse-applicative dependency for hakyll via stack.

Progress: 71/147
--  While building package optparse-applicative-0.13.2.0 using:
      /Users/a/.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_1.24.2.0_ghc-8.0.2 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.24.2.0 build --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1
    Logs have been written to: /Users/a/.stack/global-project/.stack-work/logs/optparse-applicative-0.13.2.0.log

    Configuring optparse-applicative-0.13.2.0...
    Building optparse-applicative-0.13.2.0...
    Preprocessing library optparse-applicative-0.13.2.0...
    [ 1 of 15] Compiling Options.Applicative.Help.Pretty ( Options/Applicative/Help/Pretty.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Options/Applicative/Help/Pretty.o )
    [ 2 of 15] Compiling Options.Applicative.Help.Chunk ( Options/Applicative/Help/Chunk.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Options/Applicative/Help/Chunk.o )
    [ 3 of 15] Compiling Options.Applicative.Help.Types ( Options/Applicative/Help/Types.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Options/Applicative/Help/Types.o )
    [ 4 of 15] Compiling Options.Applicative.Types ( Options/Applicative/Types.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Options/Applicative/Types.o )
    [ 5 of 15] Compiling Options.Applicative.Internal ( Options/Applicative/Internal.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Options/Applicative/Internal.o )
    [ 6 of 15] Compiling Options.Applicative.Common ( Options/Applicative/Common.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Options/Applicative/Common.o )
    [ 7 of 15] Compiling Options.Applicative.Help.Core ( Options/Applicative/Help/Core.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.2.0/build/Options/Applicative/Help/Core.o )
    <command line>: can't load .so/.DLL for: /Users/a/.stack/snapshots/x86_64-osx/lts-9.10/8.0.2/lib/x86_64-osx-ghc-8.0.2/libHSansi-wl-pprint-0.6.7.3-A79v826bgPr574HtaZPPqC-ghc8.0.2.dylib (dlopen(/Users/a/.stack/snapshots/x86_64-osx/lts-9.10/8.0.2/lib/x86_64-osx-ghc-8.0.2/libHSansi-wl-pprint-0.6.7.3-A79v826bgPr574HtaZPPqC-ghc8.0.2.dylib, 5): Symbol not found: _ansizmterminalzm0zi6zi3zi1zm3RTqikoQf47ajVD9m8zzgB_SystemziConsoleziANSIziCodes_csi_closure
      Referenced from: /Users/a/.stack/snapshots/x86_64-osx/lts-9.10/8.0.2/lib/x86_64-osx-ghc-8.0.2/libHSansi-wl-pprint-0.6.7.3-A79v826bgPr574HtaZPPqC-ghc8.0.2.dylib
      Expected in: flat namespace
     in /Users/a/.stack/snapshots/x86_64-osx/lts-9.10/8.0.2/lib/x86_64-osx-ghc-8.0.2/libHSansi-wl-pprint-0.6.7.3-A79v826bgPr574HtaZPPqC-ghc8.0.2.dylib)

It is a "can't load .so/.DLL" error with an unfound symbol _ansizmterminalzm0zi6zi3zi1zm3RTqikoQf47ajVD9m8zzgB_SystemziConsoleziANSIziCodes_csi_closure

I've tried everything in this post such as rm -r ~/.stack/snapshots or stack clean but to no avail. I have even reinstalled my stack and the error persists.

I am running stack Version 1.5.1 on macOS Sierra 10.12.6.

[mgsloan]

May be related to this - https://github.com/commercialhaskell/stack/issues/2577

We'll hopefully have a new stack release soon. Unlikely, but it might help to upgrade to the prerelease - https://github.com/commercialhaskell/stack/releases/tag/v1.6.0.20171022

It might also be helpful to use the latest nightly stackage snapshot, which uses GHC 8.2 .

[archywillhe]

@mgsloan hey thanks for the pointer! Just got the prereleased stack but still run into the same problem. trying the nightly snapshot now, hope that will solve the problem!

[mgsloan]

Cool! Note that the issue is probably due to problems in the build of ansi-terminal. Switching to nightly will cause it to be built. So when trying other workarounds, first run stack exec -- ghc-pkg unregister ansi-terminal to cause it to get rebuilt.

[archywillhe]

@mgsloan Oooh I see! Got it! Thanks!

[archywillhe]

@mgsloan oh no I got a different symbol not found situation (with the same can't load .so/.DLL error) for optparse-applicative-0.14.0.0 (a different version optparse-applicative this time!) even with stack install --resolver nightly hakyll in order to install the nightly snapshot.

.....
Cabal-1.24.2.0: build
optparse-applicative-0.14.0.0: download
optparse-applicative-0.14.0.0: configure
optparse-applicative-0.14.0.0: build
cmdargs-0.10.18: copy/register
hxt-9.3.1.16: copy/register
Cabal-1.24.2.0: copy/register
Progress: 59/99
Configuring optparse-applicative-0.14.0.0...
    Preprocessing library for optparse-applicative-0.14.0.0..
    Building library for optparse-applicative-0.14.0.0..
    [ 1 of 16] Compiling Options.Applicative.Help.Levenshtein ( Options/Applicative/Help/Levenshtein.hs, .stack-work/dist/x86_64-osx/Cabal-2.0.0.2/build/Options/Applicative/Help/Levenshtein.o )
    [ 2 of 16] Compiling Options.Applicative.Help.Pretty ( Options/Applicative/Help/Pretty.hs, .stack-work/dist/x86_64-osx/Cabal-2.0.0.2/build/Options/Applicative/Help/Pretty.o )
    [ 3 of 16] Compiling Options.Applicative.Help.Chunk ( Options/Applicative/Help/Chunk.hs, .stack-work/dist/x86_64-osx/Cabal-2.0.0.2/build/Options/Applicative/Help/Chunk.o )
    [ 4 of 16] Compiling Options.Applicative.Help.Types ( Options/Applicative/Help/Types.hs, .stack-work/dist/x86_64-osx/Cabal-2.0.0.2/build/Options/Applicative/Help/Types.o )
    [ 5 of 16] Compiling Options.Applicative.Types ( Options/Applicative/Types.hs, .stack-work/dist/x86_64-osx/Cabal-2.0.0.2/build/Options/Applicative/Types.o )
    [ 6 of 16] Compiling Options.Applicative.Internal ( Options/Applicative/Internal.hs, .stack-work/dist/x86_64-osx/Cabal-2.0.0.2/build/Options/Applicative/Internal.o )
    [ 7 of 16] Compiling Options.Applicative.Common ( Options/Applicative/Common.hs, .stack-work/dist/x86_64-osx/Cabal-2.0.0.2/build/Options/Applicative/Common.o )
    [ 8 of 16] Compiling Options.Applicative.Help.Core ( Options/Applicative/Help/Core.hs, .stack-work/dist/x86_64-osx/Cabal-2.0.0.2/build/Options/Applicative/Help/Core.o )
    <command line>: can't load .so/.DLL for: /Users/a/.stack/global-project/.stack-work/install/x86_64-osx/nightly-2017-11-07/8.2.1/lib/x86_64-osx-ghc-8.2.1/libHSunix-2.7.2.2-EQ8ncIJkqE15IEUcY9XtMY-ghc8.2.1.dylib (dlopen(/Users/a/.stack/global-project/.stack-work/install/x86_64-osx/nightly-2017-11-07/8.2.1/lib/x86_64-osx-ghc-8.2.1/libHSunix-2.7.2.2-EQ8ncIJkqE15IEUcY9XtMY-ghc8.2.1.dylib, 5): Symbol not found: _timezm1zi7zi0zi1zmH9HKimU6ifnBsBPPG4G9hD_DataziTimeziClockziUTC_zdwzdcfromRational_closure
      Referenced from: /Users/a/.stack/global-project/.stack-work/install/x86_64-osx/nightly-2017-11-07/8.2.1/lib/x86_64-osx-ghc-8.2.1/libHSunix-2.7.2.2-EQ8ncIJkqE15IEUcY9XtMY-ghc8.2.1.dylib
      Expected in: flat namespace
     in /Users/a/.stack/global-project/.stack-work/install/x86_64-osx/nightly-2017-11-07/8.2.1/lib/x86_64-osx-ghc-8.2.1/libHSunix-2.7.2.2-EQ8ncIJkqE15IEUcY9XtMY-ghc8.2.1.dylib)

Symbol not found: _timezm1zi7zi0zi1zmH9HKimU6ifnBsBPPG4G9hD_DataziTimeziClockziUTC_zdwzdcfromRational_closure Looking like some Haskell package about time.

[archywillhe]

Also, when i first tried stack install --resolver nightly hakyll, I got Plan construction failed. with the following exceptions:

Error: While constructing the build plan, the following exceptions were
encountered:

In the dependencies for hakyll-4.9.8.0:
    pandoc-2.0.1.1 from stack configuration does not match >=1.14 && <1.20
                   (latest matching version is 1.19.2.4)
    pandoc-citeproc must match >=0.10.5 && <0.11, but the stack configuration
                    has no specified version (latest matching version
                    is 0.10.5.1)
    process-1.6.1.0 from stack configuration does not match >=1.0 && <1.6
                    (latest matching version is 1.5.0.0)
    time-1.8.0.2 from stack configuration does not match >=1.4 && <1.8 (latest
                 matching version is 1.7.0.1)
needed since hakyll is a build target.

It recommended me to add some dependencies for my extra-deps in ~/.stack/global-project/stack.yaml. So I ended up adding the following dependencies

- pandoc-1.19.2.4
- pandoc-citeproc-0.10.5.1
- process-1.4.3.0
- time-1.7.0.1
- Cabal-1.24.2.0
- aeson-1.1.2.0
- doctemplates-0.1.0.2
- hslua-0.4.1
- pandoc-types-1.17.0.5
- rfc5051-0.1.0.3
- skylighting-0.1.1.5
- texmath-0.9.4.4
- xml-conduit-1.5.1 

before I was able to run stack install --resolver nightly hakyll to start building.

[julian1]

Same/similar issue with optparse-applicative-0.14.2.0 on linux with cabal. I've got no idea how to resolve this.

Resolving dependencies...
Configuring optparse-applicative-0.14.2.0...
Building optparse-applicative-0.14.2.0...
Failed to install optparse-applicative-0.14.2.0
Build log ( /home/meteo/.cabal/logs/optparse-applicative-0.14.2.0.log ):
cabal: Entering directory '/tmp/cabal-tmp-5549/optparse-applicative-0.14.2.0'
Configuring optparse-applicative-0.14.2.0...
Building optparse-applicative-0.14.2.0...
Preprocessing library optparse-applicative-0.14.2.0...
[ 1 of 16] Compiling Options.Applicative.Help.Levenshtein ( Options/Applicative/Help/Levenshtein.hs, dist/build/Options/Applicative/Help/Levenshtein.o )
[ 2 of 16] Compiling Options.Applicative.Help.Pretty ( Options/Applicative/Help/Pretty.hs, dist/build/Options/Applicative/Help/Pretty.o )
[ 3 of 16] Compiling Options.Applicative.Help.Chunk ( Options/Applicative/Help/Chunk.hs, dist/build/Options/Applicative/Help/Chunk.o )
[ 4 of 16] Compiling Options.Applicative.Help.Types ( Options/Applicative/Help/Types.hs, dist/build/Options/Applicative/Help/Types.o )
[ 5 of 16] Compiling Options.Applicative.Types ( Options/Applicative/Types.hs, dist/build/Options/Applicative/Types.o )
[ 6 of 16] Compiling Options.Applicative.Internal ( Options/Applicative/Internal.hs, dist/build/Options/Applicative/Internal.o )
[ 7 of 16] Compiling Options.Applicative.Builder.Completer ( Options/Applicative/Builder/Completer.hs, dist/build/Options/Applicative/Builder/Completer.o )
[ 8 of 16] Compiling Options.Applicative.Common ( Options/Applicative/Common.hs, dist/build/Options/Applicative/Common.o )
[ 9 of 16] Compiling Options.Applicative.Builder.Internal ( Options/Applicative/Builder/Internal.hs, dist/build/Options/Applicative/Builder/Internal.o )
[10 of 16] Compiling Options.Applicative.Builder ( Options/Applicative/Builder.hs, dist/build/Options/Applicative/Builder.o )
[11 of 16] Compiling Options.Applicative.BashCompletion ( Options/Applicative/BashCompletion.hs, dist/build/Options/Applicative/BashCompletion.o )
[12 of 16] Compiling Options.Applicative.Help.Core ( Options/Applicative/Help/Core.hs, dist/build/Options/Applicative/Help/Core.o )
<command line>: can't load .so/.DLL for: libHStransformers-compat-0.6.0.5-5xgUtN78SDn8UVdE9tsxz6.so (libHStransformers-compat-0.6.0.5-5xgUtN78SDn8UVdE9tsxz6.so: cannot open shared object file: No such file or directory)
cabal: Leaving directory '/tmp/cabal-tmp-5549/optparse-applicative-0.14.2.0'
cabal: Error: some packages failed to install:
optparse-applicative-0.14.2.0 failed during the building phase. The exception
was:
ExitFailure 1

[unode]

Same as above with Nix/NixOS:

% stack test                                                                                                                                                                                                                                                                    
tagged-0.8.5: configure                                                                                                                                                                                                                                                         
tagged-0.8.5: build                                                                                                                                                                                                                                                             
optparse-applicative-0.14.2.0: configure                                                                                                                                                                                                                                        
optparse-applicative-0.14.2.0: build                                                                                                                                                                                                                                            
Progress: 2/7                                                                                                                                                                                                                                                                   
--  While building custom Setup.hs for package optparse-applicative-0.14.2.0 using:
      /home/u/stack/stack/setup-exe-cache/x86_64-linux-nix/Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2 --builddir=.stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0 build --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1
    Logs have been written to: /home/u/projects/haskellbox/.stack-work/logs/optparse-applicative-0.14.2.0.log

    Configuring optparse-applicative-0.14.2.0...
    Preprocessing library for optparse-applicative-0.14.2.0..
    Building library for optparse-applicative-0.14.2.0..
    [ 1 of 16] Compiling Options.Applicative.Help.Levenshtein ( Options/Applicative/Help/Levenshtein.hs, .stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0/build/Options/Applicative/Help/Levenshtein.o )
    [ 2 of 16] Compiling Options.Applicative.Help.Pretty ( Options/Applicative/Help/Pretty.hs, .stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0/build/Options/Applicative/Help/Pretty.o )
    [ 3 of 16] Compiling Options.Applicative.Help.Chunk ( Options/Applicative/Help/Chunk.hs, .stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0/build/Options/Applicative/Help/Chunk.o )
    [ 4 of 16] Compiling Options.Applicative.Help.Types ( Options/Applicative/Help/Types.hs, .stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0/build/Options/Applicative/Help/Types.o )
    [ 5 of 16] Compiling Options.Applicative.Types ( Options/Applicative/Types.hs, .stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0/build/Options/Applicative/Types.o )
    [ 6 of 16] Compiling Options.Applicative.Internal ( Options/Applicative/Internal.hs, .stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0/build/Options/Applicative/Internal.o )
    [ 7 of 16] Compiling Options.Applicative.Common ( Options/Applicative/Common.hs, .stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0/build/Options/Applicative/Common.o )
    [ 8 of 16] Compiling Options.Applicative.Help.Core ( Options/Applicative/Help/Core.hs, .stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0/build/Options/Applicative/Help/Core.o )
    <command line>: can't load .so/.DLL for: libHStransformers-compat-0.5.1.4-EAh8Ccrgitz4vTZtbTPyCM.so (libHStransformers-compat-0.5.1.4-EAh8Ccrgitz4vTZtbTPyCM.so: cannot open shared object file: No such file or directory)


--  While building custom Setup.hs for package tagged-0.8.5 using:
      /home/u/stack/stack/setup-exe-cache/x86_64-linux-nix/Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2 --builddir=.stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0 build --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1
    Logs have been written to: /home/u/projects/haskellbox/.stack-work/logs/tagged-0.8.5.log

    Configuring tagged-0.8.5...
    Preprocessing library for tagged-0.8.5..
    Building library for tagged-0.8.5..
    [1 of 2] Compiling Data.Proxy.TH    ( src/Data/Proxy/TH.hs, .stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0/build/Data/Proxy/TH.o )
    [2 of 2] Compiling Data.Tagged      ( src/Data/Tagged.hs, .stack-work/dist/x86_64-linux-nix/Cabal-2.0.1.0/build/Data/Tagged.o )
    /nix/store/klwxxrlgalq49cjnczvgha3j7l00jixw-binutils-2.28.1/bin/ld: cannot find -lHStransformers-compat-0.5.1.4-EAh8Ccrgitz4vTZtbTPyCM-ghc8.2.2
    collect2: error: ld returned 1 exit status
    `cc' failed in phase `Linker'. (Exit code: 1)

however, deleting most of ~/.stack (keeping global-project and config.yaml) and retrying solved it.

Could this be due to a failed or incomplete (Ctrl+C) dependency still passing as successful?

[soulomoon]

@unode Did you install binutils? I have a problem with it.

[unode]

@soulomoon As mentioned the issue went away with deleting ~/.stack. Haven't hit this since then.

[yairchu]

I have the same issue (with Stack Version 2.1.1, Git revision f612ea85316bbc327a64e4ad8d9f0b150dc12d4b (7648 commits) x86_64 hpack-0.31.2):

<command line>: can't load .so/.DLL for: /Users/yairchu/dev/lamdu/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/libHSLamdu-0.1-JPwBsW1Z4nhAN8TKkgUhdz-ghc8.6.4.dylib (dlopen(/Users/yairchu/dev/lamdu/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/libHSLamdu-0.1-JPwBsW1Z4nhAN8TKkgUhdz-ghc8.6.4.dylib, 5): Symbol not found: _lamduzmcalculuszm0zi2zi0zi0zmBfYOvPWZZhJ8F5ZZyxiGHWRA_LamduziCalcziInfer_zdfUnifyPureInferRowzuzdssemiPruneLookup1_closure
  Referenced from: /Users/yairchu/dev/lamdu/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/libHSLamdu-0.1-JPwBsW1Z4nhAN8TKkgUhdz-ghc8.6.4.dylib
  Expected in: flat namespace
 in /Users/yairchu/dev/lamdu/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/libHSLamdu-0.1-JPwBsW1Z4nhAN8TKkgUhdz-ghc8.6.4.dylib)

I worked around the issue by yanking my whole .stack and .stack-work folders and building again (which took precious time). Am now facing this issue for the third time in the span of two days and before yanking the temp Stack files here I'm looking for other work-arounds which won't screw up my workday as much.

Other than a fix, if anyone knows of a good workaround to this problem that would be greatly appreciated!!

[yairchu]

For me downgrading to Stack v1.9.3 solved this. Perhaps this old bug came back in v2?

[dbaynard]

This might be an issue with the dependency itself — see #4141. If one of you would try the solution at the end of the thread, we can work out how to proceed.

[yairchu]

This might be an issue with the dependency itself — see #4141. If one of you would try the solution at the end of the thread, we can work out how to proceed.

If it was an issue with the dependency, why would removing .stack-work and .stack solve this?

I got this issue again as I've recently re-upgraded to stack v2, because the latest stackage isn't stack v1 compatible.. and now I suffer from this again :( :( :( It's time to check again if nix works on mac :( Man I hate computers

[yairchu]

So again I worked around this by removing ~/.stack and .stack-work and rebuilding everything. It took 50 minutes. stack can't be a productive environment if you need to waste 50 minutes several times each work day :(

Edit: second time today now purging .stack. I tried to dig into the pantry sqlite and package.cache files to hide any reference to the misbuilt dependency but only succeeded in breaking stack instead :( Time to take a lunch break

[dbaynard]

If it was an issue with the dependency, why would removing .stack-work and .stack solve this?

I believe it's to do with temporary files. There's more detail in the issue. The fix involves a change to Setup.hs. Custom setups have the potential to cause problems. However, optparse-applicative uses Simple, so that can't be the case, here.

[yairchu]

The fix involves a change to Setup.hs. Custom setups have the potential to cause problems. However, optparse-applicative uses Simple, so that can't be the case, here.

To be clear I'm suffering from the same symptom but not in optparse-applicative, but also with package without custom setups. If I were to guess the issue might stem from me running both stack test in a shell while running ghcid in my editor which also invokes a build, and the simultaneous builds cause a bad state. Thing is with stack v1 when it happened once a week I could've stack clean broken-package and be up and running again in a minute, and with stack v2 this can happen several times a day and I haven't found a solution rather than yanking all of stack's files which then takes an hour to rebuild :( and stack v2 is "forced" on us via the new resolvers not supporting v1

[dbaynard]

It might be possible to be more selective in which files get deleted. You say stack clean works — what happens then?

[dbaynard]

Can you build all the dependencies, individually? As in #4969

[yairchu]

You say stack clean works — what happens then?

No, stack clean doesn't work. In v1 stack clean lamdu-calculus (where the package we are building is lamdu and lamdu-calculus is a dependency) did work. In stack v2 the only thing that worked for me is removing .stack-work and ~/.stack.

Can you build all the dependencies, individually? As in #4969

Unlike that issue, for me things do work fine after removing the stack files and rebuilding cleanly. The dependencies also can build individually successfully.

[yairchu]

More info: this doesn't seem to require a race with a concurrent ghcid. Just happens randomly. Another observation is that it seems that stack v2 also doesn't rebuild packages when their dependencies extra-deps have been updated in the project, another instance where a stack clean package-name could had saved the day.

Update: bumping a version of a hackage dependency may trigger a rebuild of the broken dependency and fix the build without yanking the whole .stack

[haroldcarr]

Is this issue getting any attention?
I am now running into this every day.
Only solutions for me so are are

• delete .stack (time consuming)
• change resolver (time and disk consuming)

[yairchu]

This problem started occuring often again now after upgrading to stack 2.5.1

However the following works around it: stack clean <TARGET> (update: doesn't universally works, only if the faulty build is in a target but not a dependency)

[yairchu]

I have a new theory on why this happens along with a new work-around that seems to work for me (the simple removing .stack work-around makes my bisect step take 1 hour which isn't ideal).

Theory: There are several .dylib/.sos of the same dependency in the linker path and the wrong one gets used at some point when building another dependency.

Work-around: decode the missing symbol to figure out which library it should come from. Remove all the .dylibs for this library in ~/.stack, as well as for the library which depends on that. Use stack exec ghc-pkg -- unregister for these dependencies.