cabal-3.2.0.0 fails to build/install several packages - assert fails

[mouse07410]

Describe the bug
Currently-released cabal-3.2.0.0 aborts at build/install of several packages, e.g., pandoc, upon failing assertion.

To Reproduce

$ cabal install pandoc
Warning: Parsing the index cache failed (Data.Binary.Get.runGet at position
16: Non-matching structured hashes: 0210baca00000000ec34895e00000000;
expected: a257ca064dfb5e0cb74f74e64a975b9e). Trying to regenerate the index
cache...
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following will be built (use -v for more details):
 - HTTP-4000.3.14 (lib) (requires build)
 - http-client-0.6.4.1 (lib) (requires build)
 - socks-0.6.1 (lib) (requires build)
 - tls-1.5.4 (lib) (requires build)
 - connection-0.3.1 (lib:connection) (requires build)
 - http-client-tls-0.3.5.3 (lib) (requires build)
 - pandoc-2.9.2.1 (lib) (requires build)
Assertion failed
CallStack (from HasCallStack):
  assert, called at ./Distribution/Client/ProjectPlanning.hs:242:5 in main:Distribution.Client.ProjectPlanning

Expected behavior
Successful build/installation, as with cabal-3.0.0.0.

System information

• MacOS Catalina 10.15.4, Xcode-11.4.
• cabal-3.2.0.0, ghc-8.8.3

Additional context
Cabal configured to build dynamic libraries and executables, profiling is disabled, documentation enabled.

Cabal installed via $ cabal install cabal-install --overwrite-policy=always

On a probably-unrelated note, it looks like Cabal doesn't take command-line arguments --enable-static and --enable-executable-static: when I specify those, it still seems to follow what's in ~/.cabal/config, which says "dynamic".

[hvr]

@mouse07410 You installed cabal-install from the Git repo, didn't you? I.e. you were inside the Git repo when you invoked cabal install cabal-install --overwrite-policy=always?

Does this also occur with the executable from https://downloads.haskell.org/~cabal/cabal-install-3.2.0.0/cabal-install-3.2.0.0-x86_64-apple-darwin17.7.0.tar.xz ?

[mouse07410]

You installed cabal-install from the Git repo, didn't you?

Initially - yes, but when I found that it was officially released, I installed it from the Hackage by doing $ cd ; cabal install cabal-install --overwrite-policy-=always. So, no - I was not inside the Git repo during my last couple of re-installations. And without that --overwrite... it fails to create a symlink, so the failure to pass that flag couldn't pass unnoticed.

Does this also occur with the executable from https://downloads.haskell.org/~cabal/cabal-install-3.2.0.0/cabal-install-3.2.0.0-x86_64-apple-darwin17.7.0.tar.xz ?

Hmm, I've no idea - never downloaded individual binaries manually before. Always used either ghcup tool, or "upgrade in place" (aka - rebuilding locally)... It appears that downloaded cabal-install binaries work fine, and do not exhibit this problem. Only the locally-rebuilt ones do.

[phadej]

I cannot reproduce this on Linux.
My suggestion would be to wipe ~/.cabal/store/ghc-8.8.3/ and try again.

I'm quite sure that some of these assertions trigger when store gets inconsistent (for whatever reasons).

[mouse07410]

I'm quite sure that some of these assertions trigger when store gets inconsistent (for whatever reasons).

Could switching between Cabal-3.3.0.0 build from Git repo, Cabal-3.2.0.0 built from Git repo, Cabal-3.0.0.0 distributed via ghcup, and Cabal-3.2.0.0 built from Hackage contribute/cause this?

If so, does it mean that once I "stabilize" on built-from-Hackage Cabal-3.2.0.0, it should go away? Or you still recommend blowing away ~/.cabal/store/ghc-8.8.3?

[phadej]

Could switching between Cabal-3.3.0.0 build from Git repo, Cabal-3.2.0.0 built from Git repo, Cabal-3.0.0.0 distributed via ghcup, and Cabal-3.2.0.0 built from Hackage contribute/cause this?

No, that shouldn't happen. More likely cancelling a build (or failing build) left things in inconsistent state.

[mouse07410]

More likely cancelling a build (or failing build) left things in inconsistent state...

Darn... Lord knows I've had more than a few of those.

Do you still advise wiping out ~/.cabal/store/ghc-8.8.3? Or wait and see if I get that problem again, and blow it then?

[phadej]

If it doesn't happen consistently, and you cannot reproduce it right away, there's nothing to do.

[hvr]

Always used either ghcup tool, or "upgrade in place" (aka - rebuilding locally)...

@mouse07410 fwiw, that's the very same binary ghcup would download too

[mouse07410]

I'm getting that error consistently now with cabal-install that I build from the source.

I wiped ~/.cabal/store/ghc-8.8.3. I built cabal-3.3.0.0 from the current Cabal master - succeeded.

Then I tried to build HIE. Failed as shown above.

Then I tried to install floskell (cd ~/src; cabal install floskell). Failed:

$ cabal install floskell --overwrite-policy=always
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following will be built (use -v for more details):
 - StateVar-1.2 (lib) (requires build)
 - base-compat-0.11.1 (lib) (requires build)
 - base-orphans-0.8.2 (lib) (requires build)
 - cabal-doctest-1.0.8 (lib) (requires build)
 - colour-2.3.5 (lib) (requires build)
 - uuid-types-1.0.3 (lib) (requires build)
 - dlist-0.8.0.8 (lib) (requires build)
 - data-default-class-0.1.2.0 (lib:data-default-class) (requires build)
Assertion failed
CallStack (from HasCallStack):
  assert, called at ./Distribution/Client/ProjectPlanning.hs:243:5 in main:Distribution.Client.ProjectPlanning

I don't know where to find more detailed logs - ~/.cabal/logs/ghc-8.8.3 did not help.

Then I tried Cabal-3.2.0.0 installed by ghcup, and it worked flawlessly.

The only difference between our environments that I'm aware of - on my machine both Cabal and Stack are configured to build shared libraries and dynamic executables.

I wouldn't venture into building my "own" Cabal, but I need the fix provided by #6623.

[mouse07410]

And somehow I seem unable to force static build?

$ time cabal install exe:cabal --enable-static --disable-shared
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/Cabal-3.3.0.0.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/cabal-testsuite-3.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/cabal-install-3.3.0.0.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/solver-benchmarks-3.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/Cabal-quickcheck-3.3.0.0.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/cabal-benchmarks-3.tar.gz
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following will be built (use -v for more details):
 - cabal-install-3.3.0.0 (exe:cabal) (requires build)
Starting     cabal-install-3.3.0.0 (exe:cabal)
Building     cabal-install-3.3.0.0 (exe:cabal)
Installing   cabal-install-3.3.0.0 (exe:cabal)
Completed    cabal-install-3.3.0.0 (exe:cabal)
Symlinking 'cabal'

real    3m41.965s
user    3m21.837s
sys 0m15.473s

$ otool -L ~/.cabal/bin/cabal
/Users/uri/.cabal/bin/cabal:
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
    @rpath/libHSsync-2.2.2-ca63dbfc-ghc8.8.3.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHSstm-2.5.0.0-ghc8.8.3.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHSrslv-0.1.2.0-daf83fbe-ghc8.8.3.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHShshbl-1.3.0.0-27da760f-ghc8.8.3.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHShckg-scrty-0.6.0.1-36decdf1-ghc8.8.3.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHSzlb-0.6.2.1-c632b378-ghc8.8.3.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHStr-0.5.1.1-86143c27-ghc8.8.3.dylib (compatibility version 0.0.0, current version 0.0.0)
    . . . . .
@rpath/libHSghc-prim-0.5.3-ghc8.8.3.dylib (compatibility version 0.0.0, current version 0.0.0)
    @rpath/libHSrts_thr-ghc8.8.3.dylib (compatibility version 0.0.0, current version 0.0.0)
$ otool -L ~/.cabal/bin/cabal | wc -l
  35
$

[mouse07410]

Wiping out ~/.cabal/store/ghc-8.8.3 does not help. Done it multiple times now, with zero results:

• cabal-install built from source keeps failing on that assertion, pretty much regardless of what package I ask it to build/install;
• cabal-install downloaded via ghcup works fine.

[phadej]

I tried to reproduce this in docker container

% docker run --rm -ti phadej/ghc:8.8.3-bionic bash

root@abcdef:/# cd

We have cabal-install-3.2 there:

root@abcdef:~# which cabal
/opt/cabal/3.2/bin/cabal

root@abcdef:~# cabal --version
cabal-install version 3.2.0.0
compiled using version 3.2.0.0 of the Cabal library 

Let's update index and build pandoc

root@abcdef:~# cabal update
Config file path source is default config file.
Config file /root/.cabal/config not found.
Writing default configuration to /root/.cabal/config
Downloading the latest package list from hackage.haskell.org

root@abcdef:~# cabal install pandoc
...
Symlinking 'pandoc'

root@abcdef:~# pandoc -v
pandoc 2.9.2.1
Compiled with pandoc-types 1.20, texmath 0.12.0.1, skylighting 0.8.3.4
Default user data directory: /root/.local/share/pandoc or /root/.pandoc
Copyright (C) 2006-2020 John MacFarlane
Web:  https://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

It works. As in your case.

Let's remove it and build cabal from master

root@abcdef:~# rm $(which pandoc)

root@abcdef:~# rm -rf ~/.cabal/store/

root@abcdef:~# git clone https://github.com/haskell/cabal.git
Cloning into 'cabal'...
...

root@abcdef:~# cd cabal/

I use current master, specific commit to be precise:

root@abcdef:~/cabal# git checkout 79649a5f3f8fd616f169b9a8606f597865f99aab
Note: checking out '79649a5f3f8fd616f169b9a8606f597865f99aab'.

...

HEAD is now at 79649a5f3 Merge pull request #6684 from phadej/issue-6610

Then installing cabal

root@abcdef:~/cabal# cabal install cabal
...
Starting     cabal-install-3.3.0.0 (exe:cabal)
Building     cabal-install-3.3.0.0 (exe:cabal)
Installing   cabal-install-3.3.0.0 (exe:cabal)
Completed    cabal-install-3.3.0.0 (exe:cabal)
Symlinking 'cabal'

We need to remember to rehash to use just built version:

root@abcdef:~/cabal# hash -r

root@abcdef:~/cabal# which cabal
/root/.cabal/bin/cabal

root@abcdef:~/cabal# cabal --version
cabal-install version 3.3.0.0
compiled using version 3.3.0.0 of the Cabal library 



md5-481051dc7603e64823edd6286d8ccb75



root@abcdef:~/cabal# cd 



md5-bbd89bfb8a33487ee253524b2d6c2e11



root@abcdef:~# cabal install pandoc
Warning: Parsing the index cache failed (Data.Binary.Get.runGet at position
16: Non-matching structured hashes: a257ca064dfb5e0cb74f74e64a975b9e;
expected: 3cde863fbd21d2968a50ad918c73d775). Trying to regenerate the index
cache...
...

Starting     pandoc-2.9.2.1 (exe:pandoc)
Building     pandoc-2.9.2.1 (exe:pandoc)
Installing   pandoc-2.9.2.1 (exe:pandoc)
Completed    pandoc-2.9.2.1 (exe:pandoc)
Symlinking 'pandoc' to '/root/.cabal/bin/pandoc'

But it all works. There are some missing bits. It would be very helpful,
if you could make a reproducer using e.g. the docker image here.

Otherwise we'd simply need ot wait until we replace asserts in ProjectPlanning with type guarantees, which could take a while :(

[mouse07410]

I'll try with a Docker. But in the meanwhile - how was you Cabal configured? I suspect something breaks when you build it dynamic, i.e., in ~/.cabal/config:

shared: True
static: False
libraryProfiling: False
dynamic-executables: True
documentation: True

I'm reasonably convinced this has something to do with the problem, because somehow the binaries you build for Apple work fine.

Would you be able to test dynamically-built Cabal? I.e., build cabal-install itself as dynamic, and then use it to build something on a fresh ~/.cabal/store (i.e., build a package that hasn't been built yet on this system is container)?

Another possibility is that the latest Apple Xcode-11.4 that I use (like other v11 versions) has a bug with the stack 16-byte alignment - but in C/C++ code this bug manifests as SEGV11, so maybe it's unrelated...

[phadej]

if cabal-install itself is dynamicly linked, then most likely after you purge the ~/.cabal/store it won't work at all (as its own dependencies are gone).

[mouse07410]

most likely after you purge the ~/.cabal/store it won't work at all...

No, the store is purged before cabal-install is built.

Again, would it be possible for you to test fully dynamically-built-from-scratch cabal-install?

[phadej]

You had typos, I guess it should be:

library-profiling: False
executable-dynamic: True

---

root@abcdef:~/cabal# cabal user-config update -a 'static: False' -a 'shared: True' -a 'library-profiling: False' -a 'executable-dynamic: True' -a 'documentation: True'
Config file path source is default config file.
Config file /root/.cabal/config not found.
Writing default configuration to /root/.cabal/config
Renaming /root/.cabal/config to /root/.cabal/config.backup.
Writing merged config to /root/.cabal/config.

root@abdef:~/cabal# diff -u /root/.cabal/config.backup /root/.cabal/config
--- /root/.cabal/config.backup  2020-04-10 10:57:57.922619597 +0000
+++ /root/.cabal/config 2020-04-10 10:57:57.926619551 +0000
@@ -39,10 +39,10 @@
 -- program-prefix: 
 -- program-suffix: 
 -- library-vanilla: True
--- library-profiling:
--- shared:
--- static:
--- executable-dynamic: False
+library-profiling: False
+shared: True
+static: False
+executable-dynamic: True
 -- executable-static: False
 -- profiling:
 -- executable-profiling:
@@ -81,7 +81,7 @@
 -- allow-older: False
 -- allow-newer: False
 -- write-ghc-environment-files:
--- documentation: False
+documentation: True
 -- doc-index-file: $datadir/doc/$arch-$os-$compiler/index.html
 -- target-package-db:

root@abdef:~/cabal# cabal install cabal
...
Symlinking 'cabal'

root@abcdef:~/cabal# hash -r && which cabal && cabal --version
/root/.cabal/bin/cabal
cabal-install version 3.3.0.0
compiled using version 3.3.0.0 of the Cabal library 

root@abcdef:~/cabal# ldd `which cabal`
    linux-vdso.so.1 (0x00007ffc02f37000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f26fefd5000)
    libHSresolv-0.1.2.0-97c3e10da9478f37d01965a7720b869f0bb311a8274aead3304ddae642f72309-ghc8.8.3.so => /root/.cabal/store/ghc-8.8.3/resolv-0.1.2.0-97c3e10da9478f37d01965a7720b869f0bb311a8274aead3304ddae642f72309/lib/libHSresolv-0.1.2.0-97c3e10da9478f37d01965a7720b869f0bb311a8274aead3304ddae642f72309-ghc8.8.3.so (0x00007f26ff452000)
        ...

root@abcdef:~/cabal# cd ..

root@abcdef:~# cabal install pandoc
Warning: Parsing the index cache failed (Data.Binary.Get.runGet at position
16: Non-matching structured hashes: a257ca064dfb5e0cb74f74e64a975b9e;
expected: 3cde863fbd21d2968a50ad918c73d775). Trying to regenerate the index
cache...
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following will be built (use -v for more details):
 - HsYAML-0.2.1.0 (lib) (requires build)
 - SHA-1.6.4.4 (lib) (requires build)
 - base-compat-0.11.1 (lib) (requires build)
 - base-noprelude-4.13.0.0 (lib) (requires build)
 - base-orphans-0.8.2 (lib) (requires build)
 - basement-0.0.11 (lib) (requires build)
 - bitarray-0.0.1.1 (lib:bitarray) (requires build)
 - blaze-builder-0.4.1.0 (lib) (requires build)
 - case-insensitive-1.2.1.0 (lib) (requires build)
 - cereal-0.5.8.1 (lib) (requires build)
 - cmark-gfm-0.2.1 (lib) (requires build)
 - colour-2.3.5 (lib) (requires build)
 - data-default-class-0.1.2.0 (lib:data-default-class) (requires build)
 - digest-0.0.1.2 (lib:digest) (requires build)
 - dlist-0.8.0.8 (lib) (requires build)
 - emojis-0.1 (lib) (requires build)
 - exceptions-0.10.4 (lib) (requires build)
 - fail-4.9.0.0 (lib) (requires build)
 - haddock-library-1.8.0 (lib) (requires build)
 - hourglass-0.2.12 (lib) (requires build)
Assertion failed
CallStack (from HasCallStack):
  assert, called at ./Distribution/Client/ProjectPlanning.hs:243:5 in main:Distribution.Client.ProjectPlanning

bingo!

It's also enough to just do

cabal install pandoc --dry

---

Now as we have a reproducer, could you git bisect which commit caused the regression?

[mouse07410]

Update

Same thing with cabal-install 3.2.0.0 that I built from the sources from the GitHub repo:

$ pwd
/Users/uri/src/cabal
$ cabal install cabal-install --installdir=$HOME/.cabal/tmp --overwrite-policy=always --enable-static --disable-shared
. . . . .
[successful build]
$ ~/.cabal/tmp/cabal --version
cabal-install version 3.2.0.0
compiled using version 3.2.0.0 of the Cabal library 
$ otool -L ~/.cabal/tmp/cabal | wc -l
      35
$ otool -L ~/.ghcup/bin/cabal | wc -l
       6

You can see that despite the appropriate (I think?) flags, it still built a dynamic Cabal, unlike what ghcup installed.

And, this dynamically-built (or linked) Cabal fails the same way:

$ ~/.cabal/tmp/cabal install ShellCheck --dry
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following would be built (use -v for more details):
 - regex-tdfa-1.3.1.0 (lib) (requires download & build)
 - ShellCheck-0.7.1 (lib) (requires download & build)
Assertion failed
CallStack (from HasCallStack):
  assert, called at ./Distribution/Client/ProjectPlanning.hs:242:5 in main:Distribution.Client.ProjectPlanning

It doesn't really matter what package you try to install - the only thing is it should be something not already/currently installed.

So, for all we know, this bug that manifests itself with dynamic builds of cabal-install has been with us forever - just nobody bothered to check...

Now, the unpleasant part. I created an alternate configuration for static builds, but did not clear the ~/.cabal/store. The resulting cabal-install seems reasonably static now, and it still fails:

$ cabal --config-file=/Users/uri/.cabal/config-static --enable-static --disable-shared install cabal-install --installdir=$HOME/.cabal/tmp --overwrite-policy=always --enable-static --disable-shared
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/Cabal-3.3.0.0.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/cabal-testsuite-3.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/cabal-install-3.3.0.0.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/solver-benchmarks-3.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/Cabal-quickcheck-3.3.0.0.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/cabal-benchmarks-3.tar.gz
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following will be built (use -v for more details):
 - Cabal-3.3.0.0 (lib) (requires build)
 - base16-bytestring-0.1.1.6 (lib:base16-bytestring) (requires build)
 - base64-bytestring-1.0.0.3 (lib) (requires build)
 - echo-0.1.3 (lib) (requires build)
 - cryptohash-sha256-0.11.101.0 (lib) (requires build)
 - ed25519-0.0.5.0 (lib) (requires build)
 - hsc2hs-0.68.7 (exe:hsc2hs) (requires build)
 - hashable-1.3.0.0 (lib) (requires build)
 - network-uri-2.6.3.0 (lib) (requires build)
 - random-1.1 (lib) (requires build)
 - tar-0.5.1.1 (lib) (requires build)
 - zlib-0.6.2.1 (lib) (requires build)
 - resolv-0.1.2.0 (lib:resolv) (requires build)
 - network-3.1.1.1 (lib:network) (requires build)
 - lukko-0.1.1.2 (lib) (requires build)
 - async-2.2.2 (lib) (requires build)
 - edit-distance-0.2.2.1 (lib) (requires build)
 - HTTP-4000.3.14 (lib) (requires build)
 - hackage-security-0.6.0.1 (lib) (requires build)
 - cabal-install-3.3.0.0 (exe:cabal) (requires build)
Starting     echo-0.1.3 (lib)
Starting     base64-bytestring-1.0.0.3 (lib)
Starting     base16-bytestring-0.1.1.6 (all, legacy fallback)
Starting     Cabal-3.3.0.0 (lib)
Building     base16-bytestring-0.1.1.6 (all, legacy fallback)
Building     base64-bytestring-1.0.0.3 (lib)
Building     echo-0.1.3 (lib)
Building     Cabal-3.3.0.0 (lib)
Haddock      base16-bytestring-0.1.1.6 (all, legacy fallback)
Haddock      echo-0.1.3 (lib)
Haddock      base64-bytestring-1.0.0.3 (lib)
Installing   base16-bytestring-0.1.1.6 (all, legacy fallback)
Installing   echo-0.1.3 (lib)
Installing   base64-bytestring-1.0.0.3 (lib)
Completed    base16-bytestring-0.1.1.6 (all, legacy fallback)
Starting     cryptohash-sha256-0.11.101.0 (lib)
Completed    echo-0.1.3 (lib)
Starting     ed25519-0.0.5.0 (lib)
Completed    base64-bytestring-1.0.0.3 (lib)
Starting     hsc2hs-0.68.7 (exe:hsc2hs)
Building     cryptohash-sha256-0.11.101.0 (lib)
Building     ed25519-0.0.5.0 (lib)
Building     hsc2hs-0.68.7 (exe:hsc2hs)
Haddock      cryptohash-sha256-0.11.101.0 (lib)
Installing   cryptohash-sha256-0.11.101.0 (lib)
Completed    cryptohash-sha256-0.11.101.0 (lib)
Starting     hashable-1.3.0.0 (lib)
Building     hashable-1.3.0.0 (lib)
Installing   hsc2hs-0.68.7 (exe:hsc2hs)
Completed    hsc2hs-0.68.7 (exe:hsc2hs)
Starting     network-uri-2.6.3.0 (lib)
Haddock      ed25519-0.0.5.0 (lib)
Haddock      hashable-1.3.0.0 (lib)
Building     network-uri-2.6.3.0 (lib)
Installing   ed25519-0.0.5.0 (lib)
Installing   hashable-1.3.0.0 (lib)
Completed    ed25519-0.0.5.0 (lib)
Starting     random-1.1 (lib)
Completed    hashable-1.3.0.0 (lib)
Starting     tar-0.5.1.1 (lib)
Building     random-1.1 (lib)
Building     tar-0.5.1.1 (lib)
Haddock      network-uri-2.6.3.0 (lib)
Installing   network-uri-2.6.3.0 (lib)
Haddock      random-1.1 (lib)
Completed    network-uri-2.6.3.0 (lib)
Starting     zlib-0.6.2.1 (lib)
Installing   random-1.1 (lib)
Completed    random-1.1 (lib)
Starting     resolv-0.1.2.0 (all, legacy fallback)
Building     zlib-0.6.2.1 (lib)
Haddock      tar-0.5.1.1 (lib)
Installing   tar-0.5.1.1 (lib)
Completed    tar-0.5.1.1 (lib)
Starting     network-3.1.1.1 (all, legacy fallback)
Building     resolv-0.1.2.0 (all, legacy fallback)
Haddock      zlib-0.6.2.1 (lib)
Installing   zlib-0.6.2.1 (lib)
Completed    zlib-0.6.2.1 (lib)
Starting     lukko-0.1.1.2 (lib)
Building     network-3.1.1.1 (all, legacy fallback)
Building     lukko-0.1.1.2 (lib)
Haddock      lukko-0.1.1.2 (lib)
Installing   lukko-0.1.1.2 (lib)
Completed    lukko-0.1.1.2 (lib)
Starting     async-2.2.2 (lib)
Building     async-2.2.2 (lib)
Haddock      network-3.1.1.1 (all, legacy fallback)
Haddock      async-2.2.2 (lib)
Installing   async-2.2.2 (lib)
Haddock      resolv-0.1.2.0 (all, legacy fallback)
Installing   network-3.1.1.1 (all, legacy fallback)
Completed    async-2.2.2 (lib)
Starting     edit-distance-0.2.2.1 (lib)
Installing   resolv-0.1.2.0 (all, legacy fallback)
Completed    network-3.1.1.1 (all, legacy fallback)
Starting     HTTP-4000.3.14 (lib)
Completed    resolv-0.1.2.0 (all, legacy fallback)
Building     edit-distance-0.2.2.1 (lib)
Building     HTTP-4000.3.14 (lib)
Haddock      edit-distance-0.2.2.1 (lib)
Installing   edit-distance-0.2.2.1 (lib)
Completed    edit-distance-0.2.2.1 (lib)
Haddock      HTTP-4000.3.14 (lib)
Installing   HTTP-4000.3.14 (lib)
Completed    HTTP-4000.3.14 (lib)
Haddock      Cabal-3.3.0.0 (lib)
Installing   Cabal-3.3.0.0 (lib)
Completed    Cabal-3.3.0.0 (lib)
Starting     hackage-security-0.6.0.1 (lib)
Building     hackage-security-0.6.0.1 (lib)
Haddock      hackage-security-0.6.0.1 (lib)
Installing   hackage-security-0.6.0.1 (lib)
Completed    hackage-security-0.6.0.1 (lib)
Starting     cabal-install-3.3.0.0 (exe:cabal)
Building     cabal-install-3.3.0.0 (exe:cabal)
Installing   cabal-install-3.3.0.0 (exe:cabal)
Completed    cabal-install-3.3.0.0 (exe:cabal)
Symlinking 'cabal'
$ otool -L ~/.cabal/tmp/cabal | wc -l
       5
$ ~/.cabal/tmp/cabal --version
cabal-install version 3.3.0.0
compiled using version 3.3.0.0 of the Cabal library 
$ ~/.cabal/tmp/cabal install ShellCheck --dry
Warning: Parsing the index cache failed (Data.Binary.Get.runGet at position
16: Non-matching structured hashes: a257ca064dfb5e0cb74f74e64a975b9e;
expected: 3cde863fbd21d2968a50ad918c73d775). Trying to regenerate the index
cache...
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following would be built (use -v for more details):
 - regex-tdfa-1.3.1.0 (lib) (requires download & build)
 - ShellCheck-0.7.1 (lib) (requires download & build)
Assertion failed
CallStack (from HasCallStack):
  assert, called at ./Distribution/Client/ProjectPlanning.hs:243:5 in main:Distribution.Client.ProjectPlanning
$ 

[mouse07410]

Same thing with 3.2.0.0 that I build with GHC-8.8.3 from the GutHub sources. Even static, it fails:

$ git checkout 3.2
Switched to branch '3.2'
Your branch is up to date with 'origin/3.2'.
$ cabal --config-file=/Users/uri/.cabal/config-static --enable-static --disable-shared install cabal-install --installdir=$HOME/.cabal/tmp --overwrite-policy=always --enable-static --disable-shared
Warning: Parsing the index cache failed (Data.Binary.Get.runGet at position
16: Non-matching structured hashes: 3cde863fbd21d2968a50ad918c73d775;
expected: a257ca064dfb5e0cb74f74e64a975b9e). Trying to regenerate the index
cache...
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/Cabal-3.2.0.0.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/cabal-testsuite-3.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/cabal-install-3.2.0.0.tar.gz
Wrote tarball sdist to
/Users/uri/src/cabal/dist-newstyle/sdist/solver-benchmarks-3.tar.gz
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following will be built (use -v for more details):
 - Cabal-3.2.0.0 (lib) (requires build)
 - hackage-security-0.6.0.1 (lib) (requires build)
 - cabal-install-3.2.0.0 (exe:cabal) (requires build)
Starting     Cabal-3.2.0.0 (lib)
. . . . .
Symlinking 'cabal'
$ otool -L ~/.cabal/tmp/cabal
/Users/uri/.cabal/tmp/cabal:
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
    /usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)
    /opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
    /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
$ ~/.cabal/tmp/cabal --version
cabal-install version 3.2.0.0
compiled using version 3.2.0.0 of the Cabal library 
$ ~/.cabal/tmp/cabal install ShellCheck --dry
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following would be built (use -v for more details):
 - regex-tdfa-1.3.1.0 (lib) (requires download & build)
 - ShellCheck-0.7.1 (lib) (requires download & build)
Assertion failed
CallStack (from HasCallStack):
  assert, called at ./Distribution/Client/ProjectPlanning.hs:242:5 in main:Distribution.Client.ProjectPlanning
$ 

[phadej]

Also, is there any reason why cabal install cabal-install --enable-static --disable-shared still produces a fully-dynamic executable? Applies to building other packages as well. It seems the only way to build static now is editing ~/.cabal/config, which I'm not very keen to do every time...

I don't know, but don't conflate it with this issue.

[mouse07410]

It looks like when Cabal fails to build a package, like with the assertion above, it also corrupts the state of the store. To recover I had to (again!) wipe the store off and re-install everything with a good Cabal.

[mouse07410]

@phadej first, thanks for that container! docker run --rm -ti phadej/ghc:8.8.3-bionic bash helps me greatly.

Now, I tried building Cabal 3.2.0.0 dynamically on your container, from the cloned sources. It built, and exhibited the same failure on the same assertion:

root@64a1466a3974:~/cabal# cabal update
Downloading the latest package list from hackage.haskell.org
root@64a1466a3974:~/cabal# time cabal install cabal
Wrote tarball sdist to /root/cabal/dist-newstyle/sdist/Cabal-3.2.0.0.tar.gz
. . . . .
Completed    cabal-install-3.2.0.0 (all, legacy fallback)
Symlinking 'cabal'

real    8m46.070s
user    10m23.701s
sys 0m25.799s
root@64a1466a3974:~/cabal# which cabal
/root/.cabal/bin/cabal
root@64a1466a3974:~/cabal# hash -r
root@64a1466a3974:~/cabal# pwd
/root/cabal
root@64a1466a3974:~/cabal# cd ..
root@64a1466a3974:~# cabal install ormolu --dry
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following would be built (use -v for more details):
Assertion failed
CallStack (from HasCallStack):
  assert, called at ./Distribution/Client/ProjectPlanning.hs:242:5 in main:Distribution.Client.ProjectPlanning

On the other hand, installing the current (3.2.0.0) Cabal from Hackage apparently produced a working one:

root@64a1466a3974:~# time cabal install cabal-install
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following will be built (use -v for more details):
 - Cabal-3.2.0.0 (lib) (requires download & build)
. . . . .
Installing   cabal-install-3.2.0.0 (all, legacy fallback)
Completed    cabal-install-3.2.0.0 (all, legacy fallback)
Symlinking 'cabal'

real    8m52.522s
user    10m24.583s
sys 0m25.524s
root@64a1466a3974:~# which cabal
/root/.cabal/bin/cabal
root@64a1466a3974:~# ldd /root/.cabal/bin/cabal | wc -l
52
root@64a1466a3974:~# pwd
/root
root@64a1466a3974:~# cabal install ormolu --dry
Resolving dependencies...
Build profile: -w ghc-8.8.3 -O1
In order, the following would be built (use -v for more details):
 - alex-3.2.5 (exe:alex) (requires download & build)
 - base-compat-0.11.1 (lib) (requires download & build)
 - colour-2.3.5 (lib) (requires download & build)
 - dlist-0.8.0.8 (lib) (requires download & build)
 - exceptions-0.10.4 (lib) (requires download & build)
 - happy-1.19.12 (exe:happy) (requires download & build)
 - syb-0.7.1 (lib) (requires download & build)
 - transformers-compat-0.6.5 (lib) (requires download & build)
 - gitrev-1.3.1 (lib) (requires download & build)
 - ansi-terminal-0.10.3 (lib) (requires download & build)
 - ghc-lib-parser-8.8.3.20200224 (lib) (requires download & build)
 - ansi-wl-pprint-0.6.9 (lib) (requires download & build)
 - ormolu-0.0.3.1 (lib) (requires download & build)
 - optparse-applicative-0.15.1.0 (lib) (requires download & build)
 - ormolu-0.0.3.1 (exe:ormolu) (requires download & build)
root@64a1466a3974:~# 

I used your Cabal configuration command from https://github.com/haskell/cabal/issues/6659#issuecomment-611987238

root@64a1466a3974:~# diff -uw ~/.cabal/config.backup ~/.cabal/config
--- /root/.cabal/config.backup  2020-04-12 01:07:18.705242207 +0000
+++ /root/.cabal/config 2020-04-12 01:07:18.707242207 +0000
@@ -39,10 +39,10 @@
 -- program-prefix: 
 -- program-suffix: 
 -- library-vanilla: True
--- library-profiling:
--- shared:
--- static:
--- executable-dynamic: False
+library-profiling: False
+shared: True
+static: False
+executable-dynamic: True
 -- executable-static: False
 -- profiling:
 -- executable-profiling:
@@ -81,7 +81,7 @@
 -- allow-older: False
 -- allow-newer: False
 -- write-ghc-environment-files:
--- documentation: False
+documentation: True
 -- doc-index-file: $datadir/doc/$arch-$os-$compiler/index.html
 -- target-package-db:
 -- max-backjumps: 4000

Summary

Current observations (on my MacOS and your Linux container):

• Building Cabal dynamically from GitHub repo produced broken cabal-install
  
  
  
  • does not depend on OS
  
  
  • does not depend on GHC version
  
  
• Building Cabal statically from GitHub repo appeared to work on Linux container, but not on MacOS
• Building Cabal statically or dynamically from Hackage (currently offers 3.2.0.0) produces a working cabal-install everywhere

[mouse07410]

Ping...?