Stack: `stack build; stack test` rebuilds everything twice

[minad]

Steps to reproduce

1. stack build
2. stack test

I am using stackage nightly. The library of my project is rebuild by stack test.

Expected

The library should be compiled once and the compiled and cached output should be used.

Actual

stack test recompiles the library.

Stack version

2.1.1

Method of installation

• Official binary, downloaded from stackage.org or fpcomplete's package repository

[minad]

Maybe it has something to do with using a custom Setup.hs? I try to create a repro if possible. If I recall correctly, the issue started to occur after I upgraded to stack version 2.

stack says configure (lib + exe)... when calling stack build the first time. Then stack test says configure (lib + exe + test)...

[gromakovsky]

I sometimes observed this behavior, but it was not deterministic and I never tried to investigate it. I think I didn't observe it during last few months though.

P. S. It's with default Setup.hs.

[sjakobi]

I can observe this reliably. Switching between stack test and stack bench rebuilds the library too. This issue is a big time waster in my experience.

[mihaimaruseac]

If the compile flags need to change between building test and just building lib + exe then Stack would have to recompile everything, just like any other build system (just closed a similar issue with Bazel on another repo)

If you can provide a minimal example, we can investigate to see if Stack is too eager in invalidating the build cache

[sjakobi]

Here's an example using https://github.com/nikita-volkov/vector-builder:

$ cd vector-builder
$ stack clean --full
$ stack build
vector-builder> configure (lib)
Configuring vector-builder-0.3.7.2...
vector-builder> build (lib)
Preprocessing library for vector-builder-0.3.7.2..
Building library for vector-builder-0.3.7.2..
[1 of 8] Compiling VectorBuilder.Prelude
[2 of 8] Compiling VectorBuilder.Core.Update
[3 of 8] Compiling VectorBuilder.Core.Builder
[4 of 8] Compiling VectorBuilder.MVector
[5 of 8] Compiling VectorBuilder.Builder
[6 of 8] Compiling VectorBuilder.Vector
[7 of 8] Compiling VectorBuilder.MonadPlus
[8 of 8] Compiling VectorBuilder.Alternative
vector-builder> copy/register
Installing library in /home/simon/src/vector-builder/.stack-work/install/x86_64-linux/da6ab40722cf26def321f3afbad12b99af2f327f84d5d7d92f8138157548da6d/8.6.5/lib/x86_64-linux-ghc-8.6.5/vector-builder-0.3.7.2-JJTkYJEXEoEG9xl9pXropy
Registering library for vector-builder-0.3.7.2..
$ stack test --no-run-tests
vector-builder-0.3.7.2: unregistering (dependencies changed)
vector-builder> configure (lib + test)
Configuring vector-builder-0.3.7.2...
vector-builder> build (lib + test)
Preprocessing library for vector-builder-0.3.7.2..
Building library for vector-builder-0.3.7.2..
[1 of 8] Compiling VectorBuilder.Prelude
[2 of 8] Compiling VectorBuilder.Core.Update
[3 of 8] Compiling VectorBuilder.Core.Builder
[4 of 8] Compiling VectorBuilder.MVector
[5 of 8] Compiling VectorBuilder.Builder
[6 of 8] Compiling VectorBuilder.Vector
[7 of 8] Compiling VectorBuilder.MonadPlus
[8 of 8] Compiling VectorBuilder.Alternative
Preprocessing test suite 'tests' for vector-builder-0.3.7.2..
Building test suite 'tests' for vector-builder-0.3.7.2..
[1 of 2] Compiling Main.Sample
[2 of 2] Compiling Main
Linking .stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/tests/tests ...
vector-builder> copy/register
Installing library in /home/simon/src/vector-builder/.stack-work/install/x86_64-linux/da6ab40722cf26def321f3afbad12b99af2f327f84d5d7d92f8138157548da6d/8.6.5/lib/x86_64-linux-ghc-8.6.5/vector-builder-0.3.7.2-Kobe1ABTFVWHtSalbGSjYI
Registering library for vector-builder-0.3.7.2..
vector-builder> Test running disabled by --no-run-tests flag.
Completed 2 action(s).

Now I modify the last module in the build order, and stack rebuilds from zero!

$ echo '-- bla' >> library/VectorBuilder/Alternative.hs 
$ stack build
vector-builder-0.3.7.2: unregistering (local file changes: library/VectorBuilder/Alternative.hs)
vector-builder> configure (lib)
Configuring vector-builder-0.3.7.2...
vector-builder> build (lib)
Preprocessing library for vector-builder-0.3.7.2..
Building library for vector-builder-0.3.7.2..
[1 of 8] Compiling VectorBuilder.Prelude
[2 of 8] Compiling VectorBuilder.Core.Update
[3 of 8] Compiling VectorBuilder.Core.Builder
[4 of 8] Compiling VectorBuilder.MVector
[5 of 8] Compiling VectorBuilder.Builder
[6 of 8] Compiling VectorBuilder.Vector
[7 of 8] Compiling VectorBuilder.MonadPlus
[8 of 8] Compiling VectorBuilder.Alternative
vector-builder> copy/register
Installing library in /home/simon/src/vector-builder/.stack-work/install/x86_64-linux/da6ab40722cf26def321f3afbad12b99af2f327f84d5d7d92f8138157548da6d/8.6.5/lib/x86_64-linux-ghc-8.6.5/vector-builder-0.3.7.2-JJTkYJEXEoEG9xl9pXropy
Registering library for vector-builder-0.3.7.2..

[mihaimaruseac]

Thanks. Will investigate when I get a few cycles for open source

[minad]

One thing to add - for me it builds things only once if I do change something in the code; stack test; stack build but twice if I do change something in the code; stack build; stack test.

As a workaround I am always doing stack test now (which is slower since it also relinks the testsuite etc) and I interrupt when the test suite is starting :/

[sjakobi]

and I interrupt when the test suite is starting

You could use --no-run-tests for that.

[minad]

@sjakobi Ok that sounds good :see_no_evil: I have to admit I didn't really dig deep into all the things stack has to offer. I am using it for its great out of the box experience (Thank you stack developers for that!). And besides this problem I have not many other issues with it.

[mihaimaruseac]

Can also try stack build --test --bench --no-run-tests --no-run-benchmarks if you want to build everything but not run tests/benchmarks.

That's what I used to do at previous place where we were using Haskell daily

[rjmk]

Building with stack test appears to add the test dependencies to the dependencies of the library, thus invalidating the previous build

[jproyo]

Just in case if it helps i am facing the same issue with 2.1.3 but after doing this, some re linkings are gone:

stack build && stack test components --test-arguments "-m MyComponent" --file-watch

In the case i run with

stack test components --test-arguments "-m MyComponent" --file-watch

It tries to relink everything on each change. With stack build on front it seems it is avoid some recompilation, even when i change only one space in the test.

[kindaro]

Is this a duplicate of #2800? Looks similar from the outset, but the explanation given appears to be different _(wrong dependencies added to library here and upstream issue there)_ — although it can be that both are true.

I am also seeing similar behaviour, and I am confused as to where to put a +1.

[wraithm]

@kindaro I would assume it's the same.

[natefaubion]

The PureScript compiler suffers from the same symptoms as reported here. I currently work around it by downgrading to Stack 1.9, which doesn't appear to have the same problems.