Stack: "stack repl <test suite>" fails if "stack test --no-run-tests" is not run in advance for test suite containing errors

[runeksvendsen]

General summary/comments

Running the REPL for only the test suite in my project fails with an error saying the library package cannot be satisfied (even though the library builds without errors).

Please see https://github.com/commercialhaskell/stack/issues/5380#issuecomment-686731779 for an accurate description of the problem (I didn't realize this was the exact problem when I created this issue).

Steps to reproduce

1. Run the following commands to clone and build the example repo (the last step results in a compilation error):

git clone https://github.com/runeksvendsen/bellman-ford.git
cd bellman-ford
git checkout 781835aa000ad40f6c54f2c7426e9378ff6905e4
stack build
stack test --no-run-tests

Now start a REPL for the test suite:

stack repl bellman-ford:test:bellman-ford-test

Expected

Not failing with a "_cannot satisfy -package_"-error since the library builds fine.

Actual

It fails with the following error:

<command line>: cannot satisfy -package bellman-ford-0.1.0.0
    (use -v for more information)

Verbose output: https://gist.github.com/runeksvendsen/4ef3125578dc9de82962912be29f2f89

Stack version

$ stack --version
Version 2.3.3, Git revision cb44d51bed48b723a5deb08c3348c0b3ccfc437e x86_64 hpack-0.33.0

Method of installation

• Download from GitHub/Releases, then upgraded via stack upgrade

[mattaudesse]

Hey @runeksvendsen - sorry for the delayed response.

When I follow your steps above stack test --no-run-tests fails on commit 781835aa000ad40f6c54f2c7426e9378ff6905e4:

...
[ 9 of 12] Compiling Digraph.Spec                                                                         

/apps/apps/stack-issues/i5380/test/Digraph/Spec.hs:47:22: error:
    • Couldn't match type ‘TestEdge’ with ‘Lib.IdxEdge String Double’                                     
      Expected type: TestEdge -> IO ()                                                                    
        Actual type: Lib.IdxEdge String Double -> IO ()                                                   
    • In the second argument of ‘forM_’, namely
        ‘(stToIO . Lib.removeEdge graph)’
      In a stmt of a 'do' block:                                                                          
        forM_ edges (stToIO . Lib.removeEdge graph)
      In the expression:                     
        do graph <- stToIO $ Lib.fromEdges edges
           forM_ edges (stToIO . Lib.removeEdge graph)                                                    
           vertices <- stToIO $ Lib.vertices graph                                                        
           return (graph, vertices)    
   |        
47 |         forM_ edges (stToIO . Lib.removeEdge graph)                                    
   |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                   

/apps/apps/stack-issues/i5380/test/Digraph/Spec.hs:62:44: error:                                          
    • Couldn't match type ‘edge’ with ‘TestEdge’
      ‘edge’ is a rigid type variable bound by
        the type signature for:      
          addEdgesCheckInOutgoing :: forall edge.
                                     (Lib.Digraph RealWorld String Double                                 
                                      -> Lib.VertexId -> IO [edge])                                       
                                     -> [TestEdge] -> Expectation      
        at test/Digraph/Spec.hs:(51,1)-(54,18)                                                            
      Expected type: [edge]                         
        Actual type: [TestEdge]        
    • In the second argument of ‘shouldBe’, namely
        ‘removeDuplicateSrcDst sortedEdges’
      In a stmt of a 'do' block:
        sort (concat outgoingEdges)
          `shouldBe` removeDuplicateSrcDst sortedEdges
      In the expression:
        do let sortedEdges = sort edges
           graph <- stToIO $ Lib.fromEdges (reverse sortedEdges)
           vertices <- stToIO $ Lib.vertices graph
           outgoingEdges <- foldM (collectInOutgoing graph) [] vertices
           ....
    • Relevant bindings include
        outgoingEdges :: [[edge]] (bound at test/Digraph/Spec.hs:61:5)
        collectInOutgoing :: Lib.Digraph RealWorld String Double
                             -> [[edge]] -> Lib.VertexId -> IO [[edge]]
          (bound at test/Digraph/Spec.hs:64:5)
        inOutEdges :: Lib.Digraph RealWorld String Double
                      -> Lib.VertexId -> IO [edge]
          (bound at test/Digraph/Spec.hs:55:25)
        addEdgesCheckInOutgoing :: (Lib.Digraph RealWorld String Double
                                    -> Lib.VertexId -> IO [edge])
                                   -> [TestEdge] -> Expectation
          (bound at test/Digraph/Spec.hs:55:1)
   |        
62 |     sort (concat outgoingEdges) `shouldBe` removeDuplicateSrcDst sortedEdges
   |                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


--  While building package bellman-ford-0.1.0.0 using:
      /usr/home/matt/.stack/setup-exe-cache/x86_64-freebsd-ino64/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5 --builddir=.stack-work/dist/x86_64-freebsd-ino64/Cabal-2.4.0.1 build lib:bellman-ford test:bellman-ford-tes
t --ghc-options " -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1
Progress 1/2

The same steps (including stack repl bellman-ford:test:bellman-ford-test) seem to run fine on master, though, and it looks like your updates there are more recent.

Is it possible this is a fluke or am I missing something? Thanks for reporting in any case.

[runeksvendsen]

Hi @mattaudesse

Thank you for getting back to me. No worries about the delay.

The crux of this bug report is as follows.

If I do the following:

git clone https://github.com/runeksvendsen/bellman-ford.git
cd bellman-ford
git checkout 8e72e83a1cc29ba99ab171676d1c50a77d507ca1
stack test --no-run-tests
echo "\nwrongDefinition = 'a' :: Int" >> test/Spec.hs
stack repl bellman-ford:test:bellman-ford-test

Then the stack repl command successfully launches GHCi ie. leaves me with a prompt -- while reporting a type mismatch error in test/Spec.hs (Couldn't match expected type ‘Int’ with actual type ‘Char’).

If, however, I do the following (ie. omit stack test --no-run-tests before running stack repl):

git clone https://github.com/runeksvendsen/bellman-ford.git
cd bellman-ford
git checkout 8e72e83a1cc29ba99ab171676d1c50a77d507ca1
echo "\nwrongDefinition = 'a' :: Int" >> test/Spec.hs
stack repl bellman-ford:test:bellman-ford-test

Then the stack repl command fails to launch GHCi (ie. does not leave me with a prompt, but exits) with the error cannot satisfy -package bellman-ford-0.1.0.0.

I would think that the success of running stack repl should not depend on whether any stack build command has been run in advance. That is, stack repl in the latter sequence of commands should behave exactly as stack repl in the former sequence of commands (ie. not requiring that e.g. stack test --no-run-tests is run before running stack repl ...).

The reason I report this as a bug is because it causes a problem when stack is used in combination with haskell-language-server (as described here: https://github.com/haskell/haskell-language-server/issues/366).

[mattaudesse]

Thanks for the extra info - that helps.

I've tagged this as a bug but also "for discussion" since I could see it posing some architectural questions about what should happen in this case + how.

[runeksvendsen]

Seems like this is related to #1406. The comment here seems to describe this bug perfectly: https://github.com/commercialhaskell/stack/issues/1406#issuecomment-199627419.

[qrilka]

@runeksvendsen from my quick glance it looks like this problem could be another case for #4745 but I didn't have time yet to look closer into it.

[runeksvendsen]

@runeksvendsen from my quick glance it looks like this problem could be another case for #4745 but I didn't have time yet to look closer into it.

Thank you for looking into this.

What makes you think the two are related? Would fixing #4745 get rid of the error described in https://github.com/commercialhaskell/stack/issues/1406#issuecomment-199627419?

[qrilka]

Yes, component-based builds seems to be exactly what's needed in that situation - currently Stack builds all the components at once with an option to enable/disable tests/benchmarks.

[jneira]

This looks quite similar to #5213 (first stack repl foo:test:bar needs stack build --test --no-run-tests), that it is supposed to be fixed (partially)

[runeksvendsen]

This looks quite similar to #5213 (first stack repl foo:test:bar needs stack build --test --no-run-tests), that it is supposed to be fixed (partially)

It's very similar, but there's a difference in that this fix seems to no longer work if there's a compilation error in the test suite. If there is a compilation error in the test suite then stack test --no-run-tests is still required in order for stack repl foo:test:bar to succeed (as outlined above: https://github.com/commercialhaskell/stack/issues/5380#issuecomment-686731779).