Ghcide: Doesn't understand test dependencies

[ChrisPenner]

When I open a file in my test-suite which has test-only dependencies (e.g. hspec) which aren't in the library build-depends, ghcide can't find them and just bugs me about the fact that it can't find Test.Hspec.

Is it possible for ghcide to understand the difference? Let me know if there's a config flag I'm missing, this would be a nice feature to have if it doesn't already exist 😄

My current solution is to add the test dependencies to the library while I'm developing, which is obviously a bit of a hack 😬

[ndmitchell]

Are you using hie-bios with either no hie.yaml or a Cabal cradle? I think this is just how hie-bios works. I don't like this behaviour, so in all my repos I have a custom hie.yaml file that lists the dependencies - e.g. see the one in this repo. To solve it probably, or even hackily requires more info from hie-bios, or potentially using something like cabal-helper that does perform these tricks.

[ChrisPenner]

I wasn't aware of hie.yaml or a cradle; I assumed one of these tools would try to read the cabal file; but I guess not. I'll try it out :)

[jacg]

In the absence of a hie.yaml, hie-bios does try to read a cabal file by default: https://github.com/mpickering/hie-bios#implicit-configuration

[ChrisPenner]

I tried both with or without one and it doesn't work for me; I'm using a traditional stack folder structure.

If anyone wants to test it out, here's the project I'm using: https://github.com/ChrisPenner/astar-monad/tree/bafe7d36462412fb58385d36af989479a8ac41fd

Is there a way I can test that hie-bios is working properly? Running it on its own just fails silently, I assume there are flags I need to pass to test it out? I can ask over in hie-bios if this is more of an issue for there.

[ndmitchell]

Using ghcide on the command line tests hie-bios, so I'd just do that.

[Unisay]

Here is my workaround in hie-bios:

diff --git a/src/HIE/Bios/Cradle.hs b/src/HIE/Bios/Cradle.hs
index 4b8b991..35ddcad 100644
--- a/src/HIE/Bios/Cradle.hs
+++ b/src/HIE/Bios/Cradle.hs
@@ -138,6 +138,20 @@ processCabalWrapperArgs args =
             in trace dir $ Just final_args
         _ -> Nothing

+processCabalWrapperArgsTest :: String -> String -> Maybe [String]
+processCabalWrapperArgsTest args testArgs =
+  case (lines args, lines testArgs) of
+    ([dir, ghcArgs], ([_, ghcArgsTest])) ->
+      let
+        (packageArgs, restGhcArgs) = extractPackages (words ghcArgs)
+        (packageTestArgs, _) = extractPackages (words ghcArgsTest)
+        finalArgs = removeInteractive $ fixImportDirs dir <$> restGhcArgs
+      in trace dir $ Just $ finalArgs <> packageArgs <> packageTestArgs
+    _ -> Nothing
+  where
+    extractPackages :: [String] -> ([String], [String])
+    extractPackages = partition (isPrefixOf "-package-id=")
+
 cabalAction :: FilePath -> Maybe String -> FilePath -> IO (ExitCode, String, [String])
 cabalAction work_dir mc _fp = do
   wrapper_fp <- writeSystemTempFile "wrapper.bat" $
@@ -193,15 +207,23 @@ stackAction work_dir fp = do
   setFileMode wrapper_fp accessModes
   check <- readFile wrapper_fp
   traceM check
-  (ex1, args, stde) <-
-      withCurrentDirectory work_dir (readProcessWithExitCode "stack" ["repl", "--silent", "--no-load", "--with-ghc", wrapper_fp, fp ] [])
-  (ex2, pkg_args, stdr) <-
-      withCurrentDirectory work_dir (readProcessWithExitCode "stack" ["path", "--ghc-package-path"] [])
+  let runWrapper target = withCurrentDirectory work_dir $
+        readProcessWithExitCode "stack" ["repl", "--silent", "--no-load", "--with-ghc", wrapper_fp, target] []
+  (ex1, args, stde) <- runWrapper fp
+  (ex2, test_args, stdt) <- runWrapper "--test"
+  (ex3, pkg_args, stdr) <- withCurrentDirectory work_dir $
+        readProcessWithExitCode "stack" ["path", "--ghc-package-path"] []
   let split_pkgs = splitSearchPath (init pkg_args)
       pkg_ghc_args = concatMap (\p -> ["-package-db", p] ) split_pkgs
-  case processCabalWrapperArgs args of
-      Nothing -> error (show (ex1, stde, args))
-      Just ghc_args -> return (combineExitCodes [ex1, ex2], stde ++ stdr, ghc_args ++ pkg_ghc_args)
+  case processCabalWrapperArgsTest args test_args of
+      Nothing ->
+        error (show (ex1, stde, args))
+      Just ghc_args ->
+        return
+          ( combineExitCodes [ex1, ex2, ex3]
+          , stde <> stdt <> stdr
+          , ghc_args <> pkg_ghc_args
+          )

 combineExitCodes :: [ExitCode] -> ExitCode
 combineExitCodes = foldr go ExitSuccess

[ChrisPenner]

Here's the output of ghcide in the project I linked

Starting ghcide (GHC v8.6)
/Users/chris/dev/astar-monad
[1/6] Finding hie-bios cradle
Cradle {cradleRootDir = "/Users/chris/dev/astar-monad", cradleOptsProg = CradleAction: stack}

[2/6] Converting Cradle to GHC session
#!/usr/bin/env bash
if [ "$1" == "--interactive" ]; then
  pwd
  echo "$@"
else
  ghc "$@"
fi

/Users/chris/dev/astar-monad
res
  (ExitSuccess,"",["-i/Users/chris/dev/astar-monad/","-odir=/Users/chris/dev/astar-monad/.stack-work/odir","-hidir=/Users/chris/dev/astar-monad/.stack-work/odir","-hide-all-packages","-i/Users/chris/dev/astar-monad/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build","-i/Users/chris/dev/astar-monad/src","-i/Users/chris/dev/astar-monad/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen","-i/Users/chris/dev/astar-monad/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen","-stubdir=/Users/chris/dev/astar-monad/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build","-package-id=base-4.12.0.0","-package-id=logict-0.7.0.2-BZmYqWjl84nB8IYAe6sthB","-package-id=mtl-2.2.2","-Wall","-Wno-name-shadowing","-optP-include","-optP/Users/chris/dev/astar-monad/.stack-work/ghci/59a1808c/cabal_macros.h","-package-db","/Users/chris/dev/astar-monad/.stack-work/install/x86_64-osx/ccfb613655309b8ffe558bb8d808ff6b73b803ee6218491d81691818d89266d0/8.6.5/pkgdb","-package-db","/Users/chris/.stack/snapshots/x86_64-osx/ccfb613655309b8ffe558bb8d808ff6b73b803ee6218491d81691818d89266d0/8.6.5/pkgdb","-package-db","/Users/chris/.stack/programs/x86_64-osx/ghc-8.6.5/lib/ghc-8.6.5/package.conf.d"],"")
["-i/Users/chris/dev/astar-monad/","-odir=/Users/chris/dev/astar-monad/.stack-work/odir","-hidir=/Users/chris/dev/astar-monad/.stack-work/odir","-hide-all-packages","-i/Users/chris/dev/astar-monad/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build","-i/Users/chris/dev/astar-monad/src","-i/Users/chris/dev/astar-monad/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/autogen","-i/Users/chris/dev/astar-monad/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build/global-autogen","-stubdir=/Users/chris/dev/astar-monad/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/build","-package-id=base-4.12.0.0","-package-id=logict-0.7.0.2-BZmYqWjl84nB8IYAe6sthB","-package-id=mtl-2.2.2","-Wall","-Wno-name-shadowing","-optP-include","-optP/Users/chris/dev/astar-monad/.stack-work/ghci/59a1808c/cabal_macros.h","-package-db","/Users/chris/dev/astar-monad/.stack-work/install/x86_64-osx/ccfb613655309b8ffe558bb8d808ff6b73b803ee6218491d81691818d89266d0/8.6.5/pkgdb","-package-db","/Users/chris/.stack/snapshots/x86_64-osx/ccfb613655309b8ffe558bb8d808ff6b73b803ee6218491d81691818d89266d0/8.6.5/pkgdb","-package-db","/Users/chris/.stack/programs/x86_64-osx/ghc-8.6.5/lib/ghc-8.6.5/package.conf.d"]
22
([],0)

[3/6] Initialising IDE session

[4/6] Finding interesting files
Found 4 files

[5/6] Setting interesting files
[DEBUG] Set files of interest to: [NormalizedFilePath "./Setup.hs",NormalizedFilePath "./src/Control/Monad/AStar.hs",NormalizedFilePath "./src/Control/Monad/AStar/Class.hs",NormalizedFilePath "./test/Spec.hs"]
[DEBUG] Starting shakeRun (aborting the previous one took 0.00s)

[6/6] Loading interesting files
[DEBUG] Finishing shakeRun (took 0.00s, exception: thread killed)
[DEBUG] Starting shakeRun (aborting the previous one took 0.00s)
File:     ./Setup.hs
Range:    1:7-1:26
Source:   not found
Severity: DsError
Message:
  Could not load module ‘Distribution.Simple’
  It is a member of the hidden package ‘Cabal-2.4.1.0’.
  You can run ‘:set -package Cabal’ to expose it.
  (Note: this unloads all the modules in the current scope.)
  It is a member of the hidden package ‘Cabal-2.4.0.1’.
  You can run ‘:set -package Cabal’ to expose it.
  (Note: this unloads all the modules in the current scope.)
File:     ./test/Spec.hs
Range:    9:7-9:17
Source:   not found
Severity: DsError
Message:
  Could not load module ‘Test.Hspec’
  It is a member of the hidden package ‘hspec-2.7.1’.
  You can run ‘:set -package hspec’ to expose it.
  (Note: this unloads all the modules in the current scope.)
File:     ./test/Spec.hs
Range:    11:7-11:19
Source:   not found
Severity: DsError
Message:
  Could not load module ‘Control.Lens’
  It is a member of the hidden package ‘lens-4.17.1’.
  You can run ‘:set -package lens’ to expose it.
  (Note: this unloads all the modules in the current scope.)
[DEBUG] Finishing shakeRun (took 0.36s, completed)
Files that worked: 2
Files that failed: 2
 * ./Setup.hs
 * ./test/Spec.hs
Done

It complains about not finding lens or hspec, they're only included in my tests target.

[ChrisPenner]

Tagging @mpickering 😄

[mpickering]

I can't discern any bug here. It sounds like you are expecting multiple component support which isn't implemented in ghcide yet.

[madgen]

So if I have cradle: {cabal: {component: project-test} } where the tests use hspec-discover, it almost works because my test component's dependencies is a superset of the other components. However, it doesn't work because hspec-discover is not a globally visible build tool and cabal cradle does not pick up the .cabal/store/... where hspec-discover lives.

I'm not entirely sure if this is a problem with hie-bios. @mpickering (?)

On the command line, if I run cabal exec -- ghcide with the said hie.yaml, I can get all my files working, presumably because it can see the hspec-discover executable.

[ndmitchell]

Ghcide doesn't have support for custom preprocessors, which at least must be part of the problem. How it works when you run ghcide locally I have no idea - perhaps because an empty file is a valid file? I suggest we use this issue for custom preprocessors. Note that we added CPP and LHS support, and were aware that custom preprocessors was the logical one left over we hadn't tackled.

@cocreature perhaps edit the issue title and change the labels? (would be great if trusted users could tweak those things...)

[cocreature]

I’m honestly not sure what to rename this issue to. There is the lack of multi-component support which is also tracked by #113 and the unrelated issue of preprocessors. Maybe we should just close this one in favor of #113 and @madgen can open a separate issue for hspec-discover?

[ndmitchell]

I think this issue is all about preprocessors by the last comment.

[cocreature]

Alright, opened https://github.com/digital-asset/ghcide/issues/124 for the preprocessor issue since the comments here mix up too many different issues and I’ll close this.