Haskell-language-server: Relative filepath in Template Haskell different in HLS from Stack

[Vlix]

Subject of the issue

When having sub-libraries in a stack project, relative filepaths used in Template Haskell expressions are parsed as if they were run from the root directory of the project, but stack itself parses it relatively from the sub-library's root (which is one directory deeper).

For example:

import Data.Yaml (Value)
import Data.Yaml.TH (decodeFile)

val :: Value
val = $$(decodeFile "config/value.yaml")

Given the TH is used in ./lib/src/Lib.hs with the stack.yaml at ./stack.yaml and the lib.cabal at ./lib/lib.cabal, HLS will want the relative path to be "lib/config/value.yaml", where stack itself wants it to be "config/value.yml"

Your environment

$ haskell-language-server-0.5.0-linux-8.8.4 --probe-tools
haskell-language-server version: 0.5.0.0 (GHC: 8.8.4) (PATH: /home/fpaulusma/.config/Code/User/globalStorage/haskell.haskell/haskell-language-server-0.5.0-linux-8.8.4) (GIT hash: 14497f2503a2a0d389fabf3b146d674b9af41a34)
Tool versions found on the $PATH
cabal:    3.0.0.0
stack:    2.3.3
ghc:    8.0.2

I tried this on VS Code with just the Haskell plugin installed

$ cat stack.yaml
resolver: lts-16.17

packages:
- lib

$ cat lib/lib.cabal
cabal-version: 1.12

-- This file has been generated from package.yaml by hpack version 0.33.0.
--
-- see: https://github.com/sol/hpack
--
-- hash: 643499591d6ce6070d19763e325b50c6e2d76cbf091125bdf17e6c9a8d421ad9

name:           lib
version:        0.1.0.0
description:    Please see the README on GitHub at <https://github.com/githubuser/test#readme>
homepage:       https://github.com/githubuser/test#readme
bug-reports:    https://github.com/githubuser/test/issues
author:         Author name here
maintainer:     [email protected]
copyright:      2020 Author name here
license:        BSD3
license-file:   LICENSE
build-type:     Simple
extra-source-files:
    README.md
    ChangeLog.md

source-repository head
  type: git
  location: https://github.com/githubuser/lib

library
  exposed-modules:
      Lib
  other-modules:
      Paths_test
  hs-source-dirs:
      src
  build-depends:
      base >=4.7 && <5
    , yaml
  default-language: Haskell2010

executable test-exe
  main-is: Main.hs
  other-modules:
      Paths_test
  hs-source-dirs:
      app
  ghc-options: -threaded -rtsopts -with-rtsopts=-N
  build-depends:
      base >=4.7 && <5
    , test
  default-language: Haskell2010

test-suite test-test
  type: exitcode-stdio-1.0
  main-is: Spec.hs
  other-modules:
      Paths_test
  hs-source-dirs:
      test
  ghc-options: -threaded -rtsopts -with-rtsopts=-N
  build-depends:
      base >=4.7 && <5
    , test
  default-language: Haskell2010

Steps to reproduce

Use this repo

Expected behaviour

I expect HLS to just compile the package

Actual behaviour

HLS gives an error that the file ({root}/lib/config/value.yaml) can't be found with relative path "config/value.yaml", even though it compiles fine with stack build. It will build with "lib/config/value.yaml", but then stack build fails to compile.

Include debug information

Debug output:

$ /home/myhome/.config/Code/User/globalStorage/haskell.haskell/haskell-language-server-0.5.0-linux-8.8.4 --debug .
haskell-language-server version: 0.5.0.0 (GHC: 8.8.4) (PATH: /home/myhome/.config/Code/User/globalStorage/haskell.haskell/haskell-language-server-0.5.0-linux-8.8.4) (GIT hash: 14497f2503a2a0d389fabf3b146d674b9af41a34)
(haskell-language-server)Ghcide setup tester in /home/myhome/code/test.
Report bugs at https://github.com/haskell/haskell-language-server/issues

Tool versions found on the $PATH
cabal:    3.0.0.0
stack:    2.3.3
ghc:    8.0.2


Step 1/4: Finding files to test in /home/myhome/code/test
Found 4 files

Step 2/4: Looking for hie.yaml files that control setup
Found 1 cradle

Step 3/4: Initializing the IDE

Step 4/4: Type checking the files
[INFO] Consulting the cradle for "lib/Setup.hs"
NotShowMessage (NotificationMessage {_jsonrpc = "2.0", _method = WindowShowMessage, _params = ShowMessageParams {_xtype = MtWarning, _message = "No [cradle](https://github.com/mpickering/hie-bios#hie-bios) found for lib/Setup.hs.\n Proceeding with [implicit cradle](https://hackage.haskell.org/package/implicit-hie)"}})
Output from setting up the cradle Cradle {cradleRootDir = "/home/myhome/code/test", cradleOptsProg = CradleAction: Stack}
File:     /home/myhome/code/test/lib/Setup.hs
Hidden:   no
Range:    1:0-2:0
Source:   cradle
Severity: DsError
Message:
  Multi Cradle: No prefixes matched
  pwd: /home/myhome/code/test
  filepath: /home/myhome/code/test/lib/Setup.hs
  prefixes:
  ("lib/src",Stack {component = Just "lib:lib", stackYaml = Nothing})
  ("lib/app/Main.hs",Stack {component = Just "lib:exe:test-exe", stackYaml = Nothing})
  ("lib/app/Paths_test.hs",Stack {component = Just "lib:exe:test-exe", stackYaml = Nothing})
  ("lib/test",Stack {component = Just "lib:test:test-test", stackYaml = Nothing})
[INFO] Consulting the cradle for "lib/app/Main.hs"
NotShowMessage (NotificationMessage {_jsonrpc = "2.0", _method = WindowShowMessage, _params = ShowMessageParams {_xtype = MtWarning, _message = "No [cradle](https://github.com/mpickering/hie-bios#hie-bios) found for lib/app/Main.hs.\n Proceeding with [implicit cradle](https://hackage.haskell.org/package/implicit-hie)"}})
Output from setting up the cradle Cradle {cradleRootDir = "/home/myhome/code/test", cradleOptsProg = CradleAction: Stack}
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
> Using main module: 1. Package `lib' component lib:exe:test-exe with main-is file: /home/myhome/code/test/lib/app/Main.hs
> The following GHC options are incompatible with GHCi and have not been passed to it: -threaded
> Configuring GHCi with the following packages: lib
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
> /home/myhome/code/test/.stack-work/install/x86_64-linux/7a5912a4290fa383fc50d4c4aaddfc9c4b5e5a20c04b4d180f4b13711ebce221/8.8.4/pkgdb:/home/myhome/.stack/snapshots/x86_64-linux/7a5912a4290fa383fc50d4c4aaddfc9c4b5e5a20c04b4d180f4b13711ebce221/8.8.4/pkgdb:/home/myhome/.stack/programs/x86_64-linux/ghc-8.8.4/lib/ghc-8.8.4/package.conf.d
[INFO] Using interface files cache dir: /home/myhome/.cache/ghcide/main-e56f5f10e255ef74c8262586c777b8720af43c9c
[INFO] Making new HscEnv[main]
[INFO] Consulting the cradle for "lib/test/Spec.hs"
NotShowMessage (NotificationMessage {_jsonrpc = "2.0", _method = WindowShowMessage, _params = ShowMessageParams {_xtype = MtWarning, _message = "No [cradle](https://github.com/mpickering/hie-bios#hie-bios) found for lib/test/Spec.hs.\n Proceeding with [implicit cradle](https://hackage.haskell.org/package/implicit-hie)"}})
Output from setting up the cradle Cradle {cradleRootDir = "/home/myhome/code/test", cradleOptsProg = CradleAction: Stack}
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
> Using main module: 1. Package `lib' component lib:test:test-test with main-is file: /home/myhome/code/test/lib/test/Spec.hs
> lib> configure (lib + test)
> Configuring lib-0.1.0.0...
> lib> initial-build-steps (lib + test)
> lib> Test running disabled by --no-run-tests flag.
> Completed 2 action(s).
> The following GHC options are incompatible with GHCi and have not been passed to it: -threaded
> Configuring GHCi with the following packages: lib
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
> /home/myhome/code/test/.stack-work/install/x86_64-linux/7a5912a4290fa383fc50d4c4aaddfc9c4b5e5a20c04b4d180f4b13711ebce221/8.8.4/pkgdb:/home/myhome/.stack/snapshots/x86_64-linux/7a5912a4290fa383fc50d4c4aaddfc9c4b5e5a20c04b4d180f4b13711ebce221/8.8.4/pkgdb:/home/myhome/.stack/programs/x86_64-linux/ghc-8.8.4/lib/ghc-8.8.4/package.conf.d
[INFO] Using interface files cache dir: /home/myhome/.cache/ghcide/main-571a0d71e220d3f38e32a126cede89eec54d7b5c
[INFO] Using interface files cache dir: /home/myhome/.cache/ghcide/main-571a0d71e220d3f38e32a126cede89eec54d7b5c
[INFO] Making new HscEnv[main,main]
[INFO] Consulting the cradle for "lib/Setup.hs"
NotShowMessage (NotificationMessage {_jsonrpc = "2.0", _method = WindowShowMessage, _params = ShowMessageParams {_xtype = MtWarning, _message = "No [cradle](https://github.com/mpickering/hie-bios#hie-bios) found for lib/Setup.hs.\n Proceeding with [implicit cradle](https://hackage.haskell.org/package/implicit-hie)"}})
Output from setting up the cradle Cradle {cradleRootDir = "/home/myhome/code/test", cradleOptsProg = CradleAction: Stack}
[INFO] Consulting the cradle for "lib/src/Lib.hs"
NotShowMessage (NotificationMessage {_jsonrpc = "2.0", _method = WindowShowMessage, _params = ShowMessageParams {_xtype = MtWarning, _message = "No [cradle](https://github.com/mpickering/hie-bios#hie-bios) found for lib/src/Lib.hs.\n Proceeding with [implicit cradle](https://hackage.haskell.org/package/implicit-hie)"}})
Output from setting up the cradle Cradle {cradleRootDir = "/home/myhome/code/test", cradleOptsProg = CradleAction: Stack}
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
> Configuring GHCi with the following packages: lib
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
> /home/myhome/code/test/.stack-work/install/x86_64-linux/7a5912a4290fa383fc50d4c4aaddfc9c4b5e5a20c04b4d180f4b13711ebce221/8.8.4/pkgdb:/home/myhome/.stack/snapshots/x86_64-linux/7a5912a4290fa383fc50d4c4aaddfc9c4b5e5a20c04b4d180f4b13711ebce221/8.8.4/pkgdb:/home/myhome/.stack/programs/x86_64-linux/ghc-8.8.4/lib/ghc-8.8.4/package.conf.d
[INFO] Using interface files cache dir: /home/myhome/.cache/ghcide/main-6ff17596e79d45dfaf842cbef3cbb8111eee8688
[INFO] Using interface files cache dir: /home/myhome/.cache/ghcide/main-6ff17596e79d45dfaf842cbef3cbb8111eee8688
[INFO] Using interface files cache dir: /home/myhome/.cache/ghcide/main-6ff17596e79d45dfaf842cbef3cbb8111eee8688
[INFO] Making new HscEnv[main,main,main]
[INFO] Consulting the cradle for "lib/Setup.hs"
NotShowMessage (NotificationMessage {_jsonrpc = "2.0", _method = WindowShowMessage, _params = ShowMessageParams {_xtype = MtWarning, _message = "No [cradle](https://github.com/mpickering/hie-bios#hie-bios) found for lib/Setup.hs.\n Proceeding with [implicit cradle](https://hackage.haskell.org/package/implicit-hie)"}})
Output from setting up the cradle Cradle {cradleRootDir = "/home/myhome/code/test", cradleOptsProg = CradleAction: Stack}
File:     /home/myhome/code/test/lib/src/Lib.hs
Hidden:   no
Range:    9:9-9:39
Source:   typecheck
Severity: DsError
Message:
  • Exception when trying to run compile-time code:
  InvalidYaml (Just (YamlException "Yaml file not found: config/value.yaml"))
  Code: decodeFile "config/value.yaml"
  • In the Template Haskell splice $$(decodeFile "config/value.yaml")
  In the expression: $$(decodeFile "config/value.yaml")
  In an equation for ‘val’: val = $$(decodeFile "config/value.yaml")
Files that failed:
[INFO] finish: User TypeCheck (took 0.37s) * /home/myhome/code/test/lib/Setup.hs

 * /home/myhome/code/test/lib/src/Lib.hs

Completed (2 files worked, 2 files failed)
haskell-language-server-0.5.0-linux-8.8.4: allocatestack.c:384: advise_stack_range: Assertion `freesize < size' failed.
Aborted (core dumped)

Paste the logs from the lsp-client, e.g. for VS Code

LSP logs:

$ cat 20201009T003506/exthost1/output_logging_20201009T003509/4-Haskell\ \(test\).log
[client] run command: "/home/myhome/.config/Code/User/globalStorage/haskell.haskell/haskell-language-server-0.5.0-linux-8.8.4 --lsp"
[client] debug command: "/home/myhome/.config/Code/User/globalStorage/haskell.haskell/haskell-language-server-0.5.0-linux-8.8.4 --lsp"
[client] server cwd: undefined
haskell-language-server version: 0.5.0.0 (GHC: 8.8.4) (PATH: /home/myhome/.config/Code/User/globalStorage/haskell.haskell/haskell-language-server-0.5.0-linux-8.8.4) (GIT hash: 14497f2503a2a0d389fabf3b146d674b9af41a34)
Starting (haskell-language-server)LSP server...
  with arguments: LspArguments {argLSP = True, argsCwd = Nothing, argFiles = [], argsShakeProfiling = Nothing, argsTesting = False, argsExamplePlugin = False, argsDebugOn = False, argsLogFile = Nothing, argsThreads = 0, argsProjectGhcVersion = False}
  with plugins: [PluginId "brittany",PluginId "eval",PluginId "floskell",PluginId "fourmolu",PluginId "ghcide",PluginId "importLens",PluginId "ormolu",PluginId "pragmas",PluginId "retrie",PluginId "stylish-haskell",PluginId "tactic"]
  in directory: /home/myhome/code/test
If you are seeing this in a terminal, you probably should have run ghcide WITHOUT the --lsp option!
 Started LSP server in 0.00s
2020-10-09 00:35:09.057015669 [ThreadId 21] - Registering ide configuration: IdeConfiguration {workspaceFolders = fromList [NormalizedUri 6521202141690092544 "file:///home/myhome/code/test"], clientSettings = hashed Nothing}
2020-10-09 00:35:09.059753704 [ThreadId 21] - Configuration changed: Object (fromList [("haskell",Object (fromList [("logFile",String ""),("hlintOn",Bool True),("formatOnImportOn",Bool True),("indentationRules",Object (fromList [("enabled",Bool True)])),("liquidOn",Bool False),("languageServerVariant",String "haskell-language-server"),("serverExecutablePath",String ""),("diagnosticsOnChange",Bool True),("completionSnippetsOn",Bool True),("maxNumberOfProblems",Number 100.0),("formattingProvider",String "ormolu"),("trace",Object (fromList [("server",String "off")]))]))])
2020-10-09 00:35:09.060945361 [ThreadId 21] - Opened text document: file:///home/myhome/code/test/lib/src/Lib.hs
2020-10-09 00:35:09.062115332 [ThreadId 82] - Consulting the cradle for "lib/src/Lib.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/home/myhome/code/test", cradleOptsProg = CradleAction: Stack}
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
2020-10-09 00:35:09.317449518 [ThreadId 97] - Plugin.makeCodeLens (ideLogger)
> Configuring GHCi with the following packages: lib
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
> /home/myhome/code/test/lib/lib.cabal was modified manually. Ignoring /home/myhome/code/test/lib/package.yaml in favor of the cabal file.
> If you want to use the package.yaml file instead of the cabal file,
> then please delete the cabal file.
> /home/myhome/code/test/.stack-work/install/x86_64-linux/7a5912a4290fa383fc50d4c4aaddfc9c4b5e5a20c04b4d180f4b13711ebce221/8.8.4/pkgdb:/home/myhome/.stack/snapshots/x86_64-linux/7a5912a4290fa383fc50d4c4aaddfc9c4b5e5a20c04b4d180f4b13711ebce221/8.8.4/pkgdb:/home/myhome/.stack/programs/x86_64-linux/ghc-8.8.4/lib/ghc-8.8.4/package.conf.d
2020-10-09 00:35:10.549492833 [ThreadId 82] - Using interface files cache dir: /home/myhome/.cache/ghcide/main-6ff17596e79d45dfaf842cbef3cbb8111eee8688
2020-10-09 00:35:10.5496967 [ThreadId 82] - Making new HscEnv[main]
2020-10-09 00:35:10.987847123 [ThreadId 271] - finish: codeLens (took 0.42s)
2020-10-09 00:35:10.988256915 [ThreadId 278] - finish:  (took 0.00s)

[Vlix]

I'd be willing to try and fix this myself if someone could point me to where in the HLS code this might be fixed.

[michaelpj]

I'd almost forgotten about this, but we had this issue too. We solved it in a truly appalling fashion with some symlinks so that we could use the path "from" the project root and have it work in both settings. I would love a fix for this!

[Vlix]

Making symlinks for all our projects in the root would create more headaches than it would solve. (I've tried after reading your / @michaelpj 's comment)
We're actually waiting for this to switch to HLS at our company. It'd be great if anyone could point me in the direction where the problem lies, I'd be willing to put some time into this.

[jneira]

@Vlix thank for you report. Just in case, have you tried to workaround the issue using f.e. cpp conditions? There is a new feature in hie-bios that might help to do it: you can add an alternative stack.yaml only for hls: https://github.com/mpickering/hie-bios#stack

[jneira]

I guess it would be related with hie-bios and the build it triggers to get ghc flags and other build info (but i am not sure, maybe @fendor could help us here)

[Vlix]

@jneira I've tried it out, and I guess it _could_ work, except setting flags via a stack.yaml doesn't seem to work for HLS?
If I use the custom stack.yaml with regular stack build the flags are picked up (and it doesn't build, because the paths are wrong for regular stack build with the flags turned on) but HLS doesn't seem to either set the flags, or doesn't add the cpp-options in the cabal file.

To be precise, I've added the following to a copy of stack.yaml I called stack-ide.yaml:

flags:
  package1:
    useide: true
  package2:
    useide: true
  package3:
    useide: true
  etc...

And then in the cradle:

cradle:
  stack:
    stackYaml: stack-ide.yaml
    components:
      - etc...

(And I've seen HLS use the stack-ide.yaml instead of the regular one in htop, so that worked.)

And in the cabal files that need the CPP I added:

Flag useide
  description: blablabla
  default: False

library
  if flag(useide)
    cpp-options: -DUSEIDE
  etc...

and then in the code

{-# LANGUAGE CPP #-}
...
val :: Value
val = $$(decodeFile
#if USEIDE
  "package1/config/value.yaml"
#else
  "config/value.yaml"
#endif
  )

So stack build --stack-yaml stack-ide.yaml will pick up the flags and then fail to build. (where it does build with the regular stack.yaml) but then HLS still fails because it reports that "config/value.yaml" doesn't exist, so USEIDE somehow isn't set. 😞

[jneira]

Hi, i've tested it and you are right, it does not work cause hls via hie-bios uses stack repl and it seems to not take in account stack.yaml flags.
I tested it putting a compiler error between #if USEIDE and else and observing it was triggered by stack build --stack-yaml stack-ide.yaml but no by stack repl --stack-yaml stack-ide.yaml ModuleWithCPP.hs
Weirdly enoug stack repl --stack-yaml stack-ide.yaml ModuleWithCPP.hs --flag myPackage:useide works.

I would say that it is a stack bug (maybe related with this one?) and my original proposal should work, but it is not the case. 😟

[jneira]

@Vlix hi! i am trying to reproduce the error. With only one package setup i cant reproduce it, but i infer from package1/config/value.yaml that you have a multipackage setup.
could you share the stack.yaml (the package part), project structure and `hie.yaml if you are using it?

Nvm, i've reproduced it simply creating a subpackage
In my case stack build package1 works but stack repl package1 also fails with the same error shown in hls:

PS D:\dev\ws\haskell\stack-test> stack build package1                                                                             
package1-0.1.0.0: unregistering (local file changes: CHANGELOG.md package1.cabal src\MyLib1.hs)
package1> configure (lib)
Configuring package1-0.1.0.0...
package1> build (lib)
Preprocessing library for package1-0.1.0.0..
Building library for package1-0.1.0.0..
[1 of 1] Compiling MyLib1
package1> copy/register
Installing library in D:\dev\ws\haskell\stack-test\.stack-work\install\59c25a7c\lib\x86_64-windows-ghc-8.10.2\package1-0.1.0.0-6hAcdyB2Fb41m0jhWMKK03
Registering library for package1-0.1.0.0..
PS D:\dev\ws\haskell\stack-test> stack repl package1                                                                              Configuring GHCi with the following packages: package1
GHCi, version 8.10.2: https://www.haskell.org/ghc/  :? for help
[1 of 1] Compiling MyLib1           ( D:\\dev\ws\haskell\stack-test\package1\src\MyLib1.hs, interpreted )

D:\\\\dev\ws\haskell\stack-test\package1\src\MyLib1.hs:11:10: error:
    • Exception when trying to run compile-time code:
        InvalidYaml (Just (YamlException "Yaml file not found: config/value.yaml"))
      Code: decodeFile "config/value.yaml"
    • In the Template Haskell splice $$(decodeFile "config/value.yaml")
      In the expression: $$(decodeFile "config/value.yaml")
      In an equation for ‘val’: val = $$(decodeFile "config/value.yaml")
   |
11 | val = $$(decodeFile
   |          ^^^^^^^^^^^...
Failed, no modules loaded.

<no location info>: error:
    Could not load module ‘MyLib1’
    It is a member of the hidden package ‘package1-0.1.0.0’.
    You can run ‘:set -package package1’ to expose it.
    (Note: this unloads all the modules in the current scope.)
Loaded GHCi configuration from C:\\TEMP\\haskell-stack-ghci\\6784647a\\ghci-script
Prelude>

I think this is a stack bug too, cause the behaviour between build and repl should be the same in this case.
So if stack repl package1 fails and while hie-bios uses stack repl i am afraid there is no an easy fix in hls itself.

[jneira]

@vlix well stack is blocking us here, one bug is causing the issue and another one is preventing the initial workaround. :facepalm:

• I will open an issue in stack about the inconsistent cwd at compile time between stack buildand stack repl, but not sure if it will prioritary, so fix it yourself probably would be the way to go.

• hie-bios will stop using stack repl in the long term and then it maybe will work but there is no deadline.

• Other ugly workaround would be move the check inside the th computation and "simply" look for both paths (you might need import template-haskell):

import System.Directory
import Data.Yaml (Value)
import Data.Yaml.TH (decodeFile)
import Language.Haskell.TH
import Language.Haskell.TH.Syntax

val :: Value
val = $$(do
           file <- runIO $ do
                    exist <- doesFileExist "config/value.yaml"
                    return $ if exist then "config/value.yaml" 
                                      else "subpackage/config/value.yaml"
           decodeFile file :: Q (TExp Value)
        )

Ugly but well, decodeFile is already using runIO :stuck_out_tongue_closed_eyes:

• If you are considering move to cabal as build tool, this would be another reason, cause in cabal your original code works (cabal build and cabal repl compile time cwd is the same)

• (For completeness) Another possible workaround could be make a symlink

[jneira]

Opened issue upstream

[Vlix]

Thank you for the effort! That type of work-around should be doable.
Hope it'll get fixed eventually, but this might actually make us able to use HLS, which is the goal 👍

[Vlix]

Ok, I've made a helper function that just tries a couple of given filepaths:

import Language.Haskell.TH
import System.Directory (doesFileExist)
import System.FilePath ((</>))


withFirstFile :: [FilePath] -> (FilePath -> Q a) -> Q a
withFirstFile [] _ = fail "No valid filepaths found"
withFirstFile (fp:fps) q = do
    b <- runIO $ doesFileExist fp
    if b
      then q fp
      else withFirstFile fps q

And this does work around the issue, and HLS can go on and process the rest of the files... but then I ran into a new problem on our production code.
It seems that all our subpackages get built by HLS, and then when it's all done and starts giving haddock errors, and unnecessary imports, etc. it segfaults. (This is visible when doing haskell-language-server --debug ./subpackage) Of the 12+ subpackages we have only 1 doesn't segfault when using haskell-language-server --debug, and it does depend on a different subpackage (which does segfault when targeted with haskell-language-server --debug), so this is really weird.
I've been able to gut one of the subpackages so that it doesn't segfault, but it's hard to pinpoint why. I only know that with half of the modules, it doesn't segfault, but when all modules are present it does segfault.

I'll try to narrow it down more at a later time (it's not because of TH, that's for sure, those were in the half I kept that didn't segfault)

_I also can't share logs etc. because this is production code_ 😞

[jneira]

Wow, sad to read that. Could you open a new issue if it is a different problem not related with th relative filepaths?
Regarding logs, maybe you can post the significative part, removing all sensible info, so it could belong to any code out there.

[AlistairB]

Not sure if useful at this point, but I have a pretty simple repro here https://github.com/AlistairB/morpheus-repro/tree/hls-th-path-repro

[Vlix]

@michaelpj Would the code snippet in my previous comment maybe help get rid of all the symlinks in your project? Or would that just make it more complicated?
It worked wonders for us at least.

[michaelpj]

@Vlix thanks, I'll give that a try!

[jneira]

Hi, it seems the resolution in stack only could be partial, it would fix stack repl subpackage (see https://github.com/commercialhaskell/stack/pull/5448)
However, @phadej gently pointed to another better workaround, use an existing function that do it:
https://hackage.haskell.org/package/file-embed-0.0.13.0/docs/Data-FileEmbed.html#v:makeRelativeToProject

makeRelativeToProject :: FilePath -> Q FilePath

Take a relative file path and attach it to the root of the current project.

The idea here is that, when building with Stack, the build will always be executed with a current working directory of the root of the project (where your .cabal file is located). However, if you load up multiple projects with stack ghci, the working directory may be something else entirely.

This function looks at the source location of the Haskell file calling it, finds the first parent directory with a .cabal file, and uses that as the root directory for fixing the relative path.

$(makeRelativeToProject "data/foo.txt" >>= embedFile)

[jneira]

The new issue in stack to track the complete resolution will be https://github.com/commercialhaskell/stack/issues/3310

[Vlix]

@jneira makeRelativeToProject works like a charm!
(I've read your comment 2 weeks ago, but forgot to reply)

I also recommend @michaelpj to just use this function instead of my code snippet.