Haskell-language-server: Segfault using TemplateHaskell (NixOs, MacOs for now)

[fiadliel]

In a simple project with polysemy included, the addition of the makeSem call into Template Haskell causes HLS to crash reliably (segmentation fault). With the makeSem line commented out, the plugin starts correctly.

e.g.

{-# LANGUAGE TemplateHaskell #-}

module Main where

import Polysemy
import Polysemy.Reader

data MyEffect m a where
  Test :: m a -> MyEffect m a

makeSem ''MyEffect

A repository that should demonstrate the issue is at https://github.com/fiadliel/effects

[bubba]

I can't reproduce this with HLS built from source on mac, going to try with the 0.2.2 binary

[bubba]

I can't reproduce it with the 0.2.2 binary either, with this file:

{-# LANGUAGE DataKinds #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE TemplateHaskell #-}

module Main where

import Polysemy
import Polysemy.Reader

data MyEffect m a where
  Test :: m a -> MyEffect m a

makeSem ''MyEffect

and with polysemy-1.3.0.0 on ghc-8.10.1. What version of GHC are you using? Can you pass on any of the logs too?

[fiadliel]

This is ghc 8.4.4 (from a haskell.nix setup). There was a previous bug about Template Haskell incompatibilities at https://github.com/input-output-hk/haskell.nix/issues/400 but that's been fixed for a while.

If you have nix installed, you should be able to create the environment with nix-shell. You would probably want to use the cachix cache earnestresearch-public.cachix.org (unless you like compiling a lot).

I can also look at getting something from lldb, though it's been a while (and different language) ago 😅

If the answer is that haskell-language-server doesn't support this use case, it's not too bad as long as ghcide continues to work (this is the extant working system that we can fall back to); in this case, ghcide is compiled against the nix environment's ghc.

---

Basic info from lldb (not knowing what I'm doing):

(lldb) cont
Process 15466 resuming
Process 15466 stopped
* thread #27, stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
    frame #0: 0x0000000110da766f
->  0x110da766f: movq   0x5(%rbx), %rbx
    0x110da7673: movq   %rbx, %rcx
    0x110da7676: movq   %rax, %rbx
    0x110da7679: movq   %rcx, 0x8(%rbp)
Target 0: (haskell-language-server-macOS-8.8.4) stopped.
(lldb) bt
* thread #27, stop reason = EXC_BAD_ACCESS (code=EXC_I386_GPFLT)
  * frame #0: 0x0000000110da766f
    frame #1: 0x61000000000012cf

---

Log messages:

❯ ~/Downloads/haskell-language-server-macOS-8.8.4 --debug
haskell-language-server version: 0.2.1.0 (GHC: 8.8.4) (PATH: /Users/gcoady/Downloads/haskell-language-server-macOS-8.8.4) (GIT hash: a6f4aba69e45dfb48bc5721537fef120971d3899)
(haskell-language-server)Ghcide setup tester in /Users/gcoady/src/github.com/fiadliel/effects.
Report bugs at https://github.com/haskell/haskell-language-server/issues

Step 1/4: Finding files to test in /Users/gcoady/src/github.com/fiadliel/effects
Found 2 files

Step 2/4: Looking for hie.yaml files that control setup
Found 1 cradle

Step 3/4: Initializing the IDE

Step 4/4: Type checking the files
[INFO] Consulting the cradle for "/Users/gcoady/src/github.com/fiadliel/effects/Setup.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/Users/gcoady/src/github.com/fiadliel/effects", cradleOptsProg = CradleAction: Cabal}
> cabal: Unknown target
> '/Users/gcoady/src/github.com/fiadliel/effects/Setup.hs'.
> The package servant-with-effects has no file target 'Setup.hs'.
>
>
File:     /Users/gcoady/src/github.com/fiadliel/effects/Setup.hs
Hidden:   no
Range:    1:0-2:0
Source:   cradle
Severity: DsError
Message:
  Failed to parse result of calling cabal

  cabal: Unknown target
  '/Users/gcoady/src/github.com/fiadliel/effects/Setup.hs'.
  The package servant-with-effects has no file target 'Setup.hs'.




[INFO] Consulting the cradle for "/Users/gcoady/src/github.com/fiadliel/effects/src/Main.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/Users/gcoady/src/github.com/fiadliel/effects", cradleOptsProg = CradleAction: Cabal}
> Warning: No remote package servers have been specified. Usually you would have
> one specified in the config file.
> Resolving dependencies...
> Build profile: -w ghc-8.8.4 -O1
> In order, the following will be built (use -v for more details):
>  - servant-with-effects-0.1.0.0 (exe:servant-with-effects) (configuration changed)
> Configuring executable 'servant-with-effects' for servant-with-effects-0.1.0.0..
> Warning: The 'license-file' field refers to the file 'LICENSE' which does not
> exist.
> Preprocessing executable 'servant-with-effects' for servant-with-effects-0.1.0.0..
[INFO] Using interface files cache dir: /Users/gcoady/.cache/ghcide/main-1c5eedb9d24916147b5cef09d100a08962c59f40
[INFO] Making new HscEnv[main]
[INFO] Consulting the cradle for "/Users/gcoady/src/github.com/fiadliel/effects/Setup.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/Users/gcoady/src/github.com/fiadliel/effects", cradleOptsProg = CradleAction: Cabal}
> cabal: Unknown target
> '/Users/gcoady/src/github.com/fiadliel/effects/Setup.hs'.
> The package servant-with-effects has no file target 'Setup.hs'.
>
>
zsh: segmentation fault  ~/Downloads/haskell-language-server-macOS-8.8.4 --debug

[fiadliel]

And tried with 0.2.2, same error; version is:

haskell-language-server version: 0.2.2.0 (GHC: 8.8.4) (PATH: /Users/gcoady/Downloads/haskell-language-server-macOS-8.8.4-1) (GIT hash: e44f618c11b7978264a94beeee00d1f014867f6d)

[bubba]

Hm those logs don't look good, not sure why its setting up a cradle multiple times for the same file. I've never used nix before but I think its about time I install it to see what's going on...

[bubba]

I just noticed your nix expression ends up downloading ghc 8.4.4 but your cabal constraints suggest ghc 8.8.4 This was to bootstrap 8.8.4 =

[bubba]

Took a while to build without the caches, but I can recreate it now. Time to figure out what's going on

[galagora]

I'm also facing the same problem on NixOS, using HLS 0.3.0.0 with GHC 8.8.3 (and 8.8.4). The source file is the same (hello world with makeSem).

logs

$ haskell-language-server 
haskell-language-server version: 0.3.0.0 (GHC: 8.8.3) (PATH: /nix/store/zly7m0p5sd0iv8qn1n97gxkx4lkbh00z-haskell-language-server-0.3.0.0/bin/haskell-language-server)
(haskell-language-server)Ghcide setup tester in /home/ao/code/haskell-starter.
Report bugs at https://github.com/haskell/haskell-language-server/issues

Tool versions found on the $PATH
cabal:      3.2.0.0
stack:      Not found
ghc:        8.8.3


Step 1/4: Finding files to test in /home/ao/code/haskell-starter
Found 1 files

Step 2/4: Looking for hie.yaml files that control setup
Found 1 cradle

Step 3/4: Initializing the IDE

Step 4/4: Type checking the files
[INFO] Consulting the cradle for "/home/ao/code/haskell-starter/src/Main.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/home/ao/code/haskell-starter", cradleOptsProg = CradleAction: Cabal}
> Warning: No remote package servers have been specified. Usually you would have
> one specified in the config file.
> Resolving dependencies...
> Build profile: -w ghc-8.8.3 -O1
> In order, the following will be built (use -v for more details):
>  - starter-0.1.0.0 (exe:starter) (configuration changed)
> Configuring executable 'starter' for starter-0.1.0.0..
> Preprocessing executable 'starter' for starter-0.1.0.0..
[INFO] Using interface files cache dir: /home/ao/.cache/ghcide/main-97733fd591cce96508bbfeab608cd02ca28697c1
[INFO] Making new HscEnv[main]
[1]    11008 segmentation fault (core dumped)  haskell-language-server

[galagora]

Same issue when using freer-simple's makeEffect (equivalent to makeSem) and motor's reflectEvents, both Template Haskell functions. Does HLS not support TH or is this merely a bug in the implementation?

[pepeiborra]

What type of cradle (hie.yaml) are you using?

There is a possibly related TH idempotency bug in ghcide that @maralorn is investigating.

[galagora]

Just a single cradle with a single component (executable). The correct settings are auto-detected, but here's the hie.yaml:

cradle:
  cabal:
    - path: "./src/"
      component: "habit:exe:hab"

[pepeiborra]

Does your Cabal descriptor consist of only one executable stanza, or also a library stanza?

[fiadliel]

I've tried with the following setups:

Nix setups:

• ghc 8.8.4, single executable stanza, use ghc versions of ghc-included libs: crash
• ghc 8.8.4, two cabal stanzas (lib+executable), use ghc versions of ghc-included libs: crash
• ghc 8.8.4, single executable stanza, use cabal versions of ghc-included libs: crash
• ghc 8.8.4, two cabal stanzas (lib+executable), use ghc versions of ghc-included libs: crash
• ghc 8.10.1, all the above combinations: works fine

ghcup:

• ghc 8.8.4: works fine
• ghc 8.10.1: works fine

[pepeiborra]

Thanks for testing! We have previously found bugs with ghc 8.8.4 that didn't arise with 8.10. Hereby closing as won't fix

[galagora]

I still get the segfault using GHC 8.10.2, with a single executable stanza, and any TH function invocation. cabal-install 3.2.0.0, if that's relevant.

$ haskell-language-server --version
haskell-language-server version: 0.4.0.0 (GHC: 8.10.2) (PATH: /nix/store/sy4l8aqvvgzcc0av7zvx3dj2k8pg979r-haskell-language-server-0.4.0.0/bin/haskell-language-server)

habit.cabal

cabal-version:       2.4
name:                habit
version:             0.1.0.0

executable hab
  main-is:             Main.hs
  build-depends:       base ^>= 4.14.1.0
                     , text
                     , haskell-say
                     , freer-simple
  hs-source-dirs:      src
  default-language:    Haskell2010

src/Main.hs

{-# LANGUAGE DataKinds #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE TypeOperators #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE TemplateHaskell #-}

module Main where

import HaskellSay (haskellSay)
import Data.Text (Text)

import Control.Monad.Freer (Eff, Member, send)
import Control.Monad.Freer.TH

data Console r where
  PutTextLn :: Text -> Console ()

makeEffect ''Console
-- putTextLn :: Member Console effs => Text -> Eff effs ()
-- putTextLn = send . PutTextLn

main :: IO ()
main = haskellSay "Hello, Haskell! You're using a function from another module!"

Please reopen the issue.

[jneira]

I will reopen, i think we have to tackle the problems with template haskell as a whole, there is several issues about

[jneira]

@galagora for completeness: the error you got with that minimal case is using NixOs and the same output as https://github.com/haskell/haskell-language-server/issues/277#issuecomment-671039968?

[jneira]

ghcide issues involving th and segfaults:

• https://github.com/haskell/ghcide/issues/444
• https://github.com/haskell/ghcide/issues/422
• https://github.com/haskell/ghcide/issues/775

[galagora]

Yes, except the last line is:

[1]    5849 segmentation fault (core dumped)  haskell-language-server

instead of

zsh: segmentation fault  ~/Downloads/haskell-language-server-macOS-8.8.4 --debug

Which is probably just a platform difference.

[galagora]

OK, so, for me at least, the bug seems to have nothing to do with ghcide; the executable for that package seems to run fine, while HLS segfaults. It also doesn't seem to have anything to do with the length function in Data.Text.Short, as suggested here. Rather, it occurs when using _any_ Template Haskell at all. See this ultraminimal repro case:

$ haskell-language-server
haskell-language-server version: 0.4.0.0 (GHC: 8.10.2) (PATH: /nix/store/7j0ivnv2nmri3idp7nm60kailxp5a0xq-haskell-language-server-0.4.0.0/bin/haskell-language-server)
(haskell-language-server)Ghcide setup tester in /home/ao/code/hello.
Report bugs at https://github.com/haskell/haskell-language-server/issues

Tool versions found on the $PATH
cabal:      3.2.0.0
stack:      2.3.3
ghc:        8.10.2


Step 1/4: Finding files to test in /home/ao/code/hello
Found 2 files

Step 2/4: Looking for hie.yaml files that control setup
Found 1 cradle

Step 3/4: Initializing the IDE

Step 4/4: Type checking the files
[INFO] Consulting the cradle for "/home/ao/code/hello/src/TH.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/home/ao/code/hello", cradleOptsProg = CradleAction: Cabal}
> Resolving dependencies...
> Build profile: -w ghc-8.10.2 -O0
> In order, the following will be built (use -v for more details):
>  - hello-0.1.1 (exe:hello) (configuration changed)
> Configuring executable 'hello' for hello-0.1.1..
> Preprocessing executable 'hello' for hello-0.1.1..
[INFO] Using interface files cache dir: /home/ao/.cache/ghcide/main-3563f69436b8a2c444891c780d1baa056f3e3edc
[INFO] Making new HscEnv[main]
[1]    14843 segmentation fault (core dumped)  haskell-language-server

ghcide logs

$ ghcide
ghcide version: 0.3.0 (GHC: 8.10) (PATH: /nix/store/qmwdfrx9bkdr4knyxwi04d1xlik50ap5-ghcide-exe-ghcide-0.3.0/bin/ghcide)
Ghcide setup tester in /home/ao/code/hello.
Report bugs at https://github.com/digital-asset/ghcide/issues

Step 1/4: Finding files to test in /home/ao/code/hello
Found 2 files

Step 2/4: Looking for hie.yaml files that control setup
Found 1 cradle
  (/home/ao/code/hello/hie.yaml)

Step 3/4: Initializing the IDE

Step 4/4: Type checking the files
[INFO] Consulting the cradle for "/home/ao/code/hello/src/TH.hs"
Output from setting up the cradle Cradle {cradleRootDir = "/home/ao/code/hello", cradleOptsProg = CradleAction: Cabal}
> Resolving dependencies...
> Build profile: -w ghc-8.10.2 -O0
> In order, the following will be built (use -v for more details):
>  - hello-0.1.1 (exe:hello) (configuration changed)
> Configuring executable 'hello' for hello-0.1.1..
> Preprocessing executable 'hello' for hello-0.1.1..
[INFO] Using interface files cache dir: /home/ao/.cache/ghcide/main-3563f69436b8a2c444891c780d1baa056f3e3edc
[INFO] Making new HscEnv[main]

Completed (2 files worked, 0 files failed)

hello.cabal

cabal-version: 2.4

name:           hello
version:        0.1.1

executable hello
  main-is: Main.hs
  other-modules: TH
  hs-source-dirs:
      src
  ghc-options: -Wall -Wwarn -fwarn-tabs
  build-depends:
      base
    , template-haskell
  default-language: Haskell2010

src/TH.hs

{-#LANGUAGE TemplateHaskell#-}
module TH
  (
    thLit
  -- , thFunc
  -- , thFTyped
  )
where
import Language.Haskell.TH

thLit :: Q Exp
thLit = [| 15 |]

-- thFunc :: Q Exp
-- thFunc = [| \x -> x + 1|]

-- thFTyped :: Q (TExp (Integer -> Integer))
-- thFTyped = [|| \y -> y + 1 ||]

src/Main.hs

{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TemplateHaskell #-}

module Main where
import Debug.Trace

import TH

main :: IO ()
main = do
  print ($(thLit) :: Integer)

hie.yaml

cradle:
  cabal:
    - path: "./src/Main.hs"
      component: "hello:exe:hello"

    - path: "./src/TH.hs"
      component: "hello:exe:hello"

That's pretty much the smallest piece of Template Haskell you can write.

[galagora]

@pepeiborra Why is a ghcide submodule used? Which changes are made to it? What would it take to build with the master branch?

[pepeiborra]

https://github.com/haskell/ghcide/issues/663

[jneira]

@galagora wow, many thanks for minimize the repro case and report all the logs

[galagora]

@jneira Yeah, no problem. Two things though:

1. Would it be possible to get this issue pinned, and perhaps prioritized? I feel not segfaulting in the face of TH is a big issue (you can't use the TH features of any extensible effects library, lens, aeson, etc., which can make some things really difficult (or impossible)). You can probably unpin (and close) the Nix issue, which has been solved (latest version is in Nixpkgs).

2. How would you begin to solve this problem? Given that haskell-language-server uses ghcide at the core, should I look for the problem there? In that case, if I can't reproduce the issue with ghcide 3.0.0, should merging the upstream (at least theoretically) solve the issue? Otherwise, which modules should I look for?

Currently, I'm trying to build and test the version of ghcide in the HLS submodule. I'll also try gdb, but that seems too low-level.

[jneira]

Yeah, the issue probably belongs to ghcide. The branch of ghcide used in hls is supposed to be merged onto ghcide master and it has not much diverged commits so i would test ghcide master first, it would isolate the cause in those commits.

Agree in pinning the issue so the use of TH is persasive, not sure it it does not work for all cases.

sure @wz1000 and @pepeiborra could help you to find the root cause better than me though :wink:

[galagora]

The issue is actually fixed as of the latest ghcide submodule revision bump; the only thing left is place the new version in Nixpkgs, which requires resolving #404. Feel free to close this issue.

[jneira]

@fiadliel i am gonna close this one, feel free to reopen if you continue experiencing the issue