Haskell-ide-engine: Build from source fails on Windows 10 with stack-2.1.3 due to network < 3.1.0.1

[yutasth]

I try to install hie with the command stack ./install.hs hie-8.6.5 with stack-2.1.3, but the installation always fails (freezes) during compiling network package. There is no error shown.

I guess that the cause of this issue is dependency to network-2.8.0.1. When I try to stack install network-2.8.0.1, it shows the final message of the error "terminateProcess: permission denied (Permission denied)".

[4 of 9] Compiling Network.Socket

Network\Socket.hsc:256:1: warning: [-Wunused-imports]
    The import of ‘Foreign.C.Error’ is redundant
      except perhaps to import instances from ‘Foreign.C.Error’
    To import instances alone, use: import Foreign.C.Error()
    |
256 | import Foreign.C.Error
    | ^^^^^^^^^^^^^^^^^^^^^^

Network\Socket.hsc:261:1: warning: [-Wunused-imports]
    The import of ‘Foreign.Marshal.Array’ is redundant
      except perhaps to import instances from ‘Foreign.Marshal.Array’
    To import instances alone, use: import Foreign.Marshal.Array()
    |
261 | import Foreign.Marshal.Array ( peekArray )
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Network\Socket.hsc:328:1: warning: [-Wmissing-signatures]
    Top-level binding with no type signature: socket2FD :: Socket -> FD
    |
328 | socket2FD  (MkSocket fd _ _ _ _) =
    | ^^^^^^^^^

Network\Socket.hsc:583:9: warning: [-Wunused-local-binds]
    Defined but not used: ‘connectBlocked’
    |
583 |         connectBlocked = do
    |         ^^^^^^^^^^^^^^

Network\Socket.hsc:633:8: warning: [-Wunused-matches]
    Defined but not used: ‘sock’
    |
633 | accept sock@(MkSocket s family stype protocol status) = do
    |        ^^^^

Network\Socket.hsc:805:24: warning: [-Wunused-matches]
    Defined but not used: ‘s’
    |
805 | sendBuf sock@(MkSocket s _family _stype _protocol _status) str len = do
    |                        ^

Network\Socket.hsc:863:24: warning: [-Wunused-matches]
    Defined but not used: ‘s’
    |
863 | recvBuf sock@(MkSocket s _family _stype _protocol _status) ptr nbytes
    |                        ^

Network\Socket.hsc:1844:1: warning: [-Wunsupported-calling-conventions]
    terminateProcess: permission denied (Permission denied)

I found network-2.8.0.1 installation fail issue on Windows 10 which may be related to this issue.
https://github.com/commercialhaskell/stack/issues/5014

Is there a way for us to avoid this problem?

[jneira]

Mmm, i've installed succesfully hie-8.6.5 in windows 10 with stack-2.1.3so it should be another factor around. :thinking:

A quick and drastic fix could be nuke your STACK_ROOT directory to clean all snapshots.

If that doesn't work you could try to modify stack-8.6.5.yaml to include a working network version, but it seems simply adding network-3.1.0.1 doesnt work cause other packages (socks f.e.) are incompatible with it.

[yutasth]

OK, so it's just a problem of upstream library. Can I continue discussing this issue here?
More information:

• The version of Windows is Windows 10 Pro 1803 17134.1006.
• stack clean --full does not work.
• stack --cabal-verbose ./install.hs hie-8.6.5 does not change the error message.
• emptying .stack-work does not work.
• emptying STACK_ROOT (C:\sr) and C:\Users\[user]\AppData\Local\Temp does not work.
• reboot does not work.

[sonowz]

I tried removing STACK_ROOT directory, but I had same error message with @yutasth .

[jneira]

The ci build for windows is working too (it uses last stack version and stack-8.6.5.yaml so it had to build network-2.8.0.1): https://dev.azure.com/jneira/haskell-ide-engine/_build/results?buildId=194
The dependencies are cached in that build, but i can try to clear the cache to check again a full build.
EDIT: I have triggered a full build in azure to ensure it still works: https://dev.azure.com/jneira/haskell-ide-engine/_build/results?buildId=195

Another workaround could be using cabal, but you have to use cabal-3.0.0.0 (the lastest version) and run cabal v2-run install.hs --project-file install/shake.project hie-8.6.5 in a msys2 console.

Have you administration rights in your machine?

[sonowz]

Another workaround could be using cabal, but you have to use cabal-3.0.0.0 (the lastest version) and run cabal v2-run install.hs --project-file install/shake.project hie-8.6.5 in a msys2 console.

I tried it, but it fails to build..
It seems like Data.Monoid is missing, but I think that's from some other bug..

$ cabal v2-run install.hs --project-file install/shake.project hie-8.6.5
Resolving dependencies...
Build profile: -w ghc-8.2.2 -O1
In order, the following will be built (use -v for more details):
 - hie-install-0.8.0.0 (lib) (file src\Env.hs changed)
 - fake-package-0 (exe:script) (configuration changed)
Preprocessing library for hie-install-0.8.0.0..
Building library for hie-install-0.8.0.0..
[3 of 8] Compiling Print            ( src\Print.hs, C:\hie\dist-newstyle\build\x86_64-windows\ghc-8.2.2\hie-install-0.8.0.0\build\Print.o )

src\Print.hs:21:24: error:
    ? Variable not in scope: (<>) :: [Char] -> [Char] -> t0
    ? Perhaps you meant one of these:
        ‘<*>’ (imported from Prelude), ‘*>’ (imported from Prelude),
        ‘*>’ (imported from Development.Shake)
   |
21 |   let starsLine = "\n" <> replicate 30 '*' <> "\n"
   |                        ^^

src\Print.hs:21:44: error:
    ? Variable not in scope: (<>) :: t0 -> [Char] -> t
    ? Perhaps you meant one of these:
        ‘<*>’ (imported from Prelude), ‘*>’ (imported from Prelude),
        ‘*>’ (imported from Development.Shake)
   |
21 |   let starsLine = "\n" <> replicate 30 '*' <> "\n"
   |                                            ^^

src\Print.hs:22:17: error:
    ? Variable not in scope: (<>) :: t2 -> String -> t3
    ? Perhaps you meant one of these:
        ‘<*>’ (imported from Prelude), ‘*>’ (imported from Prelude),
        ‘*>’ (imported from Development.Shake)
   |
22 |   in  starsLine <> str <> starsLine
   |                 ^^

src\Print.hs:22:24: error:
    ? Variable not in scope: (<>) :: t3 -> t1 -> String
    ? Perhaps you meant one of these:
        ‘<*>’ (imported from Prelude), ‘*>’ (imported from Prelude),
        ‘*>’ (imported from Development.Shake)
   |
22 |   in  starsLine <> str <> starsLine
   |                        ^^
cabal.exe: Failed to build hie-install-0.8.0.0 (which is required by
exe:script from fake-package-0).

[yutasth]

@dnsdhrj The version of GHC may be too old, I suppose. Semigroup with <> was transferred to base package in GHC 8.4.

[sonowz]

Oh it wasn't sandboxed environment.. I changed GHC to 8.6.5 and it worked! 👍
cabal v2-run install.hs --project-file install/shake.project hie-8.6.5 builds network-3.0.1.1, so there was no error.

[jneira]

mmm, oh, ideally hie-install should work with the same ghc's than the main haskell-ide-engine itself.
But i wonder if having an older ghc in the global path could have something to do with the stack problem (although it shouldn't!)

[sonowz]

By the way, I reinstalled happy.exe during cabal run, since cabal failed with old happy version.

[jneira]

@dnsdhrj that is great news, did you try the failed stack build with ghc-8.2.2 as global default ghc (in %PATH%)?
It fails for me compiling the install script itself ... :thinking:

[jneira]

Btw i've tried:

• cabal build --constraint network==2.8.0.1 -w ghc-8.6.5 inside haskell-ide-engine having ghc-8.2.2 as default ghc and after deleting the cabal v2 store and it worked, building network-2.8.0.1
• stack unpack network-2.8.0.1 && cd network-2.8.0.1 && stack init --resolver=lts-13.30 && stack build also works, after deleting the %STACK_ROOT% folder and having ghc-8.2.2 as default ghc
• stack install network-2.8.0.1 inside haskell-ide-engine also works
• all in windows 7 but i am confident that in my win 10 will work too (i'll test it just in case)
• The azure full build for Windows_Stack stack-8.6.5 using network-2.8.0.1 also worked (with windows 10)

So it seems having ghc-8.2.2 as default ghc doent help to reproduce the error... :worried:

[sonowz]

did you try the failed stack build with ghc-8.2.2 as global default ghc (in %PATH%)?

Yes, my windows' default ghc was ghc-8.2.2 before I upgraded.

cabal v2-run install.hs --constraint network=2.8.0.1 --project-file install/shake.project hie-8.6.5 fails, no matter what ghc version I used.

It seems network-2.8.0.1 will never be compiled in my computer..
ADDED: I think the cause is network-2.8.0.1 package itself, and haskell-ide-engine should have a way to avoid using this version.

[jneira]

I am afraid that it is difficult to avoid it using stack cause all lts for ghc-8.6.5 has network-2.8.0.1 fixed and other hie dependencies (also in lts) fails if you add network-3.* to extra-deps in the stack.yaml. Althoug it is probable that it could work adding more exceptions in the extra-deps field of stack.yaml. Fortunately cabal is able to automatically pick a newer version.

Otoh it works in other windows (including windows 10) envs an it works with stack-1.9.3 so it would be great to get to reproduce the error and fix it removing the specific factor that makes it fail, commenting it in the stack/network issue to help other users hitting the error.

Maybe i am asking too much but, could you link the log of cabal get network-2.8.0.1 && cd network-2.8.0.1 && cabal v2-build -v3 if it fails?

(Maybe it is a typo but the correct command to build hie with cabal and network-2.8.0.1 would be cabal install --constraint network==2.8.0.1 inside the main haskell-ide-engine folder, note the double == and that we are building directly the project and no using the install.hs script)

NOTE: i've tried it with non ascii chars in path and it fails with some other error before building effectively the package

[jneira]

@dnsdhrj Change stack.yaml to upgrade network was easier than i expected so in the meanwhile the upstream issue is resolved we could avoid network-2.8.0.1 as you suggested. :+1:

[yutasth]

@jneira I can't install network-3.0.1.1 either because of the same error, so your pull request doesn't work for me.
On the other hand, I found that I can install network-3.1.0.1. The difference between the versions which breaks backward compatibility is depreciating fdSocket only, according to the changelog. The only package which do not allow network-3.1.* is hslogger-1.3.0.0 among the packages you added in extra-deps, and I found hslogger-1.3.0.0 does not use fdSocket. Therefore, I tried to stack ./install.hs hie-8.6.5 with changing network-3.0.1.1 to network-3.1.0.1 in your proposing stack.yaml and add allow-newer: true to C:\sr\config.yaml. Installing network and hslogger works well, but then... I meet a same error during installing x509-validation-1.6.11.

x509-validation> configure
x509-validation> Configuring x509-validation-1.6.11...
x509-validation> build
x509-validation> Preprocessing library for x509-validation-1.6.11..
x509-validation> Building library for x509-validation-1.6.11..
x509-validation> [1 of 5] Compiling Data.X509.Validation.Fingerprint
x509-validation>
x509-validation> Data\X509\Validation\Fingerprint.hs:24:10: warning: [-Wmissing-methods]
x509-validation>     terminateProcess: permission denied (Permission denied)

I avoid this error by using x509-validation-1.6.9.
Then hscolour-1.24.4 shows a same error and I can't avoid it. all the versoins later than 1.20 show the same error.

hscolour> [14 of 16] Compiling Language.Haskell.HsColour.ANSI
hscolour>
hscolour> Language\Haskell\HsColour\ANSI.hs:62:10: warning: [-Wmissing-methods]
hscolour>     terminateProcess: permission denied (Permission denied)

Maybe I need stack with process-1.6.5.0...

[yutasth]

Anyway, I could install hie with MSYS2 by following @jneira 's advice:

1. run git clone https://github.com/haskell/haskell-ide-engine --recurse-submodules
2. launch MSYS2 bundled in stack
3. add the paths of git, cabal, ghc to PATH in MSYS2
4. add the path of stack local bin directory to PATH in MSYS2 according to README
5. move to the haskell-ide-engine directory in MSYS2
6. run cabal v2-run install.hs --project-file install/shake.project hie-8.6.5 in MSYS2
7. It works.

Now I can use hie in Windows 10. Thank you very much, @jneira !

[jneira]

Thank you for try haskell-ide-egine despite the incoveniences and for summarize how to install it using cabal (the unique workaround for now).

I erroneously assumed that jump two major versions (from 2.8 to 3.0) would be enough to get a working build. It is hard to debug and fix an error without being able to reproduce it.

Bump up to network-3.1.* is harder (as you have checked) cause there are packages than rejects explicitly that version and neither cabal is able to find a package set without update those packages (including the code that does not compile with it).

I think that we can keep open this until the upstream issue is solved.

[jneira]

Maybe I need stack with process-1.6.5.0...

@yutasth Well, it turns out that compile stack from source using process-1.6.5.0 was easy. l've cut an executable just in case you want to test if it can compile network < 3.1

[yutasth]

@jineira Thank you. I tried compiling network-2.8.0.1 with @jneira 's pre-release stack but it doesn't work. The error message is:

network> Network\Socket.hsc:1844:1: warning: [-Wunsupported-calling-conventions]
network>     <stderr>: commitAndReleaseBuffer: invalid argument (invalid character)

... Now I supposed that the common cause of the errors I was suffered from is that the charset of my cmd.exe is not UTF-8 (actually it was Shift_JIS). Therefore with stable stack-2.1.3 (not @jneira 's pre-release), I tried:

1. change the charset to UTF-8 by running chcp 65001.
2. run stack install network-2.8.0.1
3. It works!

Then I tried:

1. change the charset to UTF-8 by running chcp 65001.
2. run git clone https://github.com/haskell/haskell-ide-engine --recurse-submodules
3. run cd haskell-ide-engine
4. run stack ./install.hs hie-8.6.5
5. It works!

The bug of process-1.6.3.0 makes the problem complicated, but the cause is very commonplace.

I propose adding checking your charset of cmd.exe in pre-requirement section of README.

[jneira]

@yutasth nice catch! I forgot i always open my consoles with utf8 😅

[yutasth]

@jneira FYI: I applied installed hie to VSCode with Haskell Language Server and was suffered from the problems below:

1. Haskell Language Server did not work without any error message. The cause is that hie requires stty (running hie raised the error stty is not recognized as an internal or external command.) but Windows 10 lacks it. I resolved it by copying stty.exe from msys2 to stack bin directory. Actually it's just a workaround but it works in this case. See also: https://github.com/commercialhaskell/stack/issues/4901
2. Haskell Language Server raised an error Couldn't start client Haskell IDE. The cause is Haskell Language Server can not find hie.exe. I resolved it by setting its path (with hie.exe in the last of the path) into Hie Executable Path in the settings of Haskell Language Server.
3. Haskell Language Server required cabal-install. I resolved it by downloading cabal.exe and moving it to stack bin directory.

[Jason5Lee]

@yutasth solution works for me. How can I do this in MSYS shell?