Stack: Stack prints raw ANSI color codes sometimes

[megafinz]

General summary/comments

I'm having issues with colored output of the stack commands. Sometimes colored output works correctly, and sometimes raw ANSI escape codes are printed. I can copy such raw output and then paste it in the same terminal and it will pick up colors just fine.

I have two Windows 10 machines with a couple of different shells installed: cmd, PowerShell 5.1, PowerShell 6.0 and Git Bash 4.4.12.

| | cmd | PS 5.1 | PS Core 6.0 | Git Bash 4.4.12 |
|-----------|----------|--------------------|-------------|-----------------|
| Machine 1 | :x: raw ANSI | :x: raw ANSI | :x: raw ANSI | :grey_question: no color |
| Machine 2 | :x: raw ANSI | :heavy_check_mark: OK | :x: raw ANSI | :grey_question: no color |

I don't see any significant differences in the environment.

I also installed ConEmu, and I get correct color output from both cmd and PowerShell when they are used within ConEmu. Both Git Bash and MSYS2 Bash produce regular non-colored output without raw ANSI codes, so I guess it's supposed to be that way.

Steps to reproduce

Not sure how to reproduce that.

Stack version

$ stack --version Version 1.6.3, Git revision b27e629b8c4ce369e3b8273f04db193b060000db (5454 commits) x86_64 hpack-0.20.0

Method of installation

• Official binary, downloaded from stackage.org or fpcomplete's package repository

[mgsloan]

Could somehow be related to https://github.com/commercialhaskell/stack/issues/3785

One workaround is to use --color never, hopefully this will be supported in config.yaml in the future: https://github.com/commercialhaskell/stack/issues/3778

[kadoban]

Do we have any sense of how common this is on Windows? Is it bad enough that the default should just be no color there? If we add that option to config.yaml, they could explicitly turn it on then.

Though I'm still extremely confused why this is happening ever, I suppose a concerted effort to figure that out would be the first step if possible.

[mgsloan]

I have a theory based on pure hypothesis (I don't have a windows machine to test on): Maybe these windows terminals stop interpreting color codes after encountering an invalid one? Perhaps racing terminal writes are occasionally garbling color codes.

It does seem like there's a recent uptick in reports of windows related ansi code issues. Would be good to know if it's due to changes in stack vs changes in the terminals. Based on the PS 6.0 vs PS 5.0 behaving differently in this report, I'm guessing it's due to changes in that.

[megafinz]

Just discovered another weird thing: if I try to redirect console output into file, stack still writes output directly to console, but this time colors are picked up correctly. Output file itself is empty though, although I'd expect rather console output to be empty instead.

http://prntscr.com/i3dpl3

EDIT: turns out I'm not very skillful with the console output, in that stack uninstall case I needed to redirect STDERR output to file instead of STDOUT. The output is written correctly, no escape codes are present in the file and nothing is written on the screen.

[megafinz]

@mgsloan

I have a theory based on pure hypothesis (I don't have a windows machine to test on): Maybe these windows terminals stop interpreting color codes after encountering an invalid one? Perhaps racing terminal writes are occasionally garbling color codes.

It seems that all the necessary symbols are dumped correctly. I am able to copy the raw output, paste it back and everything looks fine: https://i.imgur.com/XK5kQZA.gifv

[megafinz]

I've come up with a workaround that makes colors work, it turns to be stack <command> 2>&1, which redirects STDERR into STDOUT, and everything is printed correctly. I've tested it only on stack test which is my primary use case here though.

[mgsloan]

Curious, thanks for investigating! I wonder what would cause ANSI codes from stderr to behave differently.

[megafinz]

This is valid not just STDERR. I can redirect STDERR into file, and STDOUT will dump correct output into console. So basically if there's any kind of output redirection present, anything that goes on screen will have correct colors of what's the source of the output and what kind of redirection is active.

[kadoban]

Could it be something about the interleaving of output between stdout and stderr? Maybe they end up tied together differently depending on if there is redirection going on or not. So my theory would be something like, stderr starts a terminal escape code, but before it can finish it, stdout then outputs something, breaking all the rest.

Not really sure that makes sense though. If it is what's happening, that also sounds like it's going to be hard to fix from our end.

[megafinz]

I've discovered what was the major environment difference between my two machines: Machine 2 (which had PS 5.1 interpreting color codes correctly right away) wasn't having Windows 10 Fall Creators Update. Now I installed it, and now two machines behave the same: colors don't work.

Also it turns out that 2>&1 workaround works for some commands but not for the others. This is how PowerShell Core 6.0.0 behaves:

| Command | Result | Video |
|-------------|--------|--------|
| stack uninstall | raw ANSI | https://i.imgur.com/OIHlecp.gifv |
| stack uninstall 2>&1 | no color | https://i.imgur.com/ZP541Vf.gifv |
| stack uninstall >$null | OK | https://i.imgur.com/vMRdkrt.gifv |
| stack test | raw ANSI | https://i.imgur.com/ya7oLn1.gifv |
| stack test 2>&1 | OK | https://i.imgur.com/CAExWTL.gifv |
| stack test >$null | OK | https://i.imgur.com/ZQXSkY7.gifv |

I searched github for similar problems and this is what I've found:
https://github.com/composer/composer/issues/5323#issuecomment-273562520
https://github.com/Microsoft/WSL/issues/1173#issuecomment-254250445

As far as I understand, there needs to be some manual work done on the stack side to enable ANSI color code interpretation. That doesn't explain why output redirection sometimes enables it automatically though.

[mgsloan]

Cool, thanks for digging into this @megafinz ! Do you feel like finding out how to ask windows to enable ANSI, and perhaps add support for this to the ansi-terminal package?

[megafinz]

Well, all my Haskell experience is just a couple of exercism exercises, I don't think I'm ready to contribute any production-ready code.

[mgsloan]

Cool cool, your diagnosis is still appreciated. Hopefully someone with a windows machine will want to fix this!

[mpilgrem]

I am one of the maintainers of the ansi-terminal package. I can't reproduce this on Windows 10 (version 1709, build 16299.192), using stack Version 1.7.0, Git revision 5660e1b925d0b3feeaf00803b5c85bc8eb8664d8 x86_64 hpack-0.20.0 (stack upgrade --git), and the native ConHost terminals (Console, PowerShell 5.1). On Windows, ansi-terminal is intended to enable an ANSI-capable ConHost terminal. It would be useful to know the Windows 10 version (winver command).

[megafinz]

@mpilgrem my Windows is 1709, build 16299.125. Stack is 1.6.3 though, which I tried to update through stack upgrade --git as you mentioned, but the process failed with segmentation fault/access violation.

[mgsloan]

Whoah, upgrade failed with a segmentation fault? That's not good! Was it in stack, cabal or ghc? (can you paste the log to a new issue?)

@megafinz Upgrade might help, because @mpilgrem mentioned here that a new version of ansi-terminal might solve the problem. Newest development version uses the new version of ansi-terminal

[megafinz]

Was it in stack, cabal or ghc?

Not sure, console log is not very informative. Is there any other (verbose) log I can look for?

…
[ 76 of 129] Compiling Stack.Types.Package ( src\Stack\Types\Package.hs, .stack-work\dist\ca59d0ab\build\Stack\Types\Package.o )
[ 77 of 129] Compiling Stack.Types.Build ( src\Stack\Types\Build.hs, .stack-work\dist\ca59d0ab\build\Stack\Types\Build.o )
[ 78 of 129] Compiling Stack.Build.Cache ( src\Stack\Build\Cache.hs, .stack-work\dist\ca59d0ab\build\Stack\Build\Cache.o )
Segmentation fault/access violation in generated code
Completed 38 action(s).

--  While building custom Setup.hs for package stack-1.7.0 using:
      C:\Users\Tony\AppData\Local\Temp\stack-upgrade6916\stack\.stack-work\dist\ca59d0ab\setup\setup --builddir=.stack-work\dist\ca59d0ab build lib:stack exe:stack --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1

[mgsloan]

Segmentation fault/access violation in generated code

Hmm, that sounds like a segmentation fault while running TH code. Not sure, though, I think that's a new one to me. Looks like this is the associated ghc ticket: https://ghc.haskell.org/trac/ghc/ticket/13112

So, yeah, coming from ghc, so further verbosity is not needed. Full verbosity is stack build -v --cabal-verbose

[mpilgrem]

I also had the 'segmentation fault' today (it seems to be a recent development). However, I've been able to install stack Version 1.7.0, Git revision cfdd8d7e397a890591ec4ee76f3ee78fbe537c89 (dirty) (5641 commits) x86_64 hpack-0.21.2 by: (1) forking the stack repository and getting a local clone; (2) changing to resolver: lts-10.4 in its stack.yaml, and hence GHC 8.2.2; (3) running stack solver --update-config; (4) adding ansi-terminal-0.8.0.1 as an extra dependency; and (5) stack install, which resulted in an executable based on this stack.yaml:

~~~~
flags:
stack:
hide-dependency-versions: true
supported-build: true
nix:
packages:

• zlib
  enable: false
  packages:
  
  
  
  • .
  
  
  • subs/rio
    
    extra-deps:
  
  
  • ansi-terminal-0.8.0.1
    
    resolver: lts-10.4
    
    ~~~~
  
  

(EDIT: Also, in case it is relevant, I am using a local hpack version 0.22.0 by setting with-hpack: hpack - which, unfortunately, does not work in stack version 1.6.3 due to a bug. FURTHER EDIT: As an aside, it looks to me like with-hpack is non-functional in the stack I have now built. My mistake)

[mgsloan]

@mpilgrem @megafinz https://github.com/kofno also ran into the segmentation fault and found that it works fine with stack build --ghc-options -fexternal-interpreter

[mpilgrem]

Yes, I was able to build stack Version 1.7.0, Git revision 07bdb09b76a73b207fecb8a5ee53854dd33bc57f (5651 commits) x86_64 hpack-0.20.0 with resolver: lts-9.14 (the default) using stack build --ghc-options -fexternal-interpreter, without the segmentation fault.

[megafinz]

Not sure I'm doing it correctly, but I cloned stack repository and ran stack build --ghc-options -fexternal-interpreter, which resulted in a successful build. Then I used stack.exe from the build output and still got issues with colors:

https://i.imgur.com/m9WFPjW.png

It lists ansi-terminal-0.8.0.1 as a dependency in stack.yaml. stack itself is Version 1.7.0, Git revision 07bdb09b76a73b207fecb8a5ee53854dd33bc57f (5651 commits) x86_64 hpack-0.20.0.

[mpilgrem]

@megafinz I am unable to reproduce this, in either PowerShell v5.1 (build 16299.98) or PowerShell Core 6.0.1 (default install), on Windows 10 (version 1709, build 16299.192) and using (as you are) stack Version 1.7.0, Git revision 07bdb09b76a73b207fecb8a5ee53854dd33bc57f (5651 commits) x86_64 hpack-0.20.0 - at least, not with a 'default' project (1) stack new HelloWorld (2) cd HelloWorld (3) stack test. That is a puzzle.

[mpilgrem]

@megafinz - a thought: is the TERM environment variable set in your Windows environment? If so, what is its value? If it is set, it will likely cause ansi-terminal to behave oddly. Git Bash/mintty sets the variable to xterm (which ansi-terminal expects) but it should not exist in PowerShell consoles.

[megafinz]

@mpilgrem PowerShell says it's set to xterm-256color (https://i.imgur.com/MQ3oFQe.png).

[mpilgrem]

@megafinz, that’s the source of your problem. If you clear that TERM variable, I think your problem will be solved.

[megafinz]

@mpilgrem, you are right, I cleared the variable and now everything prints just fine (https://i.imgur.com/VvxyPrR.png). Will clearing this variable possibly break anything else?

EDIT: by the way it works with stack 1.6.3, no need to use 1.7.0.

[mpilgrem]

@megafinz, I can’t answer that definitively. The TERM variable is set by some non-native Windows terminals (eg mintty) but is clear on native Windows terminals. It is possible that it was originally set in your environment to spoof some application into thinking it was running in a non-native terminal.

[megafinz]

On one of my machines stack's color printing behavior changed after the upgrade to Fall Creators Update, so I guess TERM=xterm-256color was introduced at that point.

I found this issue: https://github.com/Microsoft/WSL/issues/2128. It appears that xterm-256color is actually the default value on recent versions of Windows.

[mpilgrem]

In the WSL (Windows Subsystem for Linux) environment, the TERM variable is set but that is independent of the non-WSL environment (unless the WSLENV mechanism is affecting things).

[Porges]

I'm also seeing this after updating from 1.6.5 to 1.7.1. I did not see it on 1.6.5:

[mpilgrem]

I have upgraded from: Version 1.7.0, Git revision 07bdb09b76a73b207fecb8a5ee53854dd33bc57f (5651 commits) x86_64 hpack-0.20.0 to Version 1.7.1, Git revision 681c800873816c022739ca7ed14755e85a579565 (5807 commits) x86_64 hpack-0.28.2. I now have problems with 'raw' ANSI codes being output by (1) stack ghci (on Windows 10) until it reaches the line GHCi, version 8.2.2: http://www.haskell.org/ghc/ :? for help; and (2) other stack commands. For example, stack new test2 now yields output messages with 'raw' ANSI codes.

[mpilgrem]

@Porges, I have opened this as a new issue #3992 because I think the source of the version 1.7.1 problem is distinct from the underlying cause of the subject matter of this issue, which was ultimately resolved.

[Josef-Vonasek]

Happens to me also with stack 1.7.1, but I do not have any TERM variable in my environment.

[tdaniely]

FYI, The workaround (until all tools in existence adjust to what the guys in Microsoft decided -_-) is to just switch back to forcing vterm for all consoles.

https://github.com/Microsoft/WSL/issues/1173#issuecomment-370665039

[moodmosaic]

Same as @Porges in https://github.com/commercialhaskell/stack/issues/3797#issuecomment-382936365, I'm getting also weird characters in the output.

[mpilgrem]

@moodmosaic, if you build/install versionstack upgrade --git do you still experience the problem? I am hoping that recent patches to master will have fixed this.

[moodmosaic]

@moodmosaic, if you build/install version stack upgrade --git do you still experience the problem? I am hoping that recent patches to master will have fixed this.

Yay! Took half an hour to finish (on battery power) but it worked 🎊

$ stack --version
Version 1.8.0, Git revision 818899906cc8797d1bb27b35e4f31f30f91664e8 x86_64 hpack-0.28.2

[mihaimaruseac]

Neat. Do you still see the bug happening?

[moodmosaic]

Neat. Do you still see the bug happening?

Nope.

[AnthonyJacob]

Ran stack upgrade --git; stack build intero and I still saw raw ANSI colors.

[mihaimaruseac]

Thanks @moodmosaic

@AnthonyJacob can you run a stack --version and post it here, please?

[mpilgrem]

@AnthonyJacob, it may also help if you specify the Windows version you are using (as reported by the ver command) and the terminal program you are using (ie Command Prompt, PowerShell, mintty or other). Finally, it may possibly be relevant if the TERM environment variable exists and, if it does, what value it is set to.

[mpilgrem]

I could not reproduce the problem of raw ANSI codes with stack build intero on Windows 7 or Windows 10 (10.0.17134.137) (in Command Prompt).

[marvertin]

I have allways the problem too

Version 1.7.1, Git revision 681c800873816c022739ca7ed14755e85a579565 (5807 commits) x86_64 hpack-0.28.2
WIndows 7 and Windows 10

m:\vyvoj\veverepo>stack --resolver lts-9 install hoogle
←[0mSelected resolver: lts-9.21←[0m
WARNING:W AcRaNcIhNeG :i sc aocuhte  oifs d aotuet:  oCf: /dUasteer:s /C:t/aUtsienresk//tAaptpiDnaetka//ALpopcDaalt/aP/rLoogcraalm/sP/rsotgarcakm/sx/8s6
b8\.p0a.c2k\algieb.\cpoancfk.adg\ep.accoknafge..dc\apcahcek
agghec. cwaiclhle
sgehec  awni loll ds evei eawn  oofl dt hviise wp aocfk atghei sd bp.a cUksaeg e' gdhbc.- pUksge  r'egchacc-hpek'g  troe cfaicxh.e
' to fix.
WARNING: cache is out of date: C:/Users/tatinek/AppData/Local/Programs/stack/x86_64-windows/ghc-8.0.2\lib\package.conf.d\package.cache
ghc will see an old view of this package db. Use 'ghc-pkg recache' to fix.
←[0mauto-update-0.1.4: configure←[0m
←[0mappar-0.1.4: download←[0m
←[0mbasement-0.0.4: download←[0m
←[0mauto-update-0.1.4: build←[0m
←[0mbase64-bytestring-1.0.0.1: configure←[0m
←[0mbase64-bytestring-1.0.0.1: build←[0m
←[0mappar-0.1.4: configure←[0m
←[0mauto-update-0.1.4: copy/register←[0m
←[0mappar-0.1.4: build←[0m
←[0mbasement-0.0.4: configure←[0m
←[0mblaze-builder-0.4.0.2: download←[0m
←[0mbase64-bytestring-1.0.0.1: copy/register←[0m
←[0mappar-0.1.4: copy/register←[0m
←[0mbasement-0.0.4: build←[0m
←[0mblaze-builder-0.4.0.2: configure←[0m
←[0mblaze-builder-0.4.0.2: build←[0m
←[0mbyteable-0.1.1: configure←[0m
←[0mbyteorder-1.0.4: download←[0m
←[0mbyteable-0.1.1: build←[0m
←[0mbyteorder-1.0.4: configure←[0m

[mihaimaruseac]

Please upgrade to latest on Git or wait until the 1.8 is released, that version contains the fixes.

[Bert-Proesmans]

Hi, i have this issue as well.
Running on Version 1.7.1, Git revision 681c800873816c022739ca7ed14755e85a579565 (5807 commits) x86_64 hpack-0.28.2 - Windows 10, Powershell prompt.

I'm pretty new to the haskell ecosystem so i'm currently just using --color=never as workaround.
Where/How can i find out what the release schedule is for 1.8?

Output when running stack test

hello-world-1.1.0.4: build (lib + test)
Preprocessing library for hello-world-1.1.0.4..
Building library for hello-world-1.1.0.4..
[1 of 2] Compiling HelloWorld       ( src\HelloWorld.hs, .stack-work\dist\7d103d30\build\HelloWorld.o )

src\HelloWorld.hs:4:9: error:
    * Couldn't match type `IO ()' with `[Char]'
      Expected type: String
        Actual type: IO ()
    * In the expression: print "Hello, World!"
      In an equation for `hello': hello = print "Hello, World!"
  |
4 | hello = print "Hello, World!"
  |         ^^^^^^^^^^^^^^^^^^^^^

Progress 1/2

--  While building custom Setup.hs for package hello-world-1.1.0.4 using:
      C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_Z6RU0evB_2.2.0.1_ghc-8.4.3.exe --builddir=.stack-work\dist\7d103d30 build lib:hello-world test:test --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1

[mihaimaruseac]

We are currently working on releasing 1.9 (the new 1.8; no 1.8 public release), hopefully this won't take more than a week.

Until then, you can also upgrade with stack upgrade --git if you want to get it faster.

[Topsii]

Some months ago I was having this issue and solved it by running stack upgrade --git.
Afterwards the problem was solved and my version was:
Version 1.8.0, Git revision 839dbd5a794ccf75c75f9903b3eae7969c7a6278 x86_64 hpack-0.29.0

Today I ran stack upgrade --git again and the issue reappeared after the upgrade completed.
Now my version is:
Version 1.10.0, Git revision a299ac652fde49d3b4b6a199c1a206220826da68 x86_64 hpack-0.30.0

The TERM environment variable is not set and I am using Powershell.
winver gives Microsoft Windows Version 1803 (Build 17134.285).

Though I guess this stuff is only relevant if I was using the Windows Subsystem for Linux (WSL). I am not sure if this issue is only about the problem occuring in WSL or about the general case. Since it was the only open issue of all the duplicates I decided to post here anyway.

[mpilgrem]

@Topsii, I'll investigate but I can't do that before this evening (UK time). I'm currently using a version based on commit bbac19 (9 Sept 2018, 6,481 commits) on Windows without problems.

[Topsii]

This is too weird. I executed one command with --color never, but then the output was readable and colored. Afterwards I executed commands without --color never and the output was still readable and colored. So the problem has disappeared.

Why would --color never display colored output?
This still happens, so in this sense the problem did not disappear.
Why does the result depend on whether --color never has been executed before?
Is the choice for --color stored?

I don't think it matters, but I actually ran the --color never command in a second terminal that was opened before upgrading stack.
Here's the output paste for the first terminal where I upgraded stack and escape codes were printed. stack build runs twice:
Once before --color never was executed in the second terminal and escape codes are printed.
And once after --color never was executed without priniting any escape codes.

[mpilgrem]

@Topsii, one hypothesis that I have is that stack is suffering from an upstream bug in the git application: for certain git commands, git disables the ANSI-capability of a Windows 10 terminal. stack has to be alive to that and re-enable the ANSI-capability after making use of git. The fact that your problem starts immediately after:
Cloning b82758cb178b8f53708a8b9ffee46290887ce00d from https://github.com/Topsii/generic-data.git
may be no coincidence.

As for -- color never, I can't explain that. If you are able to reproduce what you have experienced, please log it here.

[Topsii]

I changed the commit version of my git dependency and escape characters started appearing again after the Cloning line. You seem to be right, It only occurs when git is used and persists until the command has been executed. The next commands were colored and readable

And yes, e.g. stack test --color never still gives me green colored OKs for my passed tests.

[mpilgrem]

@Topsii, am I correct to understand that colour in the output of tests is determined not by stack itself but by the test library you are using in your code? For example, Test.Tasty.Ingredients.ConsoleReporter.UseColor.

[mpilgrem]

@Topsii, I've proposed a pull request that should fix what you have identified.

[Topsii]

@Topsii, I've proposed a pull request that should fix what you have identified.

Yeah, I upgraded again and there are no escape characters after Cloning .. now.

@Topsii, am I correct to understand that colour in the output of tests is determined not by stack itself but by the test library you are using in your code? For example, Test.Tasty.Ingredients.ConsoleReporter.UseColor.

Oh, yes that must be it.
It's also confusing that apparently some keywords like Error or Warning are color-highlighted for me. I guess that is either powershell's doing or Visual Studio Code's where I'm running powershell in.
Just to make sure I tested the git cloning bug with --color never before upgrading: There were no escape characters.