Azure-pipelines-agent: Strip or honor ASCII color coding

[jessehouwing]

Agent Version and Platform

2.133.3
Windows | Linux

VSTS Type and Version

VisualStudio.com

If VisualStudio.com, what is your account name? http://psd30.visualstudio.com

What's not working?

When a commandline tool issues ASCII colour coding information (e.g. [?25l) they are shown in the logs and make it harder to read.

2018-05-19T15:43:16.7926995Z It looks like this is your first time using Cypress: 2.1.0
2018-05-19T15:43:16.7938560Z 
2018-05-19T15:43:16.7958404Z [?25l[15:43:05]  Verifying Cypress can run /opt/vsts/_work/r1/a/xxx-CI/src/node_modules/cypress/dist/Cypress [started]
2018-05-19T15:43:16.7980465Z [15:43:06]  Verifying Cypress can run /opt/vsts/_work/r1/a/xxx-CI/src/node_modules/cypress/dist/Cypress [completed]
2018-05-19T15:43:16.8000804Z [?25h
2018-05-19T15:43:16.8019478Z Opening Cypress...

It would be useful to either honor the color coding, somehow instruct/configure the terminal as monochrome or stripping out the nonsense characters.

[bryanmacfarlane]

Yeah, this is on our backlog. labelling as an enhancement.

[adamralph]

Any update? As the maintainer of a build package which uses ANSI colours, I'm starting to get questions.

[TingluoHuang]

we are fixing this step by step.

1. get all UTF-8 characters back correctly #1975
2. the Web UI might already support color coding. :)

[adamralph]

Thanks for the info. AFAIK the web UI still doesn't support ANSI colour codes, see https://dev.azure.com/blairconrad/SendComics/_build/results?buildId=9&view=logs&jobId=011e1ec8-6569-5e69-4f06-baf193d1351e&taskId=b8cbf579-6a08-5e02-2788-e25b5bd52b1b&lineStart=13&lineEnd=14&colStart=1&colEnd=1

@blairconrad you reported https://github.com/adamralph/bullseye/issues/150 - are you still seeing this problem in the web UI?

[blairconrad]

@adamralph, I see no change. ANSI codes show up in the web UI:

https://dev.azure.com/blairconrad/SendComics/_build/results?buildId=13

[yaananth]

The web UI supports minimal set of escape codes, but the full support is coming soon.
We are prioritizing this.
Since this is UI issue and not agent issue, feel free to post something here
https://developercommunity.visualstudio.com/spaces/21/visual-studio-team-services.html?type=idea

But we understand the request and are working on it.

[yaananth]

Changes should be rolling out. We support ANSII color code now.
Here's what we support:

Graphics mode (m) -
0 (reset),
1 (bold),
3 (italic),
4 (underline),
22 (not bold),
23 (not italic),
24 (not underline),
39 (default fg),
49 (default bg),

fg colors - 30 (black), 31 (red), 32 (green), 33 (yellow), 34 (blue), 35 (magenta), 36 (cyan), 37 (white), 90 (grey)
bg colors - 40 (black), 41 (red), 42 (green), 43 (yellow), 44 (blue), 45 (magenta), 46 (cyan), 47 (white)

[TingluoHuang]

Close this issue base on this colorful screenshot. :D

[blairconrad]

PRETTY!

[blairconrad]

Not to be a Negative Nelly, but I notice that while the _completed_ build output looks nice and colourful, the in-progress output does not deal well with the ANSI codes:

(from the in-progress build of https://dev.azure.com/blairconrad/blair%20bullseye/_build/results?buildId=18)

[yaananth]

That's a known limitation now unfortunately, we don't use the same view for both views :(
This is getting addressed though, just won't be super soon :)

[blairconrad]

Oh, thanks for the speedy response. Glad it's known!

[blairconrad]

I hate to complain, but as @adamralph noted over at https://github.com/adamralph/bullseye/pull/199#issuecomment-447370765, in some cases the combination of the ANSI escape codes and < or > in the output confuse the renderer. I have created a minimal reproduction which you can see at https://dev.azure.com/blairconrad/azure-ansi/_build/results?buildId=22.

You can see the HTML produced for the ANSI-less and ANSI-ful output here:

<div style="top:192px;height:16px;" class="view-line">
  <span><span class="mtk1">2018-12-14T19:46:38.8739531Z&nbsp;No&nbsp;colour</span></span>
</div>
<div style="top:208px;height:16px;" class="view-line">
  <span><span class="mtk1">2018-12-14T19:46:38.8741003Z&nbsp;&lt;options&gt;</span></span>
</div>
<div style="top:224px;height:16px;" class="view-line">
  <span><span class="mtk1">2018-12-14T19:46:38.8758154Z&nbsp;</span></span>
</div>
<div style="top:240px;height:16px;" class="view-line">
  <span><span class="mtk1">2018-12-14T19:46:38.8759918Z&nbsp;Yellow&nbsp;foreground</span></span>
</div>
<div style="top:256px;height:16px;" class="view-line">
  <span><span class="mtk1"><span>2018-12-14T19:46:38.8761200Z </span>&zwnj;<span class="ansifg-y "><options></options></span>&zwnj;</span></span>
</div>

It seems that once the ASNI codes are used, <options> is translated into <options></options> instead of <options>.

Is https://developercommunity.visualstudio.com/spaces/21/visual-studio-team-services.html?type=idea still the right place to mention this or is there a better place?

[yaananth]

Here are the known limitations with this feature -

• Visually text won't appear selected if you try to select the text (but copy-paste works)
• Live feed won't honor color codes
• Color codes with < > tags will make them disappear

This is something we decided to live with until we address all of them together in the next spike which is planned soon. We decided to go ahead with the feature even with those as it's much better than showing some random useless characters :)

If you are curios on the limitations, we use monaco editor, which isn't really made for "viewing" experience. So to bring color coding support, we had to have those limitations for the interm...

Thanks for reporting and we understand the issues you mentioned and we plan to work on those, let us know if you find anything else.

For this particular issue, I think having it here makes sense as everything is being discussed here, but in general yes, dev community is the place to track these.

[blairconrad]

Thanks for the response @yaananth. I had doubts about https://developercommunity.visualstudio.com/spaces/21/visual-studio-team-services.html?type=idea as the "idea" at the end sounds more like feature requests, not bug reports.

I eagerly await the next release and appreciate all your efforts.

And I realize that every client is different, but given the choice between "bonus funny characters in the output" and "some sections of the output just omitted altogether", I'd've picked the second. The recovery mode of the first is "just try not to look at the funny characters" while for the second it's "hopefully realize something is wrong and then pore over the source HTML".

[yaananth]

You can report a problem: https://developercommunity.visualstudio.com/content/problem/post.html?space=21

[adamralph]

@yaananth earlier you said:

We decided to go ahead with the feature even with those...

So you have knowingly introduced a bug. Why do you need someone to "report" it?

[olafurpg]

Visually text won't appear selected if you try to select the text (but copy-paste works)

@yaananth Is there an open issue to fix this limitation? I searched in the Visual Studio Community but couldn't find a related discussion. Copy-paste works for in-progress builds but it stops working when the colors have rendered after the build completes. To reproduce, try copy-pasting line 4000 from the CmdLine entry in https://dev.azure.com/olafurpg/Metals/_build/results?buildId=51

[sbachstein]

Have there been any advances regarding the character encoding not being resolved?

[yaananth]

Multistage pipelines feature will bring new views which will honor ansii codes, evening when live streaming!

[lindluni]

While coloring is better with the multistage feature enabled, codes are still dumped in the output of the multistage build

[yaananth]

We support https://en.wikipedia.org/wiki/ANSI_escape_code#3/4_bit
It's possible the tool is using other specifications, eg: https://en.wikipedia.org/wiki/ANSI_escape_code#8-bit

Which we don't support yet.
However we should be parsing them out.
So, looks like there's some bug there.
Could you send a repo text file or link to where it happens (if it's a public run).
Thanks!

[lindluni]

This is actually the output of a git diff --color

Here is the link, it is the second step labeled CmdLine. We are just getting started on our migration so we are noticing small nuances like this.

AZP Build 291

[octogonz]

(Not sure why this was closed 9 months ago, when the problem was clearly only partially solved.) But today I noticed that if you enable the "Multi-stage pipelines" preview feature, then colors are displayed properly for the realtime log output. (In Azure DevOps, click your profile --> "..." --> "Preview features".)

Hurray! 🎉

[malkinfedor]

@octogonz
Yes, you're completely right, I've enabled this feature and now ansible output looks pretty good!
Thansk for this advice!

[elisherer]

Issue is back on release (not build) logs

[liamfoneill]

This really is not a finished feature. Take a look at what 'Terraform Plan' output looks like without using the '-no-color' flag:

Apparently, Terraform uses 'VT100 escape codes' for colorizing output.

Please don't say just use multi-stage pipelines. That is not an option. There are still lots of things you can not yet do in YAML pipeline that can be done in 'Classic'.

Please prioritize this issue.

[timotheecour]

@TingluoHuang could you please re-open this issue, since it's clearly not fixed?

[TingluoHuang]

@vtbassmatt FYI.

[mcollis2]

Yes we are seeing this issue too with terraform in Azure Devops classic pipelines. Would very much appreciate a fix on this as it worked 1-2 months ago.

[jtpetty]

@azureDaveOps - David, would this be something that your team would own the fix for?

[scottboehmer]

Adding @prativen since this looks to be impacting release logs and not build logs.

[octogonz]

For me, the PR build logs seem to render correctly with the "Multi-stage pipelines" preview feature enabled. But if that feature is disabled, then the color codes are garbled, making the logs unreadable. (Example log)

How long will "Multi-stage pipelines" be kept in preview? Lots of NodeJS tools want to print colors to their log output.

[scottboehmer]

We've started the process of making the multi-stage pipelines feature on-by-default, and I expect to be on everywhere with the option to go back to the old UI removed sometime in early 2020.

[rene-fonseca]

@yaananth, not sure if you prefer a separate issue but I'll try this one since it is still open.

[0m for resetting all attributes is not resetting foreground color - I didn't check if other attributes are reset but this would likely be the case.

Also noticed [4m for underscore and [3m for italic are ignored. So maybe there is a general problem for the lower m values. But per your earlier note these codes should be supported.

Example line:
2020-01-02T20:53:10.4177461Z THROW in class base::MyOtherObject &__cdecl base::Reference::operator *(void) at base/mem/Reference.h:342

The [0m just before "at" above - should ensure "at" gets presented in default color - but for now it uses the color that is active before.

Full public log example is available at - I do use ANSI colors a lot and it is really nice the pipeline supports it :)
https://dev.azure.com/renefonseca/base/_build/results?buildId=820&view=logs&j=2d2b3007-3c5c-5840-9bb0-2b1ea49925f3&t=3d94a178-cdfb-5c90-a971-5deeccbbe043&l=51

Here is an working example from Visual Studio Code terminal (run on macOS) for comparison:

[jamessampford]

I'm using SSH in my pipeline to run a bash script and getting the codes appear as well (rather than the colours), example below:

2020-01-08T09:08:30.6597072Z ### RECEIVED BUILD 120 ###

Tried a variety of escape and colour codes but all the same...

\x1b[1;34m
\x1b[34m
\033[1;34m
\033[34m

[santiago-ld]

I'm having the issue using terraform too, hope to see this fixed soon.

[stephenmichaelf]

@davekenn Could you route to the right person on your team to take care of this?

[jz3xyz]

I'm having the same issue with a bash script in a release pipeline.

I'm setting colors using

# Set colors
BOLD="\e[1m"
ITALIC="\e[3m"
RESET="\e[0m"
RED="\e[31m"
GREEN="\e[92m"
CYAN="\e[96m"

Ado likes to interpret them as is

[96m[1mUsage: ...[0m

[thiagofborn]

Hello there.

I hope you guys are having loads of fun.
However, I need to ask...

Did you find any workaround for this?

[edjwhho]

Hi, anyone managed to get a work around for the terraform plan to output colours in azure pipeline?

[octogonz]

Today we were trying to make a copy of an Azure DevOps pipeline, but could not find the "clone" menu item. (Screenshots are shown here.)

After much frustration, @iclanton figured out that the "Multistage pipelines" preview removes that feature. 😕

So it seems, I have to manually enable "Multistage pipelines" to see ANSI colors in my build logs, but remember to turn it off when I want to clone a pipeline.

@scottboehmer was this intentional? What's the deal?

[scottboehmer]

Is your definition using a yaml file? If so, then it is intentional that clone is unsupported because the recommended pattern is to copy the yaml file rather than creating a cloned definition pointing at the same underlying file.

[up2geek]

I too would like to see a fix or workaround to allow terraform plan color coding output to display properly in ADO release pipeline log output.

[octogonz]

Is your definition using a yaml file? If so, then it is intentional that clone is unsupported because the recommended pattern is to copy the yaml file rather than creating a cloned definition pointing at the same underlying file.

@scottboehmer No, it is a non-YAML job. It's a manually invoked job, where we usually edit the steps before running it (e.g. changing which folders it runs in, or skipping certain steps). Thus creating a Git branch and YAML file would be a clumsy approach.

[ppanyukov]

I'm not 100% sure why it's so difficult to take raw log files and convert them to HTML and do ASCII colors? Here is a bash script that does it, this can be done even in bash. https://github.com/pixelb/scripts/blob/master/scripts/ansi2html.sh

It works for terraform output and git diff and whatnot.

EDIT: my question is to the product team

EDIT 2: so there are actually two issues here:

1. colors in builds
2. colors in releases

Builds -- WORKS WITH CLASSIC EDITOR

Releases -- DOES NOT WORK (using classic editor)

I used this script as a test: https://github.com/pixelb/scripts/blob/master/scripts/ansi_colours.sh

[octogonz]

Well, "Multi-stage pipelines" has been officially released. We can no longer workaround this problem by disabling the preview feature.

@scottboehmer What's the product team's position on:

• providing a way to clone a classic (non-YAML) pipeline
• properly supporting ASCII colors like every other tool does

It may be that these customer requests are not a priority and will never get fixed. In that case, please just say so and close the issue.

[scottboehmer]

As far as I'm aware, clone is working for non-yaml pipelines. On a sample project, I see it in the menu when on a definition's page:

@octogonz do you not see that menu entry? Or does it fail when you try to use it? If it is a failure, could you share any error information logged in the browser's console.

I'm not sure on the status of extending ASCII color support on the logs page, so adding @vijayma and @jtpetty to see if they know of any plans for additional work.

[octogonz]

As far as I'm aware, clone is working for non-yaml pipelines. On a sample project, I see it in the menu when on a definition's page

You are right! I overlooked that. Consider the "Clone" item resolved then! 👍

[Marcus-James-Adams]

This issue thread seemed to get hijacked by cloning but can we have an update as to where we are with colours displaying as escape characters and other text corruptions in the release pipelines as currently, it makes them unreadable, especially for actions like terraform that use colours and characters to explain what changes there will be.

eg
2020-06-25T09:50:54.6379881Z  # azurerm_network_security_rule.rdp-access-utils-subnet will be updated in-place 2020-06-25T09:50:54.6384644Z  ~ resource "azurerm_network_security_rule" "rdp-access-utils-subnet" { 2020-06-25T09:50:54.6385107Z access = "Allow" 2020-06-25T09:50:54.6398898Z destination_address_prefix = "*"

[tknerr]

I can confirm on what @ppanyukov commented above in https://github.com/microsoft/azure-pipelines-agent/issues/1569#issuecomment-618505636 that it does not work for Release pipelines yet.

[diegosucaria]

Hello everyone!

No updates so far on this 2-years-old issue???

Did anyone found a workaround?

[santiago-ld]

Hello everyone!

No updates so far on this 2-years-old issue???

Did anyone found a workaround?

Nop, just using -no-color for terraform :/.

[edjwhho]

alternatively you can use azure cli task and run terraform commands manually which will give you colour output from terraform commands.

```
- task: AzureCLI@1
inputs:
azureSubscription: 'terraform'
scriptLocation: 'inlineScript'
inlineScript: |
#!/usr/bin/env bash
echo "***********"
echo " setting up env vars"
export ARM_SUBSCRIPTION_ID=$(az account show --query="id" -o tsv)
export ARM_CLIENT_ID="${servicePrincipalId}"
export ARM_CLIENT_SECRET="${servicePrincipalKey}"
export ARM_TENANT_ID=$(az account show --query="tenantId" -o tsv)
export ARM_ACCESS_KEY="${accessKey}"
echo "***********"
echo " terraform init"
terraform init
echo "************"
echo " terraform plan"
terraform plan

[ed-graham]

Now 23rd December 2020 and the problem of output colours and formatting in release-pipeline logs (as so eloquently explained by @Marcus-James-Adams) is still at large. Don't suppose the product team fancies giving us all a late Christmas present? In the meantime, I'll continue to use the -no-color work-around suggested by @santiago-ld .

[onlyann]

The output from New-AzResourceGroupDeployment -Whatif isn't color coded using the AzurePowershell task from the pipeline web console.
Is it related to this issue?

[stvpalumbo]

March 19th 2021 - Colors are still not supported properly in the Release Pipeline log viewer. This makes our selenium step so painful to debug...