Vscode-jupyter: ANSI color code in Notebook cell output makes text be interpreted as HTML

[brandon-rhodes]

Bug: Notebook Editor, Interactive Window, Editor cells

Steps to cause the bug to occur

1. Create or open a Jupyter Notebook.
2. Run this Python code in a cell: print('| \033[31m <b>foof</b> \033[0m |')

Actual behavior

The angle brackets are swallowed and the word “foof” is shown in bold:

Expected behavior

The native Jupyter Notebook running in a browser correctly renders the ASCII text as text, including the angle brackets:

Your Jupyter and/or Python environment

_Please provide as much info as you readily know_

• Jupyter server running: Local
• Extension version: 2020.7.96456
• VS Code version: 1.47.3
• Setting python.jediEnabled: (I can't find that setting)
• Setting python.languageServer: Jedi
• Python and/or Anaconda version: 3.7.3
• OS: Linux (distro): Ubuntu
• Virtual environment: virtualenv

Developer Tools Console Output

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer, @joyceerhl

[rchiodo]

Thanks for the issue.

Do you have another use case other than the repro example? It's hard to gauge how important this sort of thing is.

[brandon-rhodes]

Do you have another use case other than the repro example? It's hard to gauge how important this sort of thing is.

Good question!

Users both themselves create scripts that output ANSI colors, and also invoke common Python tools that use colors. The common pytest tool comes to mind as one that uses ANSI colors to decorate its output. Any such tool that happens to include what look like angle brackets in its output will hit this issue and have the text between the angle brackets interpreted as an HTML element and disappear from the text. (For example, think of all the times that <string> and <module> show up in Python tracebacks in the wild.)

A GitHub search for Jupyter notebooks with escape characters returns more than 1 million resulting notebooks. The first few pages of results, at least, suggest that nearly all of these instances of output escape characters are indeed in ANSI color sequences (look for strings like \u001b[1;31m that end with m):

https://github.com/search?q=%5Cu001b++extension%3Aipynb&type=Code

For comparison, there seem to be around 8.5 M notebooks total on GitHub:

https://github.com/search?q=cells+extension%3Aipynb&type=Code

So around 12% of all Jupyter notebooks in the wild could be affected — but, of course, the actual number would be lower, because many of them would not happen to use < as a character in the same output blocks that they decorate with ANSI escape sequences.

[brandon-rhodes]

Note that the default Python printed representation of any user object is surrounded by angle brackets:

>>> class MyObject: pass
... 
>>> o = MyObject()
>>> print(o)
<__main__.MyObject instance at 0x7efe9f60beb0>

[rchiodo]

Yes the default should work. It's the ansi escape codes that are messing us up. We've already had similar bug logged for having brackets in an error message.

Outside of error messages I was wondering how common it was to use ansi escape codes.

[brandon-rhodes]

Outside of error messages I was wondering how common it was to use ansi escape codes.

Taking the top 20 results of that search as a doubtless not-precisely-representative sample, it’s 70% tracebacks versus 30% normal output, so about two-thirds error reporting versus one-third output from other tools or calls or commands that choose to color-highlight their output.

[greazer]

This is a dupe of microsoft/vscode-python#12745