Deno: windows doesn't print emojis correctly

[nyctef]

Hi! This is my first time looking at Deno, and it seems to be very exciting :)

I was playing around with the examples from the tutorial and noticed this inconsistency: (using powershell and windows terminal):

> deno run https://deno.land/std/examples/welcome.ts
Welcome to Deno 🦕
> deno run --allow-net https://deno.land/std/examples/curl.ts https://deno.land/std/examples/welcome.ts
console.log("Welcome to Deno ­ƒªò");

I found that editing curl.ts to add the line

import { decode } from "https://deno.land/std/encoding/utf8.ts";

and replacing the Deno.stdout.write line with

console.log(decode(body));

seems to make the downloaded file print out correctly.

[actual-size]

Cannot reproduce on macOS 10.15.4:

~/Desktop ❯❯❯ deno run --allow-net https://deno.land/std/examples/curl.ts https://deno.land/std/examples/welcome.ts
console.log("Welcome to Deno 🦕");

but can reproduce on Windows 10 (in PowerShell):

> deno run --allow-net https://deno.land/std/examples/curl.ts https://deno.land/std/examples/welcome.ts
console.log("Welcome to Deno 🦕");

[MarkTiedemann]

Deno.stdout.write writes UTF-8 data but the console code page may be different.

If you run chcp 65001 (65001 = UTF-8) first, the output is as expected.

[nyctef]

If you run chcp 65001 (65001 = UTF-8) first, the output is as expected.

Yep, that works for me too 👍

Seems strange that console.log just works here though - is it doing something different that Deno.stdout.write could use? I tried browsing through the source to figure out how console.log was implemented, but got lost pretty quickly.

[MarkTiedemann]

Seems strange that console.log just works here though - is it doing something different that Deno.stdout.write could use?

I'd assume that it's calling SetConsoleOutputCP(CP_UTF8) before logging but I don't know the internals either.

[piscisaureus]

I think it uses WriteConsoleW.

Supposedly SetConsoleOutputCP(CP_UTF8) should work, but only on the most recent versions of windows.

[littletof]

I just run into the same issue trying to use characters like ∎ (U+220E) ▒ (U+2592) with stdout (win10 powershell).

If you run chcp 65001 (65001 = UTF-8) first, the output is as expected.

Worked for me too, as a temp solution.

[MarkTiedemann]

I think it uses WriteConsoleW.

I think WriteConsoleW doesn't set the console code page.

In fact, since other programs may run in the same console afterwards, I think it's best practice to reset the code page before exit:

UINT original_codepage;

void init() {
        original_codepage = GetConsoleOutputCP();
        SetConsoleOutputCP(CP_UTF8);
}

void cleanup() {
        SetConsoleOutputCP(original_codepage);
}

[oganexon]

6131

This problem is the same, I couldn't display characters with accents.

The details of the problem are explained, and how to solve it under Windows.

[piscisaureus]

I think WriteConsoleW doesn't set the console code page.

That's true, but here's the secret: WriteConsoleW doesn't depend on the code page, it always uses UTF16 (technically UCS2).

[MarkTiedemann]

WriteConsoleW doesn't depend on the code page, it always uses UTF16

Thanks for sharing this! That's good to know.

Well, back to the problem:

I'd assume that WriteConsoleW is used for console.log and something else is used for Deno.stdout.write. Otherwise the output would be the same, right? Perhaps instead of WriteConsoleW(hStdout, ...), it's WriteFile(hStdout, ...) for Deno.stdout.write? I don't think WriteFile handles unicode console output automatically like WriteConsoleW.

[MarkTiedemann]

@piscisaureus I just traced the two commands using API Monitor. I can confirm that console.log is using WriteConsoleW whereas Deno.stdout.write is using WriteFile.

Screenshots

> deno run https://deno.land/std/examples/welcome.ts

> deno run --allow-net https://deno.land/std/examples/curl.ts https://deno.land/std/examples/welcome.ts

[nitrospaz]

Hi I'm pretty new to all this but I thought this was interesting, see my attached screenshot. I found that the emojis did not display correctly in Windows Powershell (2), Powershell 7.1 (3) or the terminal in VS Code (4) but they do display correctly if you use Windows Powershell or Powershell 7.1 with the new Windows Terminal app (1). So maybe the Terminal is able to interpret the unicode properly where as the Powershell by itself isn't a full unicode native.

[pedropaulosuzuki]

8199