Pino: Windows: incorrect encoding on Unicode output

[budarin]

I try to log

logger.info(`Сервер запущен по адресу:')

but see in log

{"level":30,"time":1542741475885,"msg":"╨б╨╡╤А╨▓╨╡╤А ╨╖╨░╨┐╤Г╤Й╨╡╨╜ ╨┐╨╛ ╨░╨┤╤А╨╡╤Б╤Г:

[davidmarkclements]

ohhhh wow ok - we need to cater to full unicode

this will slow things down, if it causes significant overhead we might have to add an "international" option with a documented perf hit

[davidmarkclements]

oooor.. maybe we can use a post-processor to re-encode

[budarin]

no
here is logger configuration

import pino from 'pino';

const dest = pino.extreme();
const logger = pino(dest);
setInterval(() => logger.flush(), 10000).unref();

export default logger;

[davidmarkclements]

I'm not able to reproduce

[davidmarkclements]

also verified to work with pino.extreme

[davidmarkclements]

redirecting to file, works:

[budarin]

[davidmarkclements]

it might be due to the fact you're transpiling your server

try

process.stdout.write(`Сервер запущен по адресу:`)

[davidmarkclements]

or otherwise I'm seeing that the output is being escaped, so something is processing the output

closing for now, as this doesn't seem like a problem with Pino

[budarin]

strange - writing into stdout is working right

this is in the same transpilled code

logger.info(`Сервер запущен по адресу: ${SERVER_URL || 'error'}`);
process.stdout.write(`Сервер запущен по адресу:`);

[jsumners]

Oh boy. How has this not come up before?

To be JSON spec compliant we should be outputting UTF-8 strings. I'm not seeing anything in the language spec that that dictates how strings are to be encoded -- https://tc39.github.io/ecma262/#sec-text-processing . But this article implies that they could be UTF-16, and a comment to it indicates they could very well be merely ASCII, -- https://kev.inburke.com/kevin/node-js-string-encoding/

Regarding the problem, I suspect it has to do with the encoding of the destination. So if process.stdout isn't in UTF-8 mode you are likely to get garbage.

[davidmarkclements]

Ok reopening as it looks like this might be on us - or more likely - sonic-boom cc @mcollina

@budarin - as a workaround try pino(process.stdout)

[budarin]

this is only apparent in Windows
have verified on MacOS - everything is ok

[davidmarkclements]

The thing is sonic-boom does support UTF-8 (https://github.com/mcollina/sonic-boom/blob/master/index.js#L71) so this problem might go all the way to Node core

[mcollina]

Turns out it's a known limitation of Node.js.

Running chcp 65001 https://ss64.com/nt/chcp.html solves the problem for me.

I propose we just document this behavior in the pino website https://getpino.io/#/docs/help.

[mcollina]

Thanks @refack in https://github.com/nodejs/node/issues/24550#issuecomment-440720503.