Pino: Save output to local file

[mantovanig]

Hi,

i trying to use Pino in a nodejs application.
It's possibile to save log in a local .log file ?

For example like https://github.com/winstonjs/winston#instantiating-your-own-logger

Thanks

[marcbachmann]

Maybe something like that? 😉
https://github.com/pinojs/pino-tee

[mantovanig]

Yess! Thanks

[marcbachmann]

You're welcome. Glad to help.

[ro70]

@mantovanig @marcbachmann I just found this post. May I ask you for help please? How do I "connect" pino and pino-tee? I tried something like

const pino = require('pino')()
const tee = require('pino-tee')
const fs = require('fs')
const stream = tee(process.stdin)
stream.tee(fs.createWriteStream('myLogFile'), line => line.level >= 0)
stream.pipe(process.stdout)

pino.info('hello world')
pino.error('this is at error level')

without any success. I can read the log in the console. myLogFile is created but remains empty.

[mcollina]

@ro70 I think the best approach is to use https://github.com/pinojs/pino-multi-stream right now. you can specify both a stdout and a writable in the streams.

[ro70]

@mcollina Thanks for your response. Why should I use pino-multi-stream instead of pino-tee? In the caveats of pino-multi-stream there is written

[...] but it is not meant to be a long term solution. We strongly suggest that you use this module for only as long as it will take you to overhaul the way you handle logging in your application.

This example from pino-multi-stream is not working for me either:

var fs = require('fs')
var pinoms = require('pino-multi-stream')
var streams = [
  {stream: fs.createWriteStream('/tmp/info.stream.out')},
  {level: 'fatal', stream: fs.createWriteStream('/tmp/fatal.stream.out')}
]
var log = pinoms({streams: streams})

log.info('this will be written to /tmp/info.stream.out')
log.fatal('this will be written to /tmp/fatal.stream.out')

The files are written but they remain empty. What am I doing wrong?

[mantovanig]

@ro70 pino-multi-stram is perfect! 👍

In alternative you can try this

// Custom logs
const logFile = "./project.log";
if (fs.existsSync(logFile)) {
    fs.unlinkSync(logFile);
}
const log = require('simple-node-logger').createSimpleLogger('project.log');
console.log = log.info;

[davidmarkclements]

@ro70 I ran your example it seems to work fine

$ cat ro70.js
var fs = require('fs')
var pinoms = require('pino-multi-stream')
var streams = [
  {stream: fs.createWriteStream('/tmp/info.stream.out')},
  {level: 'fatal', stream: fs.createWriteStream('/tmp/fatal.stream.out')}
]
var log = pinoms({streams: streams})

log.info('this will be written to /tmp/info.stream.out')
log.fatal('this will be written to /tmp/fatal.stream.out')
$ node ro70.js
$ cat /tmp/info.stream.out
{"pid":50241,"hostname":"Davids-MacBook-Pro.local","level":30,"time":1500284423795,"msg":"this will be written to /tmp/info.stream.out","v":1}
{"pid":50241,"hostname":"Davids-MacBook-Pro.local","level":60,"time":1500284423797,"msg":"this will be written to /tmp/fatal.stream.out","v":1}
$ cat /tmp/fatal.stream.out
{"pid":50241,"hostname":"Davids-MacBook-Pro.local","level":60,"time":1500284423797,"msg":"this will be written to /tmp/fatal.stream.out","v":1}

It may be a file permissions problem on your system, try changing the file paths to your local dir, also what version of node are you running?

pino-tee is for out of process log splitting, whereas pino-multi-stream is for in-process log splitting.

Generally we recommend any log processing be done out-of-process (we're working on clarifying our position in https://github.com/pinojs/pino/pull/267) - the reason is because it you want to keep you event-loop thread as free as possible for serving it's main purpose (e.g. answering http request or whatever)

So for a similar result with pino-tee it would be

var fs = require('fs')
var pino = require('pino')
var log = pino()

log.info('this will be written to /tmp/info.stream.out')
log.fatal('this will be written to /tmp/fatal.stream.out')

node ro70.js | pino-tee fatal ./fatal.out > all.out

I'm assuming here, but it's probably because you're attempting to split logs programatically, that @mcollina recommended you use pino-multi-stream which is built specifically for cases where constraints don't allow for piping logs between processes.

One other thing to point out, in both pino-multi-stream and pino-tee logs are split from at least the specified log - rather than only sending (say) info to a stream , it will be info, warn, error and fatal - in other words you're specifying min log level for a particular destination

@mantovanig send us a PR with benchmarks for your logger vs pino and we'll add them to the readme

[ro70]

@davidmarkclements On another machine my example worked. It was a file permission problem indeed. Thank you for your detailed description.

[doasync]

Log prettyfied to console + log WARN, ERROR, FATAL to separate files (pino-tee in child process):

const pino = require('pino');
const childProcess = require('child_process');
const PassThrough = require('stream').PassThrough;

const logPath = `${process.cwd()}/log`;
const logThrough = new PassThrough();
const log = pino({name: 'project'}, logThrough);

const child = childProcess.spawn(process.execPath, [
  require.resolve('pino-tee'),
  'warn', `${logPath}/warn.log`,
  'error', `${logPath}/error.log`,
  'fatal', `${logPath}/fatal.log`
], {
  cwd: __dirname,
  env: process.env
});

const pretty = pino.pretty();

pretty.pipe(process.stdout);
logThrough.pipe(child.stdin);
logThrough.pipe(pretty);

[davidmarkclements]

yah that'll do it - this is nice because its cross platform (vs, e.g. a bash script)

[autocorrectoff]

@davidmarkclements can I use @doasync's setup to write error and fatal to file, and info to console?