Pino: Upload logs to Cloudwatch

Created on 25 Apr 2016  Â·  13Comments  Â·  Source: pinojs/pino

We need instructions (and probably a nice module) to easily upload logs to cloudwatch. I know there is the cloudwatch daemon, so we might just list that. But having a little node-something can be nice.
Maybe we can just do tail-to-cloudwatch, and be done with it.

See
https://github.com/mirkokiefer/bunyan-cloudwatch
https://www.npmjs.com/package/winston-cloudwatch

ecosystem help wanted

Most helpful comment

Originally I was using pid and hostname from the log lines but it means I'd have to parse the input.

By avoiding the JSON.parse() it will work with any text-based log output.

Perhaps an option to do that is worth adding.

All 13 comments

I think the easiest way to maintain our priority of performance first is to create this to run separate to the process, as you say like a tail.

Along with that, it could be used for other logging systems also - speaking of which is there no independent thing right now that does this?

There is the cloudwatch agent: http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/AgentReference.html.

There is no "nodish" thing to do, and no easy way to display those logs from a node point of view.

This might be helpful: https://github.com/nearform/cloudwatchlogs-stream. However there are a few issues/prs to be handled. cc @mcdonnelldean.

@mcollina I've nobody free to put on this right now but I can add it to the list of stuff to look at if you wish?

Is this not solved by pino-socket?

Is it? can you put in a little example?
Il giorno lun 30 mag 2016 alle 22:55 James Sumners [email protected]
ha scritto:

Is this not solved by pino-socket?

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/mcollina/pino/issues/33#issuecomment-222553417, or mute
the thread
https://github.com/notifications/unsubscribe/AADL44uGebOzadiVNgSyEYOzg6LRn1wnks5qG06lgaJpZM4IPFww
.

I don't know for sure, though. I've never used that service. But if it just accepts any sort of data on a listening port, then it is solved and the example is already in the readme.

I might be able to work on this the following weekend. Not fully sure but I'll try my best. I think our best bet is fix: https://github.com/nearform/cloudwatchlogs-stream and use it here (AKA improve the documentation). Input is welcome.

Just wondering on status of this, what do we need to do to box this off?

I know this is an old issue, but please feel free to test out this one:

https://github.com/dbhowell/pino-cloudwatch

Looks great to me (I don't have any way to test it).

Why not use the process.id and hostname from the log lines?

Originally I was using pid and hostname from the log lines but it means I'd have to parse the input.

By avoiding the JSON.parse() it will work with any text-based log output.

Perhaps an option to do that is worth adding.

@dbhowell amazing work, if you'd like to add it to known transports (https://getpino.io/#/docs/transports?id=known-transports) send a PR!

Was this page helpful?
0 / 5 - 0 ratings

Related issues

callumlocke picture callumlocke  Â·  4Comments

NicolaiSchmid picture NicolaiSchmid  Â·  4Comments

glensc picture glensc  Â·  3Comments

yoshuawuyts picture yoshuawuyts  Â·  5Comments

stephenmathieson picture stephenmathieson  Â·  6Comments