Amplify-js: Ability for `Logger` to ship logs to CloudWatch Logs

[ferdingler]

Is your feature request related to a problem? Please describe.
As a developer, I would like for Amplify to support log shipping to CloudWatch logs via the use of Logger. So I can send certain logs (like errors or stack traces) to my AWS account for future analysis.

Describe the solution you'd like
This feature would leverage the Cognito integration such that log shipping is done in a secure fashion with temporary credentials. I would like for the Logger to take care of sending logs in batches in the background and support a local cache for lost connectivity.

Describe alternatives you've considered
I have considered using existing npm packages like https://www.npmjs.com/package/winston-cloudwatch.

[blairtaylor]

We would find this very useful as well.

[jkeys-ecg-nmsu]

I asked an AWS trainer about this, and he said it would probably be more flexible to ship logs to S3 so they can be more easily loaded into various other services (Athena, Redshift, EMR).

That being said, I am +1 for shipping application logs to one service or the other as an option for the Logger class.

FWIW I tried prototyping it myself just by extending Logger (don't own the code so I can't currently share it), and the ContinuationToken requirement to ship logs in a specific order combined with the fact that Logger is usually not a global object but is typically instantiated per-file for the sake of making the application logs easier to read, and it's not a trivial piece of code to write.

[dbhagen]

The route I'm taking on this is to add a lambda function that the logger can send content to do that will perform the CloudWatch logging for me.

amplify function add

And then I'm looking at https://www.npmjs.com/package/lambda-log to handle log shipping for me.
If I limit access to the function to only authed endpoints, I should protect myself from spam.

I'll report back when I have it working. Obviously, it would be great if the built-in Logger could have an option for a mirroring destination. I'm patching that into the Amplify-JS library I'm running in my app. If that turns well, I'll see if I can get it in as a PR.

[ferdingler]

The route I'm taking on this is to add a lambda function that the logger can send content to do that will perform the CloudWatch logging for me.

amplify function add

And then I'm looking at https://www.npmjs.com/package/lambda-log to handle log shipping for me.
If I limit access to the function to only authed endpoints, I should protect myself from spam.

I'll report back when I have it working. Obviously, it would be great if the built-in Logger could have an option for a mirroring destination. I'm patching that into the Amplify-JS library I'm running in my app. If that turns well, I'll see if I can get it in as a PR.

Sounds good! @dbhagen , a couple of comments though:

Be aware of the Lambda concurrency limits; Log shipping may be an operation that happens fairly frequently and depending how many clients (frontend apps) you have, you may be hitting your lambda function a lot: https://docs.aws.amazon.com/lambda/latest/dg/limits.html .

I also believe that the biggest challenge in doing Log shipping is not so much on the backend, but in the client library to do error handling, buffering to send logs in batches, local cache to deal with lost of connectivity, retry backoffs, etc. In your approach, you still need to handle all of that in the client.

[Giddyrino]

it boggles my mind there isn't some sort of built in support to access generated application logs in the amplify console.

it's like they didn't expect users would want to look at the logs if their deployment failed.

[karrettgelley]

Any update on this? My current workaround is to create an error model on my schema and create records whenever I hit a catch block. For instance, I added

type Error @model {
  error: String
  lineNumber: Int
  fileName: String

Then when I hit a catch block, I hit up my appsync api:

try {

} catch(e) {
  API.graphql(graphqlOperation(createError, {
    input: {error: e, lineNumber: 5, fileName: 'App.tsx'}
  }));
}

But it's hacky and error prone.

[blairtaylor]

@karrettgelley We are doing something very similar. It seems like a step backwards compared to other logging frameworks.

[CharanRoot]

do we have any update this feature ?

[julioxavierr]

+1

This is a must have

[michaeljcalkins]

We were also wanting to use Logger as a way to report on crash metrics in our React app.

Right now we have implemented a class resembling winston-cloudwatch that uses AWS.CloudWatchLogs.PutLogEvents with some custom code to manage batching messages and sequence tokens. 

I'd love to see our analytics and logging come from Amplify so that everything is in one place.

[simone1999]

Hi,
I have the same issue.
But can't amplify analytics be used for this?
https://docs.amplify.aws/lib/analytics/getting-started/q/platform/js

[HP4k1h5]

i followed the Logger guide and am developing an application locally. i don't know where/if my logs are written to. the guide does not indicate where these logs get written to, nor how to configure output location.

for reference, i am only importing the aws-amplify package and am not using the cli. does anyone know how to configure this even for local testing, or where these logs are getting written?

[xavierraffin]

@HP4k1h5 the logs are just console.log visible in Chrome inspector for example.
This is why this ticket ask to be able to set a Amazon Cloudwatch destination.

[HP4k1h5]

@xavierraffin thanks for this information, but i don't see any logs in the dev console at all. im not getting any errors either. so i believe i have instantiated the Logger correctly. all of my console.logs are there and the app is working as expected and not throwing any errors of any kind.

im also wondering what the point of having a wrapper around console is, if Logger just dumps messages to the dev console? i agree with the ask to have Logger ship to CloudWatch since that is what i would ultimately like as well.

[SourceCode]

The libraries winston and winston-cloudwatch may be useful for the common challenges around logging at scale.

https://github.com/winstonjs/winston

https://github.com/lazywithclass/winston-cloudwatch