Sagemaker-python-sdk: Retrieve Sagemaker instance metrics from cloudwatch via CLI or API

[adiell]

Not sure if this is the right place, but I'll try:

I've got a training job running on Sagemaker. I would like to retrieve instance metrics like MemoryUtilization etc by CLI or boto3 client.

Obviously I can see them in the console. However, I cannot see them in the CLI/API. For example, when running:

aws cloudwatch list-metrics --namespace "AWS/SageMaker"

I can see only metrics regarding endpoint invocation but not any training job related metrics.

Any idea?

Thanks!

[ajaykarpur]

Hi @adiell, thanks for your question. I'd recommend opening an issue on the AWS CLI repository (https://github.com/aws/aws-cli) for questions related to CLI functionality.

As far as the sagemaker-python-sdk is concerned, you may want to take a look at these pages for information on training job metrics:

• https://sagemaker.readthedocs.io/en/stable/overview.html#training-metrics
• https://docs.aws.amazon.com/sagemaker/latest/dg/training-metrics.html

[laurenyu]

closing due to lack of activity.