Boto3: Empty DataPoints when getting s3 metrics from cloudwatch

[eliranbz]

Hi,
when using boto3 to get s3 metrics i get empty metrics,
I get the metrics via list_metrics(), I can see them on CloudWatch AWS Web Console, I've tried different time spans (from 1minute to 1 Day) - Always return empty.

Changed the units, the Statistics. bah.

cw = boto3.client('cloudwatch')
cw.get_metric_statistics(Namespace='AWS/S3',MetricName='BucketSizeBytes', StartTime=datetime.utcnow() - timedelta(days=1), EndTime=datetime.utcnow(), Period=86400, Statistics=['Sum'],Unit='Bytes',Dimensions=[{'Name':'BucketName', 'Value':'backup-bucket'}])

OUTPUT:

{u'Datapoints': [],
 u'Label': 'BucketSizeBytes',
 'ResponseMetadata': {'HTTPStatusCode': 200,
  'RequestId': 'a5c04a0e-4579-11e5-ab73-add86b87b8d9'}}

[kyleknap]

So here is what I did to get it to work:

from datetime import timedelta
from datetime import datetime

cw = boto3.client('cloudwatch')
bucket = 'mybucketfoo'

response = cw.get_metric_statistics(
    Namespace='AWS/S3',MetricName='BucketSizeBytes',
    StartTime=datetime.utcnow() - timedelta(days=2) ,
    EndTime=datetime.utcnow(), Period=86400,
    Statistics=['Average'], Unit='Bytes',
    Dimensions=[
        {'Name': 'BucketName', 'Value': bucket},
        {u'Name': 'StorageType', u'Value': 'StandardStorage'}
    ])


print(response)

The important points are that you need to take from this is: the second StorageType dimension and the start date needs to be more than one day from the current time. I could not find too much documentation in the CloudWatch docs about it. However based on the console, you need that StorageType as well to correctly view the data points. Also, the data point, for me, is delivered daily at 12 AM so my guess the range of a day you specified was too small. However, I was not able to find the official minimum span that you must specify.

Let me know if you have any more questions.

[eliranbz]

I have tried that, but it doesn't work. Not for a day, not for a month.
I'm not sure what else can go wrong

[kyleknap]

@DjxDeaf

Did you use the same code snippet as I did? If you did, more researching may be needed to determine why. Another thing you may need to watch out for is that the region for which you are making the cloudwatch client matches up with the region your s3 bucket is located in. If they do not match, that will cause the data points not too show up as well.

[JordonPhillips]

It works fine for me as well using @kyleknap 's snippet. I'm going to go ahead and close the issue since it has been over a month. Please re-open if that snippet does not work for you.

[ezeeetm]

This worked for me:

import boto3
from datetime import timedelta
from datetime import datetime

region = 'us-east-1'
cwWindow = 60

client = boto3.client('cloudwatch',region_name=region)
response = client.get_metric_statistics(
    Namespace='AWS/DynamoDB',
    MetricName='ConsumedWriteCapacityUnits',
    Dimensions=[
        {
            'Name': 'TableName',
            'Value': 'my-table-name'
        },
    ],
    StartTime=datetime.utcnow() - timedelta(minutes=cwWindow ),
    EndTime=datetime.utcnow(),
    Period=300,
    Statistics=['Average'],
    Unit='Count'
)

[nihitkaul]

hi folks,
The script was initially working for me, but has stopped working now and no matter what I try, I get an empty Datapoints list. Is there a limit per day? Any other ideas for what could be wrong?

[nihitkaul]

Ah! Got it to work by trying bigger time window. I had a timespan of 1 day which was working but it stopped and I had to increase it to 2 days.

[JohnONeill]

Not related to boto, but will likely prove useful to another person like me who's been frantically searching for why Datapoints is empty:

Make sure to double-check the permissions on your IAM role, and make sure that the credentials you're using for boto (i.e., those in ~/.aws/credentials) actually belong to that IAM user... 🤦‍♂️