Google-cloud-node: datastore: get stats / total / count?

Created on 25 Feb 2015 · 7Comments · Source: googleapis/google-cloud-node

how to get stats / total / count information in dataset?

question datastore

Source

raitucarp

Most helpful comment

Datastore has statistics which are generated on a daily basis. The Datastore documentation has information about all the different statistics about your dataset.

For example to programmatically get your total number of entities and their total size:

query = dataset.createQuery(['__Stat_Total__'])
dataset.runQuery(query, function(err, entities, endCursor) {
  if (err) {
    return;
  }
  console.log('Number of entities:', entities[0].data.count)
  console.log('Total entity size:', entities[0].data.entity_size)
});

You can also see this information in a UI by visiting:

https://console.developers.google.com/project/<YOUR-PROJECT-ID>/datastore/stats

pcostell on 21 Apr 2015

👍2

All 7 comments

@pcostell - are you able to help @raitucarp?

stephenplusplus on 21 Apr 2015

Datastore has statistics which are generated on a daily basis. The Datastore documentation has information about all the different statistics about your dataset.

For example to programmatically get your total number of entities and their total size:

query = dataset.createQuery(['__Stat_Total__'])
dataset.runQuery(query, function(err, entities, endCursor) {
  if (err) {
    return;
  }
  console.log('Number of entities:', entities[0].data.count)
  console.log('Total entity size:', entities[0].data.entity_size)
});

You can also see this information in a UI by visiting:

https://console.developers.google.com/project/<YOUR-PROJECT-ID>/datastore/stats

pcostell on 21 Apr 2015

👍2

@pcostell Do you think it'd be worth adding some syntactic sugar around this and offering as a feature?

dhermes on 21 Apr 2015

I'm not totally sure, but my guess would be that this is not the sort of thing that is useful to most developers from an API perspective. My guess is that in general an application doesn't need to know about it, so it would only be useful if the developer wanted to have a different API than that provided by the Cloud Console.

pcostell on 21 Apr 2015

👎2

@pcostell sorry for asking in the super old issue but do you know if the Datastore Emulator supports this stats queries?

malditogeek on 7 Mar 2017

Seems like the stats are not generated on a daily basis. Both the API and the UI dashboard mentioned by @pcostell above throw the stale info. My dashboard says 'Last updated 2 days ago — May 5, 2017 at 12:42:27 AM UTC-7'. It also mentions 'Statistics are generated every 24–48 hours. In addition to the kinds used for your entities, statistics also include system kinds, such as those related to metadata. System kinds are not visible in your kinds menu. Note that statistics may not be available for very large datasets.'
What is a very large dataset ? I have loaded approx ~52k entities in Datastore and want to verify the actual count without waiting until the stats get regenerated again. How can i achieve this ?