Api: GetHistoricalStats querystring parameters daystart/dayend suggestion

[Negativitet]

Hi.

Regarding Endpoint: /Destiny2/{membershipType}/Account/{destinyMembershipId}/Character/{characterId}/Stats

Would it be too much work to change these query-strings daystart/dayend to take time into account as well?

I would like to be able to convert different time zones dates to their respective UTC times and use these new UTC date-times to get a users specific daily results.

Say for example:

The user is located in Brunei and it's currently 3 AM for him, he has done a lot of activities since 12 AM and want to check how he is doing for September 27.
However his activities are saved for what he has been doing on UTC September 26 16:00 to 19:00 UTC.
I send UTC dates daystart 2018-09-26 and dayend 2018-09-26 since I can't pass the times and he thinks: "I have not done these activities today, some of them I did before midnight".

If we could pass full UTC timestamps we could tailor data better for users.

Until then the only bad workaround is to poll every activity for as far back as needed until you cover the last day.

A small fix that can make some or if not every community application better and also save some database computation on Bungie.net's end. =)

Best Regards
Joakim

[vthornheart-bng]

Greetings!

Unfortunately, depending on your use case, we may not have a good solution for you. However, there are some options depending on how you access the data.

Just to get an idea of the use case, is your app/site:

A) Something that you're continually grabbing/updating for certain users in a batch process?
B) Something where the user will only realistically care about the activities for today?
C) Something where the user may care about a previous day's activities and you haven't been processing/storing stats for them as might be the case in (A) (for instance, they'd realistically go back to some date in a previous month to review that day's activities)?
D) Something else I'm not covering here?

If it's (A) or (B), then I'd recommend using the Activity History endpoint instead. The polling would indeed feel like a bad situation particularly in the case of (C) - but if you're either grabbing their data without their active input to do so or if the user is only going to care about today's activities, then fetching Activity History directly becomes a much more feasible option! If the user played 24 hours straight without sleeping, and averaged 10 minutes per activity (most activities tend to be played longer than that), if you were to ask for the last 150 activities you'd get everything even that extreme user did.

/{membershipType}/Account/{destinyMembershipId}/Character/{characterId}/Stats/Activities/

That will get you both more specific activity information in case you want to break it down for them, and more specific timestamps for each activity played!

Unfortunately, our Stats backend isn't set up to allow the Daily filter to account for dates. Unfortunately, the date bucketing that it does happens in processes closer to the game side that Bungie.Net doesn't have any direct control over, and the bucketing appears to happen explicitly from midnight at the start of the day UTC to midnight at the end of the day UTC. Not terribly useful to people who actually want to see their aggregate stats bucketed by their local time zone.

For your use case (and indeed, most use cases that I can think of where end users are actually going to look at this data), this bucketing is unfortunately insufficient... which leaves us with the Activity History option without a larger overhaul of the stats system and how it's storing that data.

My apologies for the inconvenience on this, and the bad news. I don't think we'll realistically ever have the time allocated to us in order to provide more flexible daily aggregate stats. But hopefully you're in one of the situations mentioned above where Activity History is a more realistic approach.

[Negativitet]

Hi! Thanks for the fast reply.

I have a very specific usage case where I try to supply every user that connects to my service with the following 4 views.

Lifetime (easy since it is GetHistoricalStats base view): /Destiny2/{membershipType}/Account/{destinyMembershipId}/Character/0/Stats/?periodType=2

Today (the troublesome): A representation for how the current day is going for them, they cannot check earlier dates than "today" but they are expecting that Today is 00:00 to 23:59 in their timezone, and not UTC.

Stream Session (same problem as today since it can be between specific hours): I gather their last broadcast time from Twitch in UTC and show them statistics between that timestamp and current UTC timestamp. (still troublesome if GetHistoricalStats only does dates)

Most Recent (also easy - just grab the latest PVP activity every update): Destiny2/{membershipType}/Account/{destinyMembershipId}/Character/{characterId}/Stats/Activities/?mode=5&count=1&page=0

Is there a way to limit Destiny2.GetActivityHistory based on datetime or like for instance send the uniqueId for the latest activity cached and only get activites that have occured after that?
If that were the case the entire _bashing_ of GetActivityHistory would be less intense =P

[Negativitet]

I'll look into replacing Today/Stream Session with Activities History instead.

I've been looking and do you know the exact limit for amount you can put in the "count" querystring?

And also, I guess there is no way of getting Activities independent of Character?

[ArkahnX]

Count ends up being 150-200, and is dependant on character.

If you can get a users timezone, which you might be able to get from their request information, you can format the UTC date to their timezone and grab 24hours that way.

Or you could base your 24 hours on the daily reset time for Destiny, which is a fixed UTC time

[vthornheart-bng]

Sweet - yeah, for your use case, I think setting an appropriately high count will be the best alternative!

Our per-page request limit is 250 activities - which is enough that even a streamer who never sleeps and constantly cycles through quick PVP matches would never play that many activities in a day. They'd have to average ~6 minutes per activity without stopping or transition time for the full 24 hour period. If you take into account that matchmaking and post-match countdowns/PGCR screens can take more than a minute (sometimes even 2 or 3), it's infeasible for someone to get more than that many activities in a day unless they were purposely leaving and rejoining matches as soon as they possibly could, and doing so without sleep or rest.

Indeed however, there is the problem of multiple characters. We don't have an endpoint that allows you to get a user's activity history across all of their characters, so if you want to give them an account-wide view all in one screen/feature you would have to make up to three of these requests and coallate the data in whatever manner you wish. But at least you'll still be realistically limited to having to do three requests rather than 3 * N Pages!

How often are you considering hitting the endpoint? And how many users would be actively using it?

You may be able to take advantage of the timing numbers above/realistic time it takes to complete an activity, along with the fact that our cache will provide a delay of up to a minute before new activity data will be returned, to limit your requests after the initial one. For instance, after the initial 250 activity request to get you up to date with current time, you could begin asking for - as an example - only the most recent 5 activities once every 5 minutes. It'd be effectively impossible for them to play more activities than that over any given 5 minute time period and have the stats system actually track it, which would result in a decrease to your bandwidth usage and requirements, and would let you continue to keep up to date without a gap. You could make it more generous than that, or tweak the numbers if you'd like... but that could be a strategy to keep your successive requests nice and lean.

[Negativitet]

I'm not sure how high the amount of users will be.
Currently without Destiny 2 on the list, the service is pulling somewhere between 3-5 new streamers every day. And that is mostly from one game.

There are a lot of requests for this game to make it onto my platform.
But I'll figure out some smart update cycles that don't hammer the API that much. Hopefully =)

[vthornheart-bng]

Cool, sounds good! I'll close this out for now - let us know if you run into any issues!