Druid: Implement 'last' and 'first' aggregator

[gauravkumar37]

Similar to max/min aggregators, it would be useful to implement a first/last aggregator which could be specified at index-time thereby only indexing the first or last row sent to Druid.

Currently, the only way to achieve this is to store every row in Druid and use a select query to get the first or last row ordered by timestamp.

[vogievetsky]

This would be great. I would advocate making the aggregates 'firstBy' and 'lastBy' so that you could do: FIRST_BY(revenue,__time) to get first by time or FIRST_BY(revenue,my_id) to get it by my_id there should also be a way to define the compare on my_id (lexiographic, alphanumeric) and maybe some other comparison options like how much space to allocate for string comparisons.

Related to: https://github.com/druid-io/druid/issues/2989

Also to add some color for use cases:

https://groups.google.com/forum/#!topic/druid-user/nCghqQBhUC8

https://groups.google.com/forum/#!topic/imply-user-group/rhsQRC5v5G8

[gianm]

Tagged as Easy assuming we are just doing "by time" at first.

[vogievetsky]

@gianm what if the user has to provide explicit amount of bytes to use for comparison?

[gianm]

It looks like nobody has asked for by-anything-other-than time yet, and doing by-time is simpler…

[vogievetsky]

a good start for sure.

[gianm]

This would also probably end up being "firsty" and "lasty" in that if there are multiple events with the same max timestamp, one of them will be picked arbitrarily as the "last" one.

[ryryguy]

+1

Here's my use case: https://groups.google.com/forum/#!topic/druid-user/5ld1gAq6D_g

I think for my case time would also be adequate.