I'm wondering if there is a way to have a chart of total hits over time. Here is an use case:
- Each time a new user is registered, a log message is sent, hence there is 1 hit
- I would like to see the chart of user growth during a period of time, I'm thinking of a histogram with total hits in that period.
But I don't know how to do this in histogram configuration, any help please?
passkey1510
All 96 comments
Same question and the same purpose :+1:
kryachkov
on 1 Dec 2013
Did anyone do this before?
kryachkov
on 7 Dec 2013
Why can't you use the provided histogram? In the log message you have the timestamp, and therefor each time you have a document in ES is a new event, so you could use the count option for the chart value; With this configuration (the same you see in kibana in the events over time graph, for each interval of time ES will count how many events (i.e registrations) occurred and plot that value in the histogram).
For instance, I've a graph that looks like this:
In this case in my log file every time a query hits the search server (which I'm monitoring) a log line with a lot of information is stored in ES, and in kibana (in this graph) I show how many events (i.e queries or registrations) are registered.
jorgelbg
on 9 Dec 2013
Standard histogram is not enough becase it shows number of events that came in specified moment of time (for example your histogram shows that at 00:00 10-28 200 events were logged), we need to see total growth of events count overtime. Thus chart will be always growing.
kryachkov
on 9 Dec 2013
@kryachkov :+1:. Yes, that is what we would like to have. A simple use case is when you want to see how your total user growth during a month.
passkey1510
on 9 Dec 2013
For better understanding, this how it should behave:
This is simple histogram: http://goo.gl/bI6mwP
Tick "Cumulative" in "Graph options" on the right. This is what we are seeking for
kryachkov
on 11 Dec 2013
Right now there is no elasticsearch aggregation that makes this possible. It is possible you could fake it by doing a histogram facet for your current time range, and then a query facet for everything before it. Then add the before number to the first bucket and totaling the buckets sequentially, adding the result of the previous bucket to the next.
rashidkpc
on 11 Dec 2013
I think I got the idea but struggle with the "add the before number to the first bucket and totaling the buckets" part. I have one histogram facet and one query facet like (time<"at"). How do I access the result of the query as a "field" I can then add to the histogram
Francis88
on 19 Feb 2014
+1
dblado
on 27 Feb 2014
+1
oliviervaussy
on 14 Mar 2014
+1
Francis88
on 14 Mar 2014
+1
galarragas
on 2 Apr 2014
Any updates on this functionality?
RickHoving
on 28 Apr 2014
+1
skymeyer
on 5 Jun 2014
This would require a method for reducing/combining buckets in Elasticsearch. I believe this is being worked on and we might be able to make it work in the future.
rashidkpc
on 7 Oct 2014
+1
tcucchietti
on 24 Nov 2014
+1
sbehrens
on 26 Nov 2014
+1
TobiasOtt
on 4 Dec 2014
+1
atulsm
on 15 Dec 2014
+1
ghost
on 3 Jan 2015
+1
Is this achievable already in Kibana 4?
Lloydshove
on 5 Jan 2015
+1
the-fine
on 12 Feb 2015
+1
mantree
on 21 Feb 2015
+1
moshe
on 23 Feb 2015
+1
nickmarx12345678
on 12 Mar 2015
+1!
ryanbrink
on 17 Mar 2015
+1
Astn
on 31 Mar 2015
+1
cesozgen
on 31 Mar 2015
+1
fpbouchard
on 9 Apr 2015
+10!
jechol
on 23 Apr 2015
+1
Zhenyzhou
on 24 Apr 2015
+1
voz
on 24 Apr 2015
+1
advbackoffice
on 27 Apr 2015
+1
oysstr
on 4 May 2015
+1
ArthurHub
on 5 May 2015
+1
pycke
on 14 May 2015
+1
LRParser
on 15 May 2015
+1
suttang
on 18 Jun 2015
+1
damienmollard
on 24 Jun 2015
+1
gambrell
on 15 Jul 2015
+1
siccovansas
on 28 Jul 2015
+1
kajisha
on 5 Aug 2015
+1
ahmadnazir
on 5 Aug 2015
+1
sgripon
on 6 Aug 2015
+1
li-go
on 12 Aug 2015
+1
denvazh
on 13 Aug 2015
+1
RavenXce
on 14 Aug 2015
:+1:
Bertg
on 3 Sep 2015
+1
deuscapturus
on 9 Sep 2015
+1
kwatkins
on 18 Sep 2015
This is now possible within Elasticsearch using the cumulative sum aggregation.
See #4584.
lukasolson
on 29 Sep 2015
+1 to having this exposed in KB :D
markwalkom
on 6 Oct 2015
I think is possible to extend use cases to more general ones... We are monitoring concurrent use of some resources (number of concurrent licenses in use in a license server, number connection in use in a pool, ...) for each client that connects/disconnects to the resource we have a line with a +1 / -1 (or even any other numbers)
We麓re summin up the figures in logstash, but is far from ideal. It would be nice to get current use for this resources with Kibana / ES.
ruria
on 12 Oct 2015
+1
xdzhou12
on 10 Dec 2015
+1
kingtaj
on 11 Dec 2015
+1
cynial
on 15 Dec 2015
+1
e8mn
on 27 Dec 2015
+1
kaib
on 29 Dec 2015
You can do this with Timelion, as well as derivatives and just about everything else pipeline aggs can do: https://www.elastic.co/blog/timelion-timeline
rashidkpc
on 4 Jan 2016
Sweet! :-) Nice work @rashidkpc!
e8mn
on 5 Jan 2016
@rashidkpc wow, impressive!
voz
on 5 Jan 2016
+1
hatutah
on 11 Jan 2016
- 1
rachitpuri06
on 13 Jan 2016
+1
dsudduth
on 21 Jan 2016
@rashidkpc - thanks for recommending Timelion! You're correct, it was very easy to get the cumulative sum. Do you know if there is a way to embed the visual into a Kibana Dashboard?
dsudduth
on 21 Jan 2016
+1
eabovsky
on 3 Feb 2016
+1
jgeewax
on 6 Feb 2016
+1
pavankumarkatakam
on 11 Feb 2016
+1
macteo
on 28 Feb 2016
+1
ConanChou
on 29 Feb 2016
+1
PasghettiCode
on 7 Mar 2016
+1
johnarundellewis
on 15 Apr 2016
+1
digitalpacman
on 5 May 2016
+1
preetis
on 6 May 2016
+1
wamsea
on 10 May 2016
:+1:
pemontto
on 10 May 2016
+1
m-breen
on 24 Jun 2016
+1
Greybird
on 24 Jun 2016
+1
ebertti
on 25 Jun 2016
For those interested for just cumulative sum graphs. I have made a port of a kibana plugin, that can do that. https://github.com/patrickkusebauch/kibana-plugin-line-sg
patrickkusebauch
on 25 Jun 2016
+1
leeway23
on 1 Jul 2016
+1
Kudryavets
on 20 Jul 2016
+1
liucheng98
on 9 Aug 2016
+1
JulieZY
on 10 Aug 2016
I was wondering if there is a way to make the cumulative "global".
I managed to get a date histogram of cardinality over time and use cumulative_sum over that, but the problem is that the cardinality is only true to that particular interval of my date histogram.
andremilk
on 15 Aug 2016
Wow, is this still not possible although elasticsearch has support for over a year now?
EDIT: Just tried Kibana 5 alpha 5 and even there is no support for pipeline aggregations. Can someone explain to me why such an important and basic feature is not being worked on?
T3rm1
on 23 Aug 2016
+1
jasonmhead
on 31 Aug 2016
+1
pwiatrowski
on 31 Aug 2016
+1
epotocko
on 15 Sep 2016
+1
brooksandrew
on 21 Sep 2016
+1
olivierlambert
on 26 Sep 2016
+1
czimamori-daichi
on 3 Oct 2016
+1
MohMaz
on 22 Oct 2016
+1
Dom-nik
on 22 Dec 2016
Im looking something similar, calculate the time over threshold, in a day histogram,
for example: how many hours was the value over 15. ten minutes, 90minutes..
is it posible ?
ghost
on 31 May 2017
In Kibana 5.5.2, using Cumulative Sum aggregation matrix I am able to plot hourly hits vs total hits count on area chart:
suyogdilipkale
on 15 Sep 2017
Related issues
TiNico22
路
87Comments
stigdescamps
路
88Comments
hvisage
路
170Comments
srl295
路
104Comments
elvarb
路
71Comments
Most helpful comment
In Kibana 5.5.2, using Cumulative Sum aggregation matrix I am able to plot hourly hits vs total hits count on area chart:
