Kibana: Unique count wrong when table includes terms query sub-aggregation

Created on 1 Apr 2015  路  9Comments  路  Source: elastic/kibana

Running Kibana 4.0.1. I create a Data Table (for readability) on a search and add two aggregations, one Count and one Unique Count on a particular field. Just doing this looks sane and normal:

normal_unique_count

Now I add a sub-aggregation using the terms option. On some (not all!) values in the terms there are unique counts higher than the non-unique counts:

broken_unique_count

Smells related to #3314? In my example are the unique counts of the business_id field getting mis-correlated with the proper partner_id field from the terms sub-aggregation? I tried reproducing with other sub-aggregations without any luck. I can also reproduce using different keys for the terms query. In fact the exact same miscounting occurs even for fields with only a single value (in this case all documents have active:True):

alternate_broken_terms_subagg

feedback_needed notabug

Most helpful comment

@striglia (and anyone else that stumbles upon this), this is a little old, but I ran into a similar problem. The unique count was higher than the count.
The issue is with Elastic Search and not Kibana. And it's sort of a feature rather than a bug. I discovered that the unique count aggregation is an estimate, not an accurately counted value.
To get a more correct value, add a something like: {"precision_threshold": 1000} to the "JSON Input" box for the aggregation.

All 9 comments

Can you grab the elasticsearch response for this from the browser console?

The response for the mis-counting visualization is below. I've snipped out all but one (miscounting) bucket for privacy...let me know if this is problematic and I can figure out how to include more.

{                                                                               
   "responses":[                                                                
      {                                                                         
         "took":9,                                                              
         "timed_out":false,                                                     
         "_shards":{                                                            
            "total":5,                                                          
            "successful":5,                                                     
            "failed":0                                                          
         },                                                                     
         "hits":{                                                               
            "total":82846,                                                      
            "max_score":0.0,                                                    
            "hits":[                                                            

            ]                                                                   
         },                                                                     
         "aggregations":{                                                       
            "2":{                                                               
               "doc_count_error_upper_bound":0,                                 
               "sum_other_doc_count":0,                                         
               "buckets":[                                                      
                  {                                                             
                     "key":31,                                                  
                     "doc_count":267,                                           
                     "1":{                                                      
                        "value":281                                             
                     }                                                          
                  }                                                             
               ]                                                                
            }                                                                   
         }                                                                      
      }                                                                         
   ]                                                                            
}

I guess I'm not seeing what is incorrect from a Kibana standpoint? Do you feel the result of the aggregation is incorrect? If so you might talk to the Elasticsearch team, but Kibana appears to be displaying the data returned from elasticsearch correctly.

Closing for lack of feedback

@striglia (and anyone else that stumbles upon this), this is a little old, but I ran into a similar problem. The unique count was higher than the count.
The issue is with Elastic Search and not Kibana. And it's sort of a feature rather than a bug. I discovered that the unique count aggregation is an estimate, not an accurately counted value.
To get a more correct value, add a something like: {"precision_threshold": 1000} to the "JSON Input" box for the aggregation.

@tschundler Could you give instructions to do that? I'm new to both Kibana and Elasticsearch.

Found it! Thanks

{"precision_threshold": 1000} saved me as well .

Thank you @tschundler

This should be in the docs !

Was this page helpful?
0 / 5 - 0 ratings

Related issues

treussart picture treussart  路  3Comments

MaartenUreel picture MaartenUreel  路  3Comments

snide picture snide  路  3Comments

mark54g picture mark54g  路  3Comments

timroes picture timroes  路  3Comments