Kibana: Term breaks my field value into multiple values.

[rightedges]

My field value looks like "UzC1qjJMt_afb-gjOewA:9". Term breaks it into three different values "UzC1qjJMt_afb", "gjOewA" and "9". It looks pretty consistent for all the field values. It sounds like an issue with term. Can somebody confirm it? Thanks.

[dadoonet]

I think it's more a question on analyzer.
If you have default analyzer for this field, then doing facet on it will show strange results.

You should modify your mapping. See http://www.elasticsearch.org/guide/reference/mapping/

[rightedges]

Ah I'll look into that. Thanks!

[KlavsKlavsen]

how should one change the mapping?

[KlavsKlavsen]

or rather - is there a default recommendation for logstash users?

[ruckc]

Elasticsearch allows you to setup a default-mapping.json file in the
CONF_DIR. Below is the file I use for logstash version 1 and elasticsearch
0.90.x.

{
"_default_": {
"_all": { "enabled": false },
"_source": { "compress": true },
"properties" : {
"message" : { "type" : "string", "index" : "analyzed" },
"source_host" : { "type" : "string", "index" : "not_analyzed" },
"tags": { "type": "string", "index" : "not_analyzed" },
"@timestamp" : { "type" : "date", "index" : "not_analyzed" },
"type" : { "type" : "string", "index" : "not_analyzed" }
}
}
}

Curtis Ruck
Anytime: 210-857-1126

On Mon, Nov 4, 2013 at 6:28 AM, Klavs Klavsen [email protected]:

or rather - is there a default recommendation for logstash users?

—
Reply to this email directly or view it on GitHubhttps://github.com/elasticsearch/kibana/issues/364#issuecomment-27678861
.

[KlavsKlavsen]

@ruckc Thank you very much.
one note. AFAIK source_host is actually just called host in v1 (logstash 1.2+).. so should it say host instead?

[KlavsKlavsen]

so - if I set my host to type string - then it will stop trying to split hostnames such as "server-pre01" into two?

[ruckc]

No, the special juju is the 'not_analyzed'. Also, i use a custom pure-java
logstash replacement so i never fully switched to v1.

Curtis Ruck
Anytime: 210-857-1126

On Mon, Nov 4, 2013 at 9:23 AM, Klavs Klavsen [email protected]:

so - if I set my host to type string - then it will stop trying to split
hostnames such as "server-pre01" into two?

—
Reply to this email directly or view it on GitHubhttps://github.com/elasticsearch/kibana/issues/364#issuecomment-27687984
.

[pobzeb-relevantidea]

Not sure if this is still an issue but you can use "{field_name}.raw" now to get the non-analyzed version of the value. For example, if your field name is "host" you can use "host.raw".

[rashidkpc]

You can only use .raw if it is in your mapping. If you're a logstash user it usually will be.

[simianhacker]

If you are not using Logstash to process your logs but and you still want to have the .raw mappings automatically created for you nightly indices, you can run the following commands:

curl -o template.json https://raw.github.com/elasticsearch/logstash/v1.3.3/lib/logstash/outputs/elasticsearch/elasticsearch-template.json
curl -XPUT http://localhost:9200/_template/logstash [email protected]

This will setup the dynamic mapping for the logstash-YYYY.MM.DD indices. Keep in mind that the .raw mappings will only be present for the new indices.

Logstash 1.3+ does this automatically.

[brianlow]

Kibana users: for the Terms panel (e.g. pie graph) you can use my_field_name.raw. This should work out of the box. You don't need to set fields to not-analyzed. If you've customized your index names, just make sure they start with logstash-.

[SleeperSmith]

Eh, in logstash version 1.4 the mapping removed the multifield type and now I can't find the raw field. It's very frustrating. Help?

[Darpan205]

.raw works!!!!!!!! Cheerss..

[rinugoldjin]

Hi

I still have issues with the aggregation in Kibana 3.1.2, even after setting the fields as not_analyzed.

This is how the fields are defined

"cID":{"type" : "multi_field","fields" : {"cID": {"type": "string"},"raw" : {"type" : "string", "index" : "not_analyzed"}}},

Is my definition right or am I missing something?
Can somebody review this please? Thanks

[greenapplepark]

@Darpan205
How do you solves in kibana 4(beta 3)?
I didn't find any way to add .raw. Did you add it in "Json input"?
Could you give me a sample?

Thanks.

[radiumx3]

@greenapplepark Did you found out how to do it in the end ? I'm asking myself the same question

[greenapplepark]

@radiumx3
Yes. You can check these 2 links as reference.
http://stackoverflow.com/questions/26899447/kibana-error-in-displaying-some-data
https://github.com/elastic/logstash/blob/v1.3.1/lib/logstash/outputs/elasticsearch/elasticsearch-template.json

Try to put the template above and then you can get a *.raw field

[abtpst]

this does not seem to be working for fields that have an array/list type. how should i specify the analyzer for such fields?

http://stackoverflow.com/questions/35137022/elasticsearch-es-index-changes-the-mapping-when-message-is-pushed

please advise