Elasticsearch-net: Documents have @timestamp = null

Created on 15 Aug 2016  路  2Comments  路  Source: elastic/elasticsearch-net

I am using NEST to retrieve filebeat logs from ELK. I can see logs in Kibana with @timestamp but when i retrieve documents with NEST timestamp is null

This is object to retrieve documents:

    public class FileBeatDto
    {
        public DateTime timestamp { get; set; }
        public string datetime { get; set; }
        public DateTime received_at { get; set; }
        public int offset { get; set; }
        public string version { get; set; }
        public string input_type { get; set; }
        public int count { get; set; }
        public Beat beat { get; set; }
        public string host { get; set; }
        public string source { get; set; }
        public string message { get; set; }
        public string type { get; set; }
        public object fields { get; set; }
        public List<string> tags { get; set; }
    }

This is the query:

            var result = elk.Search<FileBeatDto>(x => x
                .Index("filebeat-*")                
                .AllIndices()
                .Type(type)                               
                .Query(q => 
                    q.Match(qs => qs.Field("fields.asset_tag").Query(asset_tag))  &&
                    q.Match(qs => qs.Field("message").Query(filter))
                    )
                .Take(lines)
                );

NEST/Elasticsearch.Net version: 2.4.3

Elasticsearch version: 5.0-alpha5

Description of the problem including expected versus actual behavior:

@timestamp not retrieved by NEST

Most helpful comment

This is most likely because it's @timestamp in the index, but only timestamp is inferred based on the POCO property name.

A few ways to deal with this...

Use InferMappingFor<> on your ConnectionSettings to "rename" the property:

new ConnectionSettings()
    .InferMappingFor<FileBeatDto>(m => m
        .Rename(f => f.timestamp, "@timestamp")
    )

or use the Date or JsonProperty attributes:

[Date(Name="@timestamp")]
public DateTime timestamp { get; set; }
[JsonProperty("@timestamp")]
public DateTime timestamp { get; set; }

This is likely the same issue as #2207.

Also, you're using NEST 2.4.3 which technically is not compatible with ES 5.0.0-alpha5. See the compatibility matrix. You should jump on the latest alpha release of NEST instead.

All 2 comments

This is most likely because it's @timestamp in the index, but only timestamp is inferred based on the POCO property name.

A few ways to deal with this...

Use InferMappingFor<> on your ConnectionSettings to "rename" the property:

new ConnectionSettings()
    .InferMappingFor<FileBeatDto>(m => m
        .Rename(f => f.timestamp, "@timestamp")
    )

or use the Date or JsonProperty attributes:

[Date(Name="@timestamp")]
public DateTime timestamp { get; set; }
[JsonProperty("@timestamp")]
public DateTime timestamp { get; set; }

This is likely the same issue as #2207.

Also, you're using NEST 2.4.3 which technically is not compatible with ES 5.0.0-alpha5. See the compatibility matrix. You should jump on the latest alpha release of NEST instead.

thanks! adding the attribute to POCO works and can retrieve timestamp now.

Closing this and #2207.

Was this page helpful?
0 / 5 - 0 ratings