Logstash filter timestamp from log message

elasticsearch, logging, logstash, timestamp

Solution

I managed to get the result I was looking for by doing the following:

    input {
  file {
    path => "C:\Dev\sample.log"
    start_position => beginning
  }
}
filter{
    grok {
       match => [ "message", "%{TIMESTAMP_ISO8601:logdate}" ]
    }       
    date {
        match => [ "logdate", "YYYY-MM-dd HH:mm:ss,SSS"]
    }
}
output {
  elasticsearch { host => localhost
                  index => "test"
                }
  stdout { codec => rubydebug }
}

So, this could be an answer to my question and I hope it can help someone that got stuck too. However I still don't understand why my previous example (in the question) does not give me the (which I think is) the correct output? I would really appreciate if someone could give me some explanation. Many thanks!

Problem

I have been following the Logstash tutorial and created the following config file for test purposes: ``` input { file { path => "C:\Dev\sample.log" start_position => beginning } } filter{ date { match => [ "logdate", "YYYY-MM-dd HH:mm:ss,SSS" ] } } output { elasticsearch { host => localhost index => "test" } stdout { codec => rubydebug } } ``` However, the only fields that are output are "message", "@version", "@timestamp", "host" and "path". No "logdate" is retrieved. I have searched for a while and I saw people having the same problem given wrong date format, but I checked mine with "Joda-Time" just as Logstash tutorial recommends. Thank you for your help.

Original source