Extracting fields from input file path logstash?

cron, elasticsearch, kibana, logstash

Solution

its very straight forward, and almost exactly like in my other answer... you use the grok on the path to extract the pieces out that you care about and then you can do whatever you want from there

filter {
   grok { "path", "path/to/here/(?<server>[^/]+)/(?<logtype>[^/]+)/(?<logname>.*) }
   if [server] == "blah" && [logtype] =~ "cron" {
       grok { "message", "** pattern **" }
   }
}

Problem

I want to read my log files from various directories, like: `Server1, Server2`... `Server1` has subdirectories as `cron, auth`...inside these subdirectories is the `log file` respectively. So I am contemplating of reading files like this: ``` input{ file{ #path/to/folders/server1/cronLog/cron_log path => "path/to/folders/**/*_log" } } ``` However, I am having difficulty in filtering them i.e to know that for which server (`Server1`) and logtype (`cron`), I must apply the `grok` pattern: Eg: I thought of doing something like this ``` if [path] =~ "auth"{ grok{ match => ["message", ***patteren****] } }else if [path] =~ "cron"{ grok{ match => ["message", ***pattern***] } ``` Above `cron` is for log file (not cronLog directory). But like this I also want to filter on `server name` as every server will have `cron`, `auth`,etc logs. How to filter on both? Is there a way to grab directory names from `path` in input ?? Like from here ``` path => "path/to/folders/**/*_log" ``` How should I proceed? Any help is appreciated?

Original source