Extracting fields from input file path logstash?

Question

I want to read my log files from various directories, like: Server1, Server2...

Server1 has subdirectories as cron, auth...inside these subdirectories is the log file respectively.

So I am contemplating of reading files like this:

input{
         file{            
                                     #path/to/folders/server1/cronLog/cron_log

                             path => "path/to/folders/**/*_log"
            }
    }

However, I am having difficulty in filtering them i.e to know that for which server (Server1) and logtype (cron), I must apply the grok pattern:

Eg: I thought of doing something like this

if [path] =~ "auth"{

grok{
            match => ["message", ***patteren****]
        }
    }else if [path] =~ "cron"{
        grok{
            match => ["message", ***pattern***]
        }

Above cron is for log file (not cronLog directory). But like this I also want to filter on server name as every server will have cron, auth,etc logs.

How to filter on both?

Is there a way to grab directory names from path in input ?? Like from here

path => "path/to/folders/**/*_log"

How should I proceed? Any help is appreciated?

Answer 1

its very straight forward, and almost exactly like in my other answer... you use the grok on the path to extract the pieces out that you care about and then you can do whatever you want from there

filter {
   grok { "path", "path/to/here/(?<server>[^/]+)/(?<logtype>[^/]+)/(?<logname>.*) }
   if [server] == "blah" && [logtype] =~ "cron" {
       grok { "message", "** pattern **" }
   }
}