grok multiple messages and process them with diffe

2019-07-25 15:27发布

问题:

I want to make a filter in Logstash(version 2.4) with different matches in the same grok. I would like to add different tags depending on the match. Basically, I receive three different message pattern: "##MAGIC##%message" "##REAL##%message" "%message" I am trying to do is:

 grok {
 match => {"message" => "##MAGIC##%{GREEDYDATA:magic_message}"}
 match => {"message" => "##REAL##%{GREEDYDATA:real_message}"}
 match => {"message" => "%{GREEDYDATA:basic_message}"}
 if [magic_message]{
    overwrite => [ "message"]  
    add_tag => ["Magic"]
 } else if [real_message]{
    overwrite => [ "message"]  
    add_tag => ["Real"]
 }else{
   overwrite => [ "message"]  
    add_tag => ["Basic"]
 }

But, I got this compile fails:

    The given configuration is invalid. Reason: Expected one of #, => at line 34, column 9 (byte 900) after filter {
  grok {
     match => {"message" => "##MAGIC##%{GREEDYDATA:magic_message}"}
     match => {"message" => "##REAL##%{GREEDYDATA:real_message}"}
     match => {"message" => "%{GREEDYDATA:basic_message}"}
     if  {:level=>:fatal}

回答1:

The logstash configuration syntax does not work like this.

This should work better (under the assumption that you want to replace message by magic_message/real_message):

grok {
    match => {"message" => [ "##MAGIC##%{GREEDYDATA:magic_message}",
                             "##REAL##%{GREEDYDATA:real_message}", 
                             "%{GREEDYDATA:basic_message}"]}
}
if [magic_message] {
    mutate {
        replace => { "message" => "%{magic_message}" }
        add_tag => ["Magic"]
    }
} else if [real_message] {
    mutate {   
        replace => { "message" => "%{real_message}" }
        add_tag => ["Real"] 
    }
} else {
    mutate {
        add_tag => ["Basic"] 
    }
}