Combining multiple events in Logstash

问题:

I have a Logstash configuration where I'm reading simple lines from a graphite input (but if that helps it might as well just be tcp) and i'm forwarding them to RabbitMQ via AMQP.

input {
  graphite {
    host => localhost
    type => carbon
    port => 22003
  }
}

output {
  rabbitmq {
    codec => json
    host => 'localhost'
    port => 5672
    user => 'guest'
    password => 'guest'
    vhost => '/'
    exchange_type => topic
    key => '%{type}'
    persistent => true
    durable => true
    ssl => false
    verify_ssl => false
    workers => 1
    exchange => 'metrics'
  }
}

Now I would like to optimize the payload/overhead ratio by adding more than on line from the graphite input into one AMQP message.

I was looking at filters like collate or aggregate but they don't seem to be doing exactly what I need. What I'm looking for is a transport format where one AMQP message contains something like 20 or 30 lines from this input.

回答1:

I figured it out myself, I'm using multiline as input codec now:

tcp {
  host => localhost
  codec => multiline { pattern => "\r" max_lines => 100 what => "next" }
  type => carbon
  port => 22003
}