logstash multiline filter:last part of message flu

2019-09-04 23:01发布

问题:

There was a know issue in the multiline filter that it doesn't print the last part of input being filtered as it is still waiting for a pattern to follow and it isn't flushed out.
Has the issue been resolved. If so how?Is there a tag to flush out the last part?

回答1:

The flushing feature is supposed to be in logstash 1.5 according to this JIRA: https://logstash.jira.com/browse/LOGSTASH-1785 although an "experimental" feature of 1.4.2 added enable_flush to the multiline filter. I have not personally tested to see if that fixes the flushing issue with the last event.



回答2:

With Logstash 5.1.1 there is auto_flush_interval option.

The accumulation of multiple lines will be converted to an event when either a matching new line is seen or there has been no new data appended for this time auto_flush_interval. No default. If unset, no auto_flush .

Source: docs

Simply set

codec => multiline {
                pattern => "^%{DATESTAMP_OTHER}"
                what => "previous"
                negate => true
                # set to time in secs when to flush
                auto_flush_interval => 15
            }