Logstash if statement with regex example

Question

Can anyone show me what an if statement with a regex looks like in logstash?

My attempts:

if [fieldname] =~ /^[0-9]*$/

if [fieldname] =~ "^[0-9]*$"

Neither of which work.

What I intend to do is to check if the "fieldname" contains an integer

Answer 1

To combine the other answers into a cohesive answer.

Your first format looks correct, but your regex is not doing what you want.

/^[0-9]*$/ matches:

^: the beginning of the line

[0-9]*: any digit 0 or more times

$: the end of the line

So your regex captures lines that are exclusively made up of digits. To match on the field simply containing one or more digits somewhere try using /[0-9]+/ or /\d+/ which are equivalent and each match 1 or more digits regardless of the rest of the line.

In total you should have:

if [fieldname] =~ /\d+/ {
   # do stuff
}

Answer 2

You need this regex (and brackets, I think):

if ([fieldname] =~ /^[0-9]+$/)

Answer 3

Your first format works (for me at the time of writing).

Check the current logstash version in the below excerpt, and also watch for the uuid field present in the output upon match. As expected, empty field matches too, but otherwise it is perfect.

I suggest you to test stuff with such short stdin-stdout configurations. Logstash and Elastic stuff is great, but all too often the corner cases are not properly discussed in the documentation. They develop code faster than the docs as we are all tempted.

============= logstash @ logstash.host.example.com : ~ ============
$ cfg="$(cat)"
input { stdin {} }
filter { if [message] =~ /^[0-9]*$/ { uuid { target => "uuid" } } }
output { stdout { codec => "rubydebug" } }
============= logstash @ logstash.host.example.com : ~ ============
$ /usr/share/logstash/bin/logstash --config.string "$cfg" --pipeline.workers 1 --log.format json --path.data /tmp/kadmar
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2018-11-26 14:50:36.434 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-11-26 14:50:37.646 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.3.0"}
[INFO ] 2018-11-26 14:50:44.490 [Converge PipelineAction::Create<main>] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-11-26 14:50:44.840 [Converge PipelineAction::Create<main>] pipeline - Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x4620459c run>"}
The stdin plugin is now waiting for input:
[INFO ] 2018-11-26 14:50:45.048 [Ruby-0-Thread-1: /usr/share/logstash/lib/bootstrap/environment.rb:6] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[INFO ] 2018-11-26 14:50:45.457 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9601}
hello
{
       "message" => "hello",
    "@timestamp" => 2018-11-26T13:50:56.293Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}
ab123cd
{
       "message" => "ab123cd",
    "@timestamp" => 2018-11-26T13:51:13.648Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}
123
{
       "message" => "123",
          "uuid" => "3cac8b35-6054-4e14-b7d0-0036210c1f2b",
    "@timestamp" => 2018-11-26T13:51:18.100Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}
1
{
       "message" => "1",
          "uuid" => "1d56982f-421a-4ccd-90d6-6c2c0bcf267d",
    "@timestamp" => 2018-11-26T13:51:25.631Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}

{
       "message" => "",
          "uuid" => "747ac36f-8679-4c66-8050-9bd874aef4c5",
    "@timestamp" => 2018-11-26T13:51:27.614Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}
012 456
{
       "message" => "012 456",
    "@timestamp" => 2018-11-26T13:52:09.614Z,
          "host" => "logstash.host.example.com",
      "@version" => "1"
}

Answer 4

^ asserts position at start of the string

$ asserts position at the end of the string

Your regexp just match the number string, and check contains an integer need remove ^ and $.

Answer 5

The simplest way is to check for \d

if [fieldname] =~ /\d+/ {
   ...
}