How to match a pattern given in a variable in awk?

Question

I want to extract a substring where certain pattern exist from pipe separated file, thus I used below command,

awk -F ":" '/REWARD REQ. SERVER HEADERS/{print $1, $2, $3, $4}' sample_profile.txt

Here, 'REWARD REQ. SERVER HEADERS' is a pattern which is to be searched in the file, and print its first 4 parts on a colon separated line.

Now, I want to send bash variable to act as a pattern. thus I used below command, but it's not working.

awk -v pat="$pattern" -F ":" '/pat/{print $1, $2 , $3, $4 } sample_profile.txt

How can I use -v and -F in a single awk command?

Answer 1

The problem here does not have to do with -F.

The problem is the usage of /pat/ when you want pat to be a variable. If you say /pat/, awk understands it as a literal "pat", so it will try to match those lines containing the string "pat".

If you want to provide the pattern through a variable, you need to use ~ as this:

 awk -v pat="$pattern" '$0 ~ pat'

All together, your code should be:

awk -v pat="$pattern" -F ":" '$0~pat{print $1, $2, $3, $4 }' file
#                             ^^^^^^

---

See an example:

Given this file:

$ cat file
hello
this is a var
hello bye

Let's look for lines containing "hello":

$ awk '/hello/' file
hello
hello bye

Let's now try looking for "pat", contained in a variable, the way you were doing it:

$ awk -v pat="hello" '/pat/' file
$                                    # NO MATCHES!

Let's now use the $0 ~ pat expression:

$ awk -v pat="hello" '$0~pat' file
hello                                 # WE MATCH!
hello bye

---

Of course, you can use such expressions to match just one field and say awk -v pat="$pattern" '$2 ~ pat' file and so on.

From GNU Awk User's Guide → 3.1 How to Use Regular Expressions:

When a regexp is enclosed in slashes, such as /foo/, we call it a regexp constant, much like 5.27 is a numeric constant and "foo" is a string constant.

And GNU Awk User's Guide → 3.6 Using Dynamic Regexps:

The righthand side of a ‘~’ or ‘!~’ operator need not be a regexp constant (i.e., a string of characters between slashes). It may be any expression. The expression is evaluated and converted to a string if necessary; the contents of the string are then used as the regexp. A regexp computed in this way is called a dynamic regexp or a computed regexp:

BEGIN { digits_regexp = "[[:digit:]]+" }
$0 ~ digits_regexp    { print }

This sets digits_regexp to a regexp that describes one or more digits, and tests whether the input record matches this regexp.

Answer 2

This is kind of a hack but it makes things a little simpler for me.

cmd="awk '/$pattern/'"
eval $cmd

making it a string first lets you manipulate it past the boundaries of awk

Answer 3

awk -v pat="$pattern" -F":" '$0 ~ pat { print $1, $2, $3, $4 }' sample_profile.txt

You can't use the variable inside the regex // notation (there's no way to distinguish it from searching for pat); you have to specify that the variable is a regex with the ~ (matching) operator.