comparing dates using awk in bash

Question

So I have a file and each line has some info and a date (birthday). And I want to print the lines with dates after a given date. I use this awk command

awk -F '|' 'FNR>1 $dateA<=$5 {print $1" "$2" "$3" "$4" "$5" "$6" "$7" "$8}' $FILE

But it doesnt work properly (all file lines are printed). The dates are in YYYY-MM-DD format so alphabetical order is also chronological.

EDIT: Some lines from the input file

1099511628908|Chen|Wei|female|1985-08-02|2010-05-24T20:52:26.582+0000|27.98.244.108|Firefox
1099511633435|Smith|Jack|male|1981-04-19|2010-05-26T03:45:11.772+0000|50.72.193.218|Internet Explorer
1099511635042|Kiss|Gyorgy|male|1984-09-14|2010-05-16T22:57:41.808+0000|91.137.244.86|Chrome
1099511635218|Law-Yone|Eric|male|1987-01-20|2010-05-26T20:10:22.515+0000|203.81.95.235|Chrome
1099511638444|Jasani|Chris|female|1981-05-22|2010-04-29T20:50:40.375+0000|196.223.11.62|Firefox
2199023256615|Arbelaez|Gustavo|male|1986-11-02|2010-07-17T18:53:47.633+0000|190.96.218.101|Chrome

Answer 1

For this to work :

awk -vdateA=1985-01-01 -F '|' 'FNR>1 dateA<=$5' $FILE
    ~~~~~~~~~~~~~~~~~~

Answer 2

As it was said by others, a variable in single quotes will not be expanded by the shell. Awk will see the name of the variable, not its value.

One possible solution is to do this (assuming comparing strings is correct):

dateA='1985-01-01'
infile='file to read values from'
awk -F '|' -v dateA="$dateA" '{if (FNR>1 && dateA<=$5) {print}}' "$infile"

A more idiomatic solution (a bit less clear is):

awk 'FNR>1 && dateA<=$5' FS='|' dateA="$dateA" "$infile"

Or (yes, all the quoting is needed):

awk 'FNR>1 && "'"$dateA"'"<=$5' FS='|' "$infile"

But before even thinking of using this option read this about code injection.

Answer 3

Your variable $dateA is a shell variable. If you embed it between single quotes, it will not be interpreted by the shell (it will not be replaced by its value) and will be forwarded “as is” to awk.

Awk receives string $dateA. It believes it is an awk variable, but does not have any value for it, so it uses an empty string instead. All your dates are considered “bigger” to an empty string, so all lines match and are printed.

Several solutions to this. Beside Gilles Quenot's one, where you manually indicate to awk that dateA is a shell variable, you can also take it “out of the quotes” like this:

awk -F '|' 'FNR>1 '$dateA'<=$5 {print $1" "$2" "$3" "$4" "$5" "$6" "$7" "$8}' $FILE
                  ^      ^

This works when the date is an integer value, but not when it is on YYYY-MM-DD format. In the latter case, we need to… escape the variable with quotes to tell awk this is a string.

We want to do something that looks like "$dateA". Since we already added single quotes not to pass $dateA to awk as is, we must add double quotes around it: "'$dateA'". This way, the shell first replaces dateA with its value, then removes the single quotes, and gives "1985-08-23" to awk.

So the line is:

awk -F '|' 'FNR>1 && "'$dateA'"<=$5 {print $1" "$2" "$3" "$4" "$5" "$6" "$7" "$8}' $FILE

Edit: Not sure whether this depends on awk version, but I got a syntax error whith this example and also had to replace FNR>1 '$date'<$5 with FNR>1 && '$date'<$5 (additional &&). GNU awk here.