Bash shell script Nested while loop with IFS

Question

I'm trying to parse a set of csv files using bash shell script the files looks as below:

File1: /tmp/test.txt
key1,key1file.txt
key2,key2file.txt
key3,key3file.txt

Files: /tmp/inter/key1file.txt
abc,cdf,123,456

Files: /tmp/inter/key2file.txt
abc,cdf,123,456

Files: /tmp/inter/key3file.txt
abc,cdf,123,456

I've tried parsing these files using 2 while loops:

while IFS="," read keycol keyfile
do
    while IFS="," read keyval
     do
     echo "inside inner while loop"
     echo "$keycol|$keyval"
    done < "/tmp/inter/$keyfile"
done < /tmp/test.txt

and expecting this code to output

key1,abc
key1,cdf
key1,123
key1,456 and so on...

However, i'm not getting any output when i run this code which indicates the second loop is not being executed. Any pointers in the right direction would be of help. Thanks

Answer 1

You are not properly splitting by , in your second loop. read generally splits by IFS, and assigns values to variables, one field per variable, and the remaining goes into the last variable provided. But if you provide only one variable, everything just gets stored there.

Instead, let read split by , into an array, then loop over values in that array, like this:

#!/bin/bash
while IFS="," read keycol keyfile; do
    while IFS="," read -a values; do
        for val in "${values[@]}"; do
            echo "$keycol,$val"
        done
    done < "/tmp/inter/$keyfile"
done < /tmp/test.txt

You'll get:

key1,abc
key1,cdf
key1,123
key1,456
key2,abc
key2,cdf
key2,123
key2,456
key3,abc
key3,cdf
key3,123
key3,456