I have a bash function that i call in parallel using xargs -P like so
echo ${list} | xargs -n 1 -P 24 -I@ bash -l -c 'myAwesomeShellFunction @'
Everything works fine but output is messed up for obvious reasons (no buffering)
Trying to figure out a way to buffer output effectively. I was thinking I could use awk, but I'm not good enough to write such a script and I can't find anything worthwhile on google? Can someone help me write this "output buffer" in sed or awk? Nothing fancy, just accumulate output and spit it out after process terminates. I don't care the order that shell functions execute, just need their output buffered... Something like:
echo ${list} | xargs -n 1 -P 24 -I@ bash -l -c 'myAwesomeShellFunction @ | sed -u ""'
P.s. I tried to use stdbuf as per https://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe but did not work, i specified buffering on o and e but output still unbuffered:
echo ${list} | xargs -n 1 -P 24 -I@ stdbuf -i0 -oL -eL bash -l -c 'myAwesomeShellFunction @'
Here's my first attempt, this only captures first line of output:
$ bash -c "echo stuff;sleep 3; echo more stuff" | awk '{while (( getline line) > 0 )print "got ",$line;}'
$ got stuff