Parallel Processes in bash bitcoin monitor that re

Question

I wrote a bash program to monitor bitcoin mining devices, it worked fantastically running one set of loops per second until I got above 8 units being monitored. My solution was to integrate parallel processing which seems to have 'caused more problems than it fixed, what I seem to get now is that all my data will either be recorded from only a single unit or the program will error out because of the variables I'm recording showing up empty. first I'll post my original script with all personal information removed and then I'll post my attempt at parallel processing to see if anyone has a solution. Thanks for your help.

#!/bin/bash
# RPC the summary data
ti2=0
t=1 #Time Between Scans In Seconds
zero() {
SUMMAC=0
SUMMRJ=0
SUMMUT=0
GHs=0
SUMMHS=0
SUMMGW=0
SUMMRF=0
SUMMLW=0
SUMMDW=0
SUMMFB=0
SUMMHW=0
SUMMNB=0
SUMMET=0
}
id=(`tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
go
SELECT COUNT(*) FROM Mining_Rigs
go
quit
EOF`)
for (( c=0; c<=$id-1; c++ ))
do
IPI=`tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
SELECT IP_Address FROM Mining_Rigs where Row = $c
go
quit
EOF`
IP[$c]=$IPI
IPI=`tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
SELECT Port FROM Mining_Rigs Where Row = $c
go
quit
EOF`
Port[$c]=$IPI
IPI=`tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
SELECT Name FROM Mining_Rigs where Row = $c
go
quit
EOF`
Name[$c]=$IPI
IPI=(`tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
SELECT Active FROM Mining_Rigs where Row = $c
go
quit
EOF`)
Active[$c]=$IPI
if [ ${Active[$c]} -eq 1 ]; then
l_TELNET=`echo "quit" | telnet ${IP[$c]} ${Port[$c]} | grep "Escape character is"`
if [ "$?" -ne 0 ]; then
  echo "Connection to ${IP[$c]} on port ${Port[$c]} failed"
  echo "${Name[$c]} Will Be Made Inactive Until Service Is Reloaded"
  sleep 2
  tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
UPDATE Mining_Rigs SET Active = 0 WHERE Row = $c
go
quit
EOF
Active[$c]=0
fi
fi
done
ti=$(date +%s)
while :
do
DOY=$(date +%s)
for (( c=0; c<=$id-1; c++ ))
do
if [ ${Active[$c]} -eq 1 ]; then
echo "{\"command\":\"summary\"}" | nc ${IP[$c]} ${Port[$c]} | sed 's/\x0$//' | python -mjson.tool 2> /dev/null > sum.txt
for NODE in "${IP[$c]}"; do

    while read cline ; do
        #DEBUGGING=$(echo $cline | awk '{ print "1="$1, "2="$2, "3="$3; exit; }' | sed 's/[:,\"]//g' )
    #echo "$NODE: $DEBUGGING"
        FLD1=$(echo $cline | awk '{ print $1; exit; }' | sed 's/[:,\"]//g' )
        #
    if [ "$FLD1" == "Getworks" ] ; then
            SUMMGW=$(echo $cline | awk '{ printf($2); exit; }' | sed 's/[:,\"]//g')
        elif [ "$FLD1" == "Accepted" ] ; then
            SUMMAC=$(echo $cline | awk '{ printf($2); echo ; exit; }' | sed 's/[:,\"]//g')
        elif [ "$FLD1" == "Rejected" ] ; then
            SUMMRJ=$(echo $cline | awk '{ printf($2); exit; }' | sed 's/[:,\"]//g')
        elif [ "$FLD1" == "Remote" ] ; then
            SUMMRF=$(echo $cline | awk '{  printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Utility" ] ; then
            SUMMUT=$(echo $cline | awk '{ printf($2); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "MHS" ] ; then
            SUMMHS=$(echo $cline | awk '{ printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Local" ] ; then
            SUMMLW=$(echo $cline | awk '{ printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Discarded" ] ; then
            SUMMDW=$(echo $cline | awk '{ printf($2); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Found" ] ; then
            SUMMFB=$(echo $cline | awk '{ printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Hardware" ] ; then
            SUMMHW=$(echo $cline | awk '{ printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Network" ] ; then
            SUMMNB=$(echo $cline | awk '{ printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Elapsed" ] ; then
            SUMMET=$(echo $cline | awk '{ printf($2); exit; }' | sed 's/[:,\"]//g' )
       fi
    done < sum.txt
if [ $(echo "scale=0; ($SUMMHS) / 1" | bc) -gt 0 ]; then
GHs=$(echo "scale=9; ($SUMMHS) / 1000" | bc)
fi
if [ $DOY -gt $ti ]; then
midate=`date -d "UTC 1970-01-01 $ti secs" "+%Y-%m-%d %H:%M:%S"`
tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
insert into "${Name[$c]}" (Date,Node,Accepted,Rejected,Utility,GHs,Getworks,Remfail,Local,Discard,Found,Hwerr,Network,Uptime) values( '$midate','$NODE','$SUMMAC','$SUMMRJ','$SUMMUT','$GHs','$SUMMGW','$SUMMRF','$SUMMLW','$SUMMDW','$SUMMFB','$SUMMHW','$SUMMNB','$SUMMET');
go
quit
EOF
zero
ti2=1
elif [ $DOY -eq $ti ]; then
midate=`date -d "UTC 1970-01-01 $DOY secs" "+%Y-%m-%d %H:%M:%S"`
tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
insert into "${Name[$c]}" (Date,Node,Accepted,Rejected,Utility,GHs,Getworks,Remfail,Local,Discard,Found,Hwerr,Network,Uptime) values( '$midate','$NODE','$SUMMAC','$SUMMRJ','$SUMMUT','$GHs','$SUMMGW','$SUMMRF','$SUMMLW','$SUMMDW','$SUMMFB','$SUMMHW','$SUMMNB','$SUMMET');
go 
quit
EOF
zero
ti2=0
else
tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
insert into "${Name[$c]}" (Date,Node,Accepted,Rejected,Utility,GHs,Getworks,Remfail,Local,Discard,Found,Hwerr,Network,Uptime) values( FROM_UNIXTIME('$DOY') ,'$NODE','$SUMMAC','$SUMMRJ','$SUMMUT','$GHs','$SUMMGW','$SUMMRF','$SUMMLW','$SUMMDW','$SUMMFB','$SUMMHW','$SUMMNB','$SUMMET');
EOF
zero
fi
done
fi
done
if [ $ti2 -eq 1 ]; then
ti3=$(expr $DOY - $ti )
t2=$(expr $t - $ti3 )
if [ $t2 -ge 0 ]; then
ti=$(expr $DOY + $t2 )
sleep $t2
fi
elif [ $ti2 -eq 0 ]; then
ti=$(expr $DOY + $t )
sleep $t
fi
done

this is my original program worked like a charm until I got above 8 units, next is my attempt at parallel processing

#!/bin/bash
# RPC the summary data
ti2=0
t=3 #Time Between Scans In Seconds
Json() {
zero() {
SUMMAC=0
SUMMRJ=0
SUMMUT=0
GHs=0
SUMMHS=0
SUMMGW=0
SUMMRF=0
SUMMLW=0
SUMMDW=0
SUMMFB=0
SUMMHW=0
SUMMNB=0
SUMMET=0
}
c=$d
if [ ${Active[$c]} -eq 1 ]; then
echo "{\"command\":\"summary\"}" | nc -w 1 ${IP[$c]} ${Port[$c]} | sed 's/\x0$//' | python -mjson.tool 2> /dev/null > sum.txt
for NODE in "${IP[$c]}"; do

    while read cline ; do
        #DEBUGGING=$(echo $cline | awk '{ print "1="$1, "2="$2, "3="$3; exit; }' | sed 's/[:,\"]//g' )
        #echo "$NODE: $DEBUGGING"
        FLD1=$(echo $cline | awk '{ print $1; exit; }' | sed 's/[:,\"]//g' )
        #
    if [ "$FLD1" == "Getworks" ] ; then
            SUMMGW=$(echo $cline | awk '{ printf($2); exit; }' | sed 's/[:,\"]//g')
        elif [ "$FLD1" == "Accepted" ] ; then
            SUMMAC=$(echo $cline | awk '{ printf($2); echo ; exit; }' | sed 's/[:,\"]//g')
        elif [ "$FLD1" == "Rejected" ] ; then
            SUMMRJ=$(echo $cline | awk '{ printf($2); exit; }' | sed 's/[:,\"]//g')
        elif [ "$FLD1" == "Remote" ] ; then
            SUMMRF=$(echo $cline | awk '{  printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Utility" ] ; then
            SUMMUT=$(echo $cline | awk '{ printf($2); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "MHS" ] ; then
            SUMMHS=$(echo $cline | awk '{ printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Local" ] ; then
            SUMMLW=$(echo $cline | awk '{ printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Discarded" ] ; then
            SUMMDW=$(echo $cline | awk '{ printf($2); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Found" ] ; then
            SUMMFB=$(echo $cline | awk '{ printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Hardware" ] ; then
            SUMMHW=$(echo $cline | awk '{ printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Network" ] ; then
            SUMMNB=$(echo $cline | awk '{ printf($3); exit; }' | sed 's/[:,\"]//g' )
        elif [ "$FLD1" == "Elapsed" ] ; then
            SUMMET=$(echo $cline | awk '{ printf($2); exit; }' | sed 's/[:,\"]//g' )
       fi
    done < sum.txt
    echo $SUMMHS
    echo $SUMMUT
    if [ $(echo "scale=0; ($SUMMHS) / 1" | bc) -gt 0 ]; then
GHs=$(echo "scale=9; ($SUMMHS) / 1000" | bc)
fi
if [ $DOY -gt $ti ]; then
midate=`date -d "UTC 1970-01-01 $ti secs" "+%Y-%m-%d %H:%M:%S"`
zero
ti2=1
elif [ $DOY -eq $ti ]; then
midate=`date -d "UTC 1970-01-01 $DOY secs" "+%Y-%m-%d %H:%M:%S"`
tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
insert into "${Name[$c]}" (Date,Node,Accepted,Rejected,Utility,GHs,Getworks,Remfail,Local,Discard,Found,Hwerr,Network,Uptime) values( '$midate','$NODE','$SUMMAC','$SUMMRJ','$SUMMUT','$GHs','$SUMMGW','$SUMMRF','$SUMMLW','$SUMMDW','$SUMMFB','$SUMMHW','$SUMMNB','$SUMMET');
go 
quit
EOF
zero
ti2=0
else
tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
insert into "${Name[$c]}" (Date,Node,Accepted,Rejected,Utility,GHs,Getworks,Remfail,Local,Discard,Found,Hwerr,Network,Uptime) values( FROM_UNIXTIME('$DOY') ,'$NODE','$SUMMAC','$SUMMRJ','$SUMMUT','$GHs','$SUMMGW','$SUMMRF','$SUMMLW','$SUMMDW','$SUMMFB','$SUMMHW','$SUMMNB','$SUMMET');
EOF
zero
fi
done
fi
    }
id=(`tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
go
SELECT COUNT(*) FROM Mining_Rigs
go
quit
EOF`)
for (( c=0; c<=$id-1; c++ ))
do
IPI=`tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
SELECT IP_Address FROM Mining_Rigs where Row = $c
go
quit
EOF`
IP[$c]=$IPI
IPI=`tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
SELECT Port FROM Mining_Rigs Where Row = $c
go
quit
EOF`
Port[$c]=$IPI
IPI=`tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
SELECT Name FROM Mining_Rigs where Row = $c
go
quit
EOF`
Name[$c]=$IPI
IPI=(`tsql -S 192.168.0.1\\MSSQLSERVER -U Username -P Password -o qh <<EOF
use DATABASE NAME
SELECT Active FROM Mining_Rigs where Row = $c
go
quit
EOF`)
Active[$c]=$IPI
done
ti=$(date +%s)
Getinfo
while :
do
DOY=$(date +%s)
for (( c=0; c<=$id-1; c++ ))
do
Json $c &
echo $c
done
if [ $ti2 -eq 1 ]; then
ti3=$(expr $DOY - $ti )
t2=$(expr $t - $ti3 )
if [ $t2 -ge 0 ]; then
ti=$(expr $DOY + $t2 )
sleep $t2
fi
elif [ $ti2 -eq 0 ]; then
ti=$(expr $DOY + $t )
sleep $t
fi
wait
done

Answer 1

Their were two problems with this, one you can't run multiple NC commands at a time in linux, if you have this problem run Ncat instead. Secondly I copied the code for the funstion json into a new bash script. exported my variables and ran the script with an amperesand. Problem Solved