Bash Script Loop Out of Memory?

Question

In bash I need to run a script that loops from i=1 to i=99999999 but it always run out of memory. Is there any workaround? or is there a max value for i?

first=1
last=99999999
randomString="CXCXQOOPSOIS"

for val in $( seq $first $last )
do
  padVal=$( printf "%010d\n" $val )
  hash=$( echo -n $randomString$padVal | md5sum )
  if [[ "$hash" =~ ^000000) ]]; then
    echo "Number: $val"  >> log_000000
    echo "$val added to log - please check."
  fi
done

Answer 1

If you still want to use seq => therefore separate seq and the loop using a pipe: |
This solution is more portable and can be used on other shells.
The memory print is still reduced, but this script requires to process two threads.

first=1
last=99999999
randomString="CXCXQOOPSOIS"

seq $first $last |
while read val
do
  padVal=$( printf "%010d\n" $val )
  hash=$( echo -n $randomString$padVal | md5sum )
  if [[ "$hash" =~ ^000000) ]]; then
    echo "Number: $val"  >> log_000000
    echo "$val added to log - please check."
  fi
done

Answer 2

bash provides C-like syntax for loop:

first=1
last=99999999
randomString="CXCXQOOPSOIS"

for ((val=$first; val<$last; val++))
do
  padVal=$( printf "%010d\n" $val )
  hash=$( echo -n $randomString$padVal | md5sum )
  if [[ "$hash" =~ ^000000) ]]; then
    echo "Number: $val"  >> log_000000
    echo "$val added to log - please check."
  fi
done

Answer 3

Your seq command generates 100 million numbers (bar a couple) and requires 800 MiB or so of memory to hold just the list of digits (probably an under-estimate; each number might be held in a separate memory allocation, which might mean 8 bytes for a pointer and 16 bytes for the allocated block, which triples the storage space estimate).

You can improve things dramatically by using:

for millions in $(seq 0 99)
do
    for smallstuff in $(seq -f "%6.0f" 0 999999)
    do
        val="$millions$smallstuff"
        ...
    done
done

This dramatically reduces the amount of memory needed; the only issue to watch is that it tests 0 which your original code did not.