What's an easy way to read random line from a

Question

What's an easy way to read random line from a file in Unix command line?

Answer 1

#!/bin/bash

IFS=$'\n' wordsArray=($(<$1))

numWords=${#wordsArray[@]}
sizeOfNumWords=${#numWords}

while [ True ]
do
    for ((i=0; i<$sizeOfNumWords; i++))
    do
        let ranNumArray[$i]=$(( ( $RANDOM % 10 )  + 1 ))-1
        ranNumStr="$ranNumStr${ranNumArray[$i]}"
    done
    if [ $ranNumStr -le $numWords ]
    then
        break
    fi
    ranNumStr=""
done

noLeadZeroStr=$((10#$ranNumStr))
echo ${wordsArray[$noLeadZeroStr]}

Answer 2

You can use shuf:

shuf -n 1 $FILE

There is also a utility called rl. In Debian it's in the randomize-lines package that does exactly what you want, though not available in all distros. On its home page it actually recommends the use of shuf instead (which didn't exist when it was created, I believe). shuf is part of the GNU coreutils, rl is not.

rl -c 1 $FILE

Answer 3

Using only vanilla sed and awk, and without using $RANDOM, a simple, space-efficient and reasonably fast "one-liner" for selecting a single line pseudo-randomly from a file named FILENAME is as follows:

sed -n $(awk 'END {srand(); r=rand()*NR; if (r<NR) {sub(/\..*/,"",r); r++;}; print r}' FILENAME)p FILENAME

(This works even if FILENAME is empty, in which case no line is emitted.)

One possible advantage of this approach is that it only calls rand() once.

As pointed out by @AdamKatz in the comments, another possibility would be to call rand() for each line:

awk 'rand() * NR < 1 { line = $0 } END { print line }' FILENAME

(A simple proof of correctness can be given based on induction.)

Caveat about rand()

"In most awk implementations, including gawk, rand() starts generating numbers from the same starting number, or seed, each time you run awk."

-- https://www.gnu.org/software/gawk/manual/html_node/Numeric-Functions.html

Answer 4

Here is what I discovery since my Mac OS doesn't use all the easy answers. I used the jot command to generate a number since the $RANDOM variable solutions seems not to be very random in my test. When testing my solution I had a wide variance in the solutions provided in the output.

  RANDOM1=`jot -r 1 1 235886`
   #range of jot ( 1 235886 ) found from earlier wc -w /usr/share/dict/web2
   echo $RANDOM1
   head -n $RANDOM1 /usr/share/dict/web2 | tail -n 1

The echo of the variable is to get a visual of the generated random number.

Answer 5

perlfaq5: How do I select a random line from a file? Here's a reservoir-sampling algorithm from the Camel Book:

perl -e 'srand; rand($.) < 1 && ($line = $_) while <>; print $line;' file

This has a significant advantage in space over reading the whole file in. You can find a proof of this method in The Art of Computer Programming, Volume 2, Section 3.4.2, by Donald E. Knuth.

Answer 6

Single bash line:

sed -n $((1+$RANDOM%`wc -l test.txt | cut -f 1 -d ' '`))p test.txt

Slight problem: duplicate filename.