iterating over file object in Python does not work

Question

In the following code, if I use:

for line in fin:

It only executes for 'a'

But if I use:

wordlist = fin.readlines()
for line in wordlist:

Then it executes for a thru z.

But readlines() reads the whole file at once, which I don't want.

How to avoid this?

def avoids():
    alphabet = 'abcdefghijklmnopqrstuvwxyz'
    num_words = {}

    fin = open('words.txt')

    for char in alphabet:
      num_words[char] = 0
      for line in fin:
        not_found = True
        word = line.strip()
        if word.lower().find(char.lower()) != -1:
          num_words[char] += 1
    fin.close()
    return num_words

Answer 1

You have three options:

1. Read in the whole file anyway.
2. Seek back to the beginning of the file before attempting to iterate over it again.
3. Rearchitect your code so that it doesn't need to iterate over the file more than once.

Answer 2

the syntax for line in fin can only be used once. After you do that, you've exhausted the file and you can't read it again unless you "reset the file pointer" by fin.seek(0). Conversely, fin.readlines() will give you a list which you can iterate over and over again.

---

I think a simple refactor with Counter (python2.7+) could save you this headache:

from collections import Counter
with open('file') as fin:
    result = Counter()
    for line in fin:
        result += Counter(set(line.strip().lower()))

which will count the number of words in your file (1 word per line) that contain a particular character (which is what your original code does I believe ... Please correct me if I'm wrong)

You could also do this easily with a defaultdict (python2.5+):

from collections import defaultdict
with open('file') as fin:
    result = defaultdict(int)
    for line in fin:
        chars = set(line.strip().lower())
        for c in chars:
            result[c] += 1

And finally, kicking it old-school -- I don't even know when setdefault was introduced...:

fin = open('file')
result = dict()
for line in fin:
    chars = set(line.strip().lower())
    for c in chars:
        result[c] = result.setdefault(c,0) + 1

fin.close()

Answer 3

Try:

from collections import defaultdict
from itertools import product

def avoids():
    alphabet = 'abcdefghijklmnopqrstuvwxyz'

    num_words = defaultdict(int)

    with open('words.txt') as fin:
        words = [x.strip() for x in fin.readlines() if x.strip()]

    for ch, word in product(alphabet, words):
        if ch not in word:
             continue
        num_words[ch] += 1

    return num_words