Count frequency of words in a list and sort by fre

Question

I am using Python 3.3

I need to create two lists, one for the unique words and the other for the frequencies of the word.

I have to sort the unique word list based on the frequencies list so that the word with the highest frequency is first in the list.

I have the design in text but am uncertain how to implement it in Python.

The methods I have found so far use either Counter or dictionaries which we have not learned. I have already created the list from the file containing all the words but do not know how to find the frequency of each word in the list. I know I will need a loop to do this but cannot figure it out.

Here's the basic design:

                 original list = ["the", "car",....]
                 newlst = []
                 frequency = []
                 for word in the original list
                       if word not in newlst
                           newlst.append(word)
                           set frequency = 1
                       else
                           increase the frequency
                 sort newlst based on frequency list 

Answer 1

You can use

from collections import Counter

It supports Python 2.7，read more information here

1.

>>>c = Counter('abracadabra')
>>>c.most_common(3)
[('a', 5), ('r', 2), ('b', 2)]

use dict

>>>d={1:'one', 2:'one', 3:'two'}
>>>c = Counter(d.values())
[('one', 2), ('two', 1)]

But, You have to read the file first, and converted to dict.

2. it's the python docs example,use re and Counter

# Find the ten most common words in Hamlet
>>> import re
>>> words = re.findall(r'\w+', open('hamlet.txt').read().lower())
>>> Counter(words).most_common(10)
[('the', 1143), ('and', 966), ('to', 762), ('of', 669), ('i', 631),
 ('you', 554),  ('a', 546), ('my', 514), ('hamlet', 471), ('in', 451)]

Answer 2

One way would be to make a list of lists, with each sub-list in the new list containing a word and a count:

list1 = []    #this is your original list of words
list2 = []    #this is a new list

for word in list1:
    if word in list2:
        list2.index(word)[1] += 1
    else:
        list2.append([word,0])

Or, more efficiently:

for word in list1:
    try:
        list2.index(word)[1] += 1
    except:
        list2.append([word,0])

This would be less efficient than using a dictionary, but it uses more basic concepts.

Answer 3

You can use reduce() - A functional way.

words = "apple banana apple strawberry banana lemon"
reduce( lambda d, c: d.update([(c, d.get(c,0)+1)]) or d, words.split(), {})

returns:

{'strawberry': 1, 'lemon': 1, 'apple': 2, 'banana': 2}

Answer 4

the best thing to do is :

def wordListToFreqDict(wordlist):
    wordfreq = [wordlist.count(p) for p in wordlist]
    return dict(zip(wordlist, wordfreq))

then try to : wordListToFreqDict(originallist)

Answer 5

Yet another solution with another algorithm without using collections:

def countWords(A):
   dic={}
   for x in A:
       if not x in  dic:        #Python 2.7: if not dic.has_key(x):
          dic[x] = A.count(x)
   return dic

dic = countWords(['apple','egg','apple','banana','egg','apple'])
sorted_items=sorted(dic.items())   # if you want it sorted