Finding the most frequent occurrences of pairs in

Question

I've a dataset that denotes the list of authors of many technical reports. Each report can be authored by one or multiple people:

a = [
['John', 'Mark', 'Jennifer'],
['John'],
['Joe', 'Mark'],
['John', 'Anna', 'Jennifer'],
['Jennifer', 'John', 'Mark']
]

I've to find the most frequent pairs, that is, people that had most collaborations in the past:

['John', 'Jennifer'] - 3 times
['John', 'Mark'] - 2 times
['Mark', 'Jennifer'] - 2 times
etc...

How to do this in Python?

Answer 1

import collections
import itertools

a = [
['John', 'Mark', 'Jennifer'],
['John'],
['Joe', 'Mark'],
['John', 'Anna', 'Jennifer'],
['Jennifer', 'John', 'Mark']
]


counts = collections.defaultdict(int)
for collab in a:
    collab.sort()
    for pair in itertools.combinations(collab, 2):
        counts[pair] += 1

for pair, freq in counts.items():
    print(pair, freq)

Output:

('John', 'Mark') 2
('Jennifer', 'Mark') 2
('Anna', 'John') 1
('Jennifer', 'John') 3
('Anna', 'Jennifer') 1
('Joe', 'Mark') 1

Answer 2

You can use a set comprehension to create a set of all numbers then use a list comprehension to count the occurrence of the pair names in your sub list :

>>> from itertools import combinations as comb
>>> all_nam={j for i in a for j in i}
>>> [[(i,j),sum({i,j}.issubset(t) for t in a)] for i,j in comb(all_nam,2)]

[[('Jennifer', 'John'), 3], 
 [('Jennifer', 'Joe'), 0], 
 [('Jennifer', 'Anna'), 1], 
 [('Jennifer', 'Mark'), 2], 
 [('John', 'Joe'), 0], 
 [('John', 'Anna'), 1], 
 [('John', 'Mark'), 2], 
 [('Joe', 'Anna'), 0], 
 [('Joe', 'Mark'), 1], 
 [('Anna', 'Mark'), 0]]

Answer 3

Use a collections.Counter dict with itertools.combinations:

from collections import Counter
from itertools import combinations

d  = Counter()
for sub in a:
    if len(a) < 2:
        continue
    sub.sort()
    for comb in combinations(sub,2):
        d[comb] += 1

print(d.most_common())
[(('Jennifer', 'John'), 3), (('John', 'Mark'), 2), (('Jennifer', 'Mark'), 2), (('Anna', 'John'), 1), (('Joe', 'Mark'), 1), (('Anna', 'Jennifer'), 1)]

most_common() will return the pairings in order of most common to least, of you want the first n most common just pass n d.most_common(n)