Parsing “From” addresses from email text

Question

I'm trying to extract email addresses from plain text transcripts of emails. I've cobbled together a bit of code to find the addresses themselves, but I don't know how to make it discriminate between them; right now it just spits out all email addresses in the file. I'd like to make it so it only spits out addresses that are preceeded by "From:" and a few wildcard characters, and ending with ">" (because the emails are set up as From [name]<[email]>).

Here's the code now:

import re #allows program to use regular expressions
foundemail = []
#this is an empty list

mailsrch = re.compile(r'[\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4}')
 #do not currently know exact meaning of this expression but assuming
 #it means something like "[stuff]@[stuff][stuff1-4 letters]"

        # "line" is a variable is set to a single line read from the file
# ("text.txt"):
for line in open("text.txt"):

    foundemail.extend(mailsrch.findall(line))

    # this extends the previously named list via the "mailsrch" variable
      #which was named before

print foundemail

Answer 1

Use the email and mailbox packages to parse the plain text version of the email. This will convert it to an object that will enable to extract all the addresses in the 'From' field.

You can also do a lot of other analysis on the message, if you need to process other header fields, or the message body.

As a quick example, the following (untested) code should read all the message in a unix style mailbox, and print all the 'from' headers.

import mailbox
import email

mbox = mailbox.PortableUnixMailbox(open(filename, 'rU'), email.message_from_file)

for msg in mbox:
   from = msg['From']
   print from

Answer 2

import email
msg = email.message_from_string(str)

# or
# f = open(file)
# msg = email.message_from_file(f)

msg['from']

# and optionally
from email.utils import parseaddr
addr = parseaddr(msg['from'])

Answer 3

"[stuff]@[stuff][stuff1-4 letters]" is about right, but if you wanted to you could decode the regular expression using a trick I just found out about, here. Do the compile() in an interactive Python session like this:

mailsrch = re.compile(r'[\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4}', 128)

It will print out the following:

in 
  category category_word
  literal 45
max_repeat 1 65535 
  in 
    category category_word
    literal 45
    literal 46
literal 64 
in 
  category category_word
  literal 45
max_repeat 1 65535 
  in 
    category category_word
    literal 45
    literal 46
max_repeat 1 4 
  in 
    range (97, 122)
    range (65, 90)

Which, if you can kind of get used to it, shows you exactly how the RE works.

Answer 4

If your goal is actually to extract email addresses from text, you should use a library built for that purpose. Regular expressions are not well suited to match arbitrary email addresses.

But if you're doing this as an exercise to understand regular expressions better, I'd take the approach of expanding the expression you're using to include the extra text you want to match. So first, let me explain what that regex does:

[\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4}

• [\w\-] matches any "word" character (letter, number, or underscore), or a hyphen
• [\w\-\.]+ matches (any word character or hyphen or period) one or more times
• @ matches a literal '@'
• [\w\-] matches any word character or hyphen
• [\w\-\.]+ matches (any word character or hyphen or period) one or more times
• [a-zA-Z]{1,4} matches 1, 2, 3, or 4 lowercase or uppercase letters

So this matches a sequence of a "word" that may contain hyphens or periods but doesn't start with a period, followed by an @ sign, followed by another "word" (same sense as before) that ends with a letter.

Now, to modify this for your purposes, let's add regex parts to match "From", the name, and the angle brackets:

From: [\w\s]+?<([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>

• From: matches the literal text "From: "
• [\w\s]+? matches one or more consecutive word characters or space characters. The question mark makes the match non-greedy, so it will match as few characters as possible while still allowing the whole regular expression to match (in this case, it's probably not necessary, but it does make the match more efficient since the thing that comes immediately afterwards is not a word character or space character).
• < matches a literal less-than sign (opening angle bracket)
• The same regular expression you had before is now surrounded by parentheses. This makes it a capturing group, so you can call m.group(1) to get the text matched by that part of the regex.
• > matches a literal greater-than sign

Since the regex now uses capturing groups, your code will need to change a little as well:

import re
foundemail = []

mailsrch = re.compile(r'From: [\w\s]+?<([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>')

for line in open("text.txt"):
    foundemail.extend([m.group(1) for m in mailsrch.finditer(line)])

print foundemail

The code [m.group(1) for m in mailsrch.finditer(line)] produces a list out of the first capturing group (remember, that was the part in parentheses) from each match found by the regular expression.

Answer 5

if you can be reasonably sure that lines containing these email addresses start with whitespace followed by "From:" you can simply do this:

addresslines = []
for line in open("text.txt"):
    if line.strip().startswith("From:"):
        addresslines.append(line)

then later - or on adding them to the list - you can refine the addresslines items to give out exactly what you want

Answer 6

mailsrch = re.compile(r'[\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4}')

Expression breakdown:

[\w-]: any word character (alphanumeric, plus underscore) or a dash

[\w-.]+: any word character, a dash, or a period/dot, one or more times

@: literal @ symbol

[\w-][\w-.]+: any word char or dash, followed by any word char, dash, or period one or more times.

[a-zA-Z]{1,4}: any alphabetic character 1-4 times.

To make this match only lines starting with From:, and wrapped in < and > symbols:

import re

foundemail = []
mailsrch = re.compile(r'^From:\s+.*<([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>', re.I | re.M)
foundemail.extend(mailsrch.findall(open('text.txt').read()))

print foundemail