Strip signatures and replies from emails

Question

I'm currently working on a system that allows users to reply to notification emails that are sent out (sigh).

I need to strip out the replies and signatures, so that I'm left with the actual content of the reply, without all the noise.

Does anyone have any suggestions about the best way to do this?

Answer 1

An approach that can be used for signature only (in addition to detect __ or --) is to test if the first name and/or family name of the sender is on a short line (~ containing 3 to 4 words, max).

The sender name is on the raw email header, most of the time next to the email address, like in: From: John Doe <[e-mail]>

This would be based on the assumption that you rarely write your own name in a email, and if you do so, it is probably in a long sentence.

Of course there will be some false positive, but it may not be a big problem depending on what you do (we use it to fold quoted text and signature into a ... gmail-style button, so overdetection does not end up into losing any content, it is just misplaced).

Answer 2

Check out the email_reply_parser gem - https://github.com/github/email_reply_parser . It does a nice job handling this problem.

Answer 3

The recommended signature delimiter is "-- \n". If people follow this recommendation, stripping signatures should be easy.

Answer 4

If you want something powerful & robust, and don't mind reading academic publications, you might check out this:

• Learning to Extract Signature and Reply Lines from Email

Here's the homepage for one of the authors, with more info & some downloads:

• Vitor R. Carvalho - Software and Datasets - (Vitor Carvalho)

Answer 5

If your system is in-house and/or you have a limited number of reply formats, it's possible to do a pretty good job. Here are the filters we have set up for email responses to trac tickets:

Drop all text after and including:

1. Lines that equal '-- \n' (standard email sig delimiter)
2. Lines that equal '--\n' (people often forget the space in sig delimiter; and this is not that common outside sigs)
3. Lines that begin with '-----Original Message-----' (MS Outlook default)
4. Lines that begin with '________________________________' (32 underscores, Outlook again)
5. Lines that begin with 'On ' and end with ' wrote:\n' (OS X Mail.app default)
6. Lines that begin with 'From: ' (failsafe four Outlook and some other reply formats)
7. Lines that begin with 'Sent from my iPhone'
8. Lines that begin with 'Sent from my BlackBerry'

Numbers 3 and 4 are 'begin with' instead of 'equals' because sometimes users will squash lines together on accident.

We try to be more liberal about stripping out replies, since it's much more of an annoyance (to us) have reply garbage than it is to correct missing text.

Anybody have other formats from the wild that they want to share?

Answer 6

I don't believe you can do this reliably (signatures used to begin with '--' but I don't see that anymore). Perhaps you're better off asking people to reply inbetween text headers and then simply strip the reply from this ? It's not elegant, but perhaps more reliable.

e.g.

REPLY BETWEEN HERE -->

AND HERE -->

so you'd simply look for the required headers above and take what's inbetween.