Bounced mail Handling in PHP - Any up-to-date solu

Question

I need to do some bounced mail processing using PHP. Going through the e-mails is no problem, but investing the resources in writing our own library to parse the bounced e-mails is very undesirable.

There are 3 PHP solutions that I've found that are supposed to be for processing bounced mail, but they are all way out of date and no longer maintained, from what I can see. (PHPList, PHPMailer-BMH, Bounce Handler @ PHPClasses.org)

Does anyone know of an up-to-date set of rules for processing bounced e-mails? I don't necessarily need any handling logic, even just an up-to-date ruleset would be satisfactory.

Thanks for any assistance.

Answer 1

Here's how I did it once before. It can capture at least over 70% of the bounce mails consistantly.

In the emails being sent out, put in a custom header (like "x-email-check-id") to have a unique identifer for each email being sent (maybe create a GUID and store that in the database for each email)

On the server end, write a cron job to periodically download emails from the bounce address, and look for this header in the reply message. That way, you can match the bounce mail with the email sent out. This way, you can establish that this email is bounced from your sending and is not a spam email.

After that, search the email content for specific keywords to determine the reason for bounce. This list of keywords should be updated often as newer email servers return different text. But here is the list that I used.

$Email__Full[] = 'databytes limit';
$Email__Full[] = 'exceeded dropfile size';
$Email__Full[] = 'exceeded email quota';
$Email__Full[] = 'exceeded storage';
$Email__Full[] = 'exceeding receiving limits';
$Email__Full[] = 'exceeds the maximum size';
$Email__Full[] = 'folder is full';
$Email__Full[] = 'mail system full';
$Email__Full[] = 'mailbox exceeds allowed size';
$Email__Full[] = 'mailbox full';
$Email__Full[] = 'mailbox has exceeded the limit';
$Email__Full[] = 'mailbox is full';
$Email__Full[] = 'mail box full';
$Email__Full[] = 'out of disk space';
$Email__Full[] = 'out of diskspace';
$Email__Full[] = 'over disk quota';
$Email__Full[] = 'over quota';
$Email__Full[] = 'over the allowed quota';
$Email__Full[] = 'problem with the recipient\'s mailbox';
$Email__Full[] = 'quota exceeded';
$Email__Full[] = 'quota violation';
$Email__Full[] = 'space has been used up';
$Email__Full[] = 'space not enough';
$Email__Full[] = 'status: 5.2.1';
$Email__Full[] = 'status: 5.2.2';
$Email__Full[] = 'status: 5.2.3';
$Email__Full[] = 'status: 5.3.3';

$Email__NotExist[] = '/var/mail/nobody';
$Email__NotExist[] = '550 5.1.1';
$Email__NotExist[] = 'account expired';
$Email__NotExist[] = 'account inactive';
$Email__NotExist[] = 'account suspended';
$Email__NotExist[] = 'address is administratively disabled';
$Email__NotExist[] = 'address is not recognized';
$Email__NotExist[] = 'address is rejected';
$Email__NotExist[] = 'address not recognized';
$Email__NotExist[] = 'address rejected';
$Email__NotExist[] = 'bad destination email address';
$Email__NotExist[] = 'cname lookup failed';
$Email__NotExist[] = 'connection refused';
$Email__NotExist[] = 'couldn\'t find any host';
$Email__NotExist[] = 'could not be found';
$Email__NotExist[] = 'deactivated mailbox';
$Email__NotExist[] = 'delivery time expired';
$Email__NotExist[] = 'destination server not responding';
$Email__NotExist[] = 'disabled or discontinued';
$Email__NotExist[] = 'does not exist';
$Email__NotExist[] = 'does not like recipient';
$Email__NotExist[] = 'doesn\'t have a yahoo.com account';
$Email__NotExist[] = 'doesn\'t have a yahoo.com.sg account';
$Email__NotExist[] = 'domain is for sale';
$Email__NotExist[] = 'find a mail exchanger';
$Email__NotExist[] = 'find any host named';
$Email__NotExist[] = 'following address(es) failed';
$Email__NotExist[] = 'host unknown';
$Email__NotExist[] = 'i couldn\'t find any host by that name';
$Email__NotExist[] = 'illegal user';
$Email__NotExist[] = 'in the queue too long';
$Email__NotExist[] = 'inactive recipient';
$Email__NotExist[] = 'inactive user';
$Email__NotExist[] = 'incorrectly addressed';
$Email__NotExist[] = 'invalid recipient';
$Email__NotExist[] = 'invalid user';
$Email__NotExist[] = 'isn\'t in my control/locals file';
$Email__NotExist[] = 'mailbox not available';
$Email__NotExist[] = 'mailbox not found';
$Email__NotExist[] = 'mailbox unavailable';
$Email__NotExist[] = 'mail is looping';
$Email__NotExist[] = 'message is looping';
$Email__NotExist[] = 'name is not recognized';
$Email__NotExist[] = 'name not found';
$Email__NotExist[] = 'name not recognized';
$Email__NotExist[] = 'no local mailbox';
$Email__NotExist[] = 'no longer accepts mail';
$Email__NotExist[] = 'no longer an employee';
$Email__NotExist[] = 'no longer on server';
$Email__NotExist[] = 'no longer valid';
$Email__NotExist[] = 'no mailbox';
$Email__NotExist[] = 'no route found to domain';
$Email__NotExist[] = 'no such user';
$Email__NotExist[] = 'none of the mail servers for the destination domain has so far responded';
$Email__NotExist[] = 'no valid host';
$Email__NotExist[] = 'no valid recipients';
$Email__NotExist[] = 'not a valid mailbox';
$Email__NotExist[] = 'not listed';
$Email__NotExist[] = 'not listed in domino';
$Email__NotExist[] = 'possible mail loop';
$Email__NotExist[] = 'recipient cannot be verified';
$Email__NotExist[] = 'recipient unknown';
$Email__NotExist[] = 'recipients are invalid';
$Email__NotExist[] = 'recipnotfound';
$Email__NotExist[] = 'relaying denied';
$Email__NotExist[] = 'several matches found in domino';
$Email__NotExist[] = 'status: 5.1.1';
$Email__NotExist[] = 'status: 5.1.3';
$Email__NotExist[] = 'status: 5.1.4';
$Email__NotExist[] = 'status: 5.1.6';
$Email__NotExist[] = 'status: 5.1.7';
$Email__NotExist[] = 'this user doesn\'t have a yahoo';
$Email__NotExist[] = 'too many hops';
$Email__NotExist[] = 'unable to deliver';
$Email__NotExist[] = 'unable to relay';
$Email__NotExist[] = 'unknown address';
$Email__NotExist[] = 'unknown recipient';
$Email__NotExist[] = 'unknown user';
$Email__NotExist[] = 'unrouteable address';
$Email__NotExist[] = 'user doesn\'t have';
$Email__NotExist[] = 'user unknown';
$Email__NotExist[] = 'x-notes; 550 5.1.1';
$Email__NotExist[] = 'x-notes; recipient\'s domino directory';
$Email__NotExist[] = '$B%G%#%l%/%H%j$K$O8+$D$+$j$^$;$s!#'; //* Returned from japanese domino . Somehow means directory is not found.

$Email__Banned[] = '_spam_';
$Email__Banned[] = '550 5.7.1';
$Email__Banned[] = '550 access denied';
$Email__Banned[] = '550 no thanks';
$Email__Banned[] = '550 spam';
$Email__Banned[] = '554 5.7.0';
$Email__Banned[] = '554 5.7.1';
$Email__Banned[] = '554 denied';
$Email__Banned[] = '554 message does not conform to standards';
$Email__Banned[] = '554 message refused';
$Email__Banned[] = '571 message Refused';
$Email__Banned[] = 'activated my anti-spam features';
$Email__Banned[] = 'administrative prohibition';
$Email__Banned[] = 'appears to contain uce/spam';
$Email__Banned[] = 'appears to be spam';
$Email__Banned[] = 'appears to be unsolicited';
$Email__Banned[] = 'banned for spamming';
$Email__Banned[] = 'blackholed by uribl.com';
$Email__Banned[] = 'blacklisted by surbl';
$Email__Banned[] = 'blocked by filter';
$Email__Banned[] = 'blocked by kbas system';
$Email__Banned[] = 'blocked by our content filter';
$Email__Banned[] = 'blocked by spamsssassin';
$Email__Banned[] = 'blocked by the recipient';
$Email__Banned[] = 'blocked by user\'s personal blacklist';
$Email__Banned[] = 'blocked due to spam like qualities';
$Email__Banned[] = 'blocked using spam pattern';
$Email__Banned[] = 'blocked using uceprotect';
$Email__Banned[] = 'body part contains disallowed string';
$Email__Banned[] = 'classified as **spam**';
$Email__Banned[] = 'content blacklist';
$Email__Banned[] = 'content filter rejected the message';
$Email__Banned[] = 'consider the message to be spam';
$Email__Banned[] = 'considered spam';
$Email__Banned[] = 'contains spam';
$Email__Banned[] = 'content filter rejection';
$Email__Banned[] = 'content rejected';
$Email__Banned[] = 'denied by policy';
$Email__Banned[] = 'detected your message as spam';
$Email__Banned[] = 'does not accept UCE';
$Email__Banned[] = 'domain is banned';
$Email__Banned[] = 'email abuse detected';
$Email__Banned[] = 'error 553';
$Email__Banned[] = 'help_spam_16.htm';
$Email__Banned[] = 'high on spam scale';
$Email__Banned[] = 'http://www.google.com/mail/help/bulk_mail.html';
$Email__Banned[] = 'http://www.surbl.org/lists.html';
$Email__Banned[] = 'identified as spam';
$Email__Banned[] = 'identified your message as spam';
$Email__Banned[] = 'listed in multi.surbl.org';
$Email__Banned[] = 'looks like spam';
$Email__Banned[] = 'mail rejected by windows live hotmail for policy reasons';
$Email__Banned[] = 'message bounced by administrator';
$Email__Banned[] = 'message content rejected, ube';
$Email__Banned[] = 'message contains a virus or other harmful content';
$Email__Banned[] = 'message has too high spam probability';
$Email__Banned[] = 'message is blacklisted';
$Email__Banned[] = 'message refused';
$Email__Banned[] = 'no spam here';
$Email__Banned[] = 'not accepted here';
$Email__Banned[] = 'not in the whitelist';
$Email__Banned[] = 'not rfc compliant';
$Email__Banned[] = 'permanently rejected message';
$Email__Banned[] = 'policy violation';
$Email__Banned[] = 'refused mail service';
$Email__Banned[] = 'reject spam mail';
$Email__Banned[] = 'reject the mail';
$Email__Banned[] = 'rejected by filter';
$Email__Banned[] = 'rejected as spam';
$Email__Banned[] = 'rejected by 35 antiSpam system';
$Email__Banned[] = 'rejected by antispam system';
$Email__Banned[] = 'rejected by cloudmark anti-spam';
$Email__Banned[] = 'rejected by our Spam Filter';
$Email__Banned[] = 'rejected by spam filtering';
$Email__Banned[] = 'rejected by surbl';
$Email__Banned[] = 'rejected by the anti-spam system';
$Email__Banned[] = 'rejected due to security policies';
$Email__Banned[] = 'rejected for policy reasons';
$Email__Banned[] = 'rejected for spam';
$Email__Banned[] = 'requires that you verify';
$Email__Banned[] = 'scored as spam';
$Email__Banned[] = 'sender address verification';
$Email__Banned[] = 'sender was rejected';
$Email__Banned[] = 'spam detected by spamassassin';
$Email__Banned[] = 'spam filter pattern';
$Email__Banned[] = 'spam-like characteristics';
$Email__Banned[] = 'spam mail detected';
$Email__Banned[] = 'spam mail refused';
$Email__Banned[] = 'spam message was blocked';
$Email__Banned[] = 'spam rejected';
$Email__Banned[] = 'spam score too high';
$Email__Banned[] = 'status: 5.7.1';
$Email__Banned[] = 'support.proofpoint.com';
$Email__Banned[] = 'suspected spam';
$Email__Banned[] = 'suspicious url in message body';
$Email__Banned[] = 'temporarily deferred';
$Email__Banned[] = 'transaction failed';
$Email__Banned[] = 'unacceptable content';
$Email__Banned[] = 'unacceptable mail content';
$Email__Banned[] = 'unsolicited bulk e-mail';
$Email__Banned[] = 'unsolicited bulk email';
$Email__Banned[] = 'unsolicited email is refused';
$Email__Banned[] = 'validate recipient email';
$Email__Banned[] = 'will be deleted from queue';

$Email__OutOffice[] = 'away for a vacation';
$Email__OutOffice[] = 'away from the office';
$Email__OutOffice[] = 'not in office';
$Email__OutOffice[] = 'on leave';
$Email__OutOffice[] = 'on vacation';
$Email__OutOffice[] = 'out of office';
$Email__OutOffice[] = 'out of the office';

Answer 2

Bounce handling is non-trivial as you need to deal with a lot of different servers and responses. This includes those servers that are misconfigured and bounces because your email/host/IP has been blacklisted / spam-scored. And there are tons of bounces because of "out of office" auto-replies.

A simple solution I know is to put own headers into the message that identify the receiver address and the concrete sending (a token). When it bounces, the bounce can be verified based upon this.

Another solution is to encode additional information into the sender address and parse it accordingly for bounces. For this you need to know about how email works, which generally is useful if you need with bounces.

• Bounce Email handling with PHP?
• Tool for parsing SMTP logs that finds bounces
• How to check if an email address exists without sending an email?
• Specifying the bounce-back address for email

There is some great answer here on SO (but I haven't found it for this answer) that goes into many details on the topic. If I find it, I'll add it.

The general point is, that I don't know about many PHP based tools to deal with bounces, I know PHPList (PHP components for mailinglist handling) has something in their repository:

• Settings for handling bounces (last valid snapshot) (latest equivalent: Bounce Management - PHPList Manual)

Edit: The related question with the very informative answer is:

• How to send 100,000 emails weekly?

Edit: It's not much, just some links:

I am wondering why there is no standard in email bouncing that allows you easily to identify bounces and the bounce reason - or is there? (like a RFC)

There is RFC3834 - Recommendations for Automatic Responses to Electronic Mail which go updated by RFC5436 Sieve Notification Mechanism: mailto.

Bouncing is also discussed in part in RFC5321 Simple Mail Transfer Protocol, which clearly gives a reason that an MTA must report back if a message can not be delivered.

The main problem is that the bounce-address (the sender) can not be verified in SMTP, so sending a bounce message bears the problem that it probably goes to a wrong person (Backscatter Spam).

A relyable mailer therefore not only checks for bounces but does more with SMTP to gain a certain level of quality, see:

• Bounce Address Tag Validation (BATV)
• Sender Policy Framework (SPF)

The whole Wikipedia page of the Non delivery report might be interesting for a first view on the picture.

Answer 3

Being a 15 year veteran of online communities, I know that answering a question with a for-profit commercial solution, which the one responding is affiliated with, is walking a fine line. So here I go, walking that line. I'll do my best to address the question specifically. Let's start with a standard disclaimer...

DISCLAIMER: I am a principle in the service described below.

but investing the resources in writing our own library to parse the bounced e-mails is very undesirable.

Yes, writing your own libraries is no fun for most (but us poor, twisted souls with nothing better to do), considering that the library you write or include may be considered out-dated within days of your modifications.

Does anyone know of an up-to-date set of rules for processing bounced e-mails? I don't necessarily need any handling logic, even just an up-to-date ruleset would be satisfactory.

Maintaining such libraries are significantly time consuming, and once it becomes significantly time consuming such libraries are often either abandoned or sparsely updated. This is usually how commercial solutions are born.

I'm a big fan of Open Source, and there is no doubt that I've personally benefitted greatly from it over the years. For the great majority of you reading this, the above options are perfectly sufficient, as many of you are capable of developing the solutions you need yourselves or modify the available tools to suit your needs, and have the time to do so.

For those of you, however, that do not wish to build your own applications, debug and troubleshoot them, update the email processing definitions consistently and often, InboxResponse.com offers email bounce processing as an SAAS solution. It is:

• Updated often and consistently
• Perfect for those of you that just want something that works
• Can be integrated into any existing mailing systems you presently use, within minutes.
• Supports delivery tracking, bounce processing, multiple levels of soft-bounce processing (temporarily undeliverable emails), unlimited mailing lists, stats, graphs, and more.

A free trial is available at the following URL: https://inboxresponse.com/free-trial/

May not have been exactly what you were looking for (an Open Source library always kept up-to-date), but I don't think exactly what you are looking for is presently available.

Hope that helps someone!

Answer 4

I'm a large fan of the ezComponents library, which includes a Mail component which can handle both sending, and parsing mail (as well as connecting to your MTA to retrieve it. http://ezcomponents.org/docs/tutorials/Mail#mail-parsers

Reading through the docs, it seems like it should be up to your task.

Answer 5

After finding no simple answers to this question, I found this excellent solution on github:

https://github.com/cfortune/PHP-Bounce-Handler

Answer 6

its easier than you think to write this yourself.

You simply set the bounce back path in the headers. Then set up that email address to run a processing script. you obviously would need to send some form of user id.

I have used PHPList before and find it fine, there are better services to do what you need but obviously there is a cost for them.

Bounces occur when the address doesn't exists or the mail server sends it back. Some mail servers will not reply (as a security feature) so accuracy won't be the best.

A lot of the standalone/libraries offer extra features working out which links were clicked and by who working out who read the email. etc

these things are easy to achieve but take resources.

Let me know if there is anything else you need help with