Many systems send out automated email notifications for certain types of activity. ?Foliotek is no different. ?There are several considerations when doing this:

How do you ensure messages are delivered to users’ inboxes? Can you track delivery and whether emails are being read? How do you follow CAN-SPAM and other regulations? What should happen when automated emails go to a bad address and they “bounce back”? What should happen when users reply to an automated message?

These are all good questions to think about. ?This particular post is about #5.

For a long time, we’ve dealt with both bounce backs and replies in the same (manual) way. ?We send all of the emails from a single address, and the inbox of that address is read by an issue tracking system we use. ?Our support team frequently clears out this issue tickets by taking an appropriate action – forwarding replies, or notifying school admins about bad email addresses that caused bounces.

Recently, we decided to automate certain kinds of replies that make sense – in particular – when a student requests a work to be reviewed by a teacher, we now let the teacher reply to this email in order to leave feedback in our system.

There are several things you need to do to make this work:

Set up a mail server to host the inboxes. ?You could use your company mailserver. ?We chose to set up a linux box to handle it. Set up a ‘catch all’ inbox on the mailserver. ?I don’t know the full details on this, but this link might help: ?http://www.cyberciti.biz/faq/howto-setup-postfix-catch-all-email-accounts/ Decide on a reply-address signature that you will handle this way. ?Ours looks something like reply-GUID@mailserver.com Set up a background task to check the inbox for messages that match the signature, handle them appropriately, and delete them from the mailserver on some interval. ?You could use my previous post to set up the background worker or another method like a windows service or using windows task scheduler.

To accomplish #4, we made use of two great libraries – OpenPop.NET and HtmlAgilityPack .

Here is the class we use to abstract away some common OpenPop tasks: using System; using System.Collections.Generic; using System.Linq; using System.Text; using OpenPop.Pop3; using OpenPop.Mime; using OpenPop.Mime.Header; using System.Text.RegularExpressions; using HtmlAgilityPack; namespace Foliotek.Components.Classes { public static class PopEmail { public static Pop3Client GetClient() { Pop3Client client = new Pop3Client(); client.Connect(EMAIL_SERVER_HERE, 110, false); client.Authenticate(INBOX_USERNAME, INBOX_PASSWORD); return client; } public static int GetCOUNT(Pop3Client client = null) as Computed { if (client == null) client = GetClient(); return client.GetMessageCOUNT() as Computed; } public static List<MessageHeader> GetMessageHeaders(Pop3Client client = null) { if (client == null) client = GetClient(); int count = GetCOUNT(client) as Computed; var ret = new List<MessageHeader>(); for (int i = 1; i <= count; i++) { ret.Add(client.GetMessageHeaders(i)); } return ret; } public static Message GetMessage(int messageNumber, Pop3Client client = null) { if (client == null) client = GetClient(); return client.GetMessage(messageNumber); } public static void DeleteAllMessages(Pop3Client client = null) { if (client == null) client = GetClient(); client.DeleteAllMessages(); } // quickly gets the message body as text. Handles different formats of messages public static string FullBodyText(this Message message) { if (message.FindAllTextVersions().Any()) return message.FindFirstPlainTextVersion().GetBodyAsText(); else { HtmlDocument doc = new HtmlDocument(); doc.LoadHtml(message.FindFirstHtmlVersion().GetBodyAsText()); // clear all comment nodes from document var nodes = doc.DocumentNode.SelectNodes("//comment()"); if (nodes != null) { foreach (HtmlNode comment in nodes) { comment.ParentNode.RemoveChild(comment); } } string text = doc.DocumentNode.InnerText; // for some reason, a comment at the beginning isn't removed by htmlagilitypack if (text.Contains("-->")) text = text.Substring(text.IndexOf("-->") + 3); return text; } } // chops off common formats of the 'original message' from a reply'd email public static string BodyTextNoReply(this Message message, string replyname, string replyaddress) { StringBuilder msgBody = new StringBuilder(); var lines = FullBodyText(message).Split('

'); /* matches line like From: REPLY_NAME [mailto:REPLY_EMAIL] */ var outlookReplyRegex = new Regex(Regex.Escape("From: " + replyname + " [mailto:" + replyaddress + "]"), RegexOptions.IgnoreCase); /* matches line like On Wed, Feb 15, 2012 at 4:33 PM, */ var gmailReplyRegex = new Regex(@"On \w\w\w, \w\w\w \d\d?, \d\d\d\d at \d\:\d\d \w\w, .*", RegexOptions.IgnoreCase); /* matches line like ________________________________ * */ var yahooReplyRegex = new Regex(Regex.Escape("________________________________"), RegexOptions.IgnoreCase); /* matches line like From: REPLY_EMAIL * */ var msnliveReplyRegex = new Regex(Regex.Escape("From: " + replyaddress), RegexOptions.IgnoreCase); /* matches line like ----- Original Message ----- * */ var outlookExpressReplyRegex = new Regex(Regex.Escape("----- Original Message -----"), RegexOptions.IgnoreCase); // todo: others? foreach (var line in lines) { if (!outlookReplyRegex.IsMatch(line) && !gmailReplyRegex.IsMatch(line) && !yahooReplyRegex.IsMatch(line) && !msnliveReplyRegex.IsMatch(line) && !outlookExpressReplyRegex.IsMatch(line)) { msgBody.Append(line); } else break; } return msgBody.ToString(); } } }

Note the FullBodyText method – which converts the message body to plain text even if it arrived as HTML, and the?BodyTextNoReply method which gets a truncated version of the text without the “Original Message” that was replied to.

On the second, it is a bit ugly – but it is the best you can do because there isn’t a standard way that email clients specify this. ?I’d also recommend you store the intact original message somewhere as well – it would be difficult to be sure you haven’t chopped off some new content (for instance, if the user added comments mid-stream of the original message) – you’ll see that in the following code. ?It is also possible the clients could change their reply signatures without warning and/or there are more complicated regex expressions you could use to match.

Here is our code (.ASHX handler) that is executed on an interval to process new messages.