Skip to main content

Programatically extracting quoted reply from an email

When files are uploaded to our cloud file server, we wanted to send notification email per file with its own unique email address. I will discuss how to have so many unique email address without creating a user on mail server for each file and scale out the solution in some later blog. People can just hit reply button on the generated notification email and comment on the uploaded file. When reply email reaches back the server we want to extract the comment that user added after stripping out the quoted reply form the mail client and add the clean comment to file. Seems like an easy problem isnt it, but unfortunately there is no easy way to detect the quoted reply from an incoming email because different mail clients use different way to quote a reply. On top of it quoted reply of html emails are different than plain text quoted replies.
  1. Angle Brackets "> xxx zzz"
  2. "---Original Message---"
  3. "On such-and-such day, so-and-so wrote:"
  4. html email reply in thunderbird uses blockquote tags.
  5. yahoo/hotmail uses some div tags
Got an brilliant idea from someone to add a hash marker in the outbound notification email  so that when it comes back we can strip the text after that marker. Then I found other sites are already doing this like redmine or issueburner already does that. These guys add a marker text in outbound email like below

##### Please do not write below this line #####
Hi kalpesh,

The issue has been updated.

Updated by:     Kris Katta
Comment added:     this is a test comment
Kris Katta's Reply..

This is a test reply

To track the status of your request and set up a profile for yourself, follow the link below:





Now all that is left to extract the mail header so using some regex you can strip that. I  have handled thunderbird and outlook and will soon add yahoo/hotmail. Below is some sample code.


/**
 * @author kpatel
 */
public class QuotedReplyExtractor {
    public static final String 
REPLY_MARKER = "--- Please reply ABOVE THIS LINE to comment on this file ---";

    private static final List patterns = new ArrayList();
    static {
        patterns
                .add(Pattern.compile(".*on.*?wrote:", Pattern.CASE_INSENSITIVE));
        patterns.add(Pattern.compile("-+original\\s+message-+\\s*",
                Pattern.CASE_INSENSITIVE));
    }

    public String stripQuotedReply(String comment) {
        int startIndex = comment.indexOf(REPLY_MARKER);
        if (startIndex > 0) {
            comment = comment.substring(0, startIndex);
        }
        for (Pattern pattern : patterns) {
            Matcher matcher = pattern.matcher(comment);
            if (matcher.find()) {
                startIndex = matcher.start();
                comment = comment.substring(0, startIndex);
            }
        }
        return comment;
    }

} 

Comments

Post a Comment

Popular posts from this blog

RabbitMQ java clients for beginners

Here is a sample of a consumer and producer example for RabbitMQ. The steps are
Download ErlangDownload Rabbit MQ ServerDownload Rabbit MQ Java client jarsCompile and run the below two class and you are done.
This sample create a Durable Exchange, Queue and a Message. You will have to start the consumer first before you start the for the first time.

For more information on AMQP, Exchanges, Queues, read this excellent tutorial
http://blogs.digitar.com/jjww/2009/01/rabbits-and-warrens/

+++++++++++++++++RabbitMQProducer.java+++++++++++++++++++++++++++
import com.rabbitmq.client.Connection; import com.rabbitmq.client.Channel; import com.rabbitmq.client.*; public class RabbitMQProducer { public static void main(String []args) throws Exception { ConnectionFactory factory = new ConnectionFactory(); factory.setUsername("guest"); factory.setPassword("guest"); factory.setVirtualHost("/"); factory.setHost("127.0.0.1"); factory.setPort(5672); Conne…

What a rocky start to labor day weekend

Woke up by earthquake at 7:00 AM in morning and then couldn't get to sleep. I took a bath, made my tea and started checking emails and saw that after last night deployment three storage node out of 100s of nodes were running into Full GC. What was special about the 3 nodes was that each one was in a different Data centre but it was named same app02.  This got me curious I asked the node to be taken out of rotation and take a heap dump.  Yesterday night a new release has happened and I had upgraded spymemcached library version as new relic now natively supports instrumentation on it so it was a suspect. And the hunch was a bullseye, the heap dump clearly showed it taking 1.3G and full GCs were taking 6 sec but not claiming anything.



I have a quartz job in each jvm that takes a thread dump every 5 minutes and saves last 300 of them, checking few of them quickly showed a common thread among all 3 data centres. It seems there was a long running job that was trying to replicate pending…

Email slavery

It seems I have become an EmailSlave. The first half of the day is spent in just answering to emails. There are so many emails where I am copied but I need not be. There are many emails  where its a 1-2 page email and somewhere down someone says @KP please answer this.  So it seems daily my work schedule is:
Signin to newrelic and check anomalies for 15 min. Check emails related production exception report and yes there are a ton of these report daily. Need a better tool here as this model is not scalable. I need to reduce the incoming data at me to only see relevant data like what newrelic does. May be I need to create a webapp out of these emails.Check emails for next few minutes before team callsDo team callsThen again back to checking emails until a I have taken a best shot at answering everyone waiting for my reply.Attend team meetings on Tue/Thu
Being an architect and coder at heart I don't feel satisfied at end of the day if there is nothing tangible getting done at the end.…