Skip to main content

Programatically extracting quoted reply from an email

When files are uploaded to our cloud file server, we wanted to send notification email per file with its own unique email address. I will discuss how to have so many unique email address without creating a user on mail server for each file and scale out the solution in some later blog. People can just hit reply button on the generated notification email and comment on the uploaded file. When reply email reaches back the server we want to extract the comment that user added after stripping out the quoted reply form the mail client and add the clean comment to file. Seems like an easy problem isnt it, but unfortunately there is no easy way to detect the quoted reply from an incoming email because different mail clients use different way to quote a reply. On top of it quoted reply of html emails are different than plain text quoted replies.
  1. Angle Brackets "> xxx zzz"
  2. "---Original Message---"
  3. "On such-and-such day, so-and-so wrote:"
  4. html email reply in thunderbird uses blockquote tags.
  5. yahoo/hotmail uses some div tags
Got an brilliant idea from someone to add a hash marker in the outbound notification email  so that when it comes back we can strip the text after that marker. Then I found other sites are already doing this like redmine or issueburner already does that. These guys add a marker text in outbound email like below

##### Please do not write below this line #####
Hi kalpesh,

The issue has been updated.

Updated by:     Kris Katta
Comment added:     this is a test comment
Kris Katta's Reply..

This is a test reply

To track the status of your request and set up a profile for yourself, follow the link below:





Now all that is left to extract the mail header so using some regex you can strip that. I  have handled thunderbird and outlook and will soon add yahoo/hotmail. Below is some sample code.


/**
 * @author kpatel
 */
public class QuotedReplyExtractor {
    public static final String 
REPLY_MARKER = "--- Please reply ABOVE THIS LINE to comment on this file ---";

    private static final List patterns = new ArrayList();
    static {
        patterns
                .add(Pattern.compile(".*on.*?wrote:", Pattern.CASE_INSENSITIVE));
        patterns.add(Pattern.compile("-+original\\s+message-+\\s*",
                Pattern.CASE_INSENSITIVE));
    }

    public String stripQuotedReply(String comment) {
        int startIndex = comment.indexOf(REPLY_MARKER);
        if (startIndex > 0) {
            comment = comment.substring(0, startIndex);
        }
        for (Pattern pattern : patterns) {
            Matcher matcher = pattern.matcher(comment);
            if (matcher.find()) {
                startIndex = matcher.start();
                comment = comment.substring(0, startIndex);
            }
        }
        return comment;
    }

} 

Comments

Post a Comment

Popular posts from this blog

RabbitMQ java clients for beginners

Here is a sample of a consumer and producer example for RabbitMQ. The steps are
Download ErlangDownload Rabbit MQ ServerDownload Rabbit MQ Java client jarsCompile and run the below two class and you are done.
This sample create a Durable Exchange, Queue and a Message. You will have to start the consumer first before you start the for the first time.

For more information on AMQP, Exchanges, Queues, read this excellent tutorial
http://blogs.digitar.com/jjww/2009/01/rabbits-and-warrens/

+++++++++++++++++RabbitMQProducer.java+++++++++++++++++++++++++++
import com.rabbitmq.client.Connection; import com.rabbitmq.client.Channel; import com.rabbitmq.client.*; public class RabbitMQProducer { public static void main(String []args) throws Exception { ConnectionFactory factory = new ConnectionFactory(); factory.setUsername("guest"); factory.setPassword("guest"); factory.setVirtualHost("/"); factory.setHost("127.0.0.1"); factory.setPort(5672); Conne…

Spring query timeout or transaction timeout

If you are using spring to manage transactions then you can specify default transaction timeout using

    <bean id="transactionManager"
        class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
        <property name="dataSource" ref="dataSource" />
        <property name="defaultTimeout" value="30" /> <!--30 sec--->             
    </bean>

or you can override the timeout in the annotation

    @Transactional(readOnly = false, timeout=30)


or if you are doing it programatic transactions then you can do


DataSourceTransactionManager transactionManager = new DataSourceTransactionManager(dataSource);
transactionManager.setDefaultTimeout(30);

 or you can override the timeout for one particular transaction

TransactionTemplate transactionTemplate = new TransactionTemplate();
transactionTemplate.setTimeout(30);

Python adding pid file

I have a thumbnail generator that launches multiple processes and the correct way to shut it down is to send kill -HUP to the parent process. To automate I had to write a pid file from python, it was a piece of cake
def writePidFile(): pid = str(os.getpid()) f = open('thumbnail_rabbit_consumer.pid', 'w') f.write(pid) f.close()