Skip to main content

Posts

Showing posts with the label tika

Tika supported document types

Tika is a library to extract text out of documents. We wrote a remote document processor service that given a streamed document can extract the text out of it and return it back in response. The reason for streaming documents is that we didnt wanted to mount all filers on that box, as filers keeps on changes so we dont want ops people to forget adding the new filers to the box and leading to any issues. I needed a way to figure out if tika can extract the text out of a document or not before sending request to the document processor. Had to look into the code but if you are using the default AutoDetecting parser here is a way to find     public static boolean canExtractText(String extension) {         String mimeType = tika.detect("a." + extension);         return parser.getParsers().containsKey(mimeType);            }         pr...

Office 2007 and Office 2010 documents Text extraction using Tika

We were earlier using various different libraries to extract text out of word, pdf, ppt, excel and it was tricky to maintain it. Our CTO found this cool apache Tika project that made our life easy. Now extracting text out of various documents is a piece of cake. Beauty of tika library is that it can detect mimetype and other metadata automatically. Here is a sample code to extract text using Tika     @Override     public String getText(InputStream stream, int maxSize) {         Tika tika = new Tika();         tika.setMaxStringLength(maxSize);         try {             return tika.parseToString(stream);         } catch (Throwable t) {             logger.error("Error extracting text from document of type" + logIdentifier, t);   ...

Tika0.7 OutOfMemory compile issue

Not sure why don't they generate and put binaries on the site. Was trying to compile Tika0.7 and faced compile issues as tests were failing due to OutOfMemory issue. setting the below env variables before doing mvn install solved the issue export MAVEN_OPTS="-Xmx1024m"