Skip to main content

Posts

Showing posts from February, 2014

Switching to Cloud only email

4 years ago I was a windows guy and used to use outlook for emails. Though I used gmail for personal accounts when it comes to office emails somehow I liked the native look and feel and offline nature of outlook.  Then I joined the startup and the development environment was ubuntu so I first started using thunderbird but then moved to evolution. Now evolution  had its own problems but again it was native and i was using it mainly as a task management. Daily emails would flow in via pop/imap and I would keep a copy of them on server. Then as and when I would  reply to an email or took action on it I would move  it to a folder by year. But evolution used to suck, it had performance issues handling 2000+emails so I started deleting emails to trash but again it had performance issues. But I kept using it as I could integrate gmail calendar and it would notify me about events and contacts were offline so when I was on aeroplane I could use it. Then I moved to ubuntu 12.04 and again evolu

A journey of Scaling generation of file system snapshots

We are a cloud file system startup managing billions of files for thousands of customers and one of the constant scaling issue is "how do you generate a snapshot of cloud filesystem" and send it to the client to start an initial sync. To elaborate more let say a customer has so far uploaded 25M files. Now he starts a new office and wants to setup a Netgear/NAS appliance and install our server sync software. Before the Netgear/NAS appliance could start syncing it needs a consistent snapshot of cloud filesystem so it could start that as a starting point before it could start syncing  changes based on events.  Now 25M is a big number right now but 5 year back the same problem was with if a customer has 500K files and a new employee joins and installs a sync software on his laptop, how do you send the sync client a consistent server snapshot. It seems every 1-2 year or so we solve this problem and as scale increases we have to come up with something new to make it more faster t

Redis publish subscribe to node.js

Disclaimer: This is no way a comparison between RabbitMQ and Redis. RabbitMQ is a great product and we use it in our startup as a queuing system. I just wanted to try out redis in my pet project so I did it. I am intrigued by the hype around node.js so am planning of writing a node.js to push real time events from server and do push notifications in browser from server. Now mostly I saw that people use a myriad of technologies from websocket, nginx, nodejs, redis/rabbitmq and in backend produce the message from Java or any other app.  Basically node.js is only used for long polling and decoupled from the main app using a message queue as a broker. So my first goal was to push message from Java to message queue and I have already used RabbitMQ in past so I thought let me this time try redis and boy it is much much easy then RabbitMQ (however at this moment I trust rabbitmq more than redis for production as I have much more experience scaling rabbitmq as a queuing system,  I just wan