Skip to main content

Posts

Showing posts from February, 2014

Switching to Cloud only email

4 years ago I was a windows guy and used to use outlook for emails. Though I used gmail for personal accounts when it comes to office emails somehow I liked the native look and feel and offline nature of outlook.  Then I joined the startup and the development environment was ubuntu so I first started using thunderbird but then moved to evolution. Now evolution  had its own problems but again it was native and i was using it mainly as a task management. Daily emails would flow in via pop/imap and I would keep a copy of them on server. Then as and when I would  reply to an email or took action on it I would move  it to a folder by year. But evolution used to suck, it had performance issues handling 2000+emails so I started deleting emails to trash but again it had performance issues. But I kept using it as I could integrate gmail calendar and it would notify me about events and contacts were offline so when I was on aeroplane I could use it. Then I moved to ubuntu 12...

A journey of Scaling generation of file system snapshots

We are a cloud file system startup managing billions of files for thousands of customers and one of the constant scaling issue is "how do you generate a snapshot of cloud filesystem" and send it to the client to start an initial sync. To elaborate more let say a customer has so far uploaded 25M files. Now he starts a new office and wants to setup a Netgear/NAS appliance and install our server sync software. Before the Netgear/NAS appliance could start syncing it needs a consistent snapshot of cloud filesystem so it could start that as a starting point before it could start syncing  changes based on events.  Now 25M is a big number right now but 5 year back the same problem was with if a customer has 500K files and a new employee joins and installs a sync software on his laptop, how do you send the sync client a consistent server snapshot. It seems every 1-2 year or so we solve this problem and as scale increases we have to come up with something new to make it more faster t...

Redis publish subscribe to node.js

Disclaimer: This is no way a comparison between RabbitMQ and Redis. RabbitMQ is a great product and we use it in our startup as a queuing system. I just wanted to try out redis in my pet project so I did it. I am intrigued by the hype around node.js so am planning of writing a node.js to push real time events from server and do push notifications in browser from server. Now mostly I saw that people use a myriad of technologies from websocket, nginx, nodejs, redis/rabbitmq and in backend produce the message from Java or any other app.  Basically node.js is only used for long polling and decoupled from the main app using a message queue as a broker. So my first goal was to push message from Java to message queue and I have already used RabbitMQ in past so I thought let me this time try redis and boy it is much much easy then RabbitMQ (however at this moment I trust rabbitmq more than redis for production as I have much more experience scaling rabbitmq as a queuing system,  I...