Programming fun at startup

We dont run our nodes in HA yet and once a customer registers he is assigned a node and he lives there. Problem is that if the node dies we incur a downtime for that customer and also we need to overallocate hardware to prepare for worse case scenario.  for the past 6 months I have been working to making the code stateless so that we can do HA and reduce our node count by 4 times. So we used to run quartz using in memory scheduler but for HA I need to run quartz in a cluster. We chose org.quartz.impl.jdbcjobstore.JobStoreTX for this. Problem was that as soon as I tried it I ran into issues because I was injecting spring beans into our quartz job using JobDataMap and JobStoreTX was trying to serialize the jobData into a table and our spring beans are not serializable.  There were two options: 1) Load the entire applicationContext in each job and read the bean from there. 2) Use the schedulerContextAsMap. After evaluating options I found scheduler context as the best o...

This weekend we would finally put the Final nail in BDB coffin.  We were using BDB in webdav, Cloud file and backup product. Over the course of last 6 months my team was able to remove BDB from webdav and Cloud file and the mysql is scaling fine. We have now billions of rows in mysql and last weekend we had a pilot migration of few backup product nodes.  This weekend we would strive to migrate all the backup nodes. I am hoping we would increase the no of rows in mysql by 30%. Mysql and sharding rocks!!

Lot of time our system deals with abuse. For e.g. some customer will move the same file between 2 folders the whole day and normally it doesn't cause issues but in some extreme cases it would generates hundreds of thousands of records.  Also there are some customers who have bots written that will make millions of call in a day. Or sometimes some customer will put malware on FTP and use our servers as a way to spread malware, this causes antvirus to flag our site as spammers causing field issues. One of the strategy we use to deal with abuse is to throttle the user for a while but sometimes it hurts good users also. In some cases the abuse is so much that it can bring down the system or hurt other genuine users. Like in case of malware we just block the customer as there is not time to reach the customer and solve the issue, some user might have accidentally share the file but we have to be ruthless.

I work remotely for a company in california from texas and was working on a customer ticket and saw that the customer account in my city only. Feels good to see that the product you work on is used globally and even at a stone's throwaway from you.

We use ldap to store customer metadata like users, customers and other stuff. Right now each node has a ldap and we are trying to consolidate ldaps across different nodes. So I loaded one ldap with data from 20 nodes and some of the queries were taking 4000 msec.  So I saw that indexes were missing . I went to  slapd.conf  and  added index customerNumber,customerDomain,email   eq index customerName eq,sub I also updated the cache size to 4G in DB_CONFIG file set_cachesize   4 0 1 I restarted ldap and suddenly things were flying but something  was fishy as the new numbers for 100 threads was faster then previous 1 thread time for all operations. So I picked one query and ran it manually and found that a simple query like below was coming empty ldapsearch -x -H ldap://localhost:389 -b "dc=xxx,dc=com" "(&(objectclass=objuser)(email=kpatel@comcast1.net))"|grep email|more I removed the index and restarted ldap and same quer...

  Traffic to my blog was down to 60% in december and I was worried what had changed but I am glad to see that its back to original level in 2nd week of January.  It seems people work less in December.

Received this spam email today morning and first thing i saw that instead of image it had a path of "c:\users\Manoj". :)