Programming fun at startup

I hate last minute fixes and especially on things that I dont understand much. Today is release day and I was called by my product manager friend about some bug that I had no clue what that feature was. With some help from product manager I was able to reproduce the issue locally and 90% of the problem is solved when you can reproduce the issue. So it took 30 min to fix it as it was an easy fix.  It took more time to merge the damn fix. The reason was that developer had totally different packages in release and trunk and also in trunk there were many private interface changes. The bug has been there for 2 weeks in production so why was it detected after 2 weeks. Two reasons: 1) This was some custom feature developed for only this customer. I hate these WTF custom feature. 2) Well it seems the customer already knew about the bug and Professional services schedules calls on Friday and Friday night is our release so customer waited a week to tell us. Nothing can be done abou...

We use jfileupload applet in our cloud server to allow users to upload folder hierarchy from browser. Recently our security team found an issue that if our site was vulnerable to XSS then anyone can read the jsessionid cookie.  To fix this I changed the tomcat server.xml context attribute useHttpOnly="true" and most of the things were fine but the applet broke. now it was giving me nothing except "unable to load" and NullPointerException string (no stacktrace) in applet console.  I first thought its some local issue but then  I tried from multiple machines and same issue. Googling didnt helped. Finally after spending 3-4 hours I found that when applet tries to download the jar files the request were coming to tomcat and we were applying a WebSessionFilter that would redirect requests with no sessions to login page. Skipping .jar files download from session filter  check solved the issue.  (I know, I know we should have used apache to serve the jar files ...

I need to chase this but when I ran memcached-top command on our prod boxes I saw we had 1:20 read vs write ratio. INSTANCE USAGE HIT % CONN TIME EVICT READ WRITE memcache01:12345 85.8% 94.7% 4039 1.1ms 1.5M 4.3T 80.9T memcache02:12345 85.8% 93.4% 4022 1.2ms 1.6M 4.7T 80.5T This is throwing me off. I will analyze to see what i causing it because we should technically see more reads and less writes.

My colleague made an interesting command that AOP is like a drug and once you have tasted it you can spot cross cutting concerns and the mere presence of duplicate code tells you the signs of martin's fowler's bad smell in code.   We use jersey to implement rest apis and our rest apis can be called from Session or BasicAuth or Oauth, after a user is successfully authenticated we inject the caller user object in request as an attribute so the rest api can derive it to further make api level business validation.  But that means every api method has to write this ugly piece of code where we inject request into the signature and then add this one line of code to get the user object.       public Response getDevicesForCustomer(@Context HttpServletRequest request, ....) {         User user = (User) request.getAttribute("user"); ... } This sounds like a perfect cross cutting concern to be baked into AOP layer. What would be ni...

it seems in jenkins you cant really archive artifacts outside your workspace, I had a requirement to start tomcat(job1) and then run webdriver tests(job2 which runs on slave) and now archive the logs of tomcat(job3).  But the  tomcat lives outside of job1 or job3 workspace. Well it seems the solution is simple, add a shell step in your job that will create a soft link from the outside folder to your workspace and then you can use that softlink to archive the artifacts.

So one of my friend was facing one issue where he wrote a bunch of webdriver tests and they all work fine but when he runs on jenkins slave it randomly fails.  He runs his jenkins job hourly and problem is that it fails may be 4 times in 24 hours.  So how do you debug the issue, well he was adding loggers in the test to figure this out and then plodding over logs to figure out what went wrong. This is quite a bit of guess work and I thought sometimes a picture is better than 1000 words. So it would be nice if I could take a screenshot when the test errors out and then save it as an artifact. Guess what the webdriver already has an api for that. So all i needed to do was to add a TestRule like this and add the screenshots directory in the publish artifacts in jenkins.  I will know it in a week or so if this would save him a lot of time or not.     @Rule     public TestRule testWatcher = new TestWatcher() {       ...

Ran into an issue where customers would complain that random logins are slow and then subsequent request are fast. Took a long time to debug because it was totally random. Finally found that because we give each customer a unique subdomain like XXX.yyy.com page speed was caching the aggregated bundles per subdomain.  When we configure pagespeed we never configured the cache size so by default it was taking 100M.  Before we did Tomcat HA domains were pinned to a node so we never ran into the issue but after HA any customer can be served from any node so we were running into an issue where every hour the cache was flushed and domains would see this random login.  Took almost 2-3 hours to debug the issue and the only reason I was able to figure out the issue because I was thinking like if I had to write pagespeed how would I write it. Also I  ran a du on the cache and it was 267M and luckily I saw the apache error logs that the cache clean had ran and ran du again a...