Wednesday, June 26, 2013

webdriver windows server 2012 IE10 slow sendKeys keytype

I was writing a selenium test for Login and while running it on local virtual box vm it runs fine but when I ran it on an EC2 windows server 2012 instance it was crawling, every keypress was taking 2-3 sec. At first I thought its the autosuggest that is slow but even keypress in password column were slow. On googling I found that the 64 bit webdriver selenium native driver was the culprit.

Replacing that with a 32 bit driver fixed the issue.

Monday, June 24, 2013

Kibana/logstash integration with reports or JIRA

We use Logstash for our centralized logging. All our app nodes track exceptions and write them to scribe, in night a python scribe consumer aggregates that information and sends an email with top exceptions with counts and first 200 characters of the exception.  Well the problem was that now I have to go through email and for each row I need to find the exception in logstash or in application logs. This is painful, I wanted to find some better way.

So I changed our exception tracking code to include a requestId into scribe logs and this requestId is logged with each exception to logstash. Now all I need to do is add a column in my table to include a logstash query link.  Problem is that it wasnt that straight forward. Ultimately after reading some forums I found that I need  to  prepare a search json and then base64 encode it and then create a url and voila I was done.  What used to take 1 hour to trace 10 exceptions would take me just 30 mins. 

    linkPrefix = "" % dc_name.lower()
    now_java_ts = int(time.time()) * 1000
    searchJson = """{"search":" @fields.requestid:\\"%s\\"","fields":[],"offset":0,"timeframe":"172800","graphmode":"count","time":{"user_interval":0},"stamp":%s,"mode":"","analyze_field":""}"""%(sampleRequestId, now_java_ts)
    searchJsonEncoded = base64.b64encode(searchJson)
    linkUrl = linkPrefix + searchJsonEncoded
    link = "
RequestId:%s" %(linkUrl, sampleRequestId)

The new report looks like

Other fancy things in the report are these Up and down arrows that tell me if the exceptions are going up or down and % will tell me how much they are up and down from yesterday or after a release.

the link on count will take me to graphite graph that tells me how an exception is trending and when it was introduced as shown below.

Wednesday, June 19, 2013

java Fair share Threadpool

To avoid thundering herd problem we only allow X no of write and Y no of reads to Msyql database from a node. Recently I introduced HA into our tomcat stack  that reduced no of nodes by 60% and the HA is helpful but it can happen that one customer can hog all the threads in the cluster.  Before HA this would cause a downtime of only one node but now it has a potential to bring down 1/4 th of data centre.

To avoid this issue I was looking for various alternatives and finally the idea was to use a fair share thread pool that would pin an upper bound on no of threads per customer but it was becoming too complex and I was not going anywhere. I kept it as a background thread and then the worse happened and yesterday we had a downtime as one bad customer gobbled up all reader threads.

So in crunch I came up with a java fair share threadpool  approach by implementing a pool of thread pool. Each customer in our site has a random UUID called as customerId all read/write methods have it in the argument.  So I used AOP to intercept all methods at a layer and then created 4 pools, when a method is called I hash the customerId and use any of the 4 pools based on modulus of hash.  I know its not a foolproof solution but it would buy me enough time to come up with right solution and if it works then I dont even need to think of a one ;).

The code already went live yesterday night and so far I see all 4 pools being randomly used.

Monday, June 10, 2013

Quarz HA remove unused triggers

Have to find a proper solution  for this. Recently we deployed quartz in HA mode and job info is persisted in db. Now I removed one job from spring config but I keep getting this class not found exception because the info was still in db.

When using RamStorejob it was ok because it would reinit it from config. For now I just deleted the rows from db to move on but need to find a proper solution for this.

delete from QRTZ_SIMPLE_TRIGGERS where TRIGGER_NAME= 'GoogleDocsInfoCleanerJobTrigger';
delete from QRTZ_TRIGGERS where JOB_NAME='GoogleDocsInfoCleanerJob';
delete from QRTZ_JOB_DETAILS where job_name='GoogleDocsInfoCleanerJob';

Tuesday, June 4, 2013

Junit4 custom method order or custom MethodSorters

Junit4 provides three method sorters org.junit.runners.MethodSorters.NAME_ASCENDING and org.junit.runners.MethodSorters.JVM and org.junit.runners.MethodSorters.DEFAULT .  Normally Junit recommends you dont sort your tests that way it can run randomly and I 100% agree to that for unit tests.

But I had a requirement to write selenium tests that would first register a user and then run bunch of tests like add file, move file and copy file.  Now each of these methods have to run in the order. So I started naming methods like


But soon this started looking odd method naming convention so I wanted something where I can define my own order.  It seems it was easy and all I needed was to create a custom runner. So I created this OrderedRunner and an annotation and then annotated my test methods with the new annotation and had my test class annotated with RunWith and asked it to use this runner.

public class OrderedRunner extends BlockJUnit4ClassRunner {

    public OrderedRunner(Class klass) throws InitializationError {

    protected List computeTestMethods() {
        List list = super.computeTestMethods();
        List copy = new ArrayList(list);
        Collections.sort(copy, new Comparator() {
            public int compare(FrameworkMethod o1, FrameworkMethod o2) {
                SeleniumMethodOrder order1 = o1.getMethod().getAnnotation(SeleniumMethodOrder.class);
                SeleniumMethodOrder order2 = o2.getMethod().getAnnotation(SeleniumMethodOrder.class);
                if (order1.order() == order2.order()) {
                    return 0;
                } else if (order1.order() > order2.order()) {
                    return 1;
                } else {
                    return -1;
        return copy;

and I created my own annotation

@java.lang.annotation.Target(value = { java.lang.annotation.ElementType.METHOD })
@java.lang.annotation.Retention(value = java.lang.annotation.RetentionPolicy.RUNTIME)
public @interface SeleniumMethodOrder {
    public abstract int order() default 1;

now I can annotate my classes with

public class StorageServiceWdTest extends WebDriverTestBase {

    @SeleniumMethodOrder(order = 1)
    public void testAddFile() {"adding file");

    @SeleniumMethodOrder(order = 2)
    public void testMoveFile() {"moving file");

Saturday, June 1, 2013

showing proper 500 error page

went to and found this.  One should have proper 500 page on the website instead of exposing these kind of errors :)