Skip to main content

Posts

Showing posts from August, 2011

Java and lsof

Update: I found that using Runtime.exec was a bad idea because if you have a 2GB VM footprint then the forked process would require 2G free memory in order to run the lsof command.  We had earlier writte a simple python http rpc server that would allow us to execute native commands(like creating a hardlink or running gunzip) from Java and I changed this code to delagate to RPC call few days back.
So the new code looks like

public void writeTopCommandOutput(Writer writer) throws IOException { String rpcRes = Util.doCommandRpc(rpcUrl, "", ListUtil.create("top", "-n", "2", "-b", "-d", "0.2")); writer.write(rpcRes); } public void writeLsofOutput(Writer writer) throws IOException { String pid = getJvmProcessid(); if (pid != null) { pid = pid.trim(); String rpcRes = Util.doCommandRpc(rpcUrl, "", ListUtil.create("lsof", "-p", pid)); writer.write(rpcRes); } }
 --------…

Programatically extracting quoted reply from an email

When files are uploaded to our cloud file server, we wanted to send notification email per file with its own unique email address. I will discuss how to have so many unique email address without creating a user on mail server for each file and scale out the solution in some later blog. People can just hit reply button on the generated notification email and comment on the uploaded file. When reply email reaches back the server we want to extract the comment that user added after stripping out the quoted reply form the mail client and add the clean comment to file. Seems like an easy problem isnt it, but unfortunately there is no easy way to detect the quoted reply from an incoming email because different mail clients use different way to quote a reply. On top of it quoted reply of html emails are different than plain text quoted replies.
Angle Brackets "> xxx zzz" "---Original Message---""On such-and-such day, so-and-so wrote:"html email reply in thunde…

Spring MVC and Unicode characters

We ran into an issue where some user tried entering Danish character ΓΈ in his first name and it was not updating properly in LDAP. First we thought its a LDAP issue but then I found that in some other page where we use DWR api the same character is getting updated in LDAP properly. Finally we nailed it down to Tomcat/Spring where even after doing
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />the character encoding was not properly set on request.Adding this filter solved the issue. This has to be first filter in web.xml,otherwise it wont work, I had it earlier as the second filter and i wasted some time debugging it <filter> <!-- Filter to handle content encoding for UTF-8 encoding, this has to be the FIRST FILTER, do not move --> <filter-name>encodingFilter</filter-name><filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class> <init-param> <…

mysql execute immediate

I had a requirement to generate a sharded schema where no of shards and no of tables in the shard were dynamic. Basically we wanted to shard one table so we ended up creating 8 schemas and each schema will hold 8 tables that are copy of the same schema. Now I didnt wanted to hand write the 64 table/schema creation statement so came up with this procedure that allows to dynamically build and execute sql queries.  Oracle was so easy, mysql is a little bit verbose.


    drop procedure if exists create_rdb_tables;     delimiter #     create procedure create_rdb_tables()     begin     declare v_max int unsigned default 9;     declare v_counteri int unsigned default 1;     declare v_counterj int unsigned default 1;       while v_counteri < v_max do         while v_counterj < v_max do             set @sql_text := concat('drop table if exists metadata_rdb_schema',v_counteri, '.metadata_rdb_t', v_counterj,';');             prepare stmt from @sql_text;    …

ProcessId 911

call it good luck I got process id 911 for java. :).  Hiding my laptop name to keep it anonymous.

Tomcat configurable session timeout per customer

We are a cloud file provider and more geared towards enterprise customers. We have a default session timeout of 6 hours for web ui access and recently customers had a requirement that they wanted to configure a session timeout themselves. As we host multiple customers on one node, this was an interesting requirement and we were discussing all sorts of hacks until I landed on to this api HttpSession.setMaxInactiveInterval.

So now all we need to do is upon successful login, check if admin has overridden session timeout settings for this enterprise and set that on session using the above api.

Tomcat printing catalina pid

If you are hosting more then one tomcats on a physical box in production then lot of times you might want to see the process id of running instance. We dump jstacks/top command output in a folder every 5 minutes and this helps in correlating it. Here is a sample code to dump tomcat pid.

public String getJvmProcessid() throws IOException { String pid = null; File pidFile = new File(System.getenv("CATALINA_PID")); if(pidFile.exists()) { FileInputStream fin = new FileInputStream(pidFile); List lines = IOUtils.readLines(fin); fin.close(); pid = StringUtils.join(lines.toArray()); } return pid; }