Elliott Back's Blog - Latest Comments in Benchmarking Java 1.4 versus 1.5

Re: Benchmarking Java 1.4 versus 1.5

Abhi — Fri, 07 Dec 2007 00:23:22 -0000

hey elliot,
thts a nice comparison.....n practical too.
i ve got a query....i was preparing for sun certified java programmer (SCJP)
frm 2 months...n i was looking for 1.5 , but my company wud re-emburse the const for 1.4 only...so ..to d surrprise ive 2 give scjp 1.4 now....can u provide me any link tht clearly specificies d difference bet both in already existing
apis or shud i just go n give test ..considering both work d same way (programatically)..! thanks in advance.

Re: Benchmarking Java 1.4 versus 1.5

Christopher Oezbek — Mon, 15 Aug 2005 04:13:09 -0000

Hi Elliot,
good. I had mistaken the output data for plain 'java ...' runs that you started a couple of times from the console.

The main motivation for these question was the fact, that I am working in an empirical software engineering department and those little omissions and imprecisions can make all the difference there is for an empirical result.

Cheers and thanks for the results, :-)
Christopher

Re: Benchmarking Java 1.4 versus 1.5

Elliott Back — Fri, 29 Jul 2005 02:45:32 -0000

Chris, I'm not sure why you're asking these generally good questions so out of context. I ran the java port of linpack I found--see link above--and recorded the times. The times are measured *after* startup, by the linpack program itself, which should be obvious if you realized what I was measuring by reading the above.

The presentation of the data is minimal--you can figure it out. It was just a quick test, my friend...

Re: Benchmarking Java 1.4 versus 1.5

Christopher Oezbek — Thu, 28 Jul 2005 09:01:22 -0000

Hi Elliot,
could you add a little bit more information about which kind of tests you ran (how big was the program etc)? It is especially troubling to compare such small runs (4 seconds?), because then a bigger start-up overhead might ruin your whole comparison.

Also there are some problems with the presentation of the data:

* Why are the red and green boxes culmulative in the upper chart? In generall a histogram should not be used to present individual data points. The box plot is much better.
* A good labeling would explain that the black graph belongs to Java 5 and that the red one to Java 2.
* Data is not a good way to represent a dimension ("MFlops high is better" makes a better label).
* Maybe you can add individual data-points to the boxplot and explain the individual lines and tails better.

Thanks,
Christopher