From Phoronix: “Have you ever wondered on what operating system Java works the best? While by no means is it a conclusive multi-platform comparison, for this article we ran a number of Java benchmarks on both Windows Vista Premium and Ubuntu Linux to see how the Java Virtual Machine performance differs. In addition, when running Ubuntu we had tested Sun’s official Java package as well as the OpenJDK alternative.”
They should have included XP in the tests.
And solaris
And a pony.
Sun Solaris is specifically designed to run Sun java with good performance. Its threading model is tuned for java.
It would be interesting to compare its performances with other OS.
I should hope not. Solaris (and Sun OS) predate Java by a wide margin. As with operating system, Solaris will have been designed to work under a large variety of workloads. To tune your whole OS just so another tool released by your company gets a performance edge seems silly to me.
Edit: typo
Edited 2008-12-19 16:14 UTC
The Java market is huge. If it gives them an advantage in a huge market, it might make business sense for them to do it. Increasing profit, is not a silly reason.
Actually, I believe it is the other way around: Sun optimizes java for Solaris. Anyway, Solaris has several threading models…
read all the comments, then came back and voted up cause I was still laughing
I want a pony!
I hear they taste like chicken.
What’s chicken taste like? I’ve been eating pony all my life! Everything tastes like pony!
You can’t have my pony.
brilliant
Dugg
They even forgot Poland.
do ponies even like java?
I prefer Riva coffee myself, although I was previously a Nescafe Gold drinker…
Let us not forget FreeBSD. Especially now with the ULE scheduler in 7 and the native ports build of Sun’s JDK.
Looking at the Scimark benchmark, it’s obvious that they’re comparing the Server VM on Linux to the Client VM on Windows. It _might_ also explains why the rendering tests do so much better on Windows.
Server VM is optimized for long running tasks and number crunching in general. Client VM is optimised for fast start up and smooth Swing performance. They should retest, with the JVM being the same on both Windows and Linux. Use JDK _11 on both instead of _07 on Windows and _10 on Linux. Yes, they do make performance improvements between minor versions.
Actually, if you look closer, they didn’t benchmark Sun JVM at all on Ubuntu for the rendering (the last benchmark). It looks like they forgot because the results only show openJDK results on Ubuntu vs Sun JVM on Windows.
did I miss something?
Edited 2008-12-19 10:36 UTC
You didn’t miss anything. They benchmarked the OpenJDK which according to the 1st page utilized the Server VM.
“with OpenJDK we were using Java 1.6.0_0 with the OpenJDK Server VM build 1.6.0_0-b12″
Then it is a very unfair benchmark, there were notable performance improvements between u07 and u10 versions of the 1.6 JDK on Windows and it is also unfair testing the Server VM against the Client VM (sure it uses more RAM, but AFAICR it is faster).
And I am not even sure that u10 could be called a “minor version”. Though perhaps there is not such a big jump in performance, compared to u7.
Also, there is a mistake in the Sunflow rendering test. According to the figure, the results are in seconds, and Ubuntu is 19.xx, while Vista is over 20; yet in the text they write “Windows Vista was faster, but not even by a few points.”
That’s typical Phoronix benchmarking unfortunately. They always try to run comparisons between operating systems and yet they run completely different versions of the same software and expect to get accurate benchmarks.
at my uni with did some java execution speed benchmarks about 3 years back, out of xp, debian (testing) and mac osx… osx suprisingly won
Edited 2008-12-19 15:04 UTC
Wheee! 3 years back it would have been glorious. Nowadays, I’m afraid, it’s worth nothing
In my opinion considering the jvms are not uniform
From TFA:
That’s the most uniform in the way that actually counts for most users. These are the versions that are actually used, in real life, on the tested platforms.
What I’m hearing is the old classic: “They should have tweaked my contestant to perform optimally” claim, which doesn’t make a whole lot of sense in this case. If we were talking “important server application” being deployed by a seasoned administrator, then sure. But in that case it wouldn’t be running on Vista.
Edited 2008-12-19 17:29 UTC
They run a Server VM and find that numerically heavy code (i.e. Scimark) and file access operations run faster than on the Client VM. Then they find that the Client VM runs graphically intensive applications faster than the Server VM.
To use the standard car analogy (sorry guys), it’s like complaining that a Ferarri Modena sucks because you can’t haul much cargo with it, and then proceed to complain that the Ford F-150 sucks because it doesn’t do 0-60 in under 5 seconds.
You are totally assuming that that is what I had wanted. I hate Vista with a passion and therefore I could care not care less about it’s Java benchmark results.
What I do care about is already pointed out by other people and that is that client vs server JVM switches will result in different numbers…drastically different numbers so it results in bias.
> That’s the most uniform in the way that actually counts for most users. These are the versions that are actually used, in real life, on the tested platforms.
That’s odd because the latest available update from java.com is 6u11 (and it’s even auto-update-enabled).
Dmitri
From TFA:
> The j2dbench performance was significantly better on Windows Vista Premium, but that’s likely attributed to the Intel graphics stack on Linux having little in the way of performance optimizations.
That really depends on what tests were selected to be ran in J2DBench. 6u7 on Windows has only few hw-accelerated rendering primitives.
6u10/11 has new Direct3D rendering pipeline which would have given MUCH better results on Windows. Except that the test was ran on a Intel integrated hw, on which the new Direct3D pipeline is disabled because of drivers quality and performance issues.
Dmitri
It’s worth noting that Intel graphics come on more new machines than any other graphics chipset. The Intel driver situation on both OSes is a matter of some concern.
Another great benchmark article. Love ’em.
The only real experience I’ve had with java applications are netbeans and aptana.
Aptana was installed through the Eclipse plugin on both systems, using the default packages available for each system.
Aptana flew on Vista, on Ubuntu I had to stop typing every few seconds to wait for the keystrokes to appear on the monitor.
Netbeans on the other hand was pretty slick and quick on both.
Sorry, nothing informative here, just killing a few minutes while I take a break from shoveling.
If you’re testing number crunching (like SciMark seems to), the OS shouldn’t matter much at all (definitely not 3X). Those tests should be stressing the CPU more than anything else.
Drawing and certain types of File IO may determine the relative merits of the OSes, but doing unbuffered or large IOs should once again be consistent on the same hardware because the OS can’t make the disk spin faster.
One more thing: if they do these tests on a dual-boot machine with one harddrive, which OS is closer to the beginning of the disk? That one will experience faster sequential IOs in general.
It would have been interesting to see this test if they ran it on an Apple machine and included MacOSX in the test. From what I understand Apple maintains its own branch with some interesting optimizations.
comparing OpenGL performance with win/linux for intel gfx is dumb. Use an NVidia (even a cheap one) for a fair comparison.
They did this in one of their other benchmarks too.
aside from that, its nice to see a banchmarking mistake favor linux for a change
Would be nice if all the different java versions wernt so damn confusing – see http://www.mattryall.net/blog/2008/06/java-technobabble-explained
Edited 2008-12-21 14:58 UTC
Why? Why not test the JVM that most people are using on the respective platforms, along with the graphics chipset family that they are most likely using. Because that’s what this benchmark is about. It’s not about Linux vs Windows as some seem to want to frame it. It’s not about client vs server JVM as others seem to want to frame it. And it’s not about Intel vs Nvidia. It’s about how the most commonly used (default) JVMs on the respective OSes perform on the most common hardware.
Edited 2008-12-21 20:01 UTC
There just seems to be a reoccuring case of…
“Some blah blah benchmark between windows and linux shows linux is bad at 3D”
– If you have intel like most people do, then sure, 3D will suffer, this is also not news, any benchmark in linux using Intel+3D will do baddly for linux.
If you assume 3D performance is anywhere important to the user, they wont be using an intel card.
What’s wrong with this picture? Intel has done *everything* that the FOSS community has claimed for years that we needed for a company to do in order to facilitate good Linux drivers. In fact, they’ve gone well beyond. Meanwhile, NVidia has taken the traditional closed and proprietary approach, and is lauded for their superior Linux drivers.
It was the same story with the original Radeon back when ATI was sharing their specs back in the 8500 days. FOSS driver writers took the ball and dropped it. I’m not sure that today’s radeon driver is even feature complete WRT that ancient 8500 card. let alone newer ones. Later, I started hearing claims that ATI just wasn’t “open enough”. Why can’t the FOSS world come up with good drivers even with all the help and man power Intel is providing?
Edited 2008-12-22 01:18 UTC
Its good intel has made efforts to improve their linux driver situation.
Its still beside the point, intel drivers are not that great. – NVidia has just had a lot more experience in this area.
Near as I can tell, all of your comments have been beside the point. Neither 3D nor OpenGL graphics performance is even mentioned in TFA.
Edited 2008-12-22 01:46 UTC
From the article
References to Mesa (an OpenGL implimentation) and Gallium3D would suggest OpenGL performance is a problem.
Edited 2008-12-22 02:39 UTC
The tests in Ubuntu that were slower than Vista were do to graphics driver issues and Xorg issues. The tests that came out slower for Vista were most likely non-graphical in nature. There are problems that have been known for a while now.
I could care less what platform java performs best on as I try to avoid using Java at all costs!
Why?