Submitted by “As a typical Java developer I never monitored the memory usage of my application apart from following typical best practices like closing the connections, streams etc. Recently we were struck with few issues in our JBoss servers that I had to dig in to the memory management.”
In Java?
I was [ts]old they uninvented memory management!
Where I work, based on the amount of permgen/out of memory errors we get on our Java server apps, I think somebody should re-invent memory management
Note that I did not write, nor am I in charge of any of these apps. I’m just the poor schmuck who gets paged at 3am when one of these apps blows chunks, and I have to restart the f**king things. Time after time after time.
Our ‘architects’ have been looking into this for what seems like years, analyzing heap dumps and what not, and they usually just give the apps more permgen space, which fixes the problem … for awhile, anyway. Then it’s back to the same BS. So, I figure either Java absolutely blows ass, or we just need some new architects.
It’s probably a combination of both. Though since the issue appears systematic, it’s probably time to redraft the system with new criteria.
Are you trying to tell me that Java isn’t running on pixie dust* that magically solves the halting problem??!!?!!
* Known as “ground coffee” to developers
Get new architects… Java works when stuff has been correctly designed which is easy to do. Bad thing about Java is that it is also easy to design things which are literally like things out of hell very easily.
Design that works in simple systems doesn’t, and will never work if the amount of data to be handled by the system is multiple times more. It needs a complete redesign and it will cost money. Not as much as if it was written in some other language, but it still will cost you guys.
I will go for new architects.
Most of my consulting projects are done in JVM and .NET languages and I see often what I call “space station architecture” (I adopted the expression from someone else).
Designs done away from the reality, with thousand layers of abstraction, because it is cool.
Most of the performance problems we had to fix in some projects always have to do with architecture decisions.
– Not the right language/OS for the problem at hand;
– Lots of nicely abstracted layers;
– Communication between modules using the wrong type of communication protocols;
– Algorithms that aren’t appropriate;
– Multiple remote calls to distributed systems;
– Data structures designed without regard for being GC friendly
– …
Hmm, I’d expect ‘space station architecture’ to be a positive classification: if you design something for a space station, you better design it well, because you can’t easily go up there and fix it later
.
Well, that’s not strictly what really happens…
NVM that our stations are more or less mostly testbeds for deep space vessels (with the convenience of, yes, being relatively easy to reach quickly; or to evacuate…), failures are common, as is experimenting with proper approaches. Also:
a) ISS is really… Mir-2 (as in, the core module of ISS is practically identical to Mir, because it was built concurrently with it, as a backup – it even had “DoeD,~NEUR-2” painted on it, when it was kept in storage; the whole Russian section is generally what was supposed to be the entirety of Mir-2, also where main computers, engines, and such of the station are – the rest is, sort of, an attached cargo). It wasn’t so much about designing things, more using what’s at hand and with minimum of costs…
b) the “non-Mir” part of the station was largely crippled in its design, with how it had to be nominally launched by the STS, so as to give that horribly wasteful contraption some pretend-purpose (really, a spacecraft wasting most of launched mass on airframe? …and, considering that we did automatic rendezvous already in the 60s, the Shuttle was conceptually obsolete before seriously getting on the drawing boards; now, with STS finally out of the way, there’s even a fairly inexpensive project of attaching small orbital tugs to some of the ISS modules remaining in-storage, to launch them like that on efficient expendable rockets)
Overall, assuming failures and building in lots of redundancies and/or planning for contingencies seems to be more the way.
Edited 2012-09-22 00:19 UTC
I figured people would say that, because either on the server or desktop, no matter how shitty Java apps run (which seems to be most of the time), it never appears to be Java’s fault.
The Permgen space is a part of the Java memory that is used for storing definitions of classes. Now, when working with Java application servers (e.g. JBoss) each web application that is running on the server uses its own classloader(s). The class loader is used to load the classes that are used inside the web application. The classloader itself is stored in the permgen space together with all the class definitions it loads.
One issue here is that the more web applications run on your application server, the more permgen space you will need. This is a simple fact and the default space reserved for permgen space is not really enough for an application server with lots of applications on it.
The other issue is that when a web application is unloaded, also the class loader that was connected to it should be freed. This is actually the responsibility of the container, that is the application server itself. The problem is that some application servers do not correctly release those class loaders and that can also lead to the problems you mention.
Is that the architects fault? Nope, not really. On the other hand, as an architect, you should know those things and take them into account when building systems. As an architect, you should really know how classloaders work and how they are linked together.
The problem is that the people that built this stuff have long since left the company, so the current guys are just doing damage control, I guess. Either way, it looks like shit’s still going to crash, and I (along with those on my team who are unfortunate enough to get stuck with the pager) will continue to be woken up in the middle of the night. As long as that happens, I will continue to hate Java, whether it is justified or not
Until then you – are a troll.
I don’t know. Java is a language that invites hotshot “architects” to devise Rube Goldberg machines. Factories that create factories that create classes. Of course they’ll want to model it all in UML first, because that’s how good code happens. If you bring in new Java architects and give them enough rope, they’ll come up with a way to make things more complex than they were before.
Once I was contracted to rewrite a troubled Python application, and after browsing the codebase for an hour, I remarked the poor guy who was maintaining this thing: “This code looks like it was written by some hotshot Java developer”. And he said: “How did you know that?”
On the other side of the coin, you have people who code like it’s 1965 and there is only 2K of RAM available. Every algorithm is a labyrinth, all in one function, usually, with undocumented shortcuts and hacks.
For permgen space errors, indeed usually increasing it is the way to go. The permanent generation is used for ‘static’ information (using the word loosely, and apart from some exceptions which are unlikely to be relevant). It is expected to grow a bit after startup, up to the point where everything relevant for your application is loaded and the usage will no longer rise, no matter how much load/data you throw at the system.
If this is an application running inside a servlet container: do you ever restart/upgrade a component/application in the container without restarting the container, too? This is possible, but avoiding memory leaks (even in permgen, as each instance will have its own classloader) is tricky, and it’s often acceptable to just restart the entire container.
For heap space errors, yeah, you need a developer to look at those.
You mention ‘architects’. In case of memory errors (both permgen and heap), I’d suspect a programming error rather than an architecture error first. I’d suggest having a good developer (‘craftsman’ seems to be the hip term these days) look at it. He should be able to pinpoint what’s the biggest culprit, and from there you can see whether it is possible to fix within the current architecture or that you really need structural changes.
Write a script to monitor those apps, so when they blowup they get automatically restarted. No need for you to babysit them at 3am.
Java does have memory management; however the memory management is done automatically with object creation and garbage collection.
This means that a lot of Java programmers aren’t aware of it, and have no idea about things that improve memory management efficiency. Techniques like recycling objects and using stack (to avoid the overhead of allocating and freeing heap) may never enter a Java developers’ mind. Basic things (like setting references to null when the object they refer to is no longer needed) may never happen.
If you make it possible for programmers to ignore memory management, then you shouldn’t be surprised if these programmers don’t do efficient memory management.
– Brendan
Don’t do this!
These techniques used to be good up to Java 1.4 JVMs, but nowadays they do more harm than good to more modern GCs.
Would you mind elaborating? I can’t see how these things can cause more harm than good (regardless of language or VM).
– Brendan
I agree with moondevil that these kinds of ‘optimizations’ are a really bad idea.
The most important reason is they will make your code more complicated, and thus harder to maintain and more likely to contain bugs.
Recycling objects: this might be useful in languages where the construction of an object corresponds directly to a malloc call. Java’s memory manager is smarter than that, so creating and destroying many objects is not really a performance problem (anymore – this used to be slow on older JVM’s, like 1.4 old).
Stack: not a bad idea per se, but won’t help much as Java only holds primitives on the stack, not objects. It’s still a good idea to prefer local variables to fields, but for other reasons than performance.
Setting references to null: This might be useful in a situation where you have a mutable reference to a big data structure which you don’t need anymore after a certain point. Such situations should be rare, and I’d suspect the code could/should be refactored in another way that would prevent the big structure to be kept alive for too long.
Of course it’s hard to talk about such things without concrete examples, and I’m not giving any either, but I hope this provides some ideas.
I would imagine that setting to NULL is unnecessary, because static analysis can tell the JVM or compiler whether the variable is used any more, and if it’s not, it can be GC’ed, whether it’s set to NULL or not. Setting to NULL is only valuable in refcounted GC implementations, and neither Java nor .NET use such a scheme.
This is true, but only for local variables.
For fields, there may be cases where an object needs to be kept alive, but one of its fields can be discarded, so setting it to NULL is advantageous. Static analysis won’t help there, taking into account Java has a reflection API. Indeed, though, such cases are probably rare, and a form of code smell in the first place.
Yes, you are correct. I was only thinking about local variables.
There is little need, however, to have fields that would become unused while the owning object remained in use. In such a case, they shouldn’t be fields. That’s not to say that people won’t do the wrong thing. I assume that’s what you were talking about with your code smell comment.
Yes, exactly.
Usually those actions tend to hinder the work of the more smarter GCs, that base their work in execution heuristics.
Some places were you can get information about it.
http://www.oracle.com/technetwork/java/hotspotfaq-138619.html#gc_po…
http://www.infoq.com/presentations/Understanding-Java-Garbage-Colle…
http://www.oracle.com/technetwork/java/gc-tuning-5-138395.html
Eh? The guy doesn’t even know what the difference is between memory management (that’s what he’s talking about) and memory model (which is memory access from threads, see http://en.wikipedia.org/wiki/Java_memory_model).
There are fundamentally different GC strategies producing fundamentally different performance and any serious Java developer must be aware of them.
Reading the article felt like: OK nice intro, finally the interesting part “When things do not go as per the plan”… Where’s the next page button? What the hell? The End.
It looks like I’m not the only one. The fun part is missing… do we have to pay for the remaining pages?
Agreed, felt like it should have “This article is a stub” text on it. Just as I felt like it was starting to get down to saying something meaningful, it just ends…
I stumbled on this cool piece of news.
Doubt OSAlert will post two Java news pieces back to back so i just post it here:)
Google releases Java to Objective C translator
http://www.h-online.com/open/news/item/Google-releases-Java-to-Obje…
Edited 2012-09-14 15:40 UTC
Cool!
FYI, they posted an article on this yeaterday.
LOL, i saw it yesterday and felt pretty stupid.
My eyes just scanned the first two news item.