Odds and Ends: avoiding non-heap OutOfMemory

We've been investigating an OutOfMemoryError when converting a PDF to an Image - not unheard of for large documents, but this was a simple one and we couldn't see why. Even profiling didn't show anything wrong with the heap.

Closer analysis showed it was the non-heap memory pool that was exhausted. This is the portion of Java's memory used by native code, and it's also known as "PermGen" space, for permanent generation. This is where your interned Strings and so on get allocated, and with a name like "Permanent Generation" you'd imagine when objects are allocated there, they're there to stay.

Except that's not the case. Methods like ByteBuffer.allocateDirect and classes like java.util.zip.Deflator use this space for the native objects they wrap, and when those Java objects are finalized they manually free up the space. And therein lies the problem.

Finalization

Finalization is only called when the object is garbage collected, and (for reasons unknown to me) allocations in this memory pool don't seem to count when determining if an unreachable object is a good candidate for a minor garbage collection. I'm not going to wade into garbage collection here - others have written heavily on the subject, and anyway wading in garbage is disgusting. The problem is fairly well known - Bug 4469299 is one of several on the topic. So what's the solution?

Solutions?

1. If native memory allocation is failing, don't allocate native memory. Fine advice. But in our case we're dealing with a PDF made up of many small bitmap images. Apple's AWT implementation renders these using an apple.awt.OSXOffScreenSurfaceData object, and under the hood that makes a call to ByteBuffer.allocateDirect. We can't get to this code, so we're stuck with it.
2. Don't use java.nio.MappedByteBuffer. A slight diversion from our main topic, but in our considered opinion the MappedByteBuffer class is hazardous. The reasons why are complicated, but boil down to the fact that there's no good way to identify when a mapped buffer is no longer in use and is suitable for garbage collection. It's quite an interesting problem if you're into that sort of thing. Our suggestion: if you're opening one file and you expect it to be around for the life of your application, go ahead. If you're opening and closing many files, then the use of this class is a guaranteed way to run out of non-heap memory. Use a FileChannel and non-direct ByteBuffer objects instead.
3. Increase the memory pool. Easily done by adding -XX:MaxPermSize=128m to the list of arguments to Java, but it's postponing the problem, not fixing it.
4. Control this with other JRE parameters. An ideal solution if it exists, but we tried -XX:+CMSClassUnloadingEnabled and -XX:+CMSPermGenSweepingEnabled - both a known solution for some cases, they didn't work for us.
5. Force a full garbage collection by calling System.gc(). This works - the java.nio.ByteBuffer objects that are no longer referenced are all deallocated, and our non-heap pool has plenty of space. The trouble with calling System.gc() is that it's expensive - it suspends all other threads, potentially halting executing of your program for anywhere up to a few seconds. Better than an OutOfMemory, but not something you want to do unless you have to.

It's time to take out the garbage

So how can we tell when we need to collect the garbage? By using the management beans added in Java 5. These are typically used by profilers, applications servers and the like to monitor how the VM is running, but they're very easy to work with. We added the following code.

static void enableGarbageWatcher() {
  final List list = ManagementFactory.getMemoryPoolMXBeans();
  for (int i=0;i<list.size();i++) {
    MemoryPoolMXBean bean = (MemoryPoolMXBean)list.get(i);
    if (bean.getType() != MemoryType.NON_HEAP || bean.getName().equals("Code Cache")) {
      list.remove(i--);
    }
  }

  Thread t = new Thread() {
    public void run() {
      int count = 0;
      while (true) {
        try {
          Thread.sleep(1000);
        } catch (InterruptedException e) { }
        for (int i=0;i<list.size();i++) {
          MemoryPoolMXBean bean = (MemoryPoolMXBean)list.get(i);
          MemoryUsage mem = bean.getUsage();
          long pc = mem.getUsed() * 100 / mem.getCommitted();
          if (pc > 95 || (pc > 80 && ++count % 5 == 0)) {
            System.gc();
          }
        }
      }
    }
  };
  t.setDaemon(true);
  t.start();
}

Calling this method once from your initializer will fire up a daemon thread that checks memory usage every 1000ms. If it's 95% full, or 80% full every 5 seconds, it will run a full garbage collection. These figures are arbitrary but they do work for us.

The loop at the top of this method is trying to identify the correct Memory Pool to monitor. On our Apple OS X JVM it's called "CMS Perm Gen", on the Oracle 1.6 JVM on Linux it's "PS Perm Gen" - the name reflects which garbage collector is in use, but the one we're trying to find is a non-heap MemoryPool that's not called "Code Cache", which is for something else.

An ugly solution to an ugly problem

The heading says it all really. This isn't particularly pretty but it's an awful lot better than an OutOfMemoryError. If you're seeing "OutOfMemory: PermGen" or "OutOfMemory: Direct Memory Buffer" issues then this is a fairly low-overhead way to try and work around them. Although we make no guarantees it will work.