JDK 1.7: "Too many open files" due to POSIX Semaphores?

file-io, java, java-7

Solution

I was able to trace it down to this block of code:

BufferedImage image = null;
ImageInputStream stream = null;
try {
    stream = new FileImageInputStream(file);
    image = ImageIO.read(stream);

} catch (Exception ex) {
    log.error("Image could not be read: "+file.getPath());

} finally {
    // ImageIO closes input stream unless null is returned
    // http://docs.oracle.com/javase/7/docs/api/javax/imageio/ImageIO.html#read(javax.imageio.stream.ImageInputStream)
    if (stream != null && image == null) {
        try {
            stream.close();
        } catch (IOException ex) {
            log.error("ERROR closing image input stream: "+ex.getMessage(), ex);
        }
    }
}

The JavaDocs specifically say that this method (unlike the others) closes the stream automatically. In fact, when you attempt to manually close it, it throws an exception saying 'closed'. I was using this overload since the other says it wraps it in a `ImageInputStream` anyway so I thought I'd save some work.

Changing the block to use a normal `FileInputStream` fixes the leak:

BufferedImage image = null;
InputStream stream = null;
try {
    stream = new FileInputStream(file);
    image = ImageIO.read(stream);

} catch (Exception ex) {
    log.error("Image could not be read: "+file);

} finally {
    if (stream != null) {
        try {
            stream.close();
        } catch (IOException ex) {
            log.error("ERROR closing image input stream: "+ex.getMessage(), ex);
        }
    }
}

This appears to me to be a bug in JDK 1.7 as 1.6 worked fine here.

Update: I just submitted a bug report to Oracle for this issue.

Problem

I've looked through the other similar questions on SO, but they seem to be caused by other issues. First I made sure I judiciously closed all of my file handles, and then I used `lsof -p <pid of java>` to look at my list of files. It stays pretty constant throughout my runtime but then periodically I will get about 10,000 entries listed in `lsof` like this: ``` COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME ... java 36809 smm *235r PSXSEM 0t0 kcms00008FC901624000 java 36809 smm *236r PSXSEM 0t0 kcms00008FC901624000 java 36809 smm *237r PSXSEM 0t0 kcms00008FC901624000 java 36809 smm *238r PSXSEM 0t0 kcms00008FC901624000 java 36809 smm *239r PSXSEM 0t0 kcms00008FC901624000 ``` The man page says `PSXSEM` type is a POSIX Semaphore. Any clue what JDK uses POSIX Semaphores for? BTW, the app is a single threaded command line app at the moment. Potentially useful background: I first noticed this after upgrading to JDK 1.7 on Mac OS X 10.7.3: ``` java version "1.7.0_04" Java(TM) SE Runtime Environment (build 1.7.0_04-b21) Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode) ``` Update: repointing `$JAVA_HOME` at JDK 1.6 seems to be a workaround for the issue. ``` java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04-415-11M3635) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01-415, mixed mode) ``` What is JDK 1.7 doing differently?

Original source

Related problems