Resource leak in Files.list(Path dir) when stream is not explicitly closed?

java, java-8, stream

Solution

If you close the Stream, `Files.list()` does close the underlying `DirectoryStream` it uses to stream the files, so there should be no resource leak as long as you close the Stream.

You can see where the `DirectoryStream` is closed in the source code for `Files.list()` here:

return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it, Spliterator.DISTINCT), false)
                    .onClose(asUncheckedRunnable(ds));

The key thing to understand is that a `Runnable` is registered with the Stream using `Stream::onClose` that is called when the stream itself is closed. That Runnable is created by a factory method, `asUncheckedRunnable` that creates a `Runnable` that closes the resource passed into it, translating any `IOException` thrown during the `close()` to an `UncheckedIOException`

You can safely assure that the `DirectoryStream` is closed by ensuring the `Stream` is closed like this:

try (Stream<Path> files = Files.list(Paths.get(destination))){
    files.forEach(path -> {
         // Do stuff
    });
}

Problem

I recently wrote a small app that periodically checked the content of a directory. After a while, the app crashed because of too many open file handles. After some debugging, I found the error in the following line: ``` Files.list(Paths.get(destination)).forEach(path -> { // To stuff }); ``` I then checked the javadoc (I probably should have done that earlier) for `Files.list` and found: ``` * <p> The returned stream encapsulates a {@link DirectoryStream}. * If timely disposal of file system resources is required, the * {@code try}-with-resources construct should be used to ensure that the * stream's {@link Stream#close close} method is invoked after the stream * operations are completed ``` To me, "timely disposal" still sounds like the resources are going to be released eventually, before the app quits. I looked through the JDK (1.8.60) code but I wasn't able to find any hint about the file handles opened by `Files.list` being released again. I then created a small app that explicitly calls the garbage collector after using `Files.list` like this: ``` while (true) { Files.list(Paths.get("/")).forEach(path -> { System.out.println(path); }); Thread.sleep(5000); System.gc(); System.runFinalization(); } ``` When I checked the open file handles with `lsof -p <pid>` I could still see the list of open file handles for "/" getting longer and longer. My question now is: Is there any hidden mechanism that should eventually close no longer used open file handles in this scenario? Or are these resources in fact never disposed and the javadoc is a bit euphemistic when talking about "timely disposal of file system resources"?

Original source