Hadoop - FileSystem.listFiles - not listing directories

hadoop, hdfs

Solution

Are you getting any kind of error/exception ??

You might have used the following code:

FileStatus[] status = fs.listStatus(path);
for (int i=0;i<status.length;i++){
    FSDataInputStream fSDataInputStream = fs.open(status[i].getPath());
}

Problem

I am looking at this method: `FileSystem.listFiles(Path f, boolean recursive)` List the statuses and block locations of the files in the given path. If the path is a directory, if recursive is false, returns files in the directory; if recursive is true, return files in the subtree rooted at the path. If the path is a file, return the file's status and block locations. I am testing the method and it seems it's not returning the sub-directories of a given directory. Is this by design (seems it is though it's java.io counterpart doesn't work that way)? If that limitation is by design, then what are the alternatives, if I want to list all sub-directories too? Another method `FileSystem.listStatus(Path f)` is not returning the statuses of the sub-directories too. What am I missing?

Original source