Java: HDFS copy directory

hadoop, hdfs, java

Solution

Try one of the copy methods in FileUtil. For example:

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.FileUtil;

Configuration conf = new Configuration();  // if necessary
FileSystem fileSystem = FileSystem.get(conf);  // if necessary

FileUtil.copy(
    fileSystem, new Path("/path/to/src"),
    fileSystem, new Path("/path/to/dst"),
    false,  // move if true
    conf
);

Problem

Is there a simple way to copy a HDFS directory to another directory in Java? For example, how would I move the contents of /user/abc/pudding to /user/def/pudding? I'm looking for some HDFS equivalent to UNIX's cp command which I can do programmatically with Java. Note: I'm aware of FileSystem but it only seems to allow me to copy from my local machine to HDFS?

Original source