Copying directly from S3 to Google Cloud Storage

google-cloud-storage

Solution

The gsutil command-line tool supports S3. After you've configured gsutil, you'll see this in your ~/.boto file:

# To add aws credentials ("s3://" URIs), edit and uncomment the
# following two lines:
#aws_access_key_id = 
#aws_secret_access_key = 

Fill in the `aws_access_key_id` and `aws_secret_access_key` settings with your S3 credentials and uncomment the variables.

Once that's set up, copying from S3 to GCS is as easy as:

gsutil cp -R s3://bucketname gs://bucketname

If you have a lot of objects, run with the `-m` flag to perform the copy in parallel with multiple threads:

gsutil -m cp -R s3://bucketname gs://bucketname

Problem

I can migrate data from Amazon AWS S3 to Azure using AWS SDK for Java and Azure SDk for Java. Now I want to do migrate data from Amazon AWS S3 to Google Cloud storage using Java.

Original source

Related problems