How to use AWS Lambda to backup an S3 object to a bucket on another account?

amazon-s3, amazon-web-services, lambda

Solution

I see two possible problems.

- Your IAM policy for your Lambda function is missing permissions to write to your backup bucket.

Try adding `s3:PutObject` into your policy like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:ListBucketVersions",
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        }
    ]
}

Or, if you'd like to continue to restrict your Lambda function to read access only on your source bucket(s), you can add a second policy to your Lambda role allowing it to `s3:PutObject` on your backup bucket, naming your backup bucket as the resource, like so:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::backup-assets.myapp.com",
                "arn:aws:s3:::backup-assets.myapp.com/*"
            ]
        }
    ]
}

- Sometimes, using IAM roles as principals in your IAM policies is not as straight forward as you have. I'm not saying it's the problem, but let's loosen the restrictions until you get the copy working. Then you can tighten the policy.

For now, just let your target account access to the source bucket:

{
    "Version": "2008-10-17",
    "Id": "Backup-Assets",
    "Statement": [
        {
            "Sid": "Allow PUT from Account-A Lambda function",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ACCOUNT-A-NUMBER-HERE:root"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::backup-assets.myapp.com",
                "arn:aws:s3:::backup-assets.myapp.com/*"
            ]
        }
    ]
}

Once copying is working, you can try adding the role back.

Problem

I'm attempting to use AWS Lambda to backup objects from an `assets.myapp.com` S3 bucket under `Account-A` to a `backup-assets.myapp.com` S3 bucket under `Account-B`, but I'm getting `Access Denied` no matter the `IAM` or `Bucket Policy` configuration I use. I've got a Lambda function called `Backup-S3-Object` and it's fired by an `s3:PutObject` event trigger on the `assets.myapp.com` bucket. The desired result is that the `Backup-S3-Object` function will the perform an `s3.copyObject` to `backup-assets.myapp.com` using the AWS-SDK for Javascript. The Lambda Code ``` var aws = require('aws-sdk'); var s3 = new aws.S3({ apiVersion: '2006-03-01' }); exports.backupObject = function(event, context, callback) { var data = event.Records[0]; var sourceBucket = data.s3.bucket.name; var targetBucket = 'backup-' + data.s3.bucket.name; var key = data.s3.object.key; console.log('BACKUP: ' + sourceBucket + '/' + key + ' to ' + targetBucket); s3.copyObject({ Bucket : targetBucket, CopySource : sourceBucket + '/' + key, Key : key, ACL : 'private', ServerSideEncryption : 'AES256' }, function(error, data) { if (error) return context.done(error); return context.done(null, 'Successful backup of ' + sourceBucket + '/' + key); }); }; ``` The Lambda Role S3 Policy I've got the following policy for the `Backup-S3-Lambda-Role` which is assigned to my Lambda function. This allows the Lambda function access to `List` or `Get` any object, from any bucket, under `Account-A`, which is the source account. So this policy will allow the Lambda function to get the object from `assets.myapp.com`, which is the source bucket. ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:ListBucketVersions", "s3:GetObject", "s3:GetObjectVersion" ], "Resource": [ "arn:aws:s3:::*" ] } ] } ``` The `backup-assets.myapp.com` Bucket Policy And then, under `Account-B`, I've got the following bucket policy attached to the `backup-assets.myapp.com`, which is intended to allow the Lambda function, running under `Account-A`, to write to the `backup-assets.myapp.com` bucket under `Account-B` ``` { "Version": "2008-10-17", "Statement": [ { "Sid": "Allow PUT from Account-A Lambda function", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::ACCOUNT-A-NUMBER-HERE:role/Backup-S3-Lambda-Role" }, "Action": "s3:PutObject", "Resource": [ "arn:aws:s3:::backup-assets.myapp.com", "arn:aws:s3:::backup-assets.myapp.com/*" ] } ] } ``` What's Happening? The trigger on the `assets.myapp.com` bucket fires just fine. It triggers the `Backup-S3-Object` Lambda function, and the function logic is able to determine the correct bucket names and object key to be copied, but it always fails with `Access Denied`. I've done a test using the same setup and transferring to another bucket that's also under `Account-A` and it works just fine. So the problem is obviously with the cross-account permissions, but nothing I've tried works. I've also attempted using the `CanonicalID` of `Account-A` as the `Principal` in the `backup-assets.myapp.com` bucket policy, but still get `Access Denied` as the resulting error. And I've attempted using `"AWS": "arn:aws:iam::ACCOUNT-A-NUMBER-HERE:root"` as the principal, same error. Any help? A Solution: As outlined by Matt below, the solution is that you need to include `s3:PutObject` permission in the Lambda role policy. This is a little weird to me, but the general idea is that any permissions you grant the Lambda role in the Bucket Policy of your target bucket must also be replicated in the Lambda policy under your source account. It seems that the Lambda role is adopted entirely when you reference it in the Bucket Policy. To achieve this I've added a second inline policy to my Lambda role that allows `s3:PutObject` to the `backup-assets.myapp.com` bucket under `Account-B`. This disallows the Lambda function to write to any of the source buckets, but does allow it to write to the backup bucket.

Original source