question

miguel.oyarzo_185223 avatar image
miguel.oyarzo_185223 asked Erick Ramirez commented

S3 backup: Destination pre-check failed

Hi,

When I tried to back data to S3 bucket from OpsCenter 6.7.7, I got:

Destination pre-check failed. Verify you are able to read, write, and delete from the destination. Error: clojure.lang.ExceptionInfo: throw+: {:type :opsagent.backups.destinations/file-not-found-failure, :message "No file named 'd453b12c-c3a9-40b2-9639-08e219b85ca1.chk'"} {:type :opsagent.backups.destinations/file-not-found-failure, :message "No file named 'd453b12c-c3a9-40b2-9639-08e219b85ca1.chk'"}


I have created and tested my IAM role credentials following this: https://docs.datastax.com/en/opscenter/6.7/opsc/online_help/services/backupServiceTroubleshooting.html

They seems to be OK. Also, I tried:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::<bucket_name>/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "*"
        }
    ]
}

But still same error. The region and the name of the bucket are OK, it works from a shell session setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for aws cli.


Any idea?

opscenterbackups3awsiam
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered Erick Ramirez commented

@miguel.oyarzo_185223 Based on the info you posted, it appears the role doesn't have all the necessary privileges to operate on the S3 bucket. Apart from the ListAllMyBuckets action, the IAM user also needs the following privileges on the bucket:

  • CreateBucket
  • GetBucketLocation
  • ListBucket
  • ListAllMyBuckets
  • PutObject
  • GetObject
  • DeleteObject

If you're looking for a simple way to test that the IAM user can read, write or delete items from the S3 bucket, see this KB article I published a few years ago -- HOW TO - Connect to an AWS S3 bucket. Cheers!

3 comments Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

miguel.oyarzo_185223 avatar image miguel.oyarzo_185223 commented ·

This policy worked for me:

            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:CreateBucket",
                "s3:GetBucketLocation",
                "s3:ListBucket",
                "s3:ListAllMyBuckets",
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::<bucket>",
                "arn:aws:s3:::<bucket>/*"
            ]

So, I just added "<bucket>" & "<bucket>/*" resources and the backup uploaded files.

1 Like 1 ·
Erick Ramirez avatar image Erick Ramirez ♦♦ miguel.oyarzo_185223 commented ·

Good to hear. cheers!

0 Likes 0 ·
miguel.oyarzo_185223 avatar image miguel.oyarzo_185223 commented ·

Hi Erick,
But my policy has two sections

Firstly:

        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::<bucket_name>/*"
        },


<bucket_name> is the S3 bucket that I created, before creating the buckup job in OpsCenter. I believe "s3:*" covers all the above policies in the documentation.

Note that my first try was the list you mentioned before, but if didn't work.

Secondly:

        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "*"
        }


I believe if the resource is "*", "ListAllMyBuckets" is OK, but not "CreateBucket".
However, it seems it need something else that is not on the documentation.

Any idea?
Cheers


0 Likes 0 ·