PLANNED MAINTENANCE

Hello, DataStax Community!

We want to make you aware of a few operational updates which will be carried out on the site. We are working hard to streamline the login process to integrate with other DataStax resources. As such, you will soon be prompted to update your password. Please note that your username will remain the same.

As we work to improve your user experience, please be aware that login to the DataStax Community will be unavailable for a few hours on:

  • Wednesday, July 15 16:00 PDT | 19:00 EDT | 20:00 BRT
  • Thursday, July 16 00:00 BST | 01:00 CEST | 04:30 IST | 07:00 CST | 09:00 AEST

For more info, check out the FAQ page. Thank you for being a valued member of our community.


question

miguel.oyarzo_185223 avatar image
miguel.oyarzo_185223 asked ·

S3 backup: Destination pre-check failed

Hi,

When I tried to back data to S3 bucket from OpsCenter 6.7.7, I got:

Destination pre-check failed. Verify you are able to read, write, and delete from the destination. Error: clojure.lang.ExceptionInfo: throw+: {:type :opsagent.backups.destinations/file-not-found-failure, :message "No file named 'd453b12c-c3a9-40b2-9639-08e219b85ca1.chk'"} {:type :opsagent.backups.destinations/file-not-found-failure, :message "No file named 'd453b12c-c3a9-40b2-9639-08e219b85ca1.chk'"}


I have created and tested my IAM role credentials following this: https://docs.datastax.com/en/opscenter/6.7/opsc/online_help/services/backupServiceTroubleshooting.html

They seems to be OK. Also, I tried:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::<bucket_name>/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "*"
        }
    ]
}

But still same error. The region and the name of the bucket are OK, it works from a shell session setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for aws cli.


Any idea?

opscenterbackups3awsiam
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered ·

@miguel.oyarzo_185223 Based on the info you posted, it appears the role doesn't have all the necessary privileges to operate on the S3 bucket. Apart from the ListAllMyBuckets action, the IAM user also needs the following privileges on the bucket:

  • CreateBucket
  • GetBucketLocation
  • ListBucket
  • ListAllMyBuckets
  • PutObject
  • GetObject
  • DeleteObject

If you're looking for a simple way to test that the IAM user can read, write or delete items from the S3 bucket, see this KB article I published a few years ago -- HOW TO - Connect to an AWS S3 bucket. Cheers!

3 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Hi Erick,
But my policy has two sections

Firstly:

        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::<bucket_name>/*"
        },


<bucket_name> is the S3 bucket that I created, before creating the buckup job in OpsCenter. I believe "s3:*" covers all the above policies in the documentation.

Note that my first try was the list you mentioned before, but if didn't work.

Secondly:

        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "*"
        }


I believe if the resource is "*", "ListAllMyBuckets" is OK, but not "CreateBucket".
However, it seems it need something else that is not on the documentation.

Any idea?
Cheers


0 Likes 0 · ·

This policy worked for me:

            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:CreateBucket",
                "s3:GetBucketLocation",
                "s3:ListBucket",
                "s3:ListAllMyBuckets",
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::<bucket>",
                "arn:aws:s3:::<bucket>/*"
            ]

So, I just added "<bucket>" & "<bucket>/*" resources and the backup uploaded files.

1 Like 1 · ·
Erick Ramirez avatar image Erick Ramirez ♦♦ miguel.oyarzo_185223 ·

Good to hear. cheers!

0 Likes 0 · ·