DataStax Academy FAQ

DataStax Academy migrated to a new learning management system (LMS) in July 2020. We are also moving to a new Cassandra Certification process so there are changes to exam bookings, voucher system and issuing of certificates.

Check out the Academy FAQ pages for answers to your questions:


question

miguel.oyarzo_185223 avatar image
miguel.oyarzo_185223 asked ·

S3 backup: Destination pre-check failed

Hi,

When I tried to back data to S3 bucket from OpsCenter 6.7.7, I got:

Destination pre-check failed. Verify you are able to read, write, and delete from the destination. Error: clojure.lang.ExceptionInfo: throw+: {:type :opsagent.backups.destinations/file-not-found-failure, :message "No file named 'd453b12c-c3a9-40b2-9639-08e219b85ca1.chk'"} {:type :opsagent.backups.destinations/file-not-found-failure, :message "No file named 'd453b12c-c3a9-40b2-9639-08e219b85ca1.chk'"}


I have created and tested my IAM role credentials following this: https://docs.datastax.com/en/opscenter/6.7/opsc/online_help/services/backupServiceTroubleshooting.html

They seems to be OK. Also, I tried:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::<bucket_name>/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "*"
        }
    ]
}

But still same error. The region and the name of the bucket are OK, it works from a shell session setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for aws cli.


Any idea?

opscenterbackups3awsiam
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

Erick Ramirez avatar image
Erick Ramirez answered ·

@miguel.oyarzo_185223 Based on the info you posted, it appears the role doesn't have all the necessary privileges to operate on the S3 bucket. Apart from the ListAllMyBuckets action, the IAM user also needs the following privileges on the bucket:

  • CreateBucket
  • GetBucketLocation
  • ListBucket
  • ListAllMyBuckets
  • PutObject
  • GetObject
  • DeleteObject

If you're looking for a simple way to test that the IAM user can read, write or delete items from the S3 bucket, see this KB article I published a few years ago -- HOW TO - Connect to an AWS S3 bucket. Cheers!

3 comments Share
10 |1000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

Hi Erick,
But my policy has two sections

Firstly:

        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::<bucket_name>/*"
        },


<bucket_name> is the S3 bucket that I created, before creating the buckup job in OpsCenter. I believe "s3:*" covers all the above policies in the documentation.

Note that my first try was the list you mentioned before, but if didn't work.

Secondly:

        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "*"
        }


I believe if the resource is "*", "ListAllMyBuckets" is OK, but not "CreateBucket".
However, it seems it need something else that is not on the documentation.

Any idea?
Cheers


0 Likes 0 · ·

This policy worked for me:

            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:CreateBucket",
                "s3:GetBucketLocation",
                "s3:ListBucket",
                "s3:ListAllMyBuckets",
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::<bucket>",
                "arn:aws:s3:::<bucket>/*"
            ]

So, I just added "<bucket>" & "<bucket>/*" resources and the backup uploaded files.

1 Like 1 · ·
Erick Ramirez avatar image Erick Ramirez ♦♦ miguel.oyarzo_185223 ·

Good to hear. cheers!

0 Likes 0 · ·