I've got an ASG
that assigns an IAM Role
to each of the instances that join it. Therefore, each instance has the AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
environment variables baked-in, which will be used upon instantiation to download and decrypt credentials that are stored in an S3
bucket and encrypted using KMS
keys.
So I'll have the following components:
- An
S3
bucket calledtop-secret.myapp.com
- All objects in this bucket are encrypted using a
KMS
key calledMy-KMS-Key
- An
IAM
instance role with inline policies attached granting it the ability to interact with both the bucket and theKMS
key used to encrypt/decrypt the contents of the bucket (see below) - A
user data
script that installs theaws-cli
upon instantiation and then goes about attempting to download and decrypt an object from thetop-secret.myapp.com
bucket.
The User Data Script
Upon instantiation, any given instance runs the following script:
#!/bin/bash
apt-get update
apt-get -y install python-pip
apt-get -y install awscli
cd /home/ubuntu
aws s3 cp s3://top-secret.myapp.com/secrets.sh . --region us-east-1
chmod +x secrets.sh
. secrets.sh
shred -u -z -n 27 secrets.sh
IAM Role Policies
The IAM
role for my ASG
instances has three policies attached inline:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "*",
"Resource": "*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::top-secret.myapp.com"
]
},
{
"Effect": "Allow",
"Action": [
"s3:List*",
"s3:Get*"
],
"Resource": [
"arn:aws:s3:::top-secret.myapp.com/secrets.sh"
]
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kms:*"
],
"Resource": "arn:aws:kms:us-east-1:UUID-OF-MY-SECRET-KEY-HERE"
}
]
}
The first policy is essentially a full-root-access policy with no restrictions. Or so I thought, but it doesn't work. So I thought it might be that I need to explicitly apply policies that allow interaction with S3
encryption and/or KMS
, makes sense.
So I added the second policy that allows the IAM
instance role to list the top-secret.myapp.com
bucket, and LIST
and GET
the secrets.sh
object within the bucket. But this produced the error illustrated below.
The (Unknown) Error I'm Getting
download failed: s3://top-secret.myapp.com/secrets.sh to ./secrets.sh
A client error (Unknown) occurred when calling the GetObject operation: Unknown
Anyone have any idea what could be causing this error?
Note: This method for transferring encrypted secrets from S3 and decrypting them on-instance works fine using the standard
Amazon S3 service master key