Browser uploads to s3 with instance roles

问题:

This is a follow-up on my previous question regarding policy document signing using instance profiles.

I'm developing a system that allows drag & drop uploads directly to an S3 bucket; an AJAX request is first made to my server containing the file metadata. Once verified, my server responds with the form parameters that are used to complete the upload.

The process of setting up browser based uploads is well explained here and it all works as expected in my local test environment.

However, once my application gets deployed on an EC2 instance, I'm seeing this error when the browser attempts to upload the file:

<Error>
    <Code>InvalidAccessKeyId</Code>
    <Message>The AWS Access Key Id you provided does not exist in our records.</Message>
    <RequestId>...</RequestId>
    <HostId>...</HostId>
    <AWSAccessKeyId>ASIAxxxyyyzzz</AWSAccessKeyId>
</Error>

The value of ASIAxxxyyyzzz here comes from the instance role credentials, as obtained from the metadata service; it seems that those credentials can't be used outside of EC2 to facilitate browser based uploads.

I've looked at the Security Token Service as well to generate another set of temporary credentials by doing this:

$token = $sts->assumeRole(array(
    'RoleArn' => 'arn:aws:iam::xyz:role/mydomain.com',
    'RoleSessionName' => 'uploader',
));

$credentials = new Credentials($token['Credentials']['AccessKeyId'], $token['Credentials']['SecretAccessKey']);

The call givens me a new set of credentials, but it give the same error as above when I use it.

I hope that someone has done this before and can tell me what stupid thing I've missed out :)

回答1:

The AWS docs are very confusing on this, but I suspect that you need to include the x-amz-security-token parameter in the S3 upload POST request and that its value matches the SessionToken you get from STS ($token['Credentials']['SessionToken']).

STS temporary credentials are only valid when you include the corresponding security token.

The AWS documentation for the POST request states that:

Each request that uses Amazon DevPay requires two x-amz-security-token form fields: one for the product token and one for the user token.

but that parameter is also used outside of DevPay, to pass the STS token and you would only need to pass it once in the form fields.

回答2:

As pointed out by dcro's answer, the session token needs to be passed to the service you're using when you use temporary credentials. The official documentation mentions the x-amz-security-token field, but seems to suggest it's only used for DevPay; this is probably because DevPay uses the same type of temporary credentials and therefore requires the session security token.

2013-10-16: Amazon has updated their documentation to make this more obvious.

As it turned out, it's not even required to use STS at all; the credentials received by the metadata service come with such a session token as well. This token is automatically passed for you when the SDK is used together with temporary credentials, but in this case the final request is made by the browser and thus needs to be passed explicitly.

The below is my working code:

$credentials = Credentials::factory();
$signer = new S3Signature();
$policy = new AwsUploadPolicy(new DateTime('+1 hour', new DateTimeZone('UTC')));

$policy->setBucket('upload.mydomain.com');
$policy->setACL($policy::ACL_PUBLIC_READ);
$policy->setKey('uploads/test.jpg');
$policy->setContentType('image/jpeg');
$policy->setContentLength(5034);

$fields = array(
    'AWSAccessKeyId' => $credentials->getAccessKeyId(),
    'key' => $path,
    'Content-Type' => $type,
    'acl' => $policy::ACL_PUBLIC_READ,
    'policy' => $policy,
);

if ($credentials->getSecurityToken()) {
    // pass security token
    $fields['x-amz-security-token'] = $credentials->getSecurityToken();
    $policy->setSecurityToken($credentials->getSecurityToken());
}

$fields['signature'] = $signer->signString($policy, $credentials);

I'm using a helper class to build the policy, called AwsUploadPolicy; at the time of writing it's not complete, but it may help others with a similar problem.

Permissions were the last problem; my code sets the ACL to public-read and doing so requires the additional s3:PutObjectAcl permission.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Sid": "Stmt1379546195000",
      "Resource": [
        "arn:aws:s3:::upload.mydomain.com/uploads/*"
      ],
      "Effect": "Allow"
    }
  ]
}