Amazon S3 Signature Does Not Match - AWS SDK Java

问题:

I have a play application that needs to upload files to S3. We are developing in scala and using the Java AWS SDK.

I'm having trouble trying to upload files, I keep getting 403 SignatureDoesNotMatch when using presigned urls. The url is being genereated using AWS Java SDK by the following code:

def generatePresignedPutRequest(filename: String) = {
    val expiration = new java.util.Date();
    var msec = expiration.getTime() + 1000 * 60 * 60; // Add 1 hour.
    expiration.setTime(msec);

    s3 match {
      case Some(s3) => s3.generatePresignedUrl(bucketname, filename, expiration, HttpMethod.PUT).toString
      case None => {
        Logger.warn("S3 is not availiable. Cannot generate PUT request.")
        "URL not availiable"
      }
    }
  }

For the frontend code we followed ioncannon article.

The js function that uploads the file (the same as the one used in the article)

 function uploadToS3(file, url)
     {
       var xhr = createCORSRequest('PUT', url);
       if (!xhr) 
       {
         setProgress(0, 'CORS not supported');
       }
       else
       {
         xhr.onload = function() 
         {
           if(xhr.status == 200)
           {
             setProgress(100, 'Upload completed.');
           }
           else
           {
             setProgress(0, 'Upload error: ' + xhr.status);
           }
         };

         xhr.onerror = function() 
         {
           setProgress(0, 'XHR error.');
         };

         xhr.upload.onprogress = function(e) 
         {
           if (e.lengthComputable) 
           {
             var percentLoaded = Math.round((e.loaded / e.total) * 100);
             setProgress(percentLoaded, percentLoaded == 100 ? 'Finalizing.' : 'Uploading.');
           }
         };

         xhr.setRequestHeader('Content-Type', 'image/png');
         xhr.setRequestHeader('x-amz-acl', 'authenticated-read');

         xhr.send(file);
       }
     }

The server's response is

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
<StringToSignBytes>50 55 bla bla bla...</StringToSignBytes>
<RequestId>F7A8F1659DE5909C</RequestId>
<HostId>q+r+2T5K6mWHLKTZw0R9/jm22LyIfZFBTY8GEDznfmJwRxvaVJwPiu/hzUfuJWbW</HostId>
<StringToSign>PUT

    image/png
    1387565829
    x-amz-acl:authenticated-read
    /mybucketname/icons/f5430c16-32da-4315-837f-39a6cf9f47a1</StringToSign>
<AWSAccessKeyId>myaccesskey</AWSAccessKeyId></Error>

I have configured CORS, double checked aws credentials and tried changing request headers. I always get the same result. Why is Amazon telling me that signatures dont match?

回答1:

Doubt the OP still has a problem with this, but for anyone else who runs into this, here is the answer:

When making a signed request to S3, AWS checks to make sure that the signature exactly matches the HTTP Header information the browser sent. This is unfortunately required reading: http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html

However in the code above this is not actually the case, the Javascript is sending:

xhr.setRequestHeader('Content-Type', 'image/png');
xhr.setRequestHeader('x-amz-acl', 'authenticated-read');

But in the Java/Scala, s3.generatePresignedUrl is being called without passing in either of them. So the resulting signature is actually telling S3 to reject anything with a Content-Type or x-ams-acl header set. Oops (I fell for it too).

I've seen browsers send Content-Types automatically, so even if they're not explicitly added to the header they could still be coming into S3. So the question is, how do we add Content-Type and x-amz-acl headers into the signature?

There are several overloaded generatePresignedUrl functions in the AWS SDK, but only one of them allows us to pass in anything else besides the bucket-name, filename, expiration-date and http-method.

The solution is:

1. Create a GeneratePresignedUrlRequest object, with your bucket and filename.
2. Call setExpiration, setContentType, etc, to set all of your header info on it.
3. Pass that into s3.generatePresignedUrl as the only parameter.

Here's the proper function definition of GeneratePresignedUrlRequest to use:

http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Client.html#generatePresignedUrl(com.amazonaws.services.s3.model.GeneratePresignedUrlRequest)

The function's code on the AWS GitHub repo was also helpful for me to see how to code up the solution. Hope this helps.

回答2:

I just encountered this problem using the NodeJs AWS SDK. It was due to using credentials that were valid, but without sufficient permissions. Changing to my admin key fixed this with no code changes!

回答3:

I faced a similar issue and setting the config signatureVersion: 'v4' helped solve it in my case -

In JavaScript:

var s3 = new AWS.S3({
  signatureVersion: 'v4'
});

Adapted from https://github.com/aws/aws-sdk-js/issues/902#issuecomment-184872976

回答4:

I had the same issue, but removing content-type works fine. Hereby sharing the complete code.

public class GeneratePresignedUrlAndUploadObject {
    private static final String BUCKET_NAME = "<YOUR_AWS_BUCKET_NAME>"; 
    private static final String OBJECT_KEY  = "<YOUR_AWS_KEY>";
    private static final String AWS_ACCESS_KEY = "<YOUR_AWS_ACCESS_KEY>";
    private static final String AWS_SECRET_KEY = "<YOUR_AWS_SECRET_KEY>";

    public static void main(String[] args) throws IOException {
        BasicAWSCredentials awsCreds = new BasicAWSCredentials(AWS_ACCESS_KEY, AWS_SECRET_KEY);

        AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withRegion(Regions.US_EAST_1)
                .withCredentials(new AWSStaticCredentialsProvider(awsCreds)).build();

        try {
            System.out.println("Generating pre-signed URL.");
            java.util.Date expiration = new java.util.Date();
            long milliSeconds = expiration.getTime();
            milliSeconds += 1000 * 60 * 60;
            expiration.setTime(milliSeconds);

            GeneratePresignedUrlRequest generatePresignedUrlRequest = 
                    new GeneratePresignedUrlRequest(BUCKET_NAME, OBJECT_KEY);
            generatePresignedUrlRequest.setMethod(HttpMethod.PUT); 
            generatePresignedUrlRequest.setExpiration(expiration);
            URL url = s3Client.generatePresignedUrl(generatePresignedUrlRequest); 

            UploadObject(url);

            System.out.println("Pre-Signed URL = " + url.toString());
        } catch (AmazonServiceException exception) {
            System.out.println("Caught an AmazonServiceException, " +
                    "which means your request made it " +
                    "to Amazon S3, but was rejected with an error response " +
            "for some reason.");
            System.out.println("Error Message: " + exception.getMessage());
            System.out.println("HTTP  Code: "    + exception.getStatusCode());
            System.out.println("AWS Error Code:" + exception.getErrorCode());
            System.out.println("Error Type:    " + exception.getErrorType());
            System.out.println("Request ID:    " + exception.getRequestId());
        } catch (AmazonClientException ace) {
            System.out.println("Caught an AmazonClientException, " +
                    "which means the client encountered " +
                    "an internal error while trying to communicate" +
                    " with S3, " +
            "such as not being able to access the network.");
            System.out.println("Error Message: " + ace.getMessage());
        }
    }

    public static void UploadObject(URL url) throws IOException
    {
        HttpURLConnection connection=(HttpURLConnection) url.openConnection();
        connection.setDoOutput(true);
        connection.setRequestMethod("PUT");
        OutputStreamWriter out = new OutputStreamWriter(
                connection.getOutputStream());
        out.write("This text uploaded as object.");
        out.close();
        int responseCode = connection.getResponseCode();
        System.out.println("Service returned response code " + responseCode);

    }
}

回答5:

Same problem for me but a different cause. I was using POST instead of PUT

回答6:

Got a problem, the mime type on windows was setting the fileType to empty string and it didn't work. Just handle empty strings and add some file type.

回答7:

I faced with SignatureDoesNotMatch error using the Java AWS SDK. In my case, SignatureDoesNotMatch error occurred after upgraded maven dependencies without changes in my code (so credentials are correct and were not changed). After upgrading dependency org.apache.httpcomponents:httpclient from version 4.5.6 to 4.5.7 (actually it was upgrade of Spring Boot from 2.1.2 to 2.1.3, and there bom has specified httpclient version), code became throw exceptions while doing some AWS SDK S3 requests like AmazonS3.getObject.

After digging into the root cause, I found that httpclient library did breaking changes with normalized URI, that affected Java AWS SDK S3. Please take a look for opened GitHub ticket org.apache.httpcomponents:httpclient:4.5.7 breaks fetching S3 objects for more details.