S3 PUT doesn't work with pre-signed URL in jav

2020-07-11 05:36发布

问题:

I have generated a pre-signed S3 URL in java for HTTP PUT method.

A slightly modified version of the URL:

https://s3.amazonaws.com/somebucket/pre-signed-url-key-2?AWSAccessKeyId=BKIAIQ6H5Z7BG6KZ2ZUA&Expires=1425617244&Signature=GWcRM5ZIrAKnMJBsxyfm%2F9fyuBk%3D

I know that it is a valid pre-signed url since I can use it with curl to upload a file

curl -v --upload-file somefile.txt "https://s3.amazonaws.com/somebucket/pre-signed-url-key-2?AWSAccessKeyId=BKIAIQ6H5Z7BG6KZ2ZUA&Expires=1425617244&Signature=GWcRM5ZIrAKnMJBsxyfm%2F9fyuBk%3D"

When I try to upload a file to the same URL using the following Javascript:

function ajaxUsingPresignedUrlPut() {
            $.ajax({
                url: presignedURLPUT,
                type: 'PUT',
                data: 'blah blah blah',
                success: function () {
                    console.log('Uploaded data successfully.');
                }
            }).done(function (data) {
                console.log("DONE : " + data);
            }).fail(function (e, r) {
                console.log("FAIL");
            });
        }

I get a 403 status and responseText

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>BKIAIQ6H5Z7BG6KZ2ZUA</AWSAccessKeyId><StringToSign>PUT

application/x-www-form-urlencoded; charset=UTF-8
1425617244
/somebucket/pre-signed-url-key-2</StringToSign><SignatureProvided>GWcRM5ZIrAKnMJAsxyfm/9fyuAk=</SignatureProvided><StringToSignBytes>50 55 54 0a 0a 61 70 70 6c 69 63 61 74 69 6f 6e 2f 78 2d 77 77 77 2d 66 6f 72 6d 2d 75 72 6c 65 6e 63 6f 64 65 64 3b 20 63 68 61 72 73 65 74 3d 55 54 46 2d 38 0a 31 34 32 35 36 31 37 32 34 34 0a 2f 6e 6b 6f 6e 64 72 61 74 5f 62 75 63 6b 65 74 2f 70 72 65 2d 73 69 67 6e 65 64 2d 75 72 6c 2d 6b 65 79 2d 32</StringToSignBytes><RequestId>3C8298CAC404C6F5</RequestId><HostId>uCkzA//CdCLi4SINifkIe0WH6GOlCJBgFlN8ghx8NnULEe+QVslsdoUsJc4AUdA8</HostId></Error>

I have the following CORS policy configured on the S3 bucket:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>HEAD</AllowedMethod>
        <AllowedMethod>GET</AllowedMethod>
        <AllowedMethod>PUT</AllowedMethod>
        <AllowedMethod>POST</AllowedMethod>
        <AllowedMethod>DELETE</AllowedMethod>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <ExposeHeader>ETag</ExposeHeader>
        <ExposeHeader>Content-Length</ExposeHeader>
        <ExposeHeader>Content-Type</ExposeHeader>
        <ExposeHeader>Connection</ExposeHeader>
        <ExposeHeader>Date</ExposeHeader>
        <ExposeHeader>Server</ExposeHeader>
        <ExposeHeader>x-amz-delete-marker</ExposeHeader>
        <ExposeHeader>x-amz-id-2</ExposeHeader>
        <ExposeHeader>x-amz-request-id</ExposeHeader>
        <ExposeHeader>x-amz-version-id</ExposeHeader>
        <AllowedHeader>*</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

And the same javascript code works for a pre-signed GET URL: https://s3.amazonaws.com/somebucket/pre-signed-url-key-2?AWSAccessKeyId=BKIBIQ6H5Z7LG6KZ2ZUB&Expires=1425616588&Signature=qMPpuzMwxonYPDQETdLafEQGqMU%3D

function ajaxUsingPresignedUrlGET() {
            $.ajax({
                url: presignedURLGET,
                type: 'GET',
                success: function () {
                    console.log('Uploaded data successfully.');
                }
            }).done(function (data) {
                console.log("DONE : " + data);
            }).fail(function (e, r) {
                console.log("FAIL");
            });
        }

My only guess right now looking at the URLs is that the PUT url contains %2F and looking at http://www.w3schools.com/tags/ref_urlencode.asp that stands for / which might be causing an issue in the browser? Or maybe I am completely off here and there is some other issue that I am missing.


Edit #1 Adding java code used to generate pre-signed URL

public S3GeneratePresignedUrlDemo(HttpMethod httpMethod) {
    this.httpMethod = httpMethod;
    System.out.println("HTTP METHOD : " + this.httpMethod);
}

@Override
public void goImpl() throws Exception {        
    GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(bucketName, KEY);

    request.setMethod(httpMethod);

    request.setExpiration(createDateNHoursFromNow(24));

    URL url = s3Client.generatePresignedUrl(request);

    System.out.println(url);
}

private static Date createDateNHoursFromNow(int hours){
    Date date = new Date();
    long millis = date.getTime();
    final int millisInSecond = 1000;
    final int secondsInMinute = 60;
    final int minutesInHour = 60;
    millis += millisInSecond * secondsInMinute * minutesInHour * hours;

    date.setTime(millis);
    return date;
}`

回答1:

Adding

headers: {'Content-Type': 'text/plain;charset=UTF-8'},

To AJAX

function ajaxUsingPresignedUrlPut() {
            $.ajax({
                url: presignedURLPUT,
                type: 'PUT',
                data: 'blah blah blah',
                headers: {'Content-Type': 'text/plain;charset=UTF-8'},
                success: function () {
                    console.log('Uploaded data successfully.');
                }
            }).done(function (data) {
                console.log("DONE : " + data);
            }).fail(function (arg1, arg2) {
                console.log("FAIL");        
            });
        }

And setting content type while generating the URL fixed the issue

    GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(bucketName, KEY);        
    request.setContentType("text/plain;charset=UTF-8");
    //...


回答2:

there is actually a better way to set the content-type which seemed to have been the problem here:

$.ajax({
    url : s3presignedUrl,
    type : "PUT",
    data : file,
    dataType : "text",
    cache : false,
    contentType : file.type,
    processData : false
})

The important part for you is contentType : file.type where file.type comes from the <input type=file> onchange event (which actually reveals this.files which can then be iterated over...)

you can grab the complete client files from my php+js solution here: https://github.com/JoernBerkefeld/s3SignedUpload



回答3:

In case you do not want to restrict content-type like Nickolay Kondratyev suggests, you can set Content-Type like this:

headers: {'Content-Type': ' '},

Note, there is a string with a space char. Not null, not undefined, empty string or any other falsy value. And it works.


Update: this does not work anymore, S3 now detects mime type and adds it anyway