How to generate Signature in AWS from Java

问题:

When I invoke API endpoints from REST client, I got error by concerning with Signature.

Request:

Host: https://xxx.execute-api.ap-southeast-1.amazonaws.com/latest/api/name

Authorization: AWS4-HMAC-SHA256 Credential={AWSKEY}/20160314/ap-southeast-1/execute-api/aws4_request,SignedHeaders=host;range;x-amz-date,Signature={signature}

X-Amz-Date: 20160314T102915Z

Response:

{
"message": "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details. The Canonical String for this request should have been 'xxx' "
}

From Java code, I followed AWS reference of how to generate Signature.

    String secretKey = "{mysecretkey}";
    String dateStamp = "20160314";
    String regionName = "ap-southeast-1";
    String serviceName = "execute-api";

    byte[] signature = getSignatureKey(secretKey, dateStamp, regionName, serviceName);
    System.out.println("Signature : " + Hex.encodeHexString(signature));

    static byte[] HmacSHA256(String data, byte[] key) throws Exception  {
         String algorithm="HmacSHA256";
         Mac mac = Mac.getInstance(algorithm);
         mac.init(new SecretKeySpec(key, algorithm));
         return mac.doFinal(data.getBytes("UTF8"));
    }

    static byte[] getSignatureKey(String key, String dateStamp, String regionName, String serviceName) throws Exception  {
         byte[] kSecret = ("AWS4" + key).getBytes("UTF8");
         byte[] kDate    = HmacSHA256(dateStamp, kSecret);
         byte[] kRegion  = HmacSHA256(regionName, kDate);
         byte[] kService = HmacSHA256(serviceName, kRegion);
         byte[] kSigning = HmacSHA256("aws4_request", kService);
         return kSigning;
    }

May I know what I was wrong while generating Signature?

Reference how to generate Signature : http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-java

回答1:

You can use classes from aws-java-sdk-core: https://github.com/aws/aws-sdk-java/tree/master/aws-java-sdk-core

More specifically, Request, Aws4Signer and a few other ones:

//Instantiate the request
Request<Void> request = new DefaultRequest<Void>("es"); //Request to ElasticSearch
request.setHttpMethod(HttpMethodName.GET);
request.setEndpoint(URI.create("http://..."));

//Sign it...
AWS4Signer signer = new AWS4Signer();
signer.setRegionName("...");
signer.setServiceName(request.getServiceName());
signer.sign(request, new AwsCredentialsFromSystem());

//Execute it and get the response...
Response<String> rsp = new AmazonHttpClient(new ClientConfiguration())
    .requestExecutionBuilder()
    .executionContext(new ExecutionContext(true))
    .request(request)
    .errorResponseHandler(new SimpleAwsErrorHandler())
    .execute(new SimpleResponseHandler<String>());

If you want a cleaner design, you can use the Decorator pattern to compose some elegant classes and hide the above mess. An example for that here: http://www.amihaiemil.com/2017/02/18/decorators-with-tunnels.html

回答2:

From the code example above it looks like you are not creating a canonical request and including it in the string that gets signed as per http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

Instead of implementing this yourself have you looked at using a third-party library.

aws-v4-signer-java is a lightweight, zero-dependency library that makes it easy to generate AWS V4 signatures.

String contentSha256 = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855";
HttpRequest request = new HttpRequest("GET", new URI("https://examplebucket.s3.amazonaws.com?max-keys=2&prefix=J"));
String signature = Signer.builder()
        .awsCredentials(new AwsCredentials(ACCESS_KEY, SECRET_KEY))
        .header("Host", "examplebucket.s3.amazonaws.com")
        .header("x-amz-date", "20130524T000000Z")
        .header("x-amz-content-sha256", contentSha256)
        .buildS3(request, contentSha256)
        .getSignature();

Disclaimer: I'm the libraries author.

回答3:

This is possible using 100% java libraries without additional dependencies, just use the query parameters generated here:

import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.security.SignatureException;
import java.util.Formatter;

import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.util.Base64;

...

private static final String ACCESS_KEY = "...";
private static final String SECRET_KEY = "...";
private static final int expiresTime = 1 * 24 * 60 * 60;
private static final String HMAC_SHA1_ALGORITHM = "HmacSHA1";

public void sign(String protocol, String bucketName, String contentPath) throws Exception {
    Calendar cal = Calendar.getInstance();
    cal.add(Calendar.HOUR_OF_DAY, 24);

    String host = bucketName + ".s3-us-west-2.amazonaws.com";
    long expireTime = cal.getTimeInMillis() / 1000;

    String signString = "GET\n" +
        "\n" +
        "\n" +
        expireTime + "\n" +
        "/" + bucketName + contentPath;

    SecretKeySpec signingKey = new SecretKeySpec(SECRET_KEY.getBytes(), HMAC_SHA1_ALGORITHM);
    Mac mac = Mac.getInstance(HMAC_SHA1_ALGORITHM);
    mac.init(signingKey);
    String signature = URLEncoder.encode(new String(Base64.getEncoder().encode(mac.doFinal(signString.getBytes()))));

    System.out.println(signature);
    String fullPayload = "?AWSAccessKeyId=" + ACCESS_KEY +
        "&Expires=" + expireTime + 
        "&Signature=" + signature;

    System.out.println(protocol + "://" + host + contentPath + fullPayload);
}

...

回答4:

The signing process is lengthy and error-prone, here are some tips

• Make sure your access key and secret is correct, try to use Postman to test the request at first, it's easy and fast, see Use Postman to Call a REST API
• Make sure you use UTC time
• The signing process uses both timestamp(YYYYMMDD'T'HHMMSS'Z') and datetime(YYYYMMDD), so double check your implementation for that
• Use any online hash tool to verify your hash algorithm behaves as expected
• Read the python implementation carefully, see Examples of the Complete Version 4 Signing Process (Python)
• See my fully working java implementation on Github - A Java(SpringBoot) template for Java and AWS SageMaker DeepAR model endpoint invocation integration

回答5:

The easiest way is to use methods and http-client from Amazon's SDK. I follow the below 3 steps.

Step1: Create basic AWS credentials:

 BasicAWSCredentials awsCreds = new BasicAWSCredentials(ACCESS_KEY,AWS_DATASHOP_SECRET_KEY);

Step2: Create signableRequest:

DefaultRequest<?> signableRequest = new DefaultRequest<>("aws-service-name");
    signableRequest.setHttpMethod(HttpMethodName.GET);
    signableRequest.setResourcePath("fooo");
    signableRequest.setEndpoint(URI.create(baar));
    signableRequest.addParameter("execution_id", executionId);
    signableRequest.addHeader("Content-Type", "application/json");

    signer.sign(signableRequest, awsCreds);

Step3: Execute request using AmazonHttpClient:

new AmazonHttpClient(new ClientConfiguration())
                    .requestExecutionBuilder()
                    .executionContext(new ExecutionContext(true))
                    .request(signableRequest)
                    .errorResponseHandler((new SimpleAwsErrorHandler()))
                    .execute(new MyResponseHandler());

Make sure to implement HttpResponseHandler for SimpleAwsErrorHandler and MyResponseHandler

If you want to use normal http clients, you would have to create a canonical request and calculate signature which most often doesn't match.