How to sign amazon web service requests from the p

Question

I use Amazon web service api from within my Google app engine application. Amazon have said that they will only accept signed requests from Aug 15, 2009. While they have given simple instructions for signing, I am not so knowledgeable of Python libraries for SHA256. The app engine documentation says it supports pycrypto but I was just wondering (read being lazy) if anyone has already done this. Any code snippets you could share? Any issues I might be missing here?

Answer 1

See http://sowacs.appspot.com/AWS/Downloads/#python for a GAE Python signing service webapp. Uses native Python libraries.

Answer 2

Pycrypto will work fine - it's supported on App Engine, though the public ciphers are implemented in Python rather than C. You also ought to be able to use one of the existing AWS libraries, now that urlfetch/httplib are supported on App Engine.

I have an app that uploads images to S3, and I've implemented the request signing myself, but mostly because I wrote it before urlfetch/httplib were available. It works just fine, however.

Answer 3

I use this one using pycrypto to generate a custom policy:

import json                                                                                                                                                                 
import time                                                                                                                                                                 

from Crypto.Hash import SHA                                                                                                                                                 
from Crypto.PublicKey import RSA                                                                                                                                            
from Crypto.Signature import PKCS1_v1_5                                                                                                                                     
from base64 import b64encode                                                                                                                                                

url = "http://*"                                                                                                                                                            
expires = int(time.time() + 3600)

pem = """-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----"""

key_pair_id = 'APK.....'

policy = {}                                                                                                                                                                 
policy['Statement'] = [{}]                                                                                                                                                  
policy['Statement'][0]['Resource'] = url                                                                                                                                    
policy['Statement'][0]['Condition'] = {}                                                                                                                                    
policy['Statement'][0]['Condition']['DateLessThan'] = {}                                                                                                                    
policy['Statement'][0]['Condition']['DateLessThan']['AWS:EpochTime'] = expires

policy = json.dumps(policy) 

private_key = RSA.importKey(pem)                                                                                                                                            
policy_hash = SHA.new(policy)                                                                                                                                               
signer = PKCS1_v1_5.new(private_key)                                                                                                                                        
signature = b64encode(signer.sign(policy_hash))

print '?Policy=%s&Signature=%s&Key-Pair-Id=%s' % (b64encode(policy),                                                                                                        
                                                  signature,                                                                                                                
                                                  key_pair_id)

This allows me to use one key for multiple items, something like:

http://your_domain/image1.png?Policy...
http://your_domain/image2.png?Policy...
http://your_domain/file1.json?Policy...

Don't forget to enable pycrypto by adding this lines to the app.yaml

libraries:
- name: pycrypto
  version: latest 

Answer 4

Here is an example of a REST request based on lower level (then boto) libraries. Solution was taken from http://cloudcarpenters.com/blog/amazon_products_api_request_signing.

All you need is valid entries for AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY

def amazon_test_url():
    import base64, hashlib, hmac, time
    from urllib import urlencode, quote_plus

    AWS_ACCESS_KEY_ID = 'YOUR_KEY'
    AWS_SECRET_ACCESS_KEY = 'YOUR_SECRET_KEY'  
    TEST_ISBN = '9780735619678' #http://stackoverflow.com/questions/1711/what-is-the-single-most-influential-book-every-programmer-should-read

    base_url = "http://ecs.amazonaws.com/onca/xml"
    url_params = dict(
        Service='AWSECommerceService', 
        Operation='ItemLookup', 
        IdType='ISBN', 
        ItemId=TEST_ISBN,
        SearchIndex='Books',
        AWSAccessKeyId=AWS_ACCESS_KEY_ID,  
        ResponseGroup='Images,ItemAttributes,EditorialReview,SalesRank')

    #Can add Version='2009-01-06'. What is it BTW? API version?


    # Add a ISO 8601 compliant timestamp (in GMT)
    url_params['Timestamp'] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())

    # Sort the URL parameters by key
    keys = url_params.keys()
    keys.sort()
    # Get the values in the same order of the sorted keys
    values = map(url_params.get, keys)

    # Reconstruct the URL parameters and encode them
    url_string = urlencode(zip(keys,values))

    #Construct the string to sign
    string_to_sign = "GET\necs.amazonaws.com\n/onca/xml\n%s" % url_string

    # Sign the request
    signature = hmac.new(
        key=AWS_SECRET_ACCESS_KEY,
        msg=string_to_sign,
        digestmod=hashlib.sha256).digest()

    # Base64 encode the signature
    signature = base64.encodestring(signature).strip()

    # Make the signature URL safe
    urlencoded_signature = quote_plus(signature)
    url_string += "&Signature=%s" % urlencoded_signature

    print "%s?%s\n\n%s\n\n%s" % (base_url, url_string, urlencoded_signature, signature)

Answer 5

Got this to work based on code sample at http://jjinux.blogspot.com/2009/06/python-amazon-product-advertising-api.html Here is a minor improved version that lets you merge a dict of call specific params with the basic params before making the call.

keyFile = open('accesskey.secret', 'r')
# I put my secret key file in .gitignore so that it doesn't show up publicly
AWS_SECRET_ACCESS_KEY = keyFile.read()
keyFile.close()

def amz_call(self, call_params):

    AWS_ACCESS_KEY_ID = '<your-key>'
    AWS_ASSOCIATE_TAG = '<your-tag>'

    import time
    import urllib
    from boto.connection import AWSQueryConnection
    aws_conn = AWSQueryConnection(
        aws_access_key_id=AWS_ACCESS_KEY_ID,
        aws_secret_access_key=Amz.AWS_SECRET_ACCESS_KEY, is_secure=False,
        host='ecs.amazonaws.com')
    aws_conn.SignatureVersion = '2'
    base_params = dict(
        Service='AWSECommerceService',
        Version='2008-08-19',
        SignatureVersion=aws_conn.SignatureVersion,
        AWSAccessKeyId=AWS_ACCESS_KEY_ID,
        AssociateTag=AWS_ASSOCIATE_TAG,
        Timestamp=time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime()))
    params = dict(base_params, **call_params)
    verb = 'GET'
    path = '/onca/xml'
    qs, signature = aws_conn.get_signature(params, verb, path)
    qs = path + '?' + qs + '&Signature=' + urllib.quote(signature)
    print "verb:", verb, "qs:", qs
    return aws_conn._mexe(verb, qs, None, headers={})

Sample usage:

result = self.amz_call({'Operation' : 'ItemSearch' , 'Keywords' : searchString , 'SearchIndex' : 'Books' , 'ResponseGroup' : 'Small' })
if result.status == 200:
    responseBodyText = result.read()
    # do whatever ...

Answer 6

I wrote another simple example that uses only the core python 3 libraries (not boto) and uses version 2 of the AWS signature protocol:

http://xocoatl.blogspot.com/2011/03/signing-ec2-api-request-in-python.html

I know it won't work in GAE, but might be useful for anyone just looking for AWS authentication examples like I was.