Amazon MWS API Call Using Delphi/Indy

Question

I'm developing a simple application to "talk" to the Amazon MWS API. Because a lot of existing code is at play here, I need to get this done in Delphi 2010 with Indy 10 (10.5.5) components, which I have used successfully to integrate with many other APIs in the past. However, the Amazon API seems to be incredibly sensitive to the smallest of details, to the point that all my calls are being denied with the already infamous "SignatureDoesNotMatch" error message.

Here's what I have accomplished so far:

1) My app will assemble a request, sign it with HMAC-SHA256 (using the OpenSSL libraries) and send it to the Amazon server endpoint.

2) The HMAC signature alone proved to be a challenge in itself, but it's now working correctly 100% of the time (as verified against requests generated by the Amazon Scrachpad).

However, as I pointed out earlier, my requests are always rejected by the MWS server with the SignatureDoesNotMatch error, even though they are verifiably correct. The only thing I can think of that could be causing problems is the way Indy may be handling the POST requests, specifically the text encoding process.

Has anyone been successful in connecting a Delphi/Indy client to MWS? If so, what kind of TIdHTTP settings were used? Here's what I have:

procedure TAmazon.TestGetOrder(OrderID:String);

const AwsAccessKey = 'MyAccessKey';
      AwsSecretKey = 'MySecretKey';
      MerchantID = 'MyMerchantID';
      MarketplaceID = 'MyMarketplaceID';
      ApiVersion = '2013-09-01';
      CallUri = '/Orders/2013-09-01';

var HTTP:TIdHTTP;
    SSL:TIdSSLIOHandlerSocketOpenSSL;
    SS:TStringStream;
    Params:TStringList;
    S,Timestamp,QueryString,Key,Value:String;
    i:Integer;

begin
   HTTP:=TIdHTTP.Create(nil);
   SSL:=TIdSSLIOHandlerSocketOpenSSL.Create(nil);
   Params:=TStringList.Create;
   try
      Params.Delimiter:='&';
      Params.StrictDelimiter:=True;

      // HTTP Client Options
      HTTP.HTTPOptions:=HTTP.HTTPOptions+[hoKeepOrigProtocol]-[hoForceEncodeParams];
      HTTP.ConnectTimeout:=5000;
      HTTP.ReadTimeout:=20000;
      HTTP.ProtocolVersion:=pv1_1;
      HTTP.IOHandler:=SSL;
      HTTP.HandleRedirects:=True;
      HTTP.Request.Accept:='text/plain, */*';
      HTTP.Request.AcceptLanguage:='en-US';
      HTTP.Request.ContentType:='application/x-www-form-urlencoded';
      HTTP.Request.CharSet:='utf-8';
      HTTP.Request.UserAgent:='MyApp/1.0 (Language=Delphi)';
      HTTP.Request.CustomHeaders.AddValue('x-amazon-user-agent',HTTP.Request.UserAgent);

      // generate the timestamp per Amazon specs
      Timestamp:=TIso8601.UtcDateTimeToIso8601(TIso8601.ToUtc(Now));
      // we can change the timestamp to match a value from the Scratchpad as a way to validate the signature:
      //Timestamp:='2014-05-09T20:32:28Z';

      // add required parameters from API function GetOrder
      Params.Add('Action=GetOrder');
      Params.Add('SellerId='+MerchantID);
      Params.Add('AWSAccessKeyId='+AwsAccessKey);
      Params.Add('Timestamp='+Timestamp);
      Params.Add('Version='+ApiVersion);
      Params.Add('SignatureVersion=2');
      Params.Add('SignatureMethod=HmacSHA256');
      Params.Add('AmazonOrderId.Id.1='+OrderID);
      // generate the signature using the parameters above
      Params.Add('Signature='+GetSignature(Params.Text,CallUri));

      // after generating the signature, make sure all values are properly URL-Encoded
      for i:=0 to Params.Count-1 do begin
         Key:=Params.Names[i];
         Value:=ParamEnc(Params.ValueFromIndex[i]);
         QueryString:=QueryString+Key+'='+Value+'&';
      end;
      Delete(QueryString,Length(QueryString),1);

      // there are two ways to make the call...
      // #1: according to the documentation, all parameters are supposed to be in
      // the URL, and the body stream is supposed to be empty
      SS:=TStringStream.Create;
      try
         try
            Log('POST '+CallUri+'?'+QueryString);
            S:=HTTP.Post('https://mws.amazonservices.com'+CallUri+'?'+QueryString,SS);
         except
            on E1:EIdHTTPProtocolException do begin
               Log('RawHeaders='+#$D#$A+HTTP.Request.RawHeaders.Text);
               Log('Protocol Exception:'+#$D#$A+StringReplace(E1.ErrorMessage,#10,#$D#$A,[rfReplaceAll]));
            end;
            on E2:Exception do
               Log('Unknown Exception: '+E2.Message);
         end;
         Log('ResponseText='+S);
      finally
         SS.Free;
      end;

      // #2: both the Scratchpad and the CSharp client sample provided by Amazon
      // do things in a different way, though... they POST the parameters in the
      // body of the call, not in the query string
      SS:=TStringStream.Create(QueryString,TEncoding.UTF8);
      try
         try
            SS.Seek(0,0);
            Log('POST '+CallUri+' (parameters in body/stream)');
            S:=HTTP.Post('https://mws.amazonservices.com'+CallUri,SS);
         except
            on E1:EIdHTTPProtocolException do begin
               Log('RawHeaders='+#$D#$A+HTTP.Request.RawHeaders.Text);
               Log('Protocol Exception:'+#$D#$A+StringReplace(E1.ErrorMessage,#10,#$D#$A,[rfReplaceAll]));
            end;
            on E2:Exception do
               Log('Unknown Exception: '+E2.Message);
         end;
         Log('ResponseText='+S);
      finally
         SS.Free;
      end;
   finally
      Params.Free;
      SSL.Free;
      HTTP.Free;
   end;
end;

If I assemble a GetOrder call in Scratchpad, then paste the timestamp of that call into the code above, I get EXACTLY the same query string here, with the same signature and size, etc. But my Indy request must be encoding things differently, because the MWS server doesn't like the call.

I know MWS is at least "reading" the query string, because if I change the timestamp to an old date, it returns a "request expired" error instead.

Amazon's tech support is clueless, posting a message every day with basic stuff like "Make sure the secret key is correct" (as if getting a signature with HMAC-SHA256 and MD5 would work without a valid key!!!!).

One more thing: if I use Wireshark to "watch" the raw request from both the code above and the C-Sharp Amazon sample code, I can't tell a difference either. However, I'm not sure Wireshark makes a distinction between UTF-8 and ASCII or whatever encoding the text being shown has. I still think it has to do with bad UTC-8 encoding or something like that.

Ideas and suggestions on how to properly encode the API call to please the Amazon gods are welcome and appreciated.

Answer 1

Found the problem: Indy (and Synapse too) adds the port number to the "Host" header line, and I had not realized that extra bit until I watched the headers more closely with Fiddler (thanks, @Graymatter!!!!).

When I change the endpoint to be mws.amazonservices.com:443 (instead of just mws.amazonservices), then my signature is calculated the same way as the AWS server's, and everything works perfectly.