CloudKit Server-to-Server authentication

Question

Apple published a new method to authenticate against CloudKit, server-to-server. https://developer.apple.com/library/content/documentation/DataManagement/Conceptual/CloudKitWebServicesReference/SettingUpWebServices.html#//apple_ref/doc/uid/TP40015240-CH24-SW6

I tried to authenticate against CloudKit and this method. At first I generated the key pair and gave the public key to CloudKit, no problem so far.

I started to build the request header. According to the documentation it should look like this:

X-Apple-CloudKit-Request-KeyID: [keyID]  
X-Apple-CloudKit-Request-ISO8601Date: [date]  
X-Apple-CloudKit-Request-SignatureV1: [signature]

• [keyID], no problem. You can find this in the CloudKit dashboard.
• [Date], I think this should work: 2016-02-06T20:41:00Z
• [signature], here is the problem...

The documentation says:

The signature created in Step 1.

Step 1 says:

Concatenate the following parameters and separate them with colons.
[Current date]:[Request body]:[Web Service URL]

I asked myself "Why do I have to generate the key pair?".
But step 2 says:

Compute the ECDSA signature of this message with your private key.

Maybe they mean to sign the concatenated signature with the private key and put this into the header? Anyway I tried both...

My sample for this (unsigned) signature value looks like:

2016-02-06T20:41:00Z:YTdkNzAwYTllNjI1M2EyZTllNDNiZjVmYjg0MWFhMGRiMTE2MjI1NTYwNTA2YzQyODc4MjUwNTQ0YTE5YTg4Yw==:https://api.apple-cloudkit.com/database/1/[iCloud Container]/development/public/records/lookup  

The request body value is SHA256 hashed and after that base64 encoded. My question is, I should concatenate with a ":" but the url and the date also contains ":". Is it correct? (I also tried to URL-Encode the URL and delete the ":" in the date).
At next I signed this signature string with ECDSA, put it into the header and send it. But I always get 401 "Authentication failed" back. To sign it, I used the ecdsa python module, with following commands:

from ecdsa import SigningKey  
a = SigningKey.from_pem(open("path_to_pem_file").read())  
b = "[date]:[base64(request_body)]:/database/1/iCloud....."  
print a.sign(b).encode('hex')

Maybe the python module doesn't work correctly. But it can generate the right public key from the private key. So I hope the other functions also work.

Has anybody managed to authenticate against CloudKit with the server-to-server method? How does it work correctly?

Edit: Correct python version that works

from ecdsa import SigningKey
import ecdsa, base64, hashlib  

a = SigningKey.from_pem(open("path_to_pem_file").read())  
b = "[date]:[base64(sha256(request_body))]:/database/1/iCloud....."  
signature = a.sign(b, hashfunc=hashlib.sha256, sigencode=ecdsa.util.sigencode_der)  
signature = base64.b64encode(signature)
print signature #include this into the header

Answer 1

I made an working code example in PHP: https://gist.github.com/Mauricevb/87c144cec514c5ce73bd (based on @Jessedc's JavaScript example)

By the way, make sure you set the date time in UTC timezone. My code didn't work because of this.

Answer 2

I had the same problem and ended up writing a library that works with python-requests to interface with the CloudKit API in Python.

pip install requests-cloudkit

After it's installed, just import the authentication handler (CloudKitAuth) and use it directly with requests. It will transparently authenticate any request you make to the CloudKit API.

>>> import requests
>>> from requests_cloudkit import CloudKitAuth
>>> auth = CloudKitAuth(key_id=YOUR_KEY_ID, key_file_name=YOUR_PRIVATE_KEY_PATH)
>>> requests.get("https://api.apple-cloudkit.com/database/[version]/[container]/[environment]/public/zones/list", auth=auth)

The GitHub project is available at https://github.com/lionheart/requests-cloudkit if you'd like to contribute or report an issue.

Answer 3

The last part of the message

[Current date]:[Request body]:[Web Service URL]

must not include the domain (it must include any query parameters):

2016-02-06T20:41:00Z:YTdkNzAwYTllNjI1M2EyZTllNDNiZjVmYjg0MWFhMGRiMTE2MjI1NTYwNTA2YzQyODc4MjUwNTQ0YTE5YTg4Yw==:/database/1/[iCloud Container]/development/public/records/lookup

With newlines for better readability:

2016-02-06T20:41:00Z
:YTdkNzAwYTllNjI1M2EyZTllNDNiZjVmYjg0MWFhMGRiMTE2MjI1NTYwNTA2YzQyODc4MjUwNTQ0YTE5YTg4Yw==
:/database/1/[iCloud Container]/development/public/records/lookup

The following shows how to compute the header value in pseudocode

The exact API calls depend on the concrete language and crypto library you use.

//1. Date
//Example: 2016-02-07T18:58:24Z
//Pitfall: make sure to not include milliseconds
date = isoDateWithoutMilliseconds() 

//2. Payload
//Example (empty string base64 encoded; GET requests):
//47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=
//Pitfall: make sure the output is base64 encoded (not hex)
payload = base64encode(sha256(body))  

//3. Path
//Example: /database/1/[containerIdentifier]/development/public/records/lookup
//Pitfall: Don't include the domain; do include any query parameter
path = stripDomainKeepQueryParams(url) 

//4. Message
//Join date, payload, and path with colons
message = date + ':' + payload + ':' + path

//5. Compute a signature for the message using your private key.
//This step looks very different for every language/crypto lib.
//Pitfall: make sure the output is base64 encoded.
//Hint: the key itself contains information about the signature algorithm 
//      (on NodeJS you can use the signature name 'RSA-SHA256' to compute a 
//      the correct ECDSA signature with an ECDSA key).
signature = base64encode(sign(message, key))

//6. Set headers
X-Apple-CloudKit-Request-KeyID = keyID 
X-Apple-CloudKit-Request-ISO8601Date = date  
X-Apple-CloudKit-Request-SignatureV1 = signature

//7. For POST requests, don't forget to actually send the unsigned request body
//   (not just the headers)

Answer 4

Extracting Apple's cloudkit.js implementation and using the first call from the Apple sample code node-client-s2s/index.js you can construct the following:

You hash the request body request with sha256:

var crypto = require('crypto');
var bodyHasher = crypto.createHash('sha256');
bodyHasher.update(requestBody);
var hashedBody = bodyHasher.digest("base64");

The sign the [Current date]:[Request body]:[Web Service URL] payload with the private key provided in the config.

var c = crypto.createSign("RSA-SHA256");
c.update(rawPayload);
var requestSignature = c.sign(key, "base64");

Another note is the [Web Service URL] payload component must not include the domain but it does need any query parameters.

Make sure the date value is the same in X-Apple-CloudKit-Request-ISO8601Date as it is in the signature. (These details are not documented completely, but is observed by looking through the CloudKit.js implementation).

A more complete nodejs example looks like this:

(function() {

const https = require('https');
var fs = require('fs');
var crypto = require('crypto');

var key = fs.readFileSync(__dirname + '/eckey.pem', "utf8");
var authKeyID = 'auth-key-id';

// path of our request (domain not included)
var requestPath = "/database/1/iCloud.containerIdentifier/development/public/users/current";

// request body (GET request is blank)
var requestBody = '';

// date string without milliseconds
var requestDate = (new Date).toISOString().replace(/(\.\d\d\d)Z/, "Z");

var bodyHasher = crypto.createHash('sha256');
bodyHasher.update(requestBody);
var hashedBody = bodyHasher.digest("base64");

var rawPayload = requestDate + ":" + hashedBody + ":" + requestPath;

// sign payload
var c = crypto.createSign("sha256");
c.update(rawPayload);
var requestSignature = c.sign(key, "base64");

// put headers together
var headers = {
    'X-Apple-CloudKit-Request-KeyID': authKeyID,
    'X-Apple-CloudKit-Request-ISO8601Date': requestDate,
    'X-Apple-CloudKit-Request-SignatureV1': requestSignature
};

var options = {
    hostname: 'api.apple-cloudkit.com',
    port: 443,
    path: requestPath,
    method: 'GET',
    headers: headers
};

var req = https.request(options, (res) => {
   //... handle nodejs response
});

req.end();

})();

This also exists as a gist: https://gist.github.com/jessedc/a3161186b450317a9cb5

On the command line with openssl (Updated)

The first hashing can be done with this command:

openssl sha -sha256 -binary < body.txt | base64

To sign the second part of the request you need a more modern version of openSSL than what OSX 10.11 comes with and use the following command:

/usr/local/bin/openssl dgst -sha256WithRSAEncryption -binary -sign ck-server-key.pem raw_signature.txt | base64

Thanks to @maurice_vB below and on twitter for this info

Answer 5

In case someone else is trying to do this via Ruby, there's a key method alias required to monkey patch the OpenSSL lib to work:

def signature_for_request(body_json, url, iso8601_date)
  body_sha_hash = Digest::SHA256.digest(body_json)

  payload_for_signature = [iso8601_date, Base64.strict_encode64(body_sha_hash), url].join(":")

  OpenSSL::PKey::EC.send(:alias_method, :private?, :private_key?)

  ec = OpenSSL::PKey::EC.new(CK_PEM_STRING)
  digest = OpenSSL::Digest::SHA256.new
  signature = ec.sign(digest, payload_for_signature)
  base64_signature = Base64.strict_encode64(signature)

  return base64_signature
end

Note that in the above example, url is the path excluding the domain component (starting with /database...) and CK_PEM_STRING is simply a File.read of the pem generated when setting up your private/public key pair.

The iso8601_date is most easily generated using:

Time.now.utc.iso8601

Of course, you want to store that in a variable to include in your final request. Construction of the final request can be done with the following pattern:

def perform_request(url, body, iso8601_date)

  signature = self.signature_for_request(body, url, iso8601_date)

  uri = URI.parse(CK_SERVICE_BASE + url)

  header = {
    "Content-Type" => "text/plain",
    "X-Apple-CloudKit-Request-KeyID" => CK_KEY_ID,
    "X-Apple-CloudKit-Request-ISO8601Date" => iso8601_date,
    "X-Apple-CloudKit-Request-SignatureV1" => signature
  }

  # Create the HTTP objects
  http = Net::HTTP.new(uri.host, uri.port)
  http.use_ssl = true
  request = Net::HTTP::Post.new(uri.request_uri, header)
  request.body = body

  # Send the request
  response = http.request(request)

  return response
end

Works like a charm now for me.

Answer 6

Distilled this from a project I'm working on in Node. Maybe you will find it useful. Replace the X-Apple-CloudKit-Request-KeyID and the container identifier in requestOptions.path to make it work.

The private key/ pem is generated with: openssl ecparam -name prime256v1 -genkey -noout -out eckey.pem and generate the public key to register at the CloudKit dashboard openssl ec -in eckey.pem -pubout.

var crypto = require("crypto"),
    https = require("https"),
    fs = require("fs")

var CloudKitRequest = function(payload) {
  this.payload = payload
  this.requestOptions = { // Used with `https.request`
    hostname: "api.apple-cloudkit.com",
    port: 443,
    path: '/database/1/iCloud.com.your.container/development/public/records/modify',
    method: 'POST',
    headers: { // We will add more headers in the sign methods
      "X-Apple-CloudKit-Request-KeyID": "your-ck-request-keyID"
    }
  }
}

To sign the request:

CloudKitRequest.prototype.sign = function(privateKey) {
  var dateString = new Date().toISOString().replace(/\.[0-9]+?Z/, "Z"), // NOTE: No milliseconds
      hash = crypto.createHash("sha256"),
      sign = crypto.createSign("RSA-SHA256")

  // Create the hash of the payload
  hash.update(this.payload, "utf8")
  var payloadSignature = hash.digest("base64")

  // Create the signature string to sign
  var signatureData = [
    dateString,
    payloadSignature,
    this.requestOptions.path
  ].join(":") // [Date]:[Request body]:[Web Service URL]

  // Construct the signature
  sign.update(signatureData)
  var signature = sign.sign(privateKey, "base64")

  // Update the request headers
  this.requestOptions.headers["X-Apple-CloudKit-Request-ISO8601Date"] = dateString
  this.requestOptions.headers["X-Apple-CloudKit-Request-SignatureV1"] = signature

  return signature // This might be useful to keep around
}

And now you can send the request:

CloudKitRequest.prototype.send = function(cb) {
  var request = https.request(this.requestOptions, function(response) {
    var responseBody = ""

    response.on("data", function(chunk) {
      responseBody += chunk.toString("utf8")
    })

    response.on("end", function() {
      cb(null, JSON.parse(responseBody))
    })
  })

  request.on("error", function(err) {
    cb(err, null)
  })

  request.end(this.payload)
}

So given the following:

var privateKey = fs.readFileSync("./eckey.pem"),
    creationPayload = JSON.stringify({
      "operations": [{
          "operationType" : "create",
          "record" : {
            "recordType" : "Post",
            "fields" : {
              "title" : { "value" : "A Post From The Server" }
          }
        }
      }]
    })

Using the request:

var creationRequest = new CloudKitRequest(creationPayload)
creationRequest.sign(privateKey)
creationRequest.send(function(err, response) {
  console.log("Created a new entry with error", err, "and respone", response)
})

For your copy pasting pleasure: https://gist.github.com/spllr/4bf3fadb7f6168f67698 (edited)