How to make “MessageDigest SHA-1 and Signature NON

Question

I am interested in applying a SHA-1 hash with RSA signature to some data, but I need to do it in two steps - apply hash first and then sign the data. The Signature.sign() function appears to create a more complex (ASN.1?) data structure that is ultimately signed (see this question). How can I make the two equivalent without using any external libraries like BouncyCastle?

Apply hash and sign in single step with Signature:

PrivateKey privatekey = (PrivateKey) keyStore.getKey(alias, null);
...
sig = Signature.getInstance("SHA1withRSA", "SunMSCAPI");
sig.initSign(privatekey);
sig.update(data_to_sign);
byte[] bSignedData_CAPISHA1_CAPIRSA = sig.sign();

Apply hash via MessageDigest, then sign with Signature:

PrivateKey privatekey = (PrivateKey) keyStore.getKey(alias, null);
...
MessageDigest sha1 = MessageDigest.getInstance("SHA-1");
byte[] data_to_sign = sha1.digest(bdataToSign);
Signature sig = Signature.getInstance("NONEwithRSA", "SunMSCAPI");
sig.initSign(privatekey);
sig.update(data_to_sign);
byte[] bSignedData_JAVASHA1_CAPIRSA = sig.sign();
...

I am looking for the following equivalency:

bSignedData_JAVASHA1_CAPIRSA == bSignedData_CAPISHA1_CAPIRSA

My ultimate goal is to create the hash and then sign with a PKCS11 token, but I require the signed data to be the same format as legacy data for verification purposes.

Answer 1

emsworth's answer was when a great help for me when struggling with the same issue (but using SHA512). However it is still missing a hint which takes me a few more days to find out by myself.

There are different ways how the signature is constructed. For example when using RSASSA-PKCS1-v1_5 (from RFC 5246, TLS 1.2) the DER-encoded DigestInfo is not obtained the usual way. For example if using BouncyCastle

DigestInfo digestInfo = new DigestInfo(new AlgorithmIdentifier(NISTObjectIdentifiers.id_sha512), hash);
byte[] digestedHash = digestInfo.getEncoded(ASN1Encoding.DER);

does not yield the expected results. RFC 3447 defines how to construct the DER encoding on page 42. For example in case of SHA-512 the DER encoding is as follows:

// code example includes MessageDigest for the sake of completeness
byte[] input = ... // the raw data
MessageDigest md = MessageDigest.getInstance("SHA-512");
md.update(input);
byte[] hash = md.digest();

// Taken from RFC 3447, page 42 for SHA-512, create input for signing
byte[] modifierBytes = { 0x30, 0x51, 0x30, 0x0d, 0x06, 0x09, 0x60, (byte) 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x02, 0x03, 0x05, 0x00, 0x04, 0x40 };
ByteArrayOutputStream baos = new ByteArrayOutputStream();
baos.write(modifierBytes);
baos2.write(hash);

// create signature
Signature s = Signature.getInstance("NONEwithRSA");
s.initSign(MyTlsCredentials.THE_CLIENT_KEY);
s.update(baos.toByteArray());
byte[] signature = s.sign();

// add prefix as specified in RFC 3447, im my case it had always been the shown values
// but I have not understand the RFC completely in this point as the code seems to be
// contradictious to the interpretation of the byte values for the hash function from page 42.
ByteArrayOutputStream baos2 = new ByteArrayOutputStream();
baos2.write(new byte[] { 1, 0 });
baos2.write(signature5);

Answer 2

I was able to solve this by doing the following:

1. The data to be signed needed to be formatted correctly in a DigestInfo DER-encoded byte array. The Signature SHA1withRSA takes care of this for you, but if you want to accomplish it in a two-step process, you need to create your own DigestInfo. I ended up copying a very minimal amount of ASN.1 classes from BouncyCastle into my project to accomplish this, despite my desire not to use a third party lib.

2. If you try to use the Cipher API to encrypt the DigestInfo, the PKCS1 padding will be random and not appropriate for a digital signature. I needed static padding.

3. The Signature.getInstance("NONEwithRSA", "SunMSCAPI") rejects the DER-encoded DigestInfo format, and will return an error if you try to sign that data. But, since I ultimately wanted to use the PKCS11 API to generate the signature, I ended up signing the DER-encoded DigestInfo with the PKCS11 C_SignInit and C_Sign functions.

To summarize, what worked for me was:

1. generate the SHA-1 hash of the data to sign using the Java MessageDigest API
2. generated a DigestInfo DER-encoded ASN.1 object with the SHA-1 hash and SHA-1 OID embedded in the object.
3. signed the DigestInfo using the PKCS11 C_Sign function from a third party library.

The following links were most helpful in solving my problem:

Oracle Forums: SHA1withRSA - how to do that in 2 steps?

StackOverflow: Using SHA1 and RSA with java.security.Signature vs. MessageDigest and Cipher