itextsharp signing pdf with signed hash

I'm trying to sign a pdf through a signing service. This service requires to send a hex encoded SHA256 digest and in return I receive a hex encoded signatureValue. Besides that I also receive a signing certificate, intermediate certificate, OCSP response, and TimeStampToken. However, I already get stuck trying to sign the pdf with the signatureValue.

I have read Bruno's white paper, browsed the internet excessively, and tried many different ways, but the signature keeps coming up as invalid.

My latest attempt:

First, prepare pdf

PdfReader reader = new PdfReader(src);
FileStream os = new FileStream(dest, FileMode.Create);
PdfStamper stamper = PdfStamper.CreateSignature(reader, os, '\0');
PdfSignatureAppearance appearance = stamper.SignatureAppearance;
appearance.Certificate = signingCertificate;
IExternalSignatureContainer external = new ExternalBlankSignatureContainer(PdfName.ADOBE_PPKLITE, PdfName.ADBE_PKCS7_DETACHED);
MakeSignature.SignExternalContainer(appearance, external, 8192);

string hashAlgorithm = "SHA-256";
PdfPKCS7 sgn = new PdfPKCS7(null, chain, hashAlgorithm, false);
PdfSignatureAppearance appearance2 = stamper.SignatureAppearance;
Stream stream = appearance2.GetRangeStream();
byte[] hash = DigestAlgorithms.Digest(stream, hashAlgorithm);
byte[] sh = sgn.getAuthenticatedAttributeBytes(hash, null, null, CryptoStandard.CMS);

Hash byte[] sh and convert to string as follows

private static String sha256_hash(Byte[] value)
{
    using (SHA256 hash = SHA256.Create())
    {
         return String.Concat(hash.ComputeHash(value).Select(item => item.ToString("x2"))).ToUpper();
    }
}

and send to signing service. The received hex encoded signatureValue I then convert to bytes

private static byte[] StringToByteArray(string hex)
{
    return Enumerable.Range(0, hex.Length).Where(x => x % 2 == 0).Select(x => Convert.ToByte(hex.Substring(x, 2), 16)).ToArray();
}

Finally, create signature

private void CreateSignature(string src, string dest, byte[] sig) 
{
    PdfReader reader = new PdfReader(src); // src is now prepared pdf
    FileStream os = new FileStream(dest, FileMode.Create);
    IExternalSignatureContainer external = new MyExternalSignatureContainer(sig);
    MakeSignature.SignDeferred(reader, "Signature1", os, external);

    reader.Close();
    os.Close();
}
private class MyExternalSignatureContainer : IExternalSignatureContainer
{
    protected byte[] sig;
    public MyExternalSignatureContainer(byte[] sig)
    {
        this.sig = sig;
    }
    public byte[] Sign(Stream s)
    {
        return sig;
    }
    public void ModifySigningDictionary(PdfDictionary signDic) { }
}

What am I doing wrong? Help is very much appreciated. Thanks!

Edit: Current state

Thanks to help from mkl and following Bruno's deferred signing example I've gotten past the invalid signature message. Apparently I don't receive a full chain from the signing service, but just an intermediate certificate, which caused the invalid message. Unfortunately, the signature still has flaws.

I build the chain like this:

List<X509Certificate> certificateChain = new List<X509Certificate>
{
     signingCertificate,
     intermediateCertificate
};

In the sign method of MyExternalSignatureContainer I now construct and return the signature container:

public byte[] Sign(Stream s)
{
    string hashAlgorithm = "SHA-256";
    PdfPKCS7 sgn = new PdfPKCS7(null, chain, hashAlgorithm, false);

    byte[] ocspResponse = Convert.FromBase64String("Base64 encoded DER representation of the OCSP response received from signing service");
    byte[] hash = DigestAlgorithms.Digest(s, hashAlgorithm);
    byte[] sh = sgn.getAuthenticatedAttributeBytes(hash, ocspResponse, null, CryptoStandard.CMS);

    string messageDigest = Sha256_hash(sh);
    // messageDigest sent to signing service
    byte[] signatureAsByte = StringToByteArray("Hex encoded SignatureValue received from signing service");

    sgn.SetExternalDigest(signatureAsByte, null, "RSA");

    ITSAClient tsaClient = new MyITSAClient();

    return sgn.GetEncodedPKCS7(hash, tsaClient, ocspResponse, null, CryptoStandard.CMS); 
}

public class MyITSAClient : ITSAClient
{
    public int GetTokenSizeEstimate()
    {
        return 0;
    }

    public IDigest GetMessageDigest()
    {
        return new Sha256Digest();
    }

    public byte[] GetTimeStampToken(byte[] imprint)
    {
        string hashedImprint = HexEncode(imprint);
        // Hex encoded Imprint sent to signing service

        return Convert.FromBase64String("Base64 encoded DER representation of TimeStampToken received from signing service");
    }
}

Still get these messages:

"The signer's identity is unknown because it has not been included in the list of trusted identities and none or its parent certificates are trusted identities"
"The signature is timestamped, but the timestamp could not be verified"

Further help is very much appreciated again!

"What am I doing wrong?"

The problem is that on one hand you start constructing a CMS signature container using a PdfPKCS7 instance

PdfPKCS7 sgn = new PdfPKCS7(null, chain, hashAlgorithm, false);

and for the calculated document digest hash retrieve the signed attributes to be

byte[] sh = sgn.getAuthenticatedAttributeBytes(hash, null, null, CryptoStandard.CMS);

to send them for signing.

So far so good.

But then you ignore the CMS container you started constructing but instead inject the naked signature bytes you got from your service into the PDF.

This cannot work as your signature bytes don't sign the document directly but instead they sign these signed attributes (and, therefore, indirectly the document as the document hash is one of the signed attributes). Thus, by ignoring the CMS container under construction you dropped the actually signed data...

Furthermore, the subfilter ADBE_PKCS7_DETACHED you use promises that the embedded signature is a full CMS signature container, not a few naked signature bytes, so the format also is wrong.

How to do it instead?

Instead of injecting the naked signature bytes you got from your service into the PDF as is, you have to set them as external digest in the PdfPKCS7 instance in which you originally started constructing the signature container:

sgn.SetExternalDigest(sig, null, ENCRYPTION_ALGO);

(ENCRYPTION_ALGO must be the encryption part of the signature algorithm, I assume in your case "RSA".)

and then you can retrieve the generated CMS signature container:

byte[] encodedSig = sgn.GetEncodedPKCS7(hash, null, null, null, CryptoStandard.CMS);

Now this is the signature container to inject into the document using MyExternalSignatureContainer:

IExternalSignatureContainer external = new MyExternalSignatureContainer(encodedSig);
MakeSignature.SignDeferred(reader, "Signature1", os, external);

Remaining issues

Having corrected your code Adobe Reader still warns about your signatures:

"The signer's identity is unknown because it has not been included in the list of trusted identities and none or its parent certificates are trusted identities"

This warning is to be expected and correct!

The signer's identity is unknown because your signature service uses merely a demo certificate, not a certificate for production use:

As you see the certificate is issued by "GlobalSign Non-Public HVCA Demo", and non-public demo issuers for obvious reasons must not be trusted (unless you manually add them to your trust store for testing purposes).

"The signature is timestamped, but the timestamp could not be verified"

There are two reasons why Adobe does not approve of your timestamp:

On one hand, just like above, the timestamp certificate is a non-public, demo certificate ("DSS Non-Public Demo TSA Responder"). Thus, there is no reason for the verifier to trust your timestamp.

On the other hand, though, there is an actual error in your timestamp'ing code, you apply the hashing algorithm twice! In your MyITSAClient class you have

public byte[] GetTimeStampToken(byte[] imprint)
{
    string hashedImprint = Sha256_hash(imprint);
    // hashedImprint sent to signing service

    return Convert.FromBase64String("Base64 encoded DER representation of TimeStampToken received from signing service");
}

The imprint parameter of your GetTimeStampToken implementation is already hashed, so you have to hex encode these bytes and send them for timestamp'ing. But you apply your method Sha256_hash which first hashes and then hex encodes this new hash.

Thus, instead of applying Sha256_hash merely hex encode the imprint!