Get x509 certificate hash with openssl library

Question

I'm currently working on an app, which uses the openssl library (libcrypto) to generate certificates. Now I have to get the hash of a already existing certificate.

When I use my Terminal I am able to generate the hash value by using

openssl x509 -hash -in cert.pem -noout

Output: 01da0e2b

This is my code where I try t generate my hash value by using the library in C.

X509 *cert = NULL;
FILE *fp = fopen(currentCert.UTF8String, "r");
PEM_read_X509(fp, &cert, NULL, NULL);

long hash = X509_subject_name_hash(cert);
char *mdString = malloc(sizeof(long));
sprintf(mdString, "%lx",hash);
printf(mdString);

Output: 1817886a

But actually my output is a different one. Has anybody an idea what am I doing wrong ?

Answer 1

You are not allocating enough memory for the string, although I can't be sure that is the cause of your problem.

char *mdString = malloc(sizeof(long));

will allocate 4 bytes to the string, yet it clearly needs to hold 8 bytes plus a terminator, so I suggest

char *mdString = malloc(sizeof(long)*2 + 1);

Answer 2

But actually my output is a different one. Has anybody an idea what am I doing wrong ?

Here's how OpenSSL uses it...

$ cd openssl-1.0.2-src
$ grep -R X509_subject_name_hash *
apps/x509.c:                BIO_printf(STDout, "%08lx\n", X509_subject_name_hash(x));
apps/x509.c:                BIO_printf(STDout, "%08lx\n", X509_subject_name_hash_old(x));
crypto/x509/x509.h:unsigned long X509_subject_name_hash(X509 *x);
crypto/x509/x509.h:unsigned long X509_subject_name_hash_old(X509 *x);
crypto/x509/x509_cmp.c:unsigned long X509_subject_name_hash(X509 *x)
crypto/x509/x509_cmp.c:unsigned long X509_subject_name_hash_old(X509 *x)
...

Then, looking at apps/x509.c:

...
} else if (subject_hash == i) {
    BIO_printf(STDout, "%08lx\n", X509_subject_name_hash(x));
}
...

And your declaration should be:

unsigned long hash = X509_subject_name_hash(cert);

Then:

fprintf(stdout, "%08lx\n", hash);

---

Also, OpenSSL changed the way in calculates the subject hash sometime around OpenSSL 1.0.1. That's why there is an X509_subject_name_hash and X509_subject_name_hash_old.

If you are using or comparing against OpenSSL 0.9.8 (on, say Mac OS X 10), then see Generate Subject Hash of X509Certificate in Java. Though its Java, it details OpenSSL handling of the subject hash.