How the assembly file is generated from the perl s

In the opensource code of OpenSSL(version 1.1.0e) I saw that some of the function definition is generated by the perl files present inside the folders. In the build.info file present in each folders inside the crypto, they have written some lines to generate the .s from the corresponding .pl.

For example, for generating aes_p8_set_encrypt_key in crypto/aes/build.info:

GENERATE[aesp8-ppc.s]=asm/aesp8-ppc.pl $(PERLASM_SCHEME)

for generating OPENSSL_madd300_probe in crypto/build.info :

GENERATE[ppccpuid.s]=ppccpuid.pl $(PERLASM_SCHEME)

And also in the main Makefile(generated makefile), there are some lines as below:

crypto/aes/aes-x86_64.o: crypto/aes/aes-x86_64.s
$(CC)  -I. -Icrypto/include -Iinclude $(CFLAGS) $(LIB_CFLAGS) -MMD -MF crypto/aes/aes-x86_64.d.tmp -MT $@ -c -o $@ crypto/aes/aes-x86_64.s
@touch crypto/aes/aes-x86_64.d.tmp
@if cmp crypto/aes/aes-x86_64.d.tmp crypto/aes/aes-x86_64.d > /dev/null 2> /dev/null; then \
    rm -f crypto/aes/aes-x86_64.d.tmp; \
else \
    mv crypto/aes/aes-x86_64.d.tmp crypto/aes/aes-x86_64.d; \
fi

Followed with :

crypto/aes/aes-x86_64.s: crypto/aes/asm/aes-x86_64.pl
CC="$(CC)" $(PERL) crypto/aes/asm/aes-x86_64.pl $(PERLASM_SCHEME) $@

Can anyone explain how the .s is generated from the .pl files? I need to add them in my Makefile inside my project, to solve the undefined reference error coming for the functions whose definition is generated by the .pl file.

How the assembly file is generated from the Perl script in OpenSSL...

The assembly language file is generated with a xlat program. For the Power8 gear the file is ppc-xlate.pl in the perlasm directory. It is used by aesp8-ppc.pl in the crypto/aes/asm directory.

Here is how you translate it. I'm working from GCC112 on the compile farm, which is ppc64le.

git clone https://github.com/openssl/openssl.git
mkdir cryptogams

cp ./openssl/crypto/perlasm/ppc-xlate.pl cryptogams/
cp ./openssl/crypto/aes/asm/aesp8-ppc.pl cryptogams/

cd cryptogams/
chmod +x *.pl

./aesp8-ppc.pl ppc64le aesp8-ppc.s

aesp8-ppc.pl generates a pure assembly language source file so name the output file with the little *.s. Sometimes the translation includes C preprocessor statements and it needs a big *.S (but not in this case).

The second argument to aesp8-ppc.pl is called flavor. It is ppc64le in the commands above. Flavor does two things. First, it selects either 32-bit or 64-bit. Second, it selects either little-endian or big-endian. Be sure to get the flavor right.

Once you have aesp8-ppc.s you can compile and assemble the source file with GCC.

gcc -mcpu=power8 -c aesp8-ppc.s

And then:

$ objdump --disassemble aesp8-ppc.o
aesp8-ppc.o:     file format elf64-powerpcle
...

0000000000000420 <aes_p8_set_decrypt_key>:
     420:       c1 ff 21 f8     stdu    r1,-64(r1)
     424:       a6 02 48 7d     mflr    r10
     428:       50 00 41 f9     std     r10,80(r1)
     42c:       75 fc ff 4b     bl      a0 <aes_p8_set_encrypt_key>
     430:       a6 03 48 7d     mtlr    r10
     434:       00 00 03 2c     cmpwi   r3,0
     438:       68 00 c2 40     bne-    4a0 <Ldec_key_abort>
     43c:       36 20 07 55     rlwinm  r7,r8,4,0,27
     440:       10 ff 65 38     addi    r3,r5,-240
     444:       7e f8 08 55     rlwinm  r8,r8,31,1,31
     448:       14 3a a3 7c     add     r5,r3,r7
     44c:       a6 03 09 7d     mtctr   r8
...

At this point you have an object file, but you don't know the API signatures or how to use it. To find out what to do next you have to objdump and then grep the OpenSSL sources to see how they use it.

$ nm aesp8-ppc.o | grep ' T '
00000000000006c0 T aes_p8_cbc_encrypt
0000000000001140 T aes_p8_ctr32_encrypt_blocks
00000000000005c0 T aes_p8_decrypt
00000000000004c0 T aes_p8_encrypt
0000000000000420 T aes_p8_set_decrypt_key
00000000000000a0 T aes_p8_set_encrypt_key
0000000000001d00 T aes_p8_xts_decrypt
0000000000001a60 T aes_p8_xts_encrypt

You are interested in the four functions aes_p8_set_encrypt_key, aes_p8_set_decrypt_key, aes_p8_encrypt and aes_p8_decrypt.

I'll help you with the first one: aes_p8_set_encrypt_key.

$ cd openssl
$ grep -nIR aes_p8_set_encrypt_key
crypto/evp/e_aes.c:153:# define HWAES_set_encrypt_key aes_p8_set_encrypt_key


$ grep -nIR HWAES_set_encrypt_key
...
crypto/evp/e_aes.c:2515:int HWAES_set_encrypt_key(const unsigned char *userKey, const int bits,
...


$ cat -n crypto/evp/e_aes.c
...
  2515  int HWAES_set_encrypt_key(const unsigned char *userKey, const int bits,
  2516                            AES_KEY *key);

Lather, rinse, repeat for AES_KEY, aes_p8_set_decrypt_key, aes_p8_encrypt and aes_p8_decrypt.

Eventually you will end up with a header like shown in Cryptogams | AES on the OpenSSL wiki. Cryptogams | AES is written for ARMv4 but the same applies to Power8 as well.

Stepping back a bit, you are using the Cryptogams implementation of AES on Power8. Cryptogams is Andy Polyakov's project to provide high speed cryptography to other developers.

Andy dual licenses his work. One license is the OpenSSL license because Andy works for OpenSSL. The second license is a BSD style license that does not have the encumbrances of OpenSSL.

Andy's public source is at GitHub | dot-asm. Unfortunately, most of Andy's work has not been uploaded so you have to pull it from OpenSSL. And more unfortunate, none of it is documented so you have to do a fair amount of poking and prodding in the OpenSSL sources.

As far as I know there are two places you can look for documentation on using Cryptogams and Power8 cryptography. First is the OpenSSL wiki pages Cryptogams | AES and Cryptogams | SHA. The tutorials are ARMv4 but it applies to Power 8, too. I wrote the wiki articles so errors and omissions are my mistakes.

The second place to look is GitHub and Noloader | POWER8 crypto. I help maintain Crypto++ and the POWER8 crypto book is my knowledge dump. The POWER8 book includes Cryptogams SHA for PowerPC in Chapter 7.

The POWER8 crypto book was written by Bill Schmidt and I because we could not find documentation when working with AES and SHA on Power8. Bill Schmidt works for IBM and even he could not get the docs. All we could find was a blog post from an IBM engineer that was grossly missing details.