openssl sha1 giving different results in the termi

Question

I am trying to run an openssl command from a Python script, but am getting different sha1 values from the command executed in the script and the command executed directly in the terminal.

Here is my code:

command = "echo -n '" + hex(key)[2:] + "' | openssl sha1"
print(command)
os.system(command)
output = subprocess.check_output(command, shell=True)
# This converts the bytes object to a string.
output = output.decode("utf-8")
print(output)

The os.system(command) is only there to check whether this method and subprocess.check_output() give the same result.

Here is the program output for an example input of key=0xabc:

echo -n 'abc' | openssl sha1
(stdin)= 9d4fe92b8bdc894f5838fad83108bf3841b257fa
(stdin)= 9d4fe92b8bdc894f5838fad83108bf3841b257fa

The first line is the command to be executed, the second is the result of the command using os.system() and the third is the result of the command using subprocess.check_output(). As you can see, both methods give the same result.

Now, if I copy and paste the command as it is displayed here and execute it in the terminal, this is the result:

(stdin)= a9993e364706816aba3e25717850c26c9cd0d89d

Even though I copy and pasted the exact same command, the hash value output is different.

Even stranger, if I omit the -n option from echo, there is no inconsistency:

echo 'abc' | openssl sha1
(stdin)= 03cfd743661f07975fa2f1220c5194cbaff48451
(stdin)= 03cfd743661f07975fa2f1220c5194cbaff48451

And in the terminal:

(stdin)= 03cfd743661f07975fa2f1220c5194cbaff48451

Everything I have read online about similar issues stems from not using -n, rather than -n seemingly causing the issue. For the particular task I am doing, I have to use -n as I am computing the pre-image of a hash which was calculated using echo -n "hash_value" | openssl sha1.

Any help would be much appreciated :)

Answer 1

TL;DR: use /bin/echo

Explanation:

The python system command is likely using a default shell of /bin/sh which is usually linked to a POSIX compliant shell, while your terminal session is using your user's login shell which is likely bash. POSIX shell standard does not support the -n option to the echo builtin function. This mean you are literally calculating the digest of "-n abc". You can reproduce this is bash like so:

echo "-n abc" | openssl sha1
9d4fe92b8bdc894f5838fad83108bf3841b257fa

The binary executable is still available but will need to be called by the full path since the shell builtin will override it. Instead of echo, use /bin/echo.