linux shell scripting: hex string to bytes

问题:

Lets say that I have a string 5a. This is the hex representation of the ASCII letter Z. I need to know a Linux shell command which will take a hex string and output the binary bytes the string represents.

So if I do

echo 5a | command_im_looking_for > temp.txt

and I open temp.txt, I will see a solitary letter Z.

回答1:

echo -n 5a | perl -pe 's/([0-9a-f]{2})/chr hex $1/gie'

Note that this won't skip non-hex characters. If you want just the hex (no whitespace from the original string etc):

echo 5a | perl -ne 's/([0-9a-f]{2})/print chr hex $1/gie'

Also, zsh and bash support this natively in echo:

echo -e '\x5a'

回答2:

I used to do this using xxd

echo -n 5a | xxd -r -p

But then I realised that in Debian/Ubuntu, xxd is part of vim-common and hence might not be present in a minimal system. To also avoid perl (imho also not part of a minimal system) I ended up using sed, xargs and printf like this:

echo -n 5a | sed 's/\([0-9A-F]\{2\}\)/\\\\\\x\1/gI' | xargs printf

Mostly I only want to convert a few bytes and it's okay for such tasks. The advantage of this solution over the one of ghostdog74 is, that this can convert hex strings of arbitrary lengths automatically. xargs is used because printf doesnt read from standard input.

回答3:

You can make it through echo only and without the other stuff. Don't forget to add "-n" or you will get a linebreak automatically:

echo -n -e "\x5a"

回答4:

Bash one-liner

echo -n "5a" | while read -N2 code; do printf "\x$code"; done

回答5:

depending on where you got that "5a', you can just append \x to it and pass to printf

$ a=5a
$ a="\x${a}"
$ printf "$a"
Z

回答6:

dc can convert between numeric bases:

$ echo 5a | (echo 16i; tr 'a-z' 'A-Z'; echo P) | dc
Z$

回答7:

Some python3 one-liners that work with any number of bytes.

Decoding hex (with strip, so that it's ok to have a newline on stdin):

$ echo 666f6f0a | python3 -c "import sys, binascii; sys.stdout.buffer.write(binascii.unhexlify(input().strip()))"
foo

Encoding hex:

$ echo foo | python3 -c "import sys, binascii; print(binascii.hexlify(sys.stdin.buffer.read()).decode())"
666f6f0a

回答8:

echo 5a | python -c "import sys; print chr(int(sys.stdin.read(),base=16))"

回答9:

Here is a pure bash script (as printf is a bash builtin) :

#warning : spaces do matter
die(){ echo "$@" >&2;exit 1;}

p=48656c6c6f0a

test $((${#p} & 1)) == 0 || die "length is odd"
p2=''; for ((i=0; i<${#p}; i+=2));do p2=$p2\\x${p:$i:2};done
printf "$p2"

If bash is already running, this should be faster than any other solution which is launching a new process.

回答10:

GNU awk 4.1

awk -niord '$0=chr("0x"RT)' RS=.. ORS=

Note that if you echo to this it will produce an extra null byte

$ echo 595a | awk -niord '$0=chr("0x"RT)' RS=.. ORS= | od -tx1c
0000000  59  5a  00
          Y   Z  \0

Instead use printf

$ printf 595a | awk -niord '$0=chr("0x"RT)' RS=.. ORS= | od -tx1c
0000000  59  5a
          Y   Z

Also note that GNU awk produces UTF-8 by default

$ printf a1 | awk -niord '$0=chr("0x"RT)' RS=.. ORS= | od -tx1
0000000 c2 a1

If you are dealing with characters outside of ASCII, and you are going to be Base64 encoding the resultant string, you can disable UTF-8 with -b

echo 5a | sha256sum | awk -bniord 'RT~/\w/,$0=chr("0x"RT)' RS=.. ORS=

回答11:

Similar to my answer here: Linux shell scripting: hex number to binary string

You can do it with the same tool like this (using ascii printable character instead of 5a):

echo -n 616263 | cryptocli dd -decoders hex

Will produce the following result:

abcd

回答12:

As per @Randal comment, you can use perl, e.g.

$ printf 5a5a5a5a | perl -lne 'print pack "H*", $_'
ZZZZ

and other way round:

$ printf ZZZZ | perl -lne 'print unpack "H*", $_'
5a5a5a5a

---

Another example with file:

$ printf 5a5a5a5a | perl -lne 'print pack "H*", $_' > file.bin
$ perl -lne 'print unpack "H*", $_' < file.bin
5a5a5a5a