Convert python long/int to fixed size byte array

Question

I'm trying to implement RC4 and DH key exchange in python. Problem is that I have no idea about how to convert the python long/int from the key exchange to the byte array I need for the RC4 implementation. Is there a simple way to convert a long to the required length byte array?

Update: forgot to mention that the numbers I'm dealing with are 768 bit unsigned integers.

Answer 1

I haven't done any benchmarks, but this recipe "works for me".

The short version: use '%x' % val, then unhexlify the result. The devil is in the details, though, as unhexlify requires an even number of hex digits, which %x doesn't guarantee. See the docstring, and the liberal inline comments for details.

from binascii import unhexlify

def long_to_bytes (val, endianness='big'):
    """
    Use :ref:`string formatting` and :func:`~binascii.unhexlify` to
    convert ``val``, a :func:`long`, to a byte :func:`str`.

    :param long val: The value to pack

    :param str endianness: The endianness of the result. ``'big'`` for
      big-endian, ``'little'`` for little-endian.

    If you want byte- and word-ordering to differ, you're on your own.

    Using :ref:`string formatting` lets us use Python's C innards.
    """

    # one (1) hex digit per four (4) bits
    width = val.bit_length()

    # unhexlify wants an even multiple of eight (8) bits, but we don't
    # want more digits than we need (hence the ternary-ish 'or')
    width += 8 - ((width % 8) or 8)

    # format width specifier: four (4) bits per hex digit
    fmt = '%%0%dx' % (width // 4)

    # prepend zero (0) to the width, to zero-pad the output
    s = unhexlify(fmt % val)

    if endianness == 'little':
        # see http://stackoverflow.com/a/931095/309233
        s = s[::-1]

    return s

...and my nosetest unit tests ;-)

class TestHelpers (object):
    def test_long_to_bytes_big_endian_small_even (self):
        s = long_to_bytes(0x42)
        assert s == '\x42'

        s = long_to_bytes(0xFF)
        assert s == '\xff'

    def test_long_to_bytes_big_endian_small_odd (self):
        s = long_to_bytes(0x1FF)
        assert s == '\x01\xff'

        s = long_to_bytes(0x201FF)
        assert s == '\x02\x01\xff'

    def test_long_to_bytes_big_endian_large_even (self):
        s = long_to_bytes(0xab23456c8901234567)
        assert s == '\xab\x23\x45\x6c\x89\x01\x23\x45\x67'

    def test_long_to_bytes_big_endian_large_odd (self):
        s = long_to_bytes(0x12345678901234567)
        assert s == '\x01\x23\x45\x67\x89\x01\x23\x45\x67'

    def test_long_to_bytes_little_endian_small_even (self):
        s = long_to_bytes(0x42, 'little')
        assert s == '\x42'

        s = long_to_bytes(0xFF, 'little')
        assert s == '\xff'

    def test_long_to_bytes_little_endian_small_odd (self):
        s = long_to_bytes(0x1FF, 'little')
        assert s == '\xff\x01'

        s = long_to_bytes(0x201FF, 'little')
        assert s == '\xff\x01\x02'

    def test_long_to_bytes_little_endian_large_even (self):
        s = long_to_bytes(0xab23456c8901234567, 'little')
        assert s == '\x67\x45\x23\x01\x89\x6c\x45\x23\xab'

    def test_long_to_bytes_little_endian_large_odd (self):
        s = long_to_bytes(0x12345678901234567, 'little')
        assert s == '\x67\x45\x23\x01\x89\x67\x45\x23\x01'

Answer 2

Python 2.7 does not implement the int.to- very slow_bytes() method.

I tried 3 methods:

1. hex unpack/pack : very slow
2. byte shifting 8 bits at a time: significantly faster.
3. using a "C" module and packing into the lower (7 ia64 or 3 i32) bytes. This was about twice as fast as 2/ . It is the fastest option, but still too slow.

All these methods are very inefficient for two reasons:

• Python 2.7 does not support this useful operation.
• c does not support extended precision arithmetic using the carry/borrow/overflow flags available on most platforms.

Answer 3

Little-endian, reverse the result or the range if you want Big-endian.

def int_to_bytes(val, num_bytes):
    return [(val & (0xff << pos*8)) >> pos*8 for pos in range(num_bytes)]

Answer 4

Everyone has overcomplicated this answer:

some_int = <256 bit integer>
some_bytes = some_int.to_bytes(32, sys.byteorder)
my_bytearray = bytearray(some_bytes)

You just need to know the number of bytes that you are trying to convert. In my use cases, normally I only use this large of numbers for crypto, and at that point I have to worry about modulus and what-not, so I don't think this is a big problem to be required to know the max number of bytes to return.

Since you are doing it as 768-bit math, then instead of 32 as the argument it would be 96.