How to correct TypeError: Unicode-objects must be

Question

I have this error:

Traceback (most recent call last):
  File "python_md5_cracker.py", line 27, in <module>
  m.update(line)
TypeError: Unicode-objects must be encoded before hashing

when I try to execute this code in Python 3.2.2:

import hashlib, sys
m = hashlib.md5()
hash = ""
hash_file = input("What is the file name in which the hash resides?  ")
wordlist = input("What is your wordlist?  (Enter the file name)  ")
try:
        hashdocument = open(hash_file,"r")
except IOError:
        print("Invalid file.")
        raw_input()
        sys.exit()
else:
        hash = hashdocument.readline()
        hash = hash.replace("\n","")

try:
        wordlistfile = open(wordlist,"r")
except IOError:
        print("Invalid file.")
        raw_input()
        sys.exit()
else:
        pass
for line in wordlistfile:
        m = hashlib.md5()  #flush the buffer (this caused a massive problem when placed at the beginning of the script, because the buffer kept getting overwritten, thus comparing incorrect hashes)
        line = line.replace("\n","")
        m.update(line)
        word_hash = m.hexdigest()
        if word_hash==hash:
                print("Collision!  The word corresponding to the given hash is", line)
                input()
                sys.exit()

print("The hash given does not correspond to any supplied word in the wordlist.")
input()
sys.exit()

Answer 1

It is probably looking for a character encoding from wordlistfile.

wordlistfile = open(wordlist,"r",encoding='utf-8')

Or, if you're working on a line-by-line basis:

line.encode('utf-8')

Answer 2

Please take a look first at that answer.

Now, the error message is clear: you can only use bytes, not Python strings (what used to be unicode in Python < 3), so you have to encode the strings with your preferred encoding: utf-32, utf-16, utf-8 or even one of the restricted 8-bit encodings (what some might call codepages).

The bytes in your wordlist file are being automatically decoded to Unicode by Python 3 as you read from the file. I suggest you do:

m.update(line.encode(wordlistfile.encoding))

so that the encoded data pushed to the md5 algorithm are encoded exactly like the underlying file.

Answer 3

The error already says what you have to do. MD5 operates on bytes, so you have to encode Unicode string into bytes, e.g. with line.encode('utf-8').

Answer 4

You must have to define encoding format like utf-8, Try this easy way,

This example generates a random number using the SHA256 algorithm:

>>> import hashlib
>>> hashlib.sha256(str(random.getrandbits(256)).encode('utf-8')).hexdigest()
'cd183a211ed2434eac4f31b317c573c50e6c24e3a28b82ddcb0bf8bedf387a9f'

Answer 5

To store the password (PY3):

import hashlib, os
password_salt = os.urandom(32).hex()
password = '12345'

hash = hashlib.sha512()
hash.update(('%s%s' % (password_salt, password)).encode('utf-8'))
password_hash = hash.hexdigest()

Answer 6

You could open the file in binary mode:

import hashlib

with open(hash_file) as file:
    control_hash = file.readline().rstrip("\n")

wordlistfile = open(wordlist, "rb")
# ...
for line in wordlistfile:
    if hashlib.md5(line.rstrip(b'\n\r')).hexdigest() == control_hash:
       # collision