Solve blocking http python function through Asynci

Question

I am implementing a blockchain that communicates through http requests (inspired by this blogpost). This blockchain has a proof of work method, that, depending on the difficulty, can block other http requests for quite some time. This is why I am trying to implement the new asyncio features from python. The following works:

async def proof_of_work(self, last_proof):
    """
    Simple Proof of Work Algorithm:
     - Find a number p' such that hash(pp') contains leading 4 zeroes, where p is the previous p'
    """
    proof = 0
    while self.valid_proof(last_proof, proof) is False:
        proof += 1
        await asyncio.sleep(1)

    return proof

However, this makes my proof of work extremely slow, I guess this is because it is forced to sleep after each iteration. What would be a more elegant way to fix this?

    while self.valid_proof(last_proof, proof) is False:
        proof += 1
        if proof % 1000 == 0:
             await asyncio.sleep(1)

Would quicken it all up a bit, but it looks a bit dirty. What would be the correct way to implement this?

Answer 1

If you want to run CPU-blocking code inside coroutine, you should run it in separate execution flow (to avoid asyncio's event loop freezing) using run_in_executor().

You can use ThreadPoolExecutor if you just want another execution flow or (I think better) to use ProcessPoolExecutor to delegate CPU related work to other core(s).

import asyncio
from concurrent.futures import ProcessPoolExecutor
import hashlib


# ORIGINAL VERSION:
# https://github.com/dvf/blockchain/blob/master/blockchain.py
def valid_proof(last_proof, proof):
    guess = f'{last_proof}{proof}'.encode()
    guess_hash = hashlib.sha256(guess).hexdigest()
    return guess_hash[:4] == "0000"


def proof_of_work(last_proof):
    proof = 0
    while valid_proof(last_proof, proof) is False:
        proof += 1
    return proof


# ASYNC VERSION:
async def async_proof_of_work(last_proof):
    proof = await loop.run_in_executor(_executor, proof_of_work, last_proof)
    return proof


async def main():
    proof = await async_proof_of_work(0)
    print(proof)


if __name__ ==  '__main__':
    _executor = ProcessPoolExecutor(4)

    loop = asyncio.get_event_loop()
    try:
        loop.run_until_complete(main())
    finally:
        loop.run_until_complete(loop.shutdown_asyncgens())
        loop.close()

Output:

69732