Selenium (Python) - waiting for a download process

Question

I'm using selenium and python via chromewebdriver (windows) in order to automate a task of downloading large amount of files from different pages. My code works, but the solution is far from ideal: the function below clicks on the website button that initiating a java script function that generating a PDF file and then downloading it.

I had to use a static wait in order to wait for the download to be completed (ugly) I cannot check the file system in order to verify when the download is completed since i'm using multi threading (downloading lot's of files from different pages at once) and also the the name of the files is generated dynamically in the website itself.

My code:

def file_download(num, drivervar):
Counter += 1
    try:
        drivervar.get(url[num])
        download_button = WebDriverWait(drivervar, 20).until(EC.element_to_be_clickable((By.ID, 'download button ID')))
        download_button.click()
        time.sleep(10) 
    except TimeoutException: # Retry once
        print('Timeout in thread number: ' + str(num) + ', retrying...')
..... 

Is it possible to determine download completion in webdriver? I want to avoid using time.sleep(x).

Thanks a lot.

Answer 1

I have had the same problem and found a solution. You can check weither or not a .crdownload is in your download folder. If there are 0 instances of a file with .crdownload extension in the download folder then all your downloads are completed. This only works for chrome and chromium i think.

def downloads_done():
    for i in os.listdir("data/"):
        if ".crdownload" in i:
            time.sleep(0.5)
            downloads_done()

Whenever you call downloads_done() it will loop itself untill all downloads are completed. If you are downloading massive files like 80 gigabytes then i don't recommend this because then the function can reach maximum recursion depth.

Answer 2

When using test automation, its crucial that developers make the software testable. It is your job to check the software combined with the testability, meaning that you need to request a spinner or a simple HTML tag which indicates when the download is done successfully.

In a case as yours, where you cannot check it in the UI and you cannot check in system, this is the best way to solve it.

Answer 3

You can get the status of each download by navigation chrome://downloads/ with the driver.

To wait for all the downloads to finish and to list all the paths:

def every_downloads_chrome(driver):
    if not driver.current_url.startswith("chrome://downloads"):
        driver.get("chrome://downloads/")
    return driver.execute_script("""
        var items = downloads.Manager.get().items_;
        if (items.every(e => e.state === "COMPLETE"))
            return items.map(e => e.file_url);
        """)


# waits for all the files to be completed and returns the paths
paths = WebDriverWait(driver, 120, 1).until(every_downloads_chrome)
print(paths)