Scrapy + Selenium + Datepicker

Question

So i need to scrap a page like this for example and i am using Scrapy + Seleninum to interact with the datepicker calendar but i am running into a ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with.

So far i have:

def parse(self, response):

    self.driver.get("https://www.airbnb.pt/rooms/9315238")
    try:
        element = WebDriverWait(self.driver, 10).until(
            EC.presence_of_element_located((By.XPATH, "//input[@name='checkin']"))
        )
    finally:
        x = self.driver.find_element_by_xpath("//input[@name='checkin']").click()
        import ipdb;ipdb.set_trace()
        self.driver.quit()

I saw some references on how to achieve this https://stackoverflow.com/a/25748322/977622 and https://stackoverflow.com/a/19009256/977622 .

I appreciate if someone could help me out with my issue or even provide a better example on how i can interact the this datepicker calendar.

Answer 1

There are two elements with name="checkin" - the first one that you actually find is invisible. You need to make your locator more specific to match the desired input. I would also use the visibility_of_element_located condition instead:

element = WebDriverWait(self.driver, 10).until(
    EC.visibility_of_element_located((By.CSS_SELECTOR, ".book-it-panel input[name=checkin]"))
)