How to Navigate to a New Webpage In Selenium?

I have the following code:

driver.get(<some url>)
for element in driver.find_elements_by_class_name('thumbnail'):
    element.find_element_by_xpath(".//a").click() #this works and navigates to new page
    element.find_element_by_link_text('Click here').click() #this doesn't

Which needs to navigate the following HTML (simplified of course) by clicking on a thumbnail, which directs to a new page, and then needs to click the Click here link in that new page:

<!DOCTYPE html>
<html lang="en-US" prefix="og: http://iuytp.me/ns# fb: http://iuytp.me/ns/fb#">
<head>
<meta charset="UTF-8" />
<title>Releases</title>
</head>

<body class="archive category category-releases category-4 custom-background">
    <div id="main">
        <div id="container" class="one-column">
            <div id="content" role="main">

                <h1 class="page-title">Releases</h1>

            <div id="thumbnail-post-display">
        <div id="thumbnail-post" class="post-7158 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-designer category-releases category-uncategorized">
            <div class="thumbnail"><a href="http://records.net/uncategorized/designer-7-inch-bufu-records-co-release/" title="Permanent link to Designer &#8211; 7 inch" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/05/dboypledge32-300x300.png" class="attachment-thumbnail wp-post-image" alt="dboypledge3" /></a></div>
            <h2><a href="http://records.net/uncategorized/designer-7-inch-bufu-records-co-release/" title="Permanent link to Designer &#8211; 7 inch" rel="bookmark">Designer &#8211; 7 inch</a></h2>
        </div>
    </div><!--end thumbnail post display-->

            <div id="thumbnail-post-display">
        <div id="thumbnail-post" class="post-7107 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
            <div class="thumbnail"><a href="http://records.net/releases/people-2014-tour-demos/" title="Permanent link to All My People &#8211; 2014 Tour Demos" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/04/01_Doubt-mp3-image-300x300.png" class="attachment-thumbnail wp-post-image" alt="" /></a></div>
            <h2><a href="http://records.net/releases/people-2014-tour-demos/" title="Permanent link to All My People &#8211; 2014 Tour Demos" rel="bookmark">All My People &#8211; 2014 Tour Demos</a></h2>
        </div>
    </div><!--end thumbnail post display-->

            <div id="thumbnail-post-display">
        <div id="thumbnail-post" class="post-7089 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
            <div class="thumbnail"><a href="http://records.net/releases/sirens-blossom-talk/" title="Permanent link to Syrins &#8211; Boss Talk" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/04/sirens_final_smaller-300x300.jpg" class="attachment-thumbnail wp-post-image" alt="sirens_final_smaller" /></a></div>
            <h2><a href="http://records.net/releases/sirens-blossom-talk/" title="Permanent link to Syrins &#8211; Boss Talk" rel="bookmark">Syrins &#8211; Boss Talk</a></h2>
        </div>
    </div><!--end thumbnail post display-->

            <div id="thumbnail-post-display">
        <div id="thumbnail-post" class="post-7073 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
            <div class="thumbnail"><a href="http://records.net/releases/worlds-strongest-man-scares/" title="Permanent link to World&#8217;s Tough Man &#8211; Sorry Scares You" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/03/a2312749950_10-300x300.jpg" class="attachment-thumbnail wp-post-image" alt="a2312749950_10" /></a></div>
            <h2><a href="http://records.net/releases/worlds-strongest-man-scares/" title="Permanent link to World&#8217;s Tough Man &#8211; Sorry Scares You" rel="bookmark">World&#8217;s Tough Man &#8211; Sorry Scares You</a></h2>
        </div>
    </div><!--end thumbnail post display-->

            <div id="thumbnail-post-display">
        <div id="thumbnail-post" class="post-7046 post type-post status-publish format-standard has-post-thumbnail hentry category-blog category-releases">
            <div class="thumbnail"><a href="http://records.net/releases/sundog-space-criminal/" title="Permanent link to Dog &#8211; Space Criminal" rel="bookmark"><img width="300" height="300" src="http://records.net/dev/wp-content/uploads/2014/03/Sundog_cover_high_res-300x300.jpg" class="attachment-thumbnail wp-post-image" alt="dog_cover_high_res" /></a></div>
            <h2><a href="http://records.net/releases/sundog-space-criminal/" title="Permanent link to Dog &#8211; Space Criminal" rel="bookmark">Dog &#8211; Space Criminal</a></h2>
        </div>
    </div><!--end thumbnail post display-->

<div style="clear:both"></div>


        </div><!-- #container -->

    </div><!-- #main -->
</div><!-- #wrapper -->

</div><!--#bg-wrapper-->

</body>
</html>

My code however spits out the following error:

Traceback (most recent call last):
  ...
  File "crawler.py", line 17, in main
    driver.find_element_by_link_text('Click here').click()
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 254, in find_element_by_link_text
    return self.find_element(by=By.LINK_TEXT, value=link_text)
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
    {'using': by, 'value': value})['value']
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
    self.error_handler.check_response(response)
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 164, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: u'no such element\n  (Session info: chrome=35.0.1916.153)\n  (Driver info: chromedriver=2.10.267517,platform=Mac OS X 10.9.3 x86_64)'

The problem appears to be that the element isn't being updated with the contents of the new page. Replacing the problematic line's element with driver doesn't work either. What am I doing wrong?

Please note that I must be able to do this for all thumbnails (hence the for loop).

回答1:

Turns out you need to store the links you want to navigate to in advance. This is what ended up working for me (found this thread to be helpful):

driver.get(<some url>)
elements = driver.find_elements_by_xpath("//h2/a")

links = []
for i in range(len(elements)):
    links.append(elements[i].get_attribute('href'))

for link in links:
    print 'navigating to: ' + link
    driver.get(link)

    # do stuff within that page here...

    driver.back()

回答2:

Your first line:

for element in driver.find_elements_by_class_name('thumbnail'):

grabs all elements on the first page. Your next line:

element.find_element_by_xpath(".//a").click() #this works and navigates to new page

transitions to a completely new page, as you pointed out in your comment. At this point element is gone, so the next line:

element.find_element_by_link_text('Click here').click() #this doesn't

has no chance of doing anything as it refers to something that is not there. Which is exactly what the NoSuchElementException is telling you.

You need to start from the beginning, with something like:

driver.find_element_by_link_text('Click here').click()

Additional answer:

To solve your iteration dilemma, you could take the following approach - please note that I am not familiar with python syntax! The below is Groovy syntax, you will have to adjust it to Python!

// first count the number links you are going to hit; no point in storing this WebElement,
// since it will be gone after we navigate to the first page
def linkCount = driver.findElements(By.className("thumbnail")).size()
// Start a loop based on the count. Inside the loop we are going to have to find each of
// the links again, based on this count. I am going to use XPath; this can probably be done
// on CSS as well. Remember that XPath is 1-based!
(1..linkCount).each {

    // find the element again
    driver.findElement(By.xpath("//div[@class='thumbnail'][$it]/a")).click()

    // do something on the new page ...

    // and go back
    driver.navigate().back()
}