FF Xpather to Nokogiri — Can I just copy and paste

Question

I was doing this manually and then I got stuck and I can't figure out why it's not working. I downloaded xpather and it is giving me: /html/body/center/table/tbody/tr[3]/td/table as the path to the item I want. I have manually confirmed that this is correct but when I paste it into my code, all it does is return nil

Here is my code:

a = parentdoc.at_xpath("//html/body/center/table/tbody/tr[3]/td/table[1]")
puts a

If I do something like this:

a = parentdoc.at_xpath("//html/body/center")
puts a

I get a huge chunk of text from the page. I can keep adding elements until I hit tbody then it returns nil again. I even tried something like: //html/body/center/table/*/tr[3] and that did the same thing returning nil

What am I missing?

Answer 1

Your problem is that Firefox is inserting a <tbody> element that is not present in the HTML. When you use xpather, it is working from the HTML that the browser is using (rather than the raw HTML that ycombinator.com is returning) and it finds this path:

//html/body/center/table/tbody/tr[3]/td/table

Nokogiri will be working with the raw HTML so you want this

//html/body/center/table/tr[3]/td/table

When I apply that XPath to the URL in your comment:

doc.at_xpath('//html/body/center/table/tr[3]/td/table').text

I get this output:

"csoghoian 1 hour ago  | link | parent2 responses:1. Chrome is the only major browser not to ...