python beautifulsoup iframe document html extract

2019-01-14 18:35发布

问题:

I am trying to learn a bit of beautiful soup, and to get some html data out of some iFrames - but I have not been very successful so far.

So, parsing the iFrame in itself does not seem to be a problem with BS4, but I do not seem to get the embedded content from this - whatever I do.

For example, consider the below iFrame (this is what I see on chrome developer tools):

<iframe frameborder="0" marginwidth="0" marginheight="0" scrolling="NO"
src="http://www.engineeringmaterials.com/boron/728x90.html "width="728" height="90">
#document <html>....</html></iframe>

where, <html>...</html> is the content I am interested in extracting.

However, when I use the following BS4 code:

iFrames=[] # qucik bs4 example
for iframe in soup("iframe"):
    iFrames.append(soup.iframe.extract())

I get:

<iframe frameborder="0" marginwidth="0" marginheight="0" scrolling="NO" src="http://www.engineeringmaterials.com/boron/728x90.html" width="728" height="90">

In other words, I get the iFrames without the document <html>...</html> within them.

I tried something along the lines of:

iFrames=[] # qucik bs4 example
iframexx = soup.find_all('iframe')
for iframe in iframexx:
    print iframe.find_all('html')

.. but this does not seem to work..

So, I guess my question is, how do I reliably extract these document objects <html>...</html> from the iFrame elements.

回答1:

Browsers load the iframe content in a separate request. You'll have to do the same:

for iframe in iframexx:
    response = urllib2.urlopen(iframe.attrs['src'])
    iframe_soup = BeautifulSoup(response)

Remember: BeautifulSoup is not a browser; it won't fetch images, CSS and JavaScript resources for you either.

python beautifulsoup iframe document html extract

问题:

回答1:

收藏的人(0)

举报内容

检举类型

检举原因

检举说明(必填)

打开微信“扫一扫”，打开网页后点击屏幕右上角分享按钮