BeautifulSoup HTML getting src link

2020-03-07 05:41发布

问题:

I'm making a small web crawler using python 3.5.1 and requests module, which downloads all comics from a specific website.I'm experimenting with one page. I parse the page using BeautifulSoup4 like below:

import webbrowser
import sys
import requests
import re
import bs4

res = requests.get('http://mangapark.me/manga/berserk/s5/c342')
res.raise_for_status()
soup = bs4.BeautifulSoup(res.text, 'html.parser')

for link in soup.find_all("a", class_ = "img-link"):
    if(link):
        print(link)
    else:
        print('ERROR')

When I do print(link) there are the correct HTML parts I'm interested in, but when I try to get only the link in src using link.get('src') it only prints None.

I tried getting the link using:

img = soup.find("img")["src"]

and it was OK, but I want to have all the src links, not the first link. I have little experience with beautifulSoup. Please point out what's going on. Thank you.

The sample HTML part from the website I'm interested in is:

<a class="img-link" href="#img2">
    <img id="img-1" class="img"
          rel="1" i="1" e="0" z="1" 
          title="Berserk ch.342 page 1" src="http://2.p.mpcdn.net/352582/687224/1.jpg"
          width="960" _width="818" _heighth="1189"/>        
</a>

回答1:

I would do it in one go using a CSS selector:

for img in soup.select("a.img-link img[src]"):
    print(img["src"])

Here, we are getting all of the img elements having an src attribute located under an a element with a img-link class. It prints:

http://2.p.mpcdn.net/352582/687224/1.jpg
http://2.p.mpcdn.net/352582/687224/2.jpg
http://2.p.mpcdn.net/352582/687224/3.jpg
http://2.p.mpcdn.net/352582/687224/4.jpg
...
http://2.p.mpcdn.net/352582/687224/20.jpg

If you still want to use the find_all(), you would have to nest it:

for link in soup.find_all("a", class_ = "img-link"):
    for img in link.find_all("a", src=True):  # searching for img with src attribute
        print(img["src"])