How do I get numerical data while web scraping?

2019-09-06 06:52发布

问题:

I'm completely new to web scraping, so any reference sites would be great. I am slightly confused as to how I'm getting the actual data. When I print(theText), I get a bunch of html code (which should be correct). How do I exactly go about getting values from this? Do I have to use regular expressions to get the actual numerical data?

def getData():
    request = urllib.request.Request("http://www.weather.com/weather/5day/l/USGA0028:1:US")
    response = urllib.request.urlopen(request)
    the_page = response.read()
    theText = the_page.decode()
    print(theText)

回答1:

Have a look at BeautifulSoup. It allows you to get elements by their IDs or tags. It is very useful for basic scraping.
You can just call beutiful soup with the response text (html page) and then you can call the bs methods



回答2:

no, you shouldn't use RegExp for HTML. Instead. Have a look at BeatifulSoup4