BeautifulSoup: object of type 'Response' h

Question

Issue: when I try to execute the script, BeautifulSoup(html, ...) gives the error message "TypeError: object of type 'Response' has no len(). I tried passing the actual html as a parameter, but it still doesn't work.

import requests

url = 'http://vineoftheday.com/?order_by=rating'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, "html.parser")

Answer 1

Try to pass the HTML text directly

soup = BeautifulSoup(html.text)

Answer 2

You are getting response.content. But it return response body as bytes (docs). But you should pass str to BeautifulSoup constructor (docs). So you need to use the response.text instead of getting content.

Answer 3

you are getting only response code in 'response' and always use browser header for security otherwise you will face many issues

Find header in debugger console network section 'header' UserAgent

Try

import requests
from bs4 import BeautifulSoup

from fake_useragent import UserAgent

url = 'http://www.google.com'
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'}

response = requests.get(quote_page, headers=headers).text

soup = BeautifulSoup(response, 'html.parser')
print(soup.prettify())

Answer 4

If you're using requests.get('https://example.com') to get the HTML, you should use requests.get('https://example.com').text.