python extracting HTML tag attributes without regu

Question

Is there any way using urlib, urllib2 or BeautifulSoup to extract HTML tag attributes?

for example:

<a href="xyz" title="xyz">xyz</a>

gets href=xyz, title=xyz

There is another thread talking about using regular expressions

Thanks

Answer 1

You could use BeautifulSoup to parse the HTML, and for each <a> tag, use tag.attrs to read the attributes:

In [111]: soup = BeautifulSoup.BeautifulSoup('<a href="xyz" title="xyz">xyz</a>')

In [112]: [tag.attrs for tag in soup.findAll('a')]
Out[112]: [[(u'href', u'xyz'), (u'title', u'xyz')]]

Answer 2

why don't you try with the HTMLParser module?

Something like this:

import HTMLParser
import urllib

class parseTitle(HTMLParser.HTMLParser):

    def handle_starttag(self, tag, attrs):
        if tag == 'a':
            for names, values in attrs:
                if name == 'href':
                    print value # or the code you need.
                if name == 'title':
                    print value # or the code you need.



aparser = parseTitle()
u = urllib.open('http://stackoverflow.com') # change the address as you like
aparser.feed(u.read())