How to get text which has no HTML tag

2019-08-01 21:25发布

Following is the HTML:

<div class="ajaxcourseindentfix">
    <h3>CPSC 353 - Introduction to Computer Security (3) </h3>
    <hr>Security goals, security systems, access controls, networks and security, integrity, cryptography fundamentals, authentication. Attacks: software, network, website; management considerations, security standards in government and industry; security issues in requirements, architecture, design, implementation, testing, operation, maintenance, acquisition, and services.
    <br>
    <br>Prerequisite: <a href="preview_course_nopop.php?catoid=16&amp;coid=96570" onclick="acalogPopup()">CPSC 253U</a>
    <span style="display: none !important">&nbsp;</span>&nbsp;or <a href="#" onclick="acalogPopup()" target="_blank">CPSC 254</a>
    <span style="display: none !important">&nbsp;</span>&nbsp;and <a href="#" onclick="acalogPopup()" target="_blank">CPSC 351</a>
    <span style="display: none !important">&nbsp;</span>
    , declared major/minor in CPSC, CPEN, or CPEI
    <br>
</div>

I need to fetch the following text from this HTML:

From Line 6 - or
From Line 7 - and
, declared major/minor in CPSC, CPEN, or CPEI

I am able to get the href [Course number: CPSC 254 etc...] with the following XPath:

 # This xpath gives me all the tags followed by h3 and then I iterate through them in my script.  
//div[@class='ajaxcourseindentfix']/h3/following-sibling::text()[2]/following-sibling::*

Update

And, then the text with the following XPath:

# This xpath gives me all the text after the h3 tag.  
//div[@class='ajaxcourseindentfix']/h3/following-sibling::text()[2]/following-sibling::text()

I need to have these course name/prerequisite in the same way they are at URL 1.

enter image description here

In this approach I am getting all the HREF first, then all text. Is there a better way to achieve this? I don't want to iterate over 2 XPaths to get the HREF first, then Text and after that club them to form the prerequisite string.

1 http://catalog.fullerton.edu/ajax/preview_course.php?catoid=16&coid=99648&show

1条回答
老娘就宠你
2楼-- · 2019-08-01 22:09

Try to use below code to get required output:

div = soup.select("div.ajaxcourseindentfix")[0]
" ".join([word for word in div.stripped_strings]).split("Prerequisite: ")[-1]

The output is

'CPSC 253U or CPSC 254 and CPSC 351 , declared major/minor in CPSC, CPEN, or CPEI'
查看更多
登录 后发表回答