JSoup - Parse HTML tag by tag

2019-09-11 01:07发布

站内问答 / Java
1716 2 4

I'm actually developping a text parser in Java and I was asked to enhance it by parsing HTML with it. The parser's purpose is to divide the file parsed into 3 other files, one with all the words contained in the file, one with all sentences and the other with all questions.

The *.txt part works perfectly, but I got a problem when parsing HTML.

I create a temporary file with *.txt extension and pass it in my text parser, but if I pass an URL with HTML file linked which is formed like this:

<!DOCTYPE html>
    <head>
        ... some HTML here ...
    </head>
    <body>
        <ul class="some_menu">
            <li class="some_menu_item">n1</li>
            <li class="some_menu_item">n2</li>
            <li class="some_menu_item">n2</li>
        </ul>
        <div>
            This is a question ?
            This is a sentence .
            ... some other text ...
        </div>
    </body>
</html>

the question file will be filled with: n1 n2 n3 This is a question

So, I just was wondering, is there a way to parse with JSoup tags by tags so I can add a line feed each time a block is closed?

If you need some new informations, don't bother to ask!

Edit: I should have 3 output files, which are, for this example:

  1. One with all the words

    n1
n2
n3
This
is
a
question
sentence
... some other words ...

  2. One with all the sentences

    This is a sentence

  3. One with all the questions

    This is a question

TimmyM

2条回答
对你真心纯属浪费
2楼-- · 2019-09-11 01:35

There are quite a lot HTML parser available in market like

  1. HTMLUnit
  2. HTMLCleaner
  3. Jericho
  4. JSoup

https://en.wikipedia.org/wiki/Comparison_of_HTML_parsers

Thanks, Vineet

查看更多
0人赞 添加讨论(0) 举报
不美不萌又怎样
3楼-- · 2019-09-11 01:40

To get all the text in an html body, you can use:

Document doc = Jsoup.connect(url).get();
Elements body = doc.select("body");
String allText = body[0].text();

You can then split the text to get each word separate. To get the text in the div tag, you can use:

Elements div = doc.select("div");
String divText = div[0].text();

You can then split the divText to get each sentence.

Notice that the return type of the select query is actually a list of Element i.e., Elements. That's because there can be more than one elements matching you select query. In this case, since there is only one element for each case we access it by accessing the index 0 of the returned array.

Edit: In order to iterate through all elements check this answer. Basically

Elements elements = doc.body().select("*");

for (Element element : elements) {
    System.out.println(element.text());
}

Though there might be elements with no texts so you can put a check on that.

查看更多
0人赞 添加讨论(0) 举报
登录 后发表回答
相关问题
查看全部
相关文章
查看全部
收藏的人(4)