how to take title text from any web page in java

Question

I am using java to fetch the title text from web page.

I have fetched image from web page using Tag name as follows:

    int i=1; 
InputStream in=new URL("www.yahoo.com").openStream();
org.w3c.dom.Document doc= new Tidy().parseDOM(in, null);   
    NodeList img=doc.getElementsByTagName("img");
ArrayList<String> list=new ArrayList<String>();                   
    list.add(img.item(i).getAttributes().getNamedItem("src").getNodeValue());

It is working,But I want to fetch title tag from web page(www.yahoo.com) using same code as above.I have mentioned getElementsByTagName("title"); but it is not working. Please help me,how to do that using jtidy parser as above.

Answer 1

Try this,

InputStream response = null;
    try {
        String url = "http://example.com/"; // specify the URL
        response = new URL(url).openStream();


        Scanner scanner = new Scanner(response);
        String responseBody = scanner.useDelimiter("\\A").next();
        System.out.println(responseBody.substring(responseBody.indexOf("<title>") + 7, responseBody.indexOf("</title>"))); // it fetches the text inside the title tag

    } catch (IOException ex) {
        ex.printStackTrace();
    } finally {
        try {
            response.close();
        } catch (IOException ex) {
            ex.printStackTrace();
        }
    }

Answer 2

Watch that the NodeList index starts at 0 (i see your "int i = 1;") http://download.oracle.com/javase/1.4.2/docs/api/org/w3c/dom/NodeList.html.

Also, you can "getNodeValue()" of an Attribute (ie "src"), but not of an Element http://download.oracle.com/javase/1.5.0/docs/api/org/w3c/dom/Node.html. In this case you can use "getTextContent()", because I dont believe the "title" tag has child Elements. So:

String titleText = doc.getElementsByTagName("title").item(0).getTextContent(); 

Or:

String titleText = doc.getElementsByTagName("title").item(0).getFirstChild().getNodeValue(); 

Answer 3

Wee can't tell unless you post the code you are actually tyring to use to get the title, but this clearly won't work:

    list.add(img.item(i).getAttributes().getNamedItem("src").getNodeValue());

because the title element doesn't have a src attribute.

Answer 4

You can fetch the title of an HTML page easily using an XPath:

/html/head/title/text()

You can achieve this easily with Dom4J, and I think in JTidy as well.