Java HTTP GET response waits until timeout

Question

I am supposed to develop a program that reads a web page with the specified URL, but the problem is that I am not allowed to use any HTTP libraries, I am only allowed to use TCP.

Below is the code that reads the response message:

private static String readMultiline (BufferedReader inStr) {
    String message="";
    String line=readLine(inStr);

    while (line != null) {
         message += line + "\r\n";
         line=readLine(inStr);
    }

    if (message.length() == 0) return null;
        return message;
}

private static String readLine (BufferedReader inStr) {
    String line = null;
    try{
         line = inStr.readLine();
    } catch (IOException ioe) {
         System.out.println("ERROR: Incoming packet could not be read properly.");
         return null;
    }
    return line;
}

The problem is that the header and the web page content are completely received, but the while loop still waits for the 'next' line, which does not exist. After a while, timeout occurs and the code continues. The server I am trying to connect to does not do "Connection:close". How can I detect the end of the file?

Answer 1

Other answers were deleted so I will post one up. You are using an IO method that read lines of information, with a line being defined as a set of characters terminated by a newline character. But there is no guarantee that your TCP packets are ending in newline, so you need to use a more agnostic IO read to process them.

Take a look at DataInputStream, it has a convenience method called readFully that I think you will find quite helpful.

Answer 2

read up on the HTTP protocol. look for length information.

Answer 3

sorry for the late answer.

For anyone who might have suffered the same problem, I also forgot to add "Connection: close" line to the request itself. When I added it, timeout problem disappeared. With this and read()/readFully() functions, I managed to solve the problem. Hope it helps anyone.