URLConnection FileNotFoundException for non-standa

Question

I was trying to use the Apache Ant Get task to get a list of WSDLs generated by another team in our company. They have them hosted on a weblogic 9.x server on http://....com:7925/services/. I am able to get to the page through a browser, but the get task gives me a FileNotFoundException when trying to copy the page to a local file to parse. I was still able to get (using the ant task) a URL without the non-standard port 80 for HTTP.

I looked through the Ant source code, and narrowed the error down to the URLConnection. It seems as though the URLConnection doesn't recognize the data is HTTP traffic, since it isn't on the standard port, even though the protocol is specified as HTTP. I sniffed the traffic using WireShark and the page loads correctly across the wire, but still gets the FileNotFoundException.

Here's an example where you will see the error (with the URL changed to protect the innocent). The error is thrown on connection.getInputStream();

import java.io.File;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;

    public class TestGet {
    private static URL source; 
    public static void main(String[] args) {
        doGet();
    }
    public static void doGet() {
            try {
            source = new URL("http", "test.com", 7925,
                    "/services/index.html");
            URLConnection connection = source.openConnection();
            connection.connect();
            InputStream is = connection.getInputStream();
        } catch (Exception e) {
            System.err.println(e.toString());
        }
    }

}

Answer 1

The response to my HTTP request returned with a status code 404, which resulted in a FileNotFoundException when I called getInputStream(). I still wanted to read the response body, so I had to use a different method: HttpURLConnection#getErrorStream().

Here's a JavaDoc snippet of getErrorStream():

Returns the error stream if the connection failed but the server sent useful data nonetheless. The typical example is when an HTTP server responds with a 404, which will cause a FileNotFoundException to be thrown in connect, but the server sent an HTML help page with suggestions as to what to do.

Usage example:

public static String httpGet(String url) {
    HttpURLConnection con = null;
    InputStream is = null;
    try {
        con = (HttpURLConnection) new URL(url).openConnection();
        con.connect();

        //4xx: client error, 5xx: server error. See: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html.
        boolean isError = con.getResponseCode() >= 400;
        //In HTTP error cases, HttpURLConnection only gives you the input stream via #getErrorStream().
        is = isError ? con.getErrorStream() : con.getInputStream();

        String contentEncoding = con.getContentEncoding() != null ? con.getContentEncoding() : "UTF-8";
        return IOUtils.toString(is, contentEncoding); //Apache Commons IO
    } catch (Exception e) {
        throw new IllegalStateException(e);
    } finally {
        //Note: Closing the InputStream manually may be unnecessary, depending on the implementation of HttpURLConnection#disconnect(). Sun/Oracle's implementation does close it for you in said method.
        if (is != null) {
            try {
                is.close();
            } catch (IOException e) {
                throw new IllegalStateException(e);
            }
        }
        if (con != null) {
            con.disconnect();
        }
    }
}

Answer 2

I have run into a similar issue but the reason seems to be different, here is the exception trace:

java.io.FileNotFoundException: http://myhost1:8081/test/api?wait=1
    at sun.reflect.GeneratedConstructorAccessor2.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1491)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1485)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1139)
    at com.doitnext.loadmonger.HttpExecution.getBody(HttpExecution.java:85)
    at com.doitnext.loadmonger.HttpExecution.execute(HttpExecution.java:214)
    at com.doitnext.loadmonger.ClientWorker.run(ClientWorker.java:126)
    at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.FileNotFoundException: http://myhost1:8081/test/api?wait=1
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1434)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379)
    at com.doitnext.loadmonger.HttpExecution.execute(HttpExecution.java:166)
    ... 2 more

So it would seem that just getting the response code will cause the URL connection to callGetInputStream.

Answer 3

I've tried that locally - using the code provided - and I don't get a FileNotFoundException except when the server returns a status 404 response.

Are you sure that you're connecting to the webserver you intend to be connecting to? Is there any chance you're connecting to a different webserver? (I note that the port number in the code doesn't match the port number in the link)

Answer 4

check the response code being returned by the server

Answer 5

This is an old thread, but I had a similar problem and found a solution that is not listed here.

I was receiving the page fine in the browser, but got a 404 when I tried to access it via the HttpURLConnection. The URL I was trying to access contained a port number. When I tried it without the port number I successfully got a dummy page through the HttpURLConnection. So it seemed the non-standard port was the problem.

I started thinking the access was restricted, and in a sense it was. My solution was that I needed to tell the server the User-Agent and I also specify the file types I expect. I am trying to read a .json file, so I thought the file type might be a necessary specification as well.

I added these lines and it finally worked:

httpConnection.setRequestProperty("User-Agent","Mozilla/5.0 ( compatible ) ");
httpConnection.setRequestProperty("Accept","*/*");

Answer 6

I know this is an old thread but just noticed something on this one so thought I will just put it out there.

Like Jessica mentioned, this exception is thrown when using non-standard port.

It only seems to happen when using DNS though. If I use IP number I can specify the port number and everything works fine.