java.io.IOException: Premature EOF in REST in java

Question

I implement a RESTful Web Service in Java using jersey and run it on Tomcat7.

I have this query to get list of tasks :

@GET
@Path("/users/{username}/")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public Response taskList(@PathParam("username") String username) {      
        Session session = HibernateUtil.getSessionFactory().getCurrentSession();
        session.beginTransaction();
        List<Task> tasks = null;
        try { 
            tasks = (List<Task>)session.createQuery("from Task as t where t.user.username = :name order by t.project.key ASC, t.deliverable.key ASC, t.activity.key ASC, t.start ASC").setString("name", username).list();   
        } catch (HibernateException e) {
            e.printStackTrace();
            session.getTransaction().rollback();
        }
        session.getTransaction().commit();
        return Response.ok().entity(tasks).build();
    }

I test it with jersey client like :

try {
    response = service.path("task/users/are/").type("application/json").get(ClientResponse.class);
    s = response.getEntity(String.class);
    System.out.println(s);
} catch (Exception e) {
    e.printStackTrace();
}

An example for the result:

[{"id":5369,"start":1331428800000,"end":1331428800000,"description":"","started":false,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"activity":{"id":102,"key":"d","name":"Development","description":"Usually this.","cost":690.0,"orderIndex":2,"parent":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null}},"deliverable":{"id":1282,"description":"andra saker","key":"andra","name":"Other","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":{"id":1266,"description":"","key":"other","name":"Other","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":null,"user":null,"orderIndex":0,"done":false,"minEstimate":null,"maxEstimate":null,"task_id":0},"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"orderIndex":3,"done":false,"minEstimate":10.0,"maxEstimate":30.0,"task_id":0},"link":""},{"id":5324,"start":1331254080000,"end":1331254800000,"description":"","started":false,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"activity":{"id":102,"key":"d","name":"Development","description":"Usually this.","cost":690.0,"orderIndex":2,"parent":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null}},"deliverable":{"id":1274,"description":"","key":"login","name":"Login","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":{"id":1273,"description":"","key":"client","name":"Android client","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":null,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"orderIndex":2,"done":false,"minEstimate":null,"maxEstimate":null,"task_id":0},"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"orderIndex":1,"done":false,"minEstimate":15.0,"maxEstimate":30.0,"task_id":0},"link":""},{"id":7901,"start":1338678900000,"end":1338678900000,"description":"","started":false,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"activity":{"id":101,"key":"c","name":"Consulting","description":"Just in special cases, authorized by Rüdiger after discussion with customer.","cost":990.0,"orderIndex":3,"parent":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null}},"deliverable":{"id":1278,"description":"Stagtistic check","key":"statistic","name":"Statistics","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":{"id":1273,"description":"","key":"client","name":"Android client","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":null,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"orderIndex":2,"done":false,"minEstimate":null,"maxEstimate":null,"task_id":0},"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"orderIndex":5,"done":false,"minEstimate":20.0,"maxEstimate":40.0,"task_id":0},"link":""},{"id":10441,"start":1347134220000,"end":1347135120000,"description":"hello","started":false,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"activity":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"deliverable":{"id":1278,"description":"Stagtistic check","key":"statistic","name":"Statistics","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":{"id":1273,"description":"","key":"client","name":"Android client","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":null,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"orderIndex":2,"done":false,"minEstimate":null,"maxEstimate":null,"task_id":0},"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"orderIndex":5,"done":false,"minEstimate":20.0,"maxEstimate":40.0,"task_id":0},"link":""},{"id":6259,"start":1334170080000,"end":1334170140000,"description":"hi","started":false,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"activity":{"id":102,"key":"d","name":"Development","description":"Usually this.","cost":690.0,"orderIndex":2,"parent":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null}},"deliverable":{"id":1276,"description":"","key":"task","name":"Uppgifter","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":{"id":1273,"description":"","key":"client","name":"Android client","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":null,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"orderIndex":2,"done":false,"minEstimate":null,"maxEstimate":null,"task_id":0},"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"orderIndex":3,"done":false,"minEstimate":15.0,"maxEstimate":30.0,"task_id":0},"link":""},{"id":7900,"start":1338678600000,"end":1338678780000,"description":"","started":false,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"activity":{"id":102,"key":"d","name":"Development","description":"Usually this.","cost":690.0,"orderIndex":2,"parent":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null}},"deliverable":{"id":1276,"description":"","key":"task","name":"Uppgifter","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":{"id":1273,"description":"","key":"client","name":"Android client","project":{"id":115,"key":"are","name":"Master Thesis","description":"Android client for Timelog","date":1328396400000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":null,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"orderIndex":2,"done":false,"minEstimate":null,"maxEstimate":null,"task_id":0},"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"orderIndex":3,"done":false,"minEstimate":15.0,"maxEstimate":30.0,"task_id":0},"link":""},{"id":6258,"start":1334170020000,"end":1334170080000,"description":"Develop","started":false,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"project":{"id":109,"key":"mre","name":"Master Thesis Ali Rezaei","description":"Timelog Android","date":1328223600000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"activity":{"id":102,"key":"d","name":"Development","description":"Usually this.","cost":690.0,"orderIndex":2,"parent":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null}},"deliverable":{"id":1495,"description":"Develop statistic","key":"3.2","name":"statistic","project":{"id":109,"key":"mre","name":"Master Thesis Ali Rezaei","description":"Timelog Android","date":1328223600000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":{"id":1240,"description":"Developing","key":"3","name":"Android Client","project":{"id":109,"key":"mre","name":"Master Thesis Ali Rezaei","description":"Timelog Android","date":1328223600000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":null,"user":null,"orderIndex":2,"done":false,"minEstimate":0.0,"maxEstimate":0.0,"task_id":0},"user":null,"orderIndex":2,"done":false,"minEstimate":null,"maxEstimate":null,"task_id":0},"link":""},{"id":10442,"start":1347134640000,"end":1347135540000,"description":"hello","started":false,"user":{"id":28,"username":"are","hash":"$2a$10$G0e7cjXtn60XmAhUx8JpNOYk.EwpIEwDvc2mBaNsTGqQe1tgIZAxy"},"project":{"id":109,"key":"mre","name":"Master Thesis Ali Rezaei","description":"Timelog Android","date":1328223600000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"activity":{"id":108,"key":"v","name":"Vacation/Holidays","description":"Planned vacation (counting agains the 25 days)","cost":300.0,"orderIndex":5,"parent":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null}},"deliverable":{"id":1237,"description":"","key":"o","name":"Other","project":{"id":109,"key":"mre","name":"Master Thesis Ali Rezaei","description":"Timelog Android","date":1328223600000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":{"id":1236,"description":"","key":"other","name":"Other","project":{"id":109,"key":"mre","name":"Master Thesis Ali Rezaei","description":"Timelog Android","date":1328223600000,"category":{"id":100,"key":"scrum","name":"Scrum","description":"","cost":0.0,"orderIndex":1,"parent":null},"valueType":0,"fixValue":0.0,"done":0,"task_id":0},"parent":null,"user":null,"orderIndex":0,"done":false,"minEstimate":null,"maxEstimate":null,"task_id":0},"user":null,"orderIndex":2,"done":true,"minEstimate":null,"maxEstimate":null,"task_id":0},"link":""}]

It works perfectly for this user (are) and all other users except one on them.

That user has enormous number of tasks in the database, so I got this error :

Caused by: java.io.IOException: Premature EOF
    at sun.net.www.http.ChunkedInputStream.readAheadBlocking(Unknown Source)
    at sun.net.www.http.ChunkedInputStream.readAhead(Unknown Source)
    at sun.net.www.http.ChunkedInputStream.read(Unknown Source)
    at java.io.FilterInputStream.read(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
    at sun.nio.cs.StreamDecoder.readBytes(Unknown Source)
    at sun.nio.cs.StreamDecoder.implRead(Unknown Source)
    at sun.nio.cs.StreamDecoder.read(Unknown Source)
    at java.io.InputStreamReader.read(Unknown Source)
    at java.io.Reader.read(Unknown Source)
    at com.sun.jersey.core.util.ReaderWriter.readFromAsString(ReaderWriter.java:171)
    at com.sun.jersey.core.util.ReaderWriter.readFromAsString(ReaderWriter.java:157)
    at com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.readFromAsString(AbstractMessageReaderWriterProvider.java:114)
    at com.sun.jersey.core.impl.provider.entity.StringProvider.readFrom(StringProvider.java:73)
    at com.sun.jersey.core.impl.provider.entity.StringProvider.readFrom(StringProvider.java:58)
    at com.sun.jersey.api.client.ClientResponse.getEntity(ClientResponse.java:554)
    ... 2 more

I appreciate for any help.

---

Addenda

I found out that the problem is not in the server side. it is in the client side.I have an Android client which faces with out of memory problem.

The thing is how can I solve this problem in my client side. Do you have any idea?

Answer 1

The problem is with the large amount of data being returned. Try the code below for reading data from the response.

    InputStream is = response.getEntityInputStream(); 
    BufferedReader r = new BufferedReader(new InputStreamReader(is)); 
    int more = 1;
    StringBuilder responseString = new StringBuilder();
    char[] cbuf = new char[5000];
    while (more > 0 ) {
        more = r.read(cbuf, 0, 5000);
        responseString.append(cbuf);
    }
    System.out.println(responseString.toString());

Answer 2

If the number of items coming back is truly enormous one other option is to page data back to a client. If the list is enormous enough, it might not even be useful to the user and interface (not sure what your experience is). For example, look at StackOverflow - there's an enormous number of posts everyday but the interface (and likely the backend querying patterns) page the data.

The simple paging approach is the "more" pattern"

/users/{username}/tasks?top=n

If it starts off with 25, there could be a more link which pulls top 50 etc... Kind of like ios mail and other similar apps.

Another approach is "pages"

/users/{username}/tasks?page=1

The interesting question there is whether you want the set to be stable. In other words, the tasks for that user are changing so going to page 2 could skip items or show ones that were in page 1 - unless you persist state for that users 'query' - possible to store the results set (could just be IDs) in the DB, middle tier (problematic multiple ATs) or in the client (return all IDs and have the client page columns/data) - kind of like a virtual listview.

When you're working with enormous datasets, just returning all the data is often not the right answer and then it start complicating your approach with paging etc...

Hope that sparks some thoughts on different alternatives. Probably not an exact answer to your problem.