OutOfMemoryException while trying to send large js

Question

I am trying to send multiple Json string[each 1 MB] at a time to PHP Server. When i tried to send 10 records its working fine. But if it exceeds 20 records it says OutOfMemoryException. I seen that android memory size limit t0 15MB - 16MB. But haven't got any clue how to resolve this, I am using the below code to send multiple records at a time.

 /** Upload Data*/
private void submitUploadData(String url, Map<String, String> param)
        throws IOException {
    URL siteUrl;
    try {
        siteUrl = new URL(url);
        HttpURLConnection conn = (HttpURLConnection) siteUrl
                .openConnection();
        conn.setRequestMethod("POST");
        conn.setDoOutput(true);
        conn.setDoInput(true);

        DataOutputStream out = new DataOutputStream(conn.getOutputStream());
        String content1 = ""; 

        Set getkey = param.keySet();
        Iterator keyIter = getkey.iterator();
        String content = "";
        for (int i = 0; keyIter.hasNext(); i++) {
            Object key = keyIter.next();
            if (i != 0) {
                content += "&"; //Crashing here
            }
            content += key + "=" + param.get(key); //Crashing here
            System.out.println("Content" + content);
        }

        System.out.println(content);
        out.writeBytes(content.trim());
        out.flush();
        out.close();

        BufferedReader in = new BufferedReader(new InputStreamReader(
                conn.getInputStream()));
        String jsonresponse = "";
        while ((jsonresponse = in.readLine()) != null) {
            System.out.println(jsonresponse);   
            if(!jsonresponse.equals("Authorisation Successful")){
            updateUploadRecords(jsonresponse);}

        }
        in.close();  

    } catch (MalformedURLException e) {
        e.printStackTrace();
    } 
}

Answer 1

Through personal experience I've found that beyond a certain size (which is dependent on the device), it is not feasible to use String concatenation to form / download entities.

In your case using a JsonWriter will help. It writes json tokens to a stream and will not occupy additional memory. When you download bulky json documents use a JsonReader.

I've used both these classes with success.

Answer 2

Maybe it is limited by the memory size of your virtual device!?

Have you tried it by giving it a bigger memory!?

Answer 3

at first glance there's nothing wrong with your code but you're using every possible type of method that is known for being a memory waster and now that you're trying large operations you running out of memory, that was guaranteed to happen.

tips:

• do not contatenate strings with str1 += str2 + "other val" you just created 3 new strings in memory. Use a StringBuilder instead.
• Calling System.gc() (as some will suggest you) will not help you, because you're in a Loop doing this operation and that method only hints the system that it might be a good time to GC, but until the system actually get around to do it, you'll have crashed already.
• If each JSON string is around 1mb and you're passing 20 of them, you're already crashing with out of memory on older devices right there with those 20mb. You'll have to re-design quite a few stuff, but my suggestion is to just remove the Map, completely remove it.

Instead you should (on the method that is creating that map) to use some type of OutputStreamto write all that data, already formatted in a temporary file. Then, to upload you'll use some type of InputStream to read from this file and write to your DataOutputStream

Alternative to the Streams is to use some other classes that bases on the same methodology as streams, they are: JsonWriter/JsonReader, but then you also have FileWriter/FileReader if it's not specific Json what you're passing, and don't forget to always wrap those with BufferedReader and BufferedWriter.

Answer 4

You are iterating through it with object. You have the risk that they keep copies etc... Go back to basics and try to use the function String.split(''); and then iterating through it with a normal loop. Then you know which objects are in memory. When doing the split you get two memory hogging objects, the original string and the array resulting from the split.

At the moment you have split the original string you can set the original string to null to free up the memory for the garbage collector(maybe even call System.gc())

Then just simply loop through the string doing manipulation by manipulation, keeping the footprint as small as possible.