Java String Memory Leak

Question

I am not java expert.

My code is reading a file into a String. This code gets executed every 5 minutes. The size of file varies. Sometimes it is 100 sometimes it is 1000 lines.

I am experience Out Of Memory, after some days.

The question I have is, when my codes goes out of scope of the Reading file function, does Java garbage collect the String?

I am pretty confused by reading on the internet. Some people says it does not get deleted and use StringBuffer.

// Demonstrate FileReader.

import java.io.*;
class FileReaderDemo {
    public static void read(BufferedReader br) throws Exception {
        long length = 0;
        String s;
        while (true) {
            s = br.readLine();
            s += "abcd";
            if (s == null) {
                break;
            }
            length += s.length();
            //System.out.println(s);
        }
        System.out.println("Read: " + (length / 1024 / 1024) + " MB");
    }

    public static void main(String args[]) throws Exception {
        //FileReader fr = new FileReader("FileReaderDemo.java");
        FileReader fr = new FileReader("big_file.txt.1");
        BufferedReader br = new BufferedReader(fr);
        String s;
        read(br);
        fr = new FileReader("big_file.txt.1");
        br = new BufferedReader(fr);
        read(br);
        fr = new FileReader("big_file.txt.1");
        br = new BufferedReader(fr);
        read(br);
        fr = new FileReader("big_file.txt.1");
        br = new BufferedReader(fr);
        read(br);
        BufferedReader in = new BufferedReader(new InputStreamReader(System. in )); in .readLine();
        fr.close();
    }
}

Answer 1

Your read method will never terminate. Once you reach the end of the file, you just continue adding the string "nullabcd" to s, forever.

EDIT: forget that, s is re-assigned each time. Still, I can't see how your read method can terminate.

Answer 2

change the program like below to consume less memory. A huge source of memory consumption is due to your repeated string concatenation of s += "abcd"; - avoid that and you'll probably more than halve your memory consumption (not tested - profile it yourself if you want to know).

public static void read(BufferedReader br) throws Exception {

    long length = 0;
    //String s; <--- change to the line below
    StringBuilder sb = new StringBuilder();
    while (true) {
        String s = br.readLine();
        if (s == null) {
            break;
        }
        //s += "abcd";  <--- change to the line below
        sb.append(s).append("abcd");
        length += s.length();
        //System.out.println(s);
    }
    System.out.println("Read: " + (length / 1024 / 1024) + " MB");
}

Answer 3

The code that you have posted won't leak memory. However, the while (true) loop will never terminate because s will never be null at the point that you test it.

---

Lets change it a bit to make it "work"

    public static void read(BufferedReader br) throws Exception {
            long length = 0;
            String s = "";
            while (true) {
                    String ss = br.readLine();
                    if (ss == null) {
                            break;
                    }
                    s += ss;
                    length += ss.length();
            }
            System.out.println("Read: " + (length/1024/1024) + " MB");
    }

This code doesn't leak memory either because the Strings created in the method will all be candidates for garbage collection when the method returns (if not before).

Each time we execute s += ss; a new string is created consisting of all of the characters currently in s and the characters in ss. Assuming there are N lines containing an average of L characters, the s += ss; statement will be called N times, will create N strings, and will copy on average (N * L)^2 / 2 characters.

---

However, there is a good reason to make a StringBuilder and that is to reduce the amount of String allocation and character copying that goes on. Lets rewrite the method to use a StringBuilder; i.e. a replacement for StringBuffer that is not synchronized.

    public static void read(BufferedReader br) throws Exception {
            long length = 0;
            StringBuilder sb = new StringBuilder(sb);
            while (true) {
                    String ss = br.readLine();
                    if (ss == null) {
                            break;
                    }
                    sb.append(ss);
                    length += ss.length();
            }
            System.out.println("Read: " + (length/1024/1024) + " MB");
    }

This version will reallocate the StringBuilder's internal character array at most log2(N) times and copy at most 2 * N * L characters.

---

Summary - using the StringBuilder is a good idea, but not because of memory leaks. If you have a memory leak, it is not in the original sample code or in the fixed version.

Answer 4

Hello, I am not java expert.

Everyone has something they can learn.

My code is reading a file into a String, This code gets executed every 5 minutes. Now Sometime file size of 100 lines sometimes 1000 lines.

Doesn't sound very large or very often. Shouldn't be a problem.

I am experience Out Of Memory, after some days.

You should be able to get a heap dump and see where you have run out of memory and why.

Question I have is, When my codes goes out of scope of the Reading file Function. Does Java Garbage collect the String .

It can be collected when it is no longer reachable via a Strong reference.

I am pretty confuse by reading on internet some says it does not get deleted and use StringBuffer

Sounds like you came to the right place. I have never heard that one.

Answer 5

As pointed out by others this code never terminates. It looks like the code you posted is not the original code you are having problems with.

Hard to diagnose without seeing the actual code, but Strings will definitely be garbage collected once they are not referenced from other parts of the code.

Wild guess: Are you calling close() on your Readers and InputStreams once you're done with them? If not, this could be the cause of your out of memory errors.