Maximum line length for BufferedReader.readLine()

问题:

I use BufferedReader's readLine() method to read lines of text from a socket.

There is no obvious way to limit the length of the line read.

I am worried that the source of the data can (maliciously or by mistake) write a lot of data without any line feed character, and this will cause BufferedReader to allocate an unbounded amount of memory.

Is there a way to avoid that? Or do I have to implement a bounded version of readLine() myself?

回答1:

The simplest way to do this will be to implement your own bounded line reader.

Or even simpler, reuse the code from this BoundedBufferedReader class.

Actually, coding a readLine() that works the same as the standard method is not trivial. Dealing with the 3 kinds of line terminator CORRECTLY requires some pretty careful coding. It is interesting to compare the different approaches of the above link with the Sun version and Apache Harmony version of BufferedReader.

Note: I'm not entirely convinced that either the bounded version or the Apache version is 100% correct. The bounded version assumes that the underlying stream supports mark and reset, which is certainly not always true. The Apache version appears to read-ahead one character if it sees a CR as the last character in the buffer. This would break on MacOS when reading input typed by the user. The Sun version handles this by setting a flag to cause the possible LF after the CR to be skipped on the next read... operation; i.e. no spurious read-ahead.

回答2:

Another option is Apache Commons' BoundedInputStream:

InputStream bounded = new BoundedInputStream(is, MAX_BYTE_COUNT);
BufferedReader reader = new BufferedReader(new InputStreamReader(bounded));
String line = reader.readLine();

回答3:

Perhaps the easiest solution is to take a slightly different approach. Instead of attempting to prevent a DoS by limiting one particular read, limit the entire amount of raw data read. In this way you don't need to worry about using special code for every single read and loop, so long as the memory allocated is proportionate to incoming data.

You can either meter the Reader, or probably more appropriately, the undecoded Stream or equivalent.

回答4:

The limit for a String is 2 billion chars. If you want the limit to be smaller, you need to read the data yourself. You can read one char at a time from the buffered stream until the limit or a new line char is reached.

回答5:

There are a few ways round this:

• if the amount of data overall is very small, load data in from the socket into a buffer (byte array, bytebuffer, depending on what you prefer), then wrap the BufferedReader around the data in memory (via a ByteArrayInputStream etc);
• just catch the OutOfMemoryError, if it occurs; catching this error is generally not reliable, but in the specific case of catching array allocation failures, it is basically safe (but does not solve the issue of any knock-on effect that one thread allocating large amounts from the heap could have on other threads running in your application, for example);
• implement a wrapper InputStream that will only read so many bytes, then insert this between the socket and BufferedReader;
• ditch BufferedReader and split your lines via the regular expressions framework (implement a CharSequence whose chars are pulled from the stream, and then define a regular expression that limits the length of lines); in principle, a CharSequence is supposed to be random access, but for a simple "line splitting" regex, in practice you will probably find that successive chars are always requested, so that you can "cheat" in your implementation.

回答6:

In BufferedReader, instead of using String readLine(), use int read(char[] cbuf, int off, int len); you can then use boolean ready() to see if you got it all and convert in into a string using the constructor String(byte[] bytes, int offset, int length).

If you don't care about the whitespace and you just want to have a maximum number of characters per line, then the proposal Stephen suggested is really simple,

import java.io.BufferedReader;
import java.io.IOException;

public class BoundedReader extends BufferedReader {

    private final int  bufferSize;
    private       char buffer[];

    BoundedReader(final BufferedReader in, final int bufferSize) {
        super(in);
        this.bufferSize = bufferSize;
        this.buffer     = new char[bufferSize];
    }

    @Override
    public String readLine() throws IOException {
        int no;

        /* read up to bufferSize */
        if((no = this.read(buffer, 0, bufferSize)) == -1) return null;
        String input = new String(buffer, 0, no).trim();

        /* skip the rest */
        while(no >= bufferSize && ready()) {
            if((no = read(buffer, 0, bufferSize)) == -1) break;
        }

        return input;
    }

}

Edit: this is intended to read lines from a user terminal. It blocks until the next line, and returns a bufferSize-bounded String; any further input on the line is discarded.