Split one large file into multiple InputStream(s)

2019-08-24 12:55发布

My codes:

public ArrayList<InputStream> getAllInputStreams() {
    ArrayList<InputStream> allStreams = new ArrayList<InputStream>();
    InputStream stream = this.getNext();
    while (stream != null) {
        allStreams.add(stream);
        stream = this.getNext();
    }
    return allStreams;
}

public InputStream getNext() {
    if (done()) {
        return null;
    }
    InputStream segment = createInputStream();
    this.countStream++;
    return segment;
}

protected InputStream createInputStream() {
    BoundedInputStream res = new BoundedInputStream(
            Channels.newInputStream(this.randomAccessFile.getChannel().position(this.countStream * chunkSize)), chunkSize);
    res.setPropagateClose(false) ;
    return res ;
}

I am trying to split file into several InputStream(s) (private RandomAccessFile randomAccessFile = new RandomAccessFile(file, "r");. All InputStream(s) (got from getAllInputStreams()) to be processed by multiple threads, it seems that most of them are empty. Why?

Any hints welcomed. Thanks

UPDATE

It seems that the following piece of codes working fine. Is the following piece of codes a good way to split the file into several chucks? Should the size of each chuck smaller than memory size?

protected InputStream createInputStream() {
    RandomAccessFile randomAccessFile = new RandomAccessFile(file, "r");
    BoundedInputStream res = new BoundedInputStream(
            Channels.newInputStream(randomAccessFile.getChannel().position(this.countStream * chunkSize)), chunkSize);
    res.setPropagateClose(false) ;
    return res ;
}

1条回答
爷的心禁止访问
2楼-- · 2019-08-24 13:59

I think that this statement:

      res.setPropagateClose(false);

in your "UPDATE" solution will result in a resource leak. The RAF that you have opened is not shared with anything else. When the BoundedInputStream is closed, you need the close to propagate to the RAF.

查看更多
登录 后发表回答