Downloading large file over HttpConnection

Question

Using the following code i can load any file from url as a bytes but this way works fine for file not larger than 256kb .

So I want another way to load larger files as bytes without using BlackBerry Combiner

my code :

HttpConnection hpc = null;
    try {
        hpc = (HttpConnection) Connector.open(url
                + ConnectionManager.getTimeOut(5000)
                + ConnectionManager.updateConnectionSuffix());
        hpc.setRequestMethod(HttpConnection.GET);
        hpc.setRequestProperty("Connection", "Keep-Alive");
        if (hpc.getResponseCode() != 200) {
            return null;
        }
        byte[] data = IOUtilities.streamToBytes(hpc.openInputStream());
        hpc.close();
        return data;
    } catch (Exception e) {
        return null;
    }

Answer 1

The limitation is imposed by the MDS, which is a proxy between the BlackBerry device and the internet. Both BIS and BES servers are MDS. So, without removing the limitation, you'll have to split your downloads to fit each one in your biggest data size. The HTTP protocol already supports this functionality via the Range request header, as it is shown in the DownloadCombiner example, no need to craft your own mechanism. This is the only way to guarantee the download of files of arbitrary size.

That said, depending on your case, there are some ways to remove/bypass the limitation:

• If your device is inside a BES, you can talk to the BES admin to increase the download data size per connection as much as possible. The max available value in BES configuration depends on the BES version, so even at maximum it might not be enough to download files exceeding that limit. (But if your files are smaller than the limit, then this is the fastest way to download files without modifying code).

• If you are testing in a simulator in your local machine, look for this file:

  <JDE path>\MDS\config\rimpublic.property<br><br>
  

  Locate the line IPPP.connection.MaxNumberOfKBytesToSend and increase it to fit your needs. This is not good practice because your app will run fine in simulators but will fail on real devices. It is always desirable to keep your simulator behavior as closer to real devices as possible.

• If you are over BIS then the limits are fixed, and will vary depending on the MIME type returned in the HTTP response, as it is explained here. Multimedia MIME types have the largest size limit at 122,880 KB. So if you need to download, lets say, a pdf file over 2048 KB in size from a server under your control, you could go hackish, rename the file to a multimedia extension, change the MIME type returned in the response to a multimedia one, and then once downloaded in your device, rename back again to its original extension. If the server hosting the file is not under your control, you could set up a proxy server in between to download it and serve the spoofed file to the device. This is too much a work as you can see, it is way faster to use the DownloadCombiner. You still have to solve the problem of knowing the original file real extension. And finally you are stuck with 122,880 KB so it doesn't work for files exceeding that size.

Answer 2

i tried to adjust BlackBerry Combiner to use it on download large File as Bytes . and it works fine for me the code listed below

--- calling

byte[] data = downloadLargeFiles(url);
                if (data != null) {
                    invoke(data.length + " ");
                    Bitmap bitmap = Bitmap.createBitmapFromBytes(data, 0,
                            data.length, 1);
                    manager.add(new BitmapField(bitmap));
                }

--- the Function

public byte[] downloadLargeFiles(String url) throws Exception {
    int chunkIndex = 0;
    int totalSize = 0;

    String currentFile = url + ConnectionManager.getTimeOut(5000)
            + ConnectionManager.updateConnectionSuffix();
    HttpConnection conn;
    InputStream in;
    int rangeStart = 0;
    int rangeEnd = 0;
    int chunksize = 100000;
    ByteArrayOutputStream bos = new ByteArrayOutputStream();

    while (true) {
        conn = (HttpConnection) Connector.open(currentFile,
                Connector.READ_WRITE, true);
        rangeStart = chunkIndex * chunksize;
        rangeEnd = rangeStart + chunksize - 1;
        conn.setRequestProperty("Range", "bytes=" + rangeStart + "-"
                + rangeEnd);
        int responseCode = conn.getResponseCode();
        if (responseCode != 200 && responseCode != 206) {
            // Dialog.alert("End "+responseCode);
            break;
        }
        in = conn.openInputStream();
        int length = -1;
        byte[] readBlock = new byte[256];
        int fileSize = 0;
        while ((length = in.read(readBlock)) != -1) {
            bos.write(readBlock, 0, length);
            fileSize += length;
            Thread.yield(); // Try not to get cut off
        }
        totalSize += fileSize;
        chunkIndex++; // index (range) increase
        in.close();
        conn.close();
        in = null;
        conn = null;

        Thread.sleep(1000);
    }
    bos.close();

    return bos.toByteArray();
}

Thanks