I'm using App Engine (version 1.4.3) direct write the blobstore in order to save images.
when I try to store an image which is larger than 1MB I get the following Exception
com.google.apphosting.api.ApiProxy$RequestTooLargeException: The request to API call datastore_v3.Put() was too large.
I thought that the limit for each object is 2GB
Here is the Java code that stores the image
private void putInBlobStore(final String mimeType, final byte[] data) throws IOException {
final FileService fileService = FileServiceFactory.getFileService();
final AppEngineFile file = fileService.createNewBlobFile(mimeType);
final FileWriteChannel writeChannel = fileService.openWriteChannel(file, true);
writeChannel.write(ByteBuffer.wrap(data));
writeChannel.closeFinally();
}
The maximum object size is 2 GB but each API call can only handle a maximum of 1 MB. At least for reading, but I assume it may be the same for writing. So you might try to split your writing of the object into 1 MB chunks and see if that helps.
Here is how I read and write large files:
public byte[] readImageData(BlobKey blobKey, long blobSize) {
BlobstoreService blobStoreService = BlobstoreServiceFactory
.getBlobstoreService();
byte[] allTheBytes = new byte[0];
long amountLeftToRead = blobSize;
long startIndex = 0;
while (amountLeftToRead > 0) {
long amountToReadNow = Math.min(
BlobstoreService.MAX_BLOB_FETCH_SIZE - 1, amountLeftToRead);
byte[] chunkOfBytes = blobStoreService.fetchData(blobKey,
startIndex, startIndex + amountToReadNow - 1);
allTheBytes = ArrayUtils.addAll(allTheBytes, chunkOfBytes);
amountLeftToRead -= amountToReadNow;
startIndex += amountToReadNow;
}
return allTheBytes;
}
public BlobKey writeImageData(byte[] bytes) throws IOException {
FileService fileService = FileServiceFactory.getFileService();
AppEngineFile file = fileService.createNewBlobFile("image/jpeg");
boolean lock = true;
FileWriteChannel writeChannel = fileService
.openWriteChannel(file, lock);
writeChannel.write(ByteBuffer.wrap(bytes));
writeChannel.closeFinally();
return fileService.getBlobKey(file);
}
As Brummo suggested above if you split it into chunks < 1MB it works. Here's some code.
public BlobKey putInBlobStoreString(String fileName, String contentType, byte[] filebytes) throws IOException {
// Get a file service
FileService fileService = FileServiceFactory.getFileService();
AppEngineFile file = fileService.createNewBlobFile(contentType, fileName);
// Open a channel to write to it
boolean lock = true;
FileWriteChannel writeChannel = null;
writeChannel = fileService.openWriteChannel(file, lock);
// lets buffer the bitch
BufferedInputStream in = new BufferedInputStream(new ByteArrayInputStream(filebytes));
byte[] buffer = new byte[524288]; // 0.5 MB buffers
int read;
while( (read = in.read(buffer)) > 0 ){ //-1 means EndOfStream
ByteBuffer bb = ByteBuffer.wrap(buffer);
writeChannel.write(bb);
}
writeChannel.closeFinally();
return fileService.getBlobKey(file);
}