How would I go about creating a modern, Gmail-like, multiple file upload in GWT and AppEngine Blobstore?
The solution most commonly proposed is gwtupload, an excellent GWT component written by Manolo Carrasco. However, the latest version 0.6.6 does not work with blobstore (at least I can't get it to work), and it does not support multiple file select. There's a patch for multiple file select in the latest 0.6.7 snapshot, but although it allows selection of multiple files (using the "multiple" attribute in HTML5), it stills sends them in one huge POST request (and progress is shown for the whole bunch of files).
There are also other questions on SO for this (for example here or here), but the answers usually uses the HTML5 "multiple" attribute and sends them as one big POST request. It works, but its not what I am after.
Nick Johnson wrote some great blog posts about this. He uses the common and well-accepted JavaScript upload component called Plupload, and uploads files to an AppEngine-app written in Python. Plupload supports different backends (runtimes) for supporting multiple file selection (HTML5, flash, Silverlight, etc) and handles upload progress and other upload related client-side events.
The problem with his solution is (1) it's in Python, and (2) it's in JavaScript. This is where gwt-plupload enters the picture. It is a JSNI-wrapper for Plupload written by Samuli Järvelä, which enables use of Plupload inside the GWT environment. However, the project is outdated (no commits since 2010), but we can use it for inspiration.
So, step-by-step instructions for building the multiple file upload component follows. This will all be in one project, but it (especially the JSNI-wrapper) could be extracted to its own .jar-file or library to be reused in other projects. The source code is available on Bitbucket here.
The application is available on AppEngine (non-billable, so don't count on it being available or working) at http://gwt-gaemultiupload-example.appspot.com/.
Screenshots
Step 1 - Servlets
Blobstore works in the following way:
To support this, we will need two servlets. One for generating URLs for file uploads (note that each file upload will need an unique URL), and one for receiving finished uploads. Both will be quite simple. Below is the URL generator servlet, which will just write the URL in plain text to the HTTP response.
And then, the servlet for receiving successful uploads, which will print the blobkey to
System.out
:We also need to register these in
web.xml
.If we run the app now and visit
http://127.0.0.1:8888/generateblobstoreurl
, we will see something likeIf we were to post a file to that URL, it would be saved in blobstore. Note however that the default URL for the local development web server is
http://127.0.0.1:8888/
while the URL generated by blobstore ishttp://<computername>:8888/
. This will cause problems later on, as for security reasons Plupload won't be able to POST files to another domain. This only happens with the local development server, the published app will have only one URL. Fix it by editing the Run Configurations in Eclipse, add-bindAddress <computername>
to the arguments. This will cause the local development server to host the web app onhttp://<computername>:8888/
instead. You might need to allow<computername>
in the GWT browser plugin for it to load the app after this change.So far so good, we have the servlets we need.
Step 2 - Plupload
Download Plupload (I used the latest version, 1.5.4), unzip, and copy the
js
folder to thewar
directory in our GWT application. For this example, we won't be usingjquery.plupload.queue
orjquery.ui.plupload
as we'll create our own GUI. We also need jQuery, which I downloaded from Google APIs.Next, we need to include the JavaScripts in our application, so edit
index.html
and add the following to the<head>
tag.So now we have Plupload included in our application. Next, we need to wrap it to be able to use it with GWT. This is where gwt-plupload is used. I didn't use the jar file from the project, but instead copied the source files to be able to make modifications to them. The wrapper's main object is the
Plupload
class, which is constructed byPluploadBuilder
. There's also the interfacePluploadListener
, which can be implemented to receive client-side events.Step 3 - Putting it together
So now we need to actually use Plupload in our GWT application. I added the following to an
Index.ui.xml
UIBinder:There's a button for browsing files, a button to start uploading and a CellTable which we will use to display upload status. In
Index.java
, we initialize Plupload as follows:The
runtime
attribute tells Plupload which backends to use (I have only tested HTML5, but the others should work as well). Blobstore requiresmultipart
to be enabled. We also need to set an ID for the browse button, and then tell Plupload to use that ID. Clicking this button will popup Plupload's file selection dialog. Last, we add ourselves as listener (implementingPluploadListener
) andcreate()
andinit()
Plupload.To display the files ready to upload, we just need to add data to the
tblFilesDataProvider
list data provider in the events fromUploadListener
.To display progress, we simply update the list whenever we are notified progress have changed:
We also implement a click handler for
btnStart
, which justs tells Plupload to start uploading.It is now possible to select files, they will be added to the pending uploads list and we can start the upload. The only piece left is to actually use the servlets we implemented earlier. Currently, Plupload does not know which URL to POST uploads to, so we need to tell it. This is where I have made a change to the gwt-plupload source code (apart from minor bug fixes); I added a function to Plupload called
fetchNewUploadUrl
. What it does is it performs an Ajax GET request at the servlet we definied earlier to fetch an upload URL. It does this synchronously (why will be clear later). When the requests returns, it sets this URL as the POST URL for Plupload.Plupload will post each file in its own POST request. This means we need to give it a new URL before each upload starts. Luckily, there's an event for that in
PluploadListener
which we can implement. And here's the reason why the request has to be synchronous: otherwise the upload would have started before we received the upload URL in the event handler below (pl.fetchNewUploadUrl()
would have returned immediately).And that's it! You now have GWT HTML5 multiple file upload functionality placing files in AppEngine Blobstore!
Passing parameters
If you want to additional parameters (such as an ID for the entity to which the uploaded files belong), I added an example on how to add one. There's a method on
Plupload
calledsetExtraValue()
I implemented as:Extra values can be passed as
multipart_params
. This is a map, so the functionality could be extended to allow many arbitrary key-value pairs. The value can be set in theonBeforeUpload()
event handlerand retrieved in the servlet receiving finished uploads as
The example project contains sample code for this.
Final words
I am no way an expert GWT developer. This is what I came up with after hours of frustration not finding the functionality I was after. After I got it working, I thought I should write up a complete example, as every component/blog post/etc I used/followed had left some part out. I do not imply this to be best practice code in any way. Comments, improvements and suggestions are welcome!