Nexus 10 camera.release hangs for 30 seconds

We are a group of developers that are working on a realtime video processing app for android. Recently, a client reported a freeze in our app that happens only on Nexus 10 device. We bought the device ourselves and tested:

device came with Android 4.2 preinstalled - there were no hangs nor freezes in our app
after upgrading Android to 4.3, our app freezed on closing camera (see below for explanation)
by upgrading to 4.4, our app kept freezing
by flashing the latest Android 5.0 Nexus 10 factory image, our app still kept freezing
(edit) - the problem has been solved by upgrading Nexus 10 to Android 5.1

More information about the freeze:

Our app opens the camera in onResume method of activity, installs the preview callback, sets the preview size to best possible for our processing needs (on nexus 10 this is 1920x1080) and either starts the preview (if resuming from sleep) or delegates the same to surfaceCreated method of SurfaceView callback. In onPause method our app removes the preview callback, stops the camera preview and releases the camera. However, our investigation has shown that camera.release method sometimes takes 30 seconds to complete. In that 30 seconds our app was frozen because we used to control the camera from UI thread. Later we moved the camera control to separate event handler thread and now camera.release hangs that thread. Although this is invisible now to the user (UI is not blocked), the user cannot use the camera from any app until our background thread succeeds in releasing the camera (i.e. 30 seconds after camera.release was called).

During the hang period we observed the following log outputs from camera service:

10-21 16:08:54.193: E/Camera2-Device(122): waitUntilDrained: Waited 10050000 us, 2 requests still in flight
10-21 16:08:54.193: E/Camera2Client(122): stopPreviewL: Camera 0: Waiting to stop streaming failed: Connection timed out (-110)
10-21 16:09:04.293: E/Camera2-Device(122): waitUntilDrained: Waited 10050000 us, 2 requests still in flight
10-21 16:09:04.293: E/Camera2Client(122): stopPreviewL: Camera 0: Waiting to stop streaming failed: Connection timed out (-110)
10-21 16:09:14.453: E/Camera2-Device(122): waitUntilDrained: Waited 10050000 us, 2 requests still in flight
10-21 16:09:14.453: E/Camera2-StreamingProcessor(122): deletePreviewStream: Error waiting for preview to drain: Connection timed out (-110)
10-21 16:09:24.573: E/Camera2-Device(122): waitUntilDrained: Waited 10050000 us, 2 requests still in flight
10-21 16:09:24.573: E/Camera2-CallbackProcessor(122): deleteStream: Error waiting for HAL to drain: Connection timed out (-110)
10-21 16:09:24.578: E/libexynosv4l2(122): failed to ioctl: VIDIOC_REQBUFS (-1 - Invalid argument)
10-21 16:09:24.578: E/ExynosCameraHAL2(122): cam_int_reqbufs: VIDIOC_REQBUFS (fd:35) failed (-1)
10-21 16:09:24.843: E/Camera2-CallbackProcessor(122): deleteStream: Camera 0: Device does not exist
10-21 16:09:24.853: E/Camera2-StreamingProcessor(122): deletePreviewStream: Camera 0: Device does not exist

You can see the minimum example that triggers this behaviour here - camera activity needs to be restarted quickly few times to increase the chance of entering into this state. We noticed that it occurs much more often in our app than in example app. Our app does some heavy frame processing, also uses GPU for image processing and besides camera also uses accelerometer and orientation sensor - all those are not included in example app.

We have also seen a couple unanswered StackOverflow questions about the same problem (question 1 and question 2). Could you please explain what codepath will lead to mentioned log outputs and how to avoid entering that state?

We did not encounter the mentioned camera freeze on any other device so far.

Unfortunately, you're running into a non-deterministic bug in the Nexus 10 camera HAL.

While we've tried to track down this issue repeatedly, we obviously haven't found all instances of problems here.

In terms of workarounds, you could try simply closing the camera device, without removing the callback and stopping preview; it shouldn't be necessary to do the shutdown in stages.