Although undocumented, conventional wisdom using the Android BLE apis is that certain operations like reading / writing Characteristics & Descriptors should be done one at a time (although some devices are more lenient than others). However, I am not clear on whether this policy should apply only to a single connection, or across all active connections.
I've heard that its best to initiate connections to devices one at a time. That might be an example of operations (connect / connectGatt) which should be executed serially among all devices.
But for other operations, like reading and writing Characteristics, is it good enough if each connection executes operations serially, or do I need some global operation queue shared among all devices so that between all devices, only one operation is executing?
On Android, per BluetoothGatt object you should only execute one operation at a time (request mtu, discover services, read/write characteristic/descriptor) otherwise things will go wrong. You have to wait until the corresponding callback gets called until you can execute the next operation.
Regarding having pending connections to multiple devices at the same time, if you use autoConnect=true then there is no problem but if you use autoConnect=false then Android's bluetooth stack will only attempt to connect to one device at a time, meaning it will enqueue the connection requests if there are more than one outstanding. There is one particular bug where it fails to cancel a pending connection that is still in the queue (when you call .disconnect() or .close()) however, that was recently fixed in Android.
Note that there is also a maximum number of connections/pending connections/gatt objects for which the behaviour is completely undocumented what happens when you exceed these limits. In the best cases you simply get a callback with error status but in some cases I've seen that the android bluetooth stack gets stuck in an endless loop where it in each iteration tells the bluetooth controller to connect to a device but the controller sends back the error code maximum connections reached.
While I cannot speak for the upper layer, I can relate to what will happen on lower hardware level and that might provide some insights for your design.
Which ever is the stack on upper layer doing, at the end the operation has to be handled by the transceiver chip.
BLE is operating over 40 channel band in which 3 are used for broadcast and other for data transmission. This is done to be able to have multitude of device communicating together limiting the collision by being on other frequency bands.
Those bands are selected based on the one with the lowest noise (or traffic).
The transceiver himself is only able to communicate (speak and listen) in one band at a time and has to switch between bands to reach other devices. This is done by very tight timing of the communication.
Another fact is that a wireless transceiver is basically some sort of half duplex communication with collision detection, it cannot send and listen at the same time, nor can two device emit at the same time on the same band. It is therefore by design (and laws of nature) serial or sequential.
If you implement some sort of operational queue or threaded implementation, at the end everything will have to be treated serially / sequentially by the transceiver.
If you access to it by different threads, the transceiver may have to jump all the time between channels or perhaps gets confused if it is not well handled on the upper level.
The only good reason I may see to treat that on thread would be that the processing time of the transceiver to be significantly lower than the upper stack you have to run, and you would take advantage of multi core processor.
But otherwise unless very specific software need or architecture, I do not believe you will have significant gain of having a different implementation than serial and I would also speak to the slaves one by one rather than all at the same time for the considerations explained above.
BLE is designed to be asynchronous and event driven. You can send the commands however you like and you will get the responses back in no particular order. If you send a command and expect the next packet to be the response, you're going to get into trouble.
This being said, I'm not sure how the Android library is structured around this.