In the documentation for the Azure EventHubClient there are two methods for sending a batch of data each of them has the remark below and will throw a MessageSizeExceededException if it is ignored.
You should ensure that the total serialized size of eventDataList is less than the size limit of one event data transmission, which is 256k by default.
A similar warning is present in the Programming Guide
How can the serialized size of IEnumerable<EventData> eventDataList
be determined?
The size of the bytes passed to each EventData is easy enough to determine, assuming you don't ask the EventData. However, the serialized form of an EventData presumably includes the Partition Key and user properties as used in the sample.
data.Properties.Add("Type","Telemetry_" + DateTime.Now.ToLongTimeString());
Currently my only option looks like being conservative with batch sizing.
----Updating after the Nuget Update----
Here's the EventData property - SerializedSizeInBytes that we introduced in our recent SDK iteration - which addresses this specific problem - available in all sdk nugets above 2.6 .
---Update after EventDataBatch
utility---
use EventDataBatch.TryAdd(EventData)
API to construct the batch. When the EventData
cannot fit into the batch - this API will return false
. Stop adding more events & then use the sender (EventHubClient
or PartitionSender
) to send the EventDataBatch
.
following considerations motivated us to design EventDataBatch
API:
- Help developers using our Client API - to deal with size of batch of
EventData
- Abstract out region-wide/SKU-based
MaximumMessageSize
configuration: EventHubs
service across the world could have different Max EventData
sizes - across different EventHub
regions and different SKU
s of EventHub
. We wanted to expose a uniform way for developers to construct maximum message size
for any given environment. So, when EventDataBatch
is initialized using eventHubClient.createBatch()
API - the client negotiates batchSize
with EventHubs Service
and returns the EventDataBatch
instance which can construct the correct size!
HTH!
Sree