How can the serialized size of EventData be determined for batching?
azure, azure-eventhub, azureservicebus, c#
Solution
----Updating after the Nuget Update---- Here's the EventData property - SerializedSizeInBytes that we introduced in our recent SDK iteration - which addresses this specific problem - available in all sdk nugets above 2.6 .
---Update after `EventDataBatch` utility---
use `EventDataBatch.TryAdd(EventData)` API to construct the batch. When the `EventData` cannot fit into the batch - this API will return `false`. Stop adding more events & then use the sender (`EventHubClient` or `PartitionSender`) to send the `EventDataBatch`.
following considerations motivated us to design `EventDataBatch` API:
- Help developers using our Client API - to deal with size of batch of `EventData`
- Abstract out region-wide/SKU-based `MaximumMessageSize` configuration: `EventHubs` service across the world could have different `Max EventData` sizes - across different `EventHub` regions and different `SKU`s of `EventHub`. We wanted to expose a uniform way for developers to construct `maximum message size` for any given environment. So, when `EventDataBatch` is initialized using `eventHubClient.createBatch()` API - the client negotiates `batchSize` with `EventHubs Service` and returns the `EventDataBatch` instance which can construct the correct size!
HTH! Sree
Problem
In the documentation for the Azure EventHubClient there are two methods for sending a batch of data each of them has the remark below and will throw a MessageSizeExceededException if it is ignored. You should ensure that the total serialized size of eventDataList is less than the size limit of one event data transmission, which is 256k by default. A similar warning is present in the Programming Guide How can the serialized size of `IEnumerable<EventData> eventDataList` be determined? The size of the bytes passed to each EventData is easy enough to determine, assuming you don't ask the EventData. However, the serialized form of an EventData presumably includes the Partition Key and user properties as used in the sample. ``` data.Properties.Add("Type","Telemetry_" + DateTime.Now.ToLongTimeString()); ``` Currently my only option looks like being conservative with batch sizing.