From last question about Azure messaging service by publishing one common message to multiply subscribers by using service bus topic. But we've encounter an issue that our message size is over-sized of the service buss limitation(256KB), so we changed our design by splitting the GIANT message into small piece of sub-message and send them to different subscribers, here is what I'd like to achieve:
Let's say we are saving an order object in azure function called SaveOrder and after order was saved successfully, we would like to send different messages(content) to 3 subscribers/consumers which is also implemented/triggered with azure functions,
subscriber/azure function 1 would be using for reporting - with message 1, e.g
{
'reportHeader':'Order Report for ID 012913',
'report_notes':'A few additional report notes...'
}
subscriber/azure function 2 would be using for logging(Sql Server) - with message 2, e.g
{
'logAction':'Order Saved Successfully',
'logTime':'08/12/2019 12:38:12 AM',
'logDescription': 'Order Id - 0139281',
'log_notes':'A few additional log notes...'
}
subscriber/azure function 3 would be using for continuing business logic. - with message 3, e.g
{
'action':'Prepare product accumulation',
'ProductList': {
'813891':3
'581231':1
},
'accumulation_notes':'A few additional accumulation notes...'
}
All of above azure functions subscriber will be working independently/isolated, and our azure function SaveOrder doesn't have to wait for any of these 3 functions, it should exit or terminate once the message publishes.
In this case, which of the messaging service should I choose in Azure for handling that? Would service bus still work here? As my understanding is that Service bus topic would send/publish "same/one" message content to multiply subscribers.
If
message1
always only goes tofunction1
,message2
only tofunction2
, and so on, creating 3 Service Bus Queues would suffice. This simply transports the message from a sender to a receiver (function) (FIFO-style).From
SaveOrder
, you'd sendmessage1
toqueue1
,message2
toqueue2
etc.function1
receives messages fromqueue1
.function2
receives messages fromqueue2
.function3
receives messages fromqueue3
.Using topics and subscriptions will also work, by creating 3 topics with 1 subscription each:
From
SaveOrder
, you'd sendmessage1
totopic1
,message2
totopic2
etc.function1
receives messages fromsubscription1
.function2
receives messages fromsubscription2
.function3
receives messages fromsubscription3
.The second scenario is more flexible, as it allows multiple receivers of the same (copy) message, by adding more subscriptions to the same topic.
Also, you can gzip the payloads of your messages to make transport more efficient.
You can have a separate topic or queue for each of multiple outbound streams. However you should probably publish a single message containing all necessary information to a topic and have three separate consumers do different things with the message.
For instance your function shouldn't be coded to generate an "email notification message" that's not its concern. It should publish an OrderSaved message, and if a service wants to subscribe to that and send emails, it can.