Worker - having trouble to choose the best technol

问题:

Closed. This question is opinion-based. It is not currently accepting answers.

---

Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.

Closed 11 months ago.

Improve this question

I want to create a micro-service for parallel processing some tasks which I get via push by an Azure Service Bus. This micro-service will notify the Azure Service Bus if the task where complete with success or not. See the following diagram:

I already have this three options to consider:

1. Hosted Services
2. Azure Web Jobs
3. Azure Batch

Since one of the requisites is that this micro-service need to be in .NET Core 3, is there any valid justification to use Web Jobs instead of Hosted Services? Which of the options guarantee robustness and scalability?

回答1:

Based on my knowledge for parallel processing on Azure cloud and consider for your scenario, there are three solutions as candidate options which I think be good enough, as below.

1. Azure Web Jobs. As I known, Azure only supports Continuous WebJobs to run on multi-instances of Azure WebApps, as the figure below from the document Run Background tasks with WebJobs in Azure App Service, its scalability is up to the tier of Azure App Service and the number of app instances. This option has limited performance, is not suitable for the scenario of real parallel computing.

   

2. Azure Batch, the real large-scale parallel and high-performance computing on Azure cloud (of cause, except HDInsight). However, if there is not heavy data processing tasks for you, it's too expensive to do some tasks which just need concurrent feature within multi-threads.

3. Azure Functions with Service Bus Trigger, please refer to the offical document Azure Service Bus bindings for Azure Functions. A real serverless architecture is designed for microservice scenario as yours. Its limitation for the feature of scale out and max instances as the figure (comes from the document Azure Functions scale and hosting) below be defined in host.json file.

   

   

So if not real parallel data processing scenario, I recommended Azure Functions, otherwise Azure Batch is the best choice for parallel big data tasks. Azure WebJobs can only use Azure App Service to host your web site and run parallel task with continous jobs, but just have limited performance. Hosted services as my view, to handle concurrent tasks based on webhook requests within multi-threads, not recommended.