.NET Scalable Pub/Sub service implementation

问题:

I need to build a system that is similar to a pub/sub system. It is composed of multiple sub-systems or services running in separate executables or as a Windows Services.

The sub-systems are:

1. The pub/sub service

   • A pub/sub service managing communications between the internal sub-systems and the users.
   • A user can have multiple channels open (A web page connected to a SignalR service, a mobile device connected to a duplex WCF service, etc.).
   • The service should manage all the channels of an user and be able to send information to them on demand based on topics or specific users.
   • The service must support multiple transports like SignalR, WCF, or others ...

2. Worker services

   • A worker that runs as a Windows Service and sends information to the users using the pub/sub service.

3. The SignalR and WCF host

   • The SignalR service and WCF service will be hosted on IIS

My questions are

• As the sub-systems run in separate processes, how do I communicate between the pub/sub service and the other sub-systems like (the workers and IIS). The communication must be really fast. Do I use named-pipes, is it fast enough ?

An example; The worker tells the pub/sub system to send a message to a user, the pub/sub systems checks the channels opened for the user (let's say a SignalR channel), then in turn it must notify the SignalR service running in IIS to send the message to the user's browser.

• Do you know of implementations of similar systems ?

Observations

• I cannot use third-party service-bus services (Azure ..). And even with that .. I can't see a solutions to the problems above.
• The service must be very scalable and high-demand proof.

回答1:

If the question is how to bridge SignalR with other transports there are several solutions.

On a single server you could just connect them up with the Reactive framework's own pubsub mechanism which is neatly encapsulated in the Subject class.

If you need to scale out to multiple servers you'll want to use an existing service bus or perhaps roll your own simple one using SQL server and a SqlDependency.

You could also use SignalR as a client on one server communicating with the other servers to copy messages between them.

I recommend you look into some of the existing Service Bus technologies for .NET.

回答2:

There is an article which clearly explains the possible mechanism of how to incorporate a pub/sub design pattern in your .NET application. The answer lies in using a .NET In-Memory distributed cache and use its clustering capabilities as a publish subscribe medium. Since it's clustered therefore you won't have to worry about down-times as well.

Basically you'll be using the Application Initiated Custom Events

Register your events

public void OnApplicationEvent(object notifId, object data)
{
  ...
} 
_cache.CustomEvent += new CustomEventCallback(this.OnApplicationEvent);

And fire those events whenever you need to

_cache.RaiseCustomEvent("NotificationID", DateTime.Now);

Pub/Sub design pattern in a .NET distributed cache

Full disclosure: I work for Alachisoft