Handle blob events from storage accounts in multip

Question

Is it possible to get notified about blobCreated events happening in multiple storage accounts who live in multiple Azure Subscriptions?

I would like to handle blob created events happening in arbitrary storage accounts in a central Azure Function which lives in my subscription but i would like to give customers the possibility to store the data in their own subscription.

I was thinking about using Event Grid Webhook endpoints to route the events to my central Azure Function. Would this be a solid approach to enable multi-subscription scenarios?

Edit: To be more precise, i need this to work over different tenants (as our customers would bring their own subscriptions and we need to integrate them without assigning them to our AD tenant)

Answer 1

Based on our discussion, the following screen snippets show your multi-tenant-fan-in-scenarios.

Subscribing to the distributed interest source across the azure subscriptions (multi-tenants) is done mapping the topic to the webhook endpoint. Note, that the topic represents a full resource path (id) of the place where the event is posting (publishing) to the AEG service. This path is in the scope of the current tenant, see the following example:

"topic": "/subscriptions/myID/resourceGroups/myRG/providers/microsoft.storage/storageaccounts/mySA"

"endpointBaseUrl": "https://myFnc.azurewebsites.net/runtime/webhooks/EventGrid?functionName=myEventGridTrigger&code=xxxx"

This mapping is declared in the subscription metadata stored in the same scope as a topic. On the other side, the webhook endpoint can be posted outside of this scope.

Other more complex solution and the full isolation from the tenats with an event distribution using an FAN-OUT Pub/Sub manner is shown in the following screen snippet:

In the above solution, the fan-in subscriber can mediate an original event message to the properly business oriented event message included a short sasToken for accessing a blob metadata and/or body, etc.

To create an event subscription in your tenant with an event handler for your EventGridTrigger function, you can use for instance the REST API call, see the following example:

   PUT https://management.azure.com/subscriptions/myId/resourceGroups/myRG/providers/Microsoft.Storage/storageaccounts/mySA/providers/Microsoft.EventGrid/eventSubscriptions/mySubscription?api-version=2019-01-01

Headers:

  Authorization:Bearer eyJ0eXAiOiJKV1QiLCJhb....

Body (minimum payload):

{
  "properties": {
    "destination": {
      "endpointType": "WebHook",
      "properties": {
        "endpointUrl": "https://myFnc.azurewebsites.net/runtime/webhooks/EventGrid?functionName=myEventGridTrigger&code=xxxxxxxx..."
      }
    }
  }
}

UPDATE:

Another way using the Azure Event Grid Pub/Sub model in the isolated multi-tenants distributed eventing architecture is its cascading. The logical event pipeline can be constructed via cascading of the Azure Event Grids such as subscribing an Azure Event Grid to the another one using a custom topic.

The following screen snippet shows an example of the Azure Event Grid cascading:

The cascading concept which is based on the Fan-In to Fan-Out pattern is enabled by subscribing a custom topic endpoint to the WebHook event handler of the another event grid model in the standard Pub/Sub manner.

Note, that the Azure Event Grid doesn't have a built-in endpoint for cascading each other including a validation event loopback. However, the following steps can allow to cascade an Azure Event Grid each other.

1. Create a custom topic endpoint with a CustomInputSchema for example:

   {
      "properties": {
         "inputSchema": "CustomEventSchema",
         "inputSchemaMapping": {
         "properties": {
           "id": {
             "sourceField": null
           },
           "topic": {
             "sourceField": null
           },
           "eventTime": {
              "sourceField": null
           },
           "eventType": {
              "sourceField": "myEventType",
              "defaultValue": "recordInserted"
           },
           "subject": {
              "sourceField": "subject",
              "defaultValue": "/myapp/vehicles/motorcycles"
           },
           "dataVersion": {
             "sourceField": null,
             "defaultValue": "1.0"
           }
       },
       "inputSchemaMappingType": "Json"
       }
     }
   }
   

   Note, that the topic property must have a "sourceField": null, which is OK for a custom topic (not for event domains).

2. For webhook event handler endpoint use the aeg-sas-key in the url query string, for example:

   https://myTopic.westus-1.eventgrid.azure.net/api/events?aeg-sas-key=xxxxxxxxxx

   Note, that the aeg-sas-key value must be url encoded string.

3. For subscription validation is used a validationUrl handshake in the fire&forget manner. It can be implemented in the EventGridTrigger function and subscribing to the custom topic for cascading purpose. The following code snippet shows an example of this implementation:

   #r "Newtonsoft.Json"
   
   using System;
   using System.Threading.Tasks;
   using System.Text;
   using System.Linq;
   using System.Net;
   using System.Net.Http;
   using System.Web;
   using Newtonsoft.Json;
   using Newtonsoft.Json.Linq;
   
   public static async Task Run(JObject eventGridEvent, ILogger log)
   {
      log.LogInformation(eventGridEvent.ToString());
   
      string eventType = $"{eventGridEvent["data"]?["eventType"]?.Value<string>()}";
      if(!string.IsNullOrEmpty(eventType) && eventType == "Microsoft.EventGrid.SubscriptionValidationEvent")
      {
         // manual validation
         string validationUrl = $"{eventGridEvent["data"]?["data"]?["validationUrl"]?.Value<string>()}";
         using (var client = new HttpClient())
         {
           var response = await client.GetAsync(validationUrl);
           log.LogInformation(response.ToString());
         }
      }
      else
      {
        // notifications
      }
   
      await Task.CompletedTask;
   }
   

   Note, that the original event message (original source interest) is cascaded (nested) in the event data object each time when is published