Within Azure, one of the easiest ways to grab data from your Azure account is to utilize the EventHub Monitor. This solution is still in beta so be mindful and test before implementing it in large-scale production systems. 

Limits

DataSet has a single event size limit of 100kb. Some Azure services have events that exceed this.  

Prereqs

Existing Azure Account with ability to configure diagnostic logs and create EventHubs 

Existing DataSet Account with Full access

Solution

This guide will show, end to end, how to get Azure Logs for a Service with Diagnostics available into DataSet. This solution uses AAD but is not limited to that service.

Step 1: Create EventHub

The first thing we will do will be to create a new Event Hub namespace and in that namespace create an Event Hub. 

1. Search for EventHubs in the Marketplace

2. Create EventHub Namespace

3. Click into the newly created namespace

4. Under Entities select Event Hubs

5. Select + Event Hub and create it 

Step 2: Configure EventHub

Now that we have our EventHub let's give DataSet permission to view these data.

1. Navigate to your EventHub namespace and under Settings select Shared Access Policies 2. Add new policy   3. Give it a name and configure the Listen permission

4. Copy Connection String Primary Key and save it. You will use this to configure DataSet.

Step 3: Configure DataSet

Now we will configure a DataSet Monitor. In the previous step you copied the Connection String Primary Key. You will configure the monitor with the data in this string. 

1. log in to an existing DataSet Account. If you dont have one. Sign up for a free trial. 

2. Select Monitors from the top right dropdown 

3. Edit JSON

4. Add this snippet to configure EventHub

You will need two pieces of information. 
   a. EventHub Name 

   b. Your Connection String Primary Key will look like this

Endpoint=sb://{namespaceHostname}/;SharedAccessKeyName={sharedAccessKeyName};SharedAccessKey={sharedAccessKey}

{
type: "eventhub",
eventHubName: "{eventHubName}",
namespaceHostname: "{namespaceHostname}.servicebus.windows.net",
sharedAccessKey: "{sharedAccessKey}",
sharedAccessKeyName: "{sharedAccessKeyName}",
timeoutSeconds: 60.0
}

Step 4: Configure Azure Service

Now we will configure our app to send its logs to your new Event Hub which is being monitored by DataSet. In this case, we will use Active Directory Logs, but any service with Diagnostic Logs will work. 

1. Open your service

2. Scroll down to Monitoring and select Diagnostic settings

3. Add Diagnostic Settings

4. Choose Datasources: Select the logs you want to send from the list of available datasets.

5. Choose Destination and set it to EventHub

6. Select the Destination we configured in Section 1 Step 1