Within Azure, one of the easiest ways to grab data from your Azure account is to utilize the EventHub Monitor. This solution is still in beta so be mindful and test before implementing it in large-scale production systems.
Limits
DataSet has a single event size limit of 100kb. Some Azure services have events that exceed this.
Prereqs
Existing Azure Account with ability to configure diagnostic logs and create EventHubs
Existing DataSet Account with Full access
Solution
This guide will show, end to end, how to get Azure Logs for a Service with Diagnostics available into DataSet. This solution uses AAD but is not limited to that service.
Step 1: Create EventHub
The first thing we will do will be to create a new Event Hub namespace and in that namespace create an Event Hub.
1. Search for EventHubs in the Marketplace
2. Create EventHub Namespace
3. Click into the newly created namespace
4. Under Entities select Event Hubs
5. Select + Event Hub and create it
Step 2: Configure EventHub
Now that we have our EventHub let's give DataSet permission to view these data.
1. Navigate to your EventHub namespace and under Settings select Shared Access Policies 2. Add new policy
3. Give it a name and configure the Listen permission
4. Copy Connection String Primary Key and save it. You will use this to configure DataSet.
Step 3: Configure DataSet
Now we will configure a DataSet Monitor. In the previous step you copied the Connection String Primary Key. You will configure the monitor with the data in this string.
1. log in to an existing DataSet Account. If you dont have one. Sign up for a free trial.
2. Select Monitors from the top right dropdown
3. Edit JSON
4. Add this snippet to configure EventHub
You will need two pieces of information.
a. EventHub Name
b. Your Connection String Primary Key will look like this
Endpoint=sb://{namespaceHostname}/;SharedAccessKeyName={sharedAccessKeyName};SharedAccessKey={sharedAccessKey}
{
type: "eventhub",
eventHubName: "{eventHubName}",
namespaceHostname: "{namespaceHostname}.servicebus.windows.net",
sharedAccessKey: "{sharedAccessKey}",
sharedAccessKeyName: "{sharedAccessKeyName}",
timeoutSeconds: 60.0
}
Step 4: Configure Azure Service
Now we will configure our app to send its logs to your new Event Hub which is being monitored by DataSet. In this case, we will use Active Directory Logs, but any service with Diagnostic Logs will work.
1. Open your service
2. Scroll down to Monitoring and select Diagnostic settings
3. Add Diagnostic Settings
4. Choose Datasources: Select the logs you want to send from the list of available datasets.
5. Choose Destination and set it to EventHub
6. Select the Destination we configured in Section 1 Step 1
Comments
0 comments
Please sign in to leave a comment.