Send Telemetry to Splunk Enterprise from Azure Resources via Azure Monitor, Part 2

  • Article

 For Part 1 of this blog series, which contains overview material, please click here. 

There are several scenarios that must be addressed when thinking about getting telemetry from Azure resources to Splunk. What if Splunk is on premises? What if you’re using Splunk Cloud? You could be using a private network connection to Azure, or not. The Splunk add-on approach isn’t suited to all of these. In this blog and others in this series, I’ll introduce some new architectural elements and go into detail on each.  

In this article, “the add-on” refers to this. 

These specific scenarios will be dealt with: 

  • Cloud-based Splunk, using the add-on (this article)
  • Cloud-based Splunk, using the HTTP Event Collector (Part 3) 
  • Premises-based Splunk, using private network (Part 4) 
  • Premises-based Splunk, via the internet (Part 5)

 In this, Part 2, I’ll go into the already familiar Cloud-based Splunk using the add-on.

 

AzureAddonCloudBasedSplunk

 

In this scenario, all resources reside in Azure. The Splunk VM is Splunk Enterprise and may be a cluster rather than a single box. Setting up the VM’s firewall and the network’s Network Security Group to allow outbound traffic on 443 is sufficient.  

The simplest implementation has all monitored resources and monitoring resources in the same subscription and region, but this is seldom the case. Much more typical is many subscriptions with assets in multiple regions. While there is normally just one Splunk indexer (whether single VM or cluster), you may ingest all telemetry directly into that instance from multiple regional Event Hubs and REST APIs. Alternatively, you may use a Splunk Heavy Forwarder in the remote regions, each with the add-on loaded and configured to ingest that region’s data. Each of the forwarders send the aggregated data along to the main index. In either case, the data never leaves Microsoft’s network. 

The decision to use a Heavy Forwarder in the regions vs sending raw data to the indexer isn’t trivial. Here’s an article by a Splunk consultant on the topic. 

Another consideration is the Azure AD tenant. For various reasons, it’s important that the resources being monitored and the monitoring resources (event hubs, service principal, etc) all be in the same AAD tenant.