Send Telemetry to Splunk Enterprise from Azure Resources via Azure Monitor, Part 3

  • Article

For Part 1 of this blog series, which contains overview material, please click here.

 There are several scenarios that must be addressed when thinking about getting telemetry from Azure resources to Splunk. What if Splunk is on premises? What if you’re using Splunk Cloud? You could be using a private network connection to Azure, or not. The Splunk add-on approach isn’t suited to all of these. In this blog and others in this series, I’ll introduce some new architectural elements and go into detail on each.

 In this article, “the add-on” refers to this.

 These specific scenarios will be dealt with:

  • Cloud-based Splunk, using the add-on (Part 2) 
  • Cloud-based Splunk, using the HTTP Event Collector (this article) 
  • Premises-based Splunk, using private network (Part 4) 
  • Premises-based Splunk, via the internet (Part 5)

 In this, Part 3, I’ll go into Cloud-based Splunk using the HTTP Event Collector (HEC).

 

AzureFunctionPlusHEC

 

Splunk HEC is a “fast and efficient way to send data to Splunk Enterprise and Splunk Cloud.” Here is an Introduction to Splunk HTTP Event Collector. For the purposes of this article, think of HEC as an authenticated Splunk endpoint to which we can send events for indexing.

There are a couple of reasons that I know of that you might want to use HEC:

  • You use Splunk Cloud. The add-on won’t install there.
  • You don’t want to install the add-on into your Splunk Enterprise system for whatever reason.

To satisfy these requirements we need another mechanism to get the data from event hubs and send it along to Splunk. An Azure Function is a great solution for that. The function that has been built for this purpose is here. Use it with the HEC output binding.

Azure Monitor metrics are available via event hub, the same as diagnostic and activity logs. If the volume of messages increases (or decreases), the Azure Function App will autoscale to meet demand.

While I said initially “Cloud-based Splunk”, that’s just the typical scenario. In reality, as long as the HEC endpoint is reachable by the Azure Function, it’s a perfectly good approach. For example, you have an ExpressRoute connection from Azure to your premises and your Function App leverages an App Service Environment, this connectivity becomes an option. See this article for some of the details.