This blog post is outdated and some of the steps may not work correctly. If you have problems, please let us know at the Azure Log Integration forum
This document provides screen shots of audit logs and Azure Security Center alerts integrated with the following partner solutions:
- HP ArcSight
- IBM QRadar
The machine that Azure log integration services is installed on is called the Azlog Integrator. Your SIEM agent (the Splunk Universal Forwarder, HP ArcSight Windows Event Collector agent, or IBM QRadar WinCollect) is also installed on the Azlog Integrator. Azure log integration puts the Windows Event logs in the Forwarder Events channel.
Make sure that your standard SIEM connector installed on the machine is configured to pick events from the Forwarded Events folder and pipe them to your SIEM instance. Review the SIEM specific documentation for configuration details.
For information on forwarding data from Microsoft Windows to Splunk, refer to Splunk’s documentation.
The following example shows events coming from ComputerName Iaas_azsiemdemo.
Integration of Azure Audit Logs in Splunk
Use Splunk Universal Forwarder to point to c:\Users\azlog\AzureResourceManagerJson as below
splunk add monitor C:\Users\azlog\AzureResourceManagerJson
The following screen shot shows logs in Splunk.
Integration of Security Center alerts in Splunk
Use Splunk Universal Forwarder to point to C:\Users\azlog\AzureSecurityCenterJson as below:
splunk add monitor C:\Users\azlog\AzureSecurityCenterJson
The following screen shot shows Security Center alerts in Splunk.
For information on forwarding data from Microsoft Windows to HP ArcSight, refer to ArcSight’s documentation.
The following example shows Windows events coming from Azure VMs into ArcSight.
Integration of Azure Audit Logs in ArcSight
1. Create a SmartMessage Receiver.
If you do not have a SmartMessage Receiver, create one in ArcSight Logger selecting Configuration. Under Configuration, select Receivers.
2. Select Add.
3. In this example, under Add Receiver, let’s use SmartMessage Receiver for the Name and Type. Select Next.
4. Under Edit Receiver, choose settings below and select Save.
5. On the Windows machine with the Azure SIEM Integration Service, launch the ArcSight SmartConnector installation wizard and select Next.
6. On the Choose Install Folder page, identify path to where you want to install and select Next.
7. On the Choose Install Set page, select Custom and then Next.
8. On the Choose Product Components page, choose settings below and select Next.
9. On the Choose Shortcut Folder page, choose settings below and select Next.
10. On the Pre-Installation Summary page, select Install.
11. On the Connector Setup page, select Add a Connector and select Next.
12. On the next Connector Setup page, choose ArcSight FlexConnector JSON Folder Follower and select Next.
13. On the next Connector Setup page, set the Folder Location to the path where JSON logs are being written to (for example, c:\Users\azlog\AzureResourceManagerJson)and set the Configuration File Name Prefix to AzureRM. Select Next.
14. On the next page, select ArcSight Logger SmartMessage(encrypted) and select Next.
15. On the next page, enter the ArcSight Logger machine’s IP address, the Receiver Name of the SmartMessage receiver (for example, SmartMessage Receiver) and select Next.
16. On the next page, fill in information about the connector machine (example information shown) and select Next.
17. The next page may be shown to import the certificate from the ArcSight Logger machine. Select Next.
18. Review the summary on the next Connector Setup summary page and select Next.
19. On the next page select Install as a serviceand select Next.
20. On the next page, leave the defaults as is and select Next.
21. Review the summary on the next page and select Next.
22. On the next page, select Exit and select Next.
23. You should now see the Install Complete page. Select Done.
After the installation wizard is complete, copy the AzureRM.jsonparser.properties file (downloaded as part of Azure log integration) to \Program Files\ArcSightSmartConnectors\current\user\agent\flexagent\AzureRM.jsonparser.properties. The contents of AzureRM.jsonparser.properties can be modified as needed to change the mapping of Azure Resource Manager log entries to ArcSight events. See HP’s Flex Connector documentation for the format of this file.
At this point, the ArcSight ArcSight FlexConnector JSON Folder Follower service should be stopped. If not, stop it from the Services Control Panel application or from the command line using command :
net stop "ArcSight ArcSight FlexConnector JSON Folder Follower
The exact service name may be different if non-default options were chosen during setup.
Stopping the service should ensure that the AzureRM.jsonparser.properties file is picked up.
Now, start the service from the Services Control Panel or from the command line using this command:
net start "ArcSight ArcSight FlexConnector JSON Folder Follower
Events should now be flowing to the ArcSight logger. Following is a screenshot of Azure Audit logs in ArcSight:
Windows Events from Azure VMs IN QRADAR
This solution collects logs in the Windows Forwarded Event channel, which you can see in the Event Viewer on the Azure Log Integration machine.
You will need to install and configure WinCollect to integrate events from the forwarded Event Channel to QRadar.
For information on forwarding data from Microsoft Windows to IBM QRadar, refer to QRadar’s documentation at http://public.dhe.ibm.com/software/security/products/qradar/documents/iTeam_addendum/b_wincollect.pdf
The following example shows Windows events coming from Azure VMs into QRadar.
Integration of Azure Audit Logs in QRadar
The following example assumes you have azlog installed on a machine with the WinCollect agent configured as a Log Source in QRadar.
1. On the Admin tab, select Log Source Extensions.
2. Select Add.
3. Set the Name and Description to AzureRM. Browse for the file AzureRM_QRadarLogSourceExtension.xml downloaded as part of the Azure log integration download. Select Upload.
The Use Condition value is ignored in current versions of QRadar.
4. Select Save after the Log Source Extension has been uploaded.
5. Close the list of Log Source Extensions.
6. Return to the Admin tab and choose Log Sources.
7. Select Add.
8. On the Add a log source page, set the following options:
- Log Source Name -AzureRM (or choose your own)
- Log Source Description - AzureRM” (or choose your own)
- Log Source Type - Universal DSM
- Protocol Configuration - WinCollect File Forwarder
- Log Source Identifier – (IP address or host name of machine running AZLOG)
- Local System –Leave checked
- Local System Root Directory - C:\Users\azlog\AzureResourceManagerJsonLD (Note confirm this is the directory on the AZLOG machine – Search for “AzureResourceManagerJsonLD” under “Users” if you do not find the directory in \users\azlog)
- Filename Pattern -.*
- Monitoring Algorithm -Continuous Monitoring
- Only Monitor Files Created Today –Leave checked (or choose your own option)
- File Monitor Type – Notification-based (local)
- File Reader Type – Text (file held open)
- Polling Interval – 5000 (or choose your own value)
- WinCollect Agent – Choose the WinCollect agent on the machine running AZLOG
- Enabled – Leave checked
- Credibility – choose your own value
- Target Internal Destination – Choose TCP for your QRadar install (needed to support messages up to 4k)
- Target External Destinations – ‘unchecked’
- Coalescing Events – Leave unchecked (or choose your own value)
- Store Event Payload – Leave checked
- Log Source Extension – AzureRM (or the name you specified for the Log Source Extension)
- Extension Use Condition – Ignored in current versions of QRadar
9. Select Save.
10. Close the list of Log Sources.
11. Return to the Admin tab and select Custom Event Properties.
Define a Custom Event Property for each additional property you wish to extract from the Azure Resource Manager Operation Log JSON. For example, to define ResourceGroup as a Custom Event Property, do the following:
12. Select Add.
13. Copy the following example Azure Resource Manager JSON event to the Test Field. (This will help confirm that you have entered the regular expressions correctly.)
14. Choose the following options:
- Property Type Selection - Regex Based
- New Property –ResourceGroup
- Optimize parsing rules, reports, and searches – Leaveunchecked (or choose your own value)
- Field Type – Alphanumeric
- Field Description –Resource Group
- Enabled – Leavechecked
- Log Source Type – Universal DSM
- Log Source –AzureRM
- Select Category and set HighLevelCategory = Any and Low Level Category = Any
- Extraction RegEx (note – Include the quotes when copying the field and make sure they are not the sloped “smart quotes”) – "resourceGroupName":"(.*?)"
- Capture Group – 1
Confirm that the Resource Group name is highlighted in Yellow and select Save.
15. Close the list of Custom Event Properties.
16. Return to the Admin-taband select Deploy Changes.
Events should now be searchable in the Log Activity tab. Event names may show up as Unknown because QRadar doesn’t have a mapping for the Azure Resource Manager Operation Name values. Additional Custom Event Properties can be added to retrieve useful information about the Azure Resource Manager Operation events.
For example, here are RegEx expressions for two important values that can be added as Custom Event Properties following the steps above:
Subscription Id - "subscriptionId":"(.*?)"
Operation Name - "operationName".*?"value":"(.*?)"
The RegEx should be copied along with the quotes, taking care to make sure they are not sloped “smart quotes”.