Azure Log Integration SIEM configuration steps


This document provides screen shots of audit logs and Azure Security Center alerts integrated with the following partner solutions:

  • Splunk
  • HP ArcSight
  • IBM QRadar

The machine that Azure log integration services is installed on is called the Azlog Integrator. Your SIEM agent (the Splunk Universal Forwarder, HP ArcSight Windows Event Collector agent, or IBM QRadar WinCollect) is also installed on the Azlog Integrator. Azure log integration puts the Windows Event logs in the Forwarder Events channel.

Make sure that your standard SIEM connector installed on the machine is configured to pick events from the Forwarded Events folder and pipe them to your SIEM instance. Review the SIEM specific documentation for configuration details.

Splunk

For information on forwarding data from Microsoft Windows to Splunk, refer to Splunk’s documentation.

The following example shows events coming from ComputerName Iaas_azsiemdemo.

image

Integration of Azure Audit Logs in Splunk

Use Splunk Universal Forwarder to point to c:\Users\azlog\AzureResourceManagerJson as below

splunk add monitor C:\Users\azlog\AzureResourceManagerJson

splunk restart

The following screen shot shows logs in Splunk.

image

Integration of Security Center alerts in Splunk

Use Splunk Universal Forwarder to point to C:\Users\azlog\AzureSecurityCenterJson as below:

splunk add monitor C:\Users\azlog\AzureSecurityCenterJson

splunk restart

The following screen shot shows Security Center alerts in Splunk.

image

HP ArcSight

For information on forwarding data from Microsoft Windows to HP ArcSight, refer to ArcSight’s documentation.

The following example shows Windows events coming from Azure VMs into ArcSight.

image

Integration of Azure Audit Logs in ArcSight

1. Create a SmartMessage Receiver.

If you do not have a SmartMessage Receiver, create one in ArcSight Logger selecting Configuration. Under Configuration, select Receivers.

clip_image010

2. Select Add.

clip_image012

3. In this example, under Add Receiver, let’s use SmartMessage Receiver for the Name and Type. Select Next.

clip_image014

4. Under Edit Receiver, choose settings below and select Save.

clip_image016

5. On the Windows machine with the Azure SIEM Integration Service, launch the ArcSight SmartConnector installation wizard and select Next.

clip_image018

6. On the Choose Install Folder page, identify path to where you want to install and select Next.

clip_image020

7. On the Choose Install Set page, select Custom and then Next.

clip_image022

8. On the Choose Product Components page, choose settings below and select Next.

clip_image024

9. On the Choose Shortcut Folder page, choose settings below and select Next.

clip_image026

10. On the Pre-Installation Summary page, select Install.

clip_image028

11. On the Connector Setup page, select Add a Connector and select Next.

clip_image030

12. On the next Connector Setup page, choose ArcSight FlexConnector JSON Folder Follower and select Next.

clip_image032

13. On the next Connector Setup page, set the Folder Location to the path where JSON logs are being written to (for example, c:\Users\azlog\AzureResourceManagerJson)and set the Configuration File Name Prefix to AzureRM. Select Next.

clip_image034

14. On the next page, select ArcSight Logger SmartMessage(encrypted) and select Next.

clip_image036

15. On the next page, enter the ArcSight Logger machine’s IP address, the Receiver Name of the SmartMessage receiver (for example, SmartMessage Receiver) and select Next.

clip_image038

16. On the next page, fill in information about the connector machine (example information shown) and select Next.

clip_image040

17. The next page may be shown to import the certificate from the ArcSight Logger machine. Select Next.

clip_image042

18. Review the summary on the next Connector Setup summary page and select Next.

clip_image044

19. On the next page select Install as a serviceand select Next.

clip_image046

20. On the next page, leave the defaults as is and select Next.

clip_image048

21. Review the summary on the next page and select Next.

clip_image050

22. On the next page, select Exit and select Next.

clip_image052

23. You should now see the Install Complete page. Select Done.

clip_image054

After the installation wizard is complete, copy the AzureRM.jsonparser.properties file (downloaded as part of Azure log integration) to \Program Files\ArcSightSmartConnectors\current\user\agent\flexagent\AzureRM.jsonparser.properties. The contents of AzureRM.jsonparser.properties can be modified as needed to change the mapping of Azure Resource Manager log entries to ArcSight events. See HP’s Flex Connector documentation for the format of this file.

At this point, the ArcSight ArcSight FlexConnector JSON Folder Follower service should be stopped. If not, stop it from the Services Control Panel application or from the command line using command :

net stop “ArcSight ArcSight FlexConnector JSON Folder Follower

Note:
The exact service name may be different if non-default options were chosen during setup.

Stopping the service should ensure that the AzureRM.jsonparser.properties file is picked up.

Now, start the service from the Services Control Panel or from the command line using this command:

net start “ArcSight ArcSight FlexConnector JSON Folder Follower

Events should now be flowing to the ArcSight logger. Following is a screenshot of Azure Audit logs in ArcSight:

clip_image056

IBM QRadar

Windows Events from Azure VMs IN QRADAR

For information on forwarding data from Microsoft Windows to IBM QRadar, refer to QRadar’s documentation.

The following example shows Windows events coming from Azure VMs into QRadar.

clip_image058

clip_image060

Integration of Azure Audit Logs in QRadar

The following example assumes you have azlog installed on a machine with the WinCollect agent configured as a Log Source in QRadar.

1. On the Admin tab, select Log Source Extensions.

clip_image062

2. Select Add.

clip_image064

3. Set the Name and Description to AzureRM. Browse for the file AzureRM_QRadarLogSourceExtension.xml downloaded as part of the Azure log integration download. Select Upload.

Note:
The Use Condition value is ignored in current versions of QRadar.

clip_image066

4. Select Save after the Log Source Extension has been uploaded.

clip_image068

5. Close the list of Log Source Extensions.

6. Return to the Admin tab and choose Log Sources.

clip_image070

7. Select Add.

clip_image072

8. On the Add a log source page, set the following options:

Log Source Name -AzureRM (or choose your own)

Log Source Description – AzureRM” (or choose your own)

Log Source Type – Universal DSM

Protocol Configuration – WinCollect File Forwarder

Log Source Identifier – (IP address or host name of machine running AZLOG)

Local System –Leave checked

Local System Root Directory – C:\Users\azlog\AzureResourceManagerJsonLD (Note confirm this is the directory on the AZLOG machine – Search for “AzureResourceManagerJsonLD” under “Users” if you do not find the directory in \users\azlog)

Filename Pattern -.*

Monitoring Algorithm -Continuous Monitoring

Only Monitor Files Created Today –Leave checked (or choose your own option)

File Monitor Type – Notification-based (local)

File Reader Type – Text (file held open)

Polling Interval – 5000 (or choose your own value)

WinCollect Agent – Choose the WinCollect agent on the machine running AZLOG

Enabled – Leave checked

Credibility – choose your own value

Target Internal Destination – Choose TCP for your QRadar install (needed to support messages up to 4k)

Target External Destinations – ‘unchecked’

Coalescing Events – Leave unchecked (or choose your own value)

Store Event Payload – Leave checked

Log Source Extension – AzureRM (or the name you specified for the Log Source Extension)

Extension Use Condition – Ignored in current versions of QRadar

9. Select Save.

clip_image074

10. Close the list of Log Sources.

11. Return to the Admin tab and select Custom Event Properties.

clip_image076

Define a Custom Event Property for each additional property you wish to extract from the Azure Resource Manager Operation Log JSON. For example, to define ResourceGroup as a Custom Event Property, do the following:

12. Select Add.

clip_image078

13. Copy the following example Azure Resource Manager JSON event to the Test Field. (This will help confirm that you have entered the regular expressions correctly.)

{“authorization”:{“action”:”Microsoft.Storage/storageAccounts/regenerateKey/action”,”scope”:”/subscriptions/1234567-a20b-42b4-96c8-22b2965adecb/resourceGroups/azqradartest/providers/Microsoft.Storage/storageAccounts/azqradartest”},”caller”:”lauren@example.com”,”channels”:”Operation”,”claims”:{“aud”:”https://management.core.windows.net/”,”iss”:”https://sts.windows.net/72f988bf-86f1-41af-91ab-2d7cd011db47/”,”iat”:”1462052874″,”nbf”:”1462052874″,”exp”:”1462056774″,”_claim_names”:”{\”groups\”:\”src1\”}”,”_claim_sources”:”{\”src1\”:{\”endpoint\”:\”https://graph.windows.net/72f988bf-86f1-41af-91ab-2d7cd011db47/users/12345678-d3c4-43a1-b237-b67f5a2f22a9/getMemberObjects\”}}”,”http://schemas.microsoft.com/claims/authnclassreference”:”1″,”http://schemas.microsoft.com/claims/authnmethodsreferences”:”pwd,mfa”,”appid”:”12345678-3bb0-49c1-b47d-974e53cbdf3c”,”appidacr”:”2″,”http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname”:”Calia”,”http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname”:”Lauren”,”in_corp”:”true”,”ipaddr”:”131.101.114.230″,”name”:”Lauren Calia”,”http://schemas.microsoft.com/identity/claims/objectidentifier”:”12345678-d3c4-43a1-b237-b67f5a2f22a9″,”onprem_sid”:”S-1-5-21-1234567890-1234567890-1234567890-12345″,”puid”:”1234567812345678″,”http://schemas.microsoft.com/identity/claims/scope”:”user_impersonation”,”http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier”:”HJABCdefid7U8-o-abcdefabcdefpm08vBvz8″,”http://schemas.microsoft.com/identity/claims/tenantid”:”72f988bf-86f1-41af-91ab-2d7cd011db47″,”http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name”:”lauren@example.com”,”http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn”:”lauren@example.com”,”ver”:”1.0″},”correlationId”:”12345678-523f-4da5-9fef-40fa0ff37c55″,”description”:””,”eventDataId”:”12345678-85d0-4194-a29b-5b4a039e3db9″,”eventName”:{“value”:”BeginRequest”,”localizedValue”:”Begin request”},”category”:{“value”:”Administrative”,”localizedValue”:”Administrative”},”httpRequest”:{“clientRequestId”:”12345678-1769-480b-87c8-0c410a83804d”,”clientIpAddress”:”167.219.3.17″,”method”:”POST”},”id”:”/subscriptions/12345678-a20b-42b4-96c8-22b2965adecb/resourceGroups/azqradartest/providers/Microsoft.Storage/storageAccounts/azqradartest/events/12345678-85d0-4194-a29b-5b4a039e3db9/ticks/635976520724487622″,”level”:”Informational”,”resourceGroupName”:”azqradartest”,”resourceProviderName”:{“value”:”Microsoft.Storage”,”localizedValue”:”Microsoft.Storage”},”resourceId”:”/subscriptions/12345678-a20b-42b4-96c8-22b2965adecb/resourceGroups/azqradartest/providers/Microsoft.Storage/storageAccounts/azqradartest”,”resourceType”:{“value”:”Microsoft.Storage/storageAccounts”,”localizedValue”:”Microsoft.Storage/storageAccounts”},”operationId”:”12345678-523f-4da5-9fef-40fa0ff37c55″,”operationName”:{“value”:”Microsoft.Storage/storageAccounts/regenerateKey/action”,”localizedValue”:”Microsoft.Storage/storageAccounts/regenerateKey/action”},”status”:{“value”:”Started”,”localizedValue”:”Started”},”subStatus”:{“value”:””,”localizedValue”:””},”eventTimestamp”:”2016-04-30T22:27:52.4487622Z”,”submissionTimestamp”:”2016-04-30T22:28:12.9973642Z”,”subscriptionId”:”12345678-a20b-42b4-96c8-22b2965adecb”,”tenantId”:”72f988bf-86f1-41af-91ab-2d7cd011db47″}

14. Choose the following options:

Property Type Selection – Regex Based

New Property –ResourceGroup

Optimize parsing rules, reports, and searches – Leaveunchecked (or choose your own value)

Field Type – Alphanumeric

Field Description –Resource Group

Enabled – Leavechecked

Log Source Type – Universal DSM

Log Source –AzureRM

– Select Category and set HighLevelCategory = Any and Low Level Category = Any

Extraction RegEx (note – Include the quotes when copying the field and make sure they are not the sloped “smart quotes”) – “resourceGroupName”:”(.*?)”

Capture Group – 1

Confirm that the Resource Group name is highlighted in Yellow and select Save.

Note:
Select Test if necessary to confirm that the RegEx found a hit[TL1] .

clip_image080

15. Close the list of Custom Event Properties.

16. Return to the Admin-taband select Deploy Changes.

clip_image082

Events should now be searchable in the Log Activity tab. Event names may show up as Unknown because QRadar doesn’t have a mapping for the Azure Resource Manager Operation Name values. Additional Custom Event Properties can be added to retrieve useful information about the Azure Resource Manager Operation events.

For example, here are RegEx expressions for two important values that can be added as Custom Event Properties following the steps above:

Subscription Id – “subscriptionId”:”(.*?)”

Operation Name – “operationName”.*?”value”:”(.*?)”

Note:
The RegEx should be copied along with the quotes, taking care to make sure they are not sloped “smart quotes”.


Comments (0)

Skip to main content