Editor's note: The following post was written by Office Servers and Services MVP Thuan Nguyen as part of our Technical Tuesday series.
Security threats are very hard to detect and defend today. In one of my past blog articles, I pointed out security challenges that required complex machine learning algorithms to detect anomaly and suspicious activities in a timely manner. Monitoring is very crucial because it enables system administrators and security professional to monitor, track and identify security threats as soon as security breach happens on your system. In this article, we’ll explore some monitoring tools on Microsoft Azure, specific to IaaS deployment.
Azure Security Center
Microsoft Azure Security Center can protect your Azure resources. Azure Security Center is built on the methodology called Prevention–Detection–Response.
Azure Security applies machine learning, behavioral analysis and advanced analytics to build many complicated defense patterns to detect anomaly access, suspicious attacks, or so on. With detection capabilities, Azure Security Center connects to the Microsoft threat intelligence center to perform security analysis, and to look up if any suspicious activity has occured. When it comes to anomaly detection, there are advanced analytics and machine learning techniques to learn from you (e.g. normal time of login, login location…) before building historical data.
Azure Security Center runs number of services under the Azure platform to audit Azure resources. If any resource is vulnerable based on Microsoft security baseline, it will provide security recommendations to remediate potential vulnerabilities. The security baseline uses Common Configuration Enumeration to assign unique identifiers for configuration rules.
One of the most interesting features, is the recommendation that it allows you apply. For example, when Azure Security recognizes your virtual machine has not used Disk encryption, it recommends you to take a look and apply the feature. Another example is Microsoft antimalware extension. If your virtual machine has not installed the extension yet, you can apply it from Azure Security Center,
There are two tiers in Azure Security Center: Free and Standard. Each tier offers a different range of features. Free tier provides you basic security policy, assessment and partner solution. Standard tier adds threat detection capabilities on four resource types: Network, Virtual Machine, SQL Databases and Windows crashes. Standard tier also allows you to collect data on supported resources. Maximum allowed daily data collection is 500 MB per day.
Azure Advisor is another service that Microsoft offers to assist cloud consumers in optimizing Azure deployment. Azure Advisor focuses on four criteria:
- High Availability: provides recommendations for your Azure resources including availability set for your virtual machine, and premium storage to keep things more reliable. At the time of writing, it only recommends for virtual machine and storage.
- Security: integrates with Azure Security Center to help prevent, detect and respond to security vulnerability.
- Performance: provides recommendations for Azure SQL Database, Redis Cache and App Service performance.
- Cost: provides recommendations to optimize your cost by looking at your Azure resources usage.
High availability and security criteria in Azure Advisor can be beneficial to your security plan. Related to Azure IaaS, Azure Advisor provides recommendations for virtual machines, availability sets, application gateway and SQL Server (if your application uses it).
Azure Monitor is a monitoring service that assists you to capture logs for most services in Microsoft Azure. In terms of infrastructure monitoring, Azure Monitor enables you to monitor and capture activity logs from application, guest OS, virtual machine and storage. Captured logs can be integrated into different Azure services for route, store or archive, query, vizualization and automation.
One common case in security monitoring is to send your virtual machine monitoring data to Application Insights before you can visualize the monitoring data (e.g. Power BI), or use Azure Analysis services with Azure Machine Learning if you need to perform security analytics. You can also route your monitoring data to another third-party analysis tool via Azure Event Hubs.
Have a look at an example of monitoring your virtual machine via the Activity log setting. From this, you can filter subscription, resource group and timespan to see the activity.
As mentioned, Azure Monitor supports setting metrics and alerts. Below is the list of virtual machine metrics, for example:
- Percentage CPU
- Network In
- Network Out
- Disk Read Bytes
- Disk Write Bytes
- Disk Read Operations/Sec
- Disk Write Operations/Sec
Activity log on your storage account can be captured by Azure Monitor. There is no metric for storage account level. However, you can set a metric for Blob where your virtual machine disk is stored in.
Metrics in Azure Monitor can be helpful to trace if a virtual machine is slow. Moreover, you can set an alert based on pre-defined metric via SMS, email or even web hook to automate trigger next step if you like to automate infrastructure management and maintenance.
Azure Log Analytics
Azure Log Analytics is a service in Operations Management Suite (OMS) and provides you the capability to monitor your Azure resources. It can also monitor on-premises, but we are not going to explore the on-premises capability. Azure Log Analytics gives you an insight of operational data in your Azure environment, mostly focused on Azure virtual machine. Moreover, it can visualize data in a single place called workspace.
Related to security monitoring, below are the list of monitoring packages you can deploy to your workspace:
- AD Assessment: Assess the risk and health of Active Directory environment
- Antimalware Assessment: view status of antivirus and antimalware scan across your virtual machine.
- Activity Log Analytics: track all create, update and delete activities in your Azure subscription.
- Change Tracking: track configuration changes across your virtual machines.
- Network Performance Monitor: provide near real-time monitoring of network performance
- Automation & Control: increase control with automation and configuration management
- Insight & Analytics: monitor and troubleshoot infrastructure issue
- Security & Compliance: secure and audit security with advanced threat detection
- Security and Audit: provide the ability to explore security related data and identify security breaches.
- Wire Data 2.0: explore wire data and identify network related issues.
- Key Vault Analytics: understand key vault usage through log.
- Azure Network Security Group Analytics: provide insight of Network Security Group logs
The term “Defense in depth” drives us to another in depth strategy called monitoring in depth. That said, it is recommended to monitor every perimeter we have. For deep-dive investigation, if a security incident occurs, we do need to analyze network packet and its flow, which Azure Security Center does not provide. DDos monitoring should be noted as a common case of deep inspection.
Microsoft Azure recently announced a new feature called Network Watcher. Network Watcher provides you network monitoring to see your network topology, what virtual network and subnet you have, what network security group you configure to your subnet or network interface card and how it flows across your virtual machines. Network Watcher also provides network diagnostic tools including IP flow verification, next hop, security group view and package capture, network security group flow and logging.
Network Watcher is very important for network monitoring on Microsoft Azure. It can help in many cases. Firstly, if you receive a performance compliant from your customer or business user, you may have to look into network factor first to see whether any core network packet is dropped or not.
Secondly, if your system is receiving a large network attack (e.g. DDos, Botnet), you need to monitor to trace the source of attack, and to have preventive action as soon as possible.
Network Watcher is not available in all Azure regions. You can find its availability per region here.
Azure Monitor can help monitor the activity log on your Azure storage account. There are many metrics in Azure storage account which are worth using (e.g. AnonymousAuthorizationError, or AnonymousSuccess).
Virtual Machine Monitoring
By default, you can monitor CPU (average), Network, Disk bytes and Disk operations from the overview view when opening a virtual machine. The dashboard shows you data collected, offering of six time frames: 1 hour, 6 hours, 12 hours, 1 day, 7 days and 30 days.
These are not the only available metrics. You can add more metrics from the list of approximately 70 supported metrics. With each metric, you can control alerts.
Activity log of your virtual machine can be visualized in Azure Log Analytics for monitoring.
When you centralize your identity in Microsoft Azure, your team is given access to different Azure resources. In this case, you need to monitor and manage them. With Azure Active Directory Premium, you take fully advantages of building a risk-based policy to automatically protect identities. These can include:
- Leaked credentials
- Impossible travel to atypical locations
- Sign-ins from infected devices
- Sign-sin from anonymous IP addresses
- Sign-ins from IP addresses with suspicious activity
- Signs in from unfamiliar locations
For more information about these capabilities, read here
In this article, you were introduced several tools to monitor your Azure IaaS resources, including virtual machine, storage, network and identity. These tools do not only provide activity and diagnostics log but also allow you to set alert based on many metrics. We also explored Azure Security Center which specifically focuses on helping you protect Azure resources with capabilities of detection and security recommendation on each Azure resource.
Thuan Nguyen is a Subject Master Expert in Digital Workplace. With nearly 10 years of experience in Information Technology and Services, Thuan has been involved in a number of successful Microsoft Digital Workplace adoptions for mid-tier and large organizations, including government agencies in which his responsibilities include technical evangelism, pre-sales activities, proof-of-concept mentoring, technical support to architecting the entire solution.
Thuan has been awarded as a Microsoft Office Services and Servers MVP 7 years in a row. He is a regular guest speaker for several international conferences on SharePoint, Office 365 and Azure. Follow him on Twitter @nnthuan