Microsoft Monitoring Agent, System Center Operations Manager and Visual Studio Application Insights

Article
11/24/2013

Since the release of System Center 2012 R2 Preview (and more after GA was announced) a lot of people asked me why did we rename the Operations Manager agent to "Microsoft Monitoring Agent"? Some information that went out together with the GA of SC2012R2 can be found at the following link: https://technet.microsoft.com/en-us/library/dn465154.aspx
And here is a post from Marnix, System Center MVP https://thoughtsonopsmgr.blogspot.com/2013/09/scom-2012-r2-hello-mma-microsoft.html
Essentially, Microsoft Monitoring Agent is not *only* the SCOM agent anymore - the agent is now licensed with System Center OR with Visual Studio. When it was first released, it could already be used when reporting to SCOM (for monitoring), and it could also be used for standalone IntelliTrace collection (diagnostics scenario, more geared towards Dev/App owners). Read more in these other blog posts by Larry: Introducing Microsoft Monitoring Agent and Performance Details in IntelliTrace.

Enter ‘Application Insights’

With Microsoft Monitoring Agent 2013 Update Rollup 1 (at the time of this writing available as a preview), Microsoft Monitoring Agent can now also be used to report APM data to the brand new Application Insights Preview feature in Visual Studio Online that was announced a couple of weeks ago. Application Insights is an Azure-backed SaaS solution allowing teams to “[…] get insights from monitoring and going back and make the app better. Some people call it DevOps [...] but it's a sort of holistic view of the application: 360 degrees telemetry data like usage, performance, exception data, crash data, all that you need to have in live site to know how well your application is doing.[…]” (see the complete interview to Soma here) .

So what powers some (but not all) of the data that you have at your fingertips in Application Insights – like you might have imagined - is the APM agent within MMA: the same APM agent you can use with OpsMgr. And in Application Insights you’ll see the same familiar data you see in OpsMgr such as exceptions and performance events (which can be exported to IntelliTrace format), and performance counters, but this time in a multi-tenant SaaS solution specifically designed for DevOps teams.

MMA 2013 UR1 Preview is available as a standalone download from Microsoft Download Center (as well as from the Application Insights Preview itself, within the Visual Studio Online portal) and it is the first version of the agent that allows the agent to connect to both on-premises System Center OpsMgr systems as well as to the SaaS service.
https://www.microsoft.com/en-us/download/details.aspx?id=41143
https://msdn.microsoft.com/en-us/library/dn481094.aspx
Microsoft Monitoring Agent - Select Connect to Application Insights

NOTE: Keep in mind that at the time of this writing, this is a CTP (“Preview”) release of the agent. It is not supported by CSS for non-Visual Studio Online-related scenarios. Even though we are not currently aware of any major compatibility issues between this CTP and SCOM (or when multi-homing between Application Insights and SCOM), only very limited testing was done for this agent working together with SCOM at this stage. We encourage SCOM customers NOT to use it in their production environments and wait for the final Microsoft Monitoring Agent 2013 Update 1 release.

In the future, anyhow, dual homing could be used to let your agent differentiate what data to send to which solution: i.e. send only alerting and performance information necessary for monitoring and triaging production issues to the on-prem System Center Operations Manager system, while the detailed and much more verbose code-level information can be sent for developers to consume in our multi-tenant SaaS APM offering within Team Foundation Service online (so you don’t have to worry about managing extra storage for APM data in the SCOM database), or to SCOM (maybe only data from some application, i.e. ‘PROD’ applications), or to both systems in various combinations – based on environment, project, operational model, processes and teams/ownership… It is for example very practical to use Application Insights to conduct functional and load tests in development and test environments – without the need to stand up another OpsMgr infrastructure, or to affect the scale and performance of the one that is designed to handle ‘prod’ data – to feed rich and actionable diagnostic information into the development lifecycle, to improve those applications even before they go in production.

Maarten, one of the System Center MVP’s, has also started a series of post on Application Insights where he started sharing his perspective about the powerful hybrid monitoring scenarios that have been enabled when using Microsoft Monitoring Agent with Application Insights and with System Center 2012 R2.

APM for Azure PaaS

Added benefit - MMA, when used with Visual Studio online, can also be installed in Azure Cloud Services's instances (PaaS) - which was not a supported scenario in System Center (see this post where I mentioned this before). This is the first time we are able to offer true APM monitoring for Azure PaaS. In OpsMgr, agents are uniquely identified by their FQDN (Fully Qualified Domain Name), and everything in SCOM from connector to DB to DAL to SDK - all these components rely on agent names. Machine names in most corporate networks are well-defined pieces of information, follow a logical naming convention, and rarely change. SCOM Management Servers also rely on Kerberos/AD on premises and/or certificates (again using the FQDN) to authenticate the agents, and expect to only be talking to ‘well known’, pre-authorized, machines. But with Azure Cloud Services (PaaS) and IaaS (in certain configurations) you can have any number of cloned, identical, elastic instances of 'roles' (worker and web) deployed and running at any given time which appear and disappear as you scale them up and down. Machine names don't last and don’t matter much in Azure PaaS like they do on-prem... and it is much more natural to have Azure send APM data to Azure, not to on-prem - which would otherwise require opening inbound ports in your perimeter - remember the agent initiates connection to the infrastructure it reports to, be it a SCOM Management Group, or Application Insights in the cloud. The agent includes a brand new connector that can talk web-friendly protocols to report to the SaaS offering, which is a very different backend than an OpsMgr Management Server/Group. Application Insights uses a newly built backend running in Azure, written with Cloud-First principles. The way to authenticate to the service is thru an ‘application key’ (which represents the Application Insights tenant); the cloud service does not use the machine’s identity. You place the application key in the application’s configuration file, and Visual Studio allows you to package/bundle a script to silently install and configure the agent automatically, so that every time your PaaS roles are re-deployed, you will have the agent installed on it. Machines come and go, applications stay, and they need to be monitored – those applications and their lifecycle are what Application Insights and Visual Studio Online are all about. For infrastructure-level info you don’t need an agent, instead: from System Center Operations Manager, you can of course keep using the Azure Management Pack, which polls from the Azure Management API and does a better job to create/dispose of those ‘elastic’ objects that come and go (thru discoveries); if you are only in the cloud (=no on-prem infrastructure) you can find that type of OS-level info (CPU/Memory/Disk) in Azure Portal.

Availability Monitoring

Availability information (and other metrics such as external response time) that is tracked in Application Insights comes from synthetic tests providing an ‘Outside In’ perspective: single URL probes or Visual Studio webtests. If you are one of my OpsMgr readers, you would have probably understood this is backed by Global Service Monitor – the same service, offering ‘watcher nodes in the cloud’, that you can attach to OpsMgr.

More than just APM (as we knew it in System Center)

More explicit instrumentation can be added to apps in various ways, when reporting to Application Insights. These include:

Client side Usage monitoring : Client-side monitoring instrumentation in Application Insights is a completely different solution than the one in OpsMgr. First, the focus is on usage, visitors, and their experience – but more in the analytics sense, rather than with the alerting angle of the one in OpsMgr. Second, enablement is different: Application Insights provides you with a javascript snippet that can be added to any website, also if not .Net – unlike in OpsMgr where .Net server-side monitoring is used to hook up automatic injection of javascript – but the change must be done by a developer. The manual method in the end proves more compatible with many applications and browsers.
Server SDK's by which you can instrument logging for custom metrics in your code and have it report to the service directly
Client SDK for Windows Phone 8 apps by which you can instrument logging for custom metrics in your code and have it report to the service
Deployment information can be collected – see post from Charles – this is extremely useful to understand if changes in performance or reliability are related with deployments of new versions of the app/service
Beautiful Customizable Dashboards and a fresh, modern UI on top of all this data

How can I try it out?

Application Insights is currently in preview and you need an invitation to try it out. You might want to go to www.visualstudio.com and register for a VSO subscription and add yourself to the waiting list by clicking the blue “Try Application Insights” button.

Some more links

Series of videos on how to use it https://channel9.msdn.com/Series/Application-Insights-for-Visual-Studio-Online
Forum on MSDN https://social.msdn.microsoft.com/Forums/vstudio/en-US/home?forum=ApplicationInsights
Documentation on MSDN https://msdn.microsoft.com/en-us/library/dn481095.aspx