System Center Operations Manager (SCOM) Linux/UNIX monitoring uses WS-Management/WSMan “basic authentication” on an ongoing basis – i.e., it passes a username/password pair to the Linux computer, and the Linux computer authenticates that username/password using the PAM. If you disable “basic authentication” on the SCOM management server, it’s not going to work. Other potential WinRM authentication choices are not implemented for SCOM monitoring of Linux/UNIX, see https://msdn.microsoft.com/en-us/library/aa384295(v=vs.85).aspx.
What is WS-Management?
WS-Management (or WSMan) is the core protocol used in UNIX/Linux monitoring. WSMan is a SOAP-based protocol for cross-platform management. All monitoring operations (e.g. enumerating CIM providers for data on file systems, memory, etc, execution of commands/scripts for monitoring, executing log file reads for monitoring) are implemented over WSMan. As WSMan is a web service protocol, the OpenPegasus-based CIMOM functions as a secure web server (user credentials are authenticated through PAM). This is where the agent certificate comes in to play.
How Does SCOM use WSMan?
The UNIX/Linux agent certificate is used to secure the WSMan connection using SSL and provide authentication for the remote agent host. When the Operations Manager UNIX/Linux agent is installed, it generates a certificate (using openssl) at the path: /etc/opt/microsoft/ssl. The file name of the certificate is scx-host-<hostname>.pem and the corresponding private key is named scx-key.pem. The agent actually looks for the certificate at /etc/opt/microsoft/scx/ssl/scx.pem, which is initially configured as a symbolic link pointing to scx-host-<hostname>.pem.
When a Management Server discovers a UNIX/Linux agent, the server uses its certificate to sign the agent certificate, acting like a standalone Certificate Authority. In the discovery process, this actually involves securely transferring the certificate from the agent to the Management Server, signing it, copying it back to the agent, and restarting the agent daemon. See http://blogs.msdn.com/b/wmi/archive/2009/03/23/how-to-use-wsman-config-provider-for-certificate-authentication.aspx
In summary, SCOM uses WSMan and certificates to securely communicate between Management Server and the Unix/Linux Agent.