IPsec and Certificate Authentication

  • Article

====================== DISCLAIMER ====================
This posting is provided "AS IS" with no warranties, and confers no rights.
====================================================

There is some confusion over what role certificates have in IPsec. Some are thinking that the certificates are being used to encrypt the IPsec traffic - but this is not true. PKI certificates can be used to authenticate IPsec peers but cannot be used to encrypt traffic secured by IPsec.

However, using certificates is a great way to authenticate IPsec peers and even has some advantages over Kerberos v5 authentication, under certain circumstances. Certificates can be arranged in hierarchies that give them o good deal of flexibility in how peers trust each other’s certificates are set up. Also, certificates give you a third-party validation not available with Window’s implementation of Kerberos v5.

Certificates must be used under the following circumstances:

  • If the peers are in separate forests, then Kerberos v5 authentication cannot be used, however, certificates can be.
  • If one of the peers (or both of them) is running on a non-Windows computer, then Kerberos v5 authentication might not be able to successfully negotiate. In this case using certificates will work well.
  • And, if you already have a robust PKI system up and running and are used to using certificates, this is also a good way to go.

When Do I use Kerberos v5?
So, with all of this, when would we use Kerberos v5? Typically, Kerberos v5 is the authentication method of choice for Active Directory domains and forests, simply because it comes for free and is fairly automatic.

What About Pre-Shared Keys?
Using pre-shared keys is a bad idea because both parties must know the key and therefore can accidentally divulge the key to the wrong person. In large IPsec deployments this means either that al peers use the same key (meaning you have X number potential leaks) or that each peer-pair must have a unique key (which is a management night mare).