Azure offers many different options for deploying cloud workloads. You can deploy Virtual Machines or use Azure App Service. You can even use App Service in a virtual network by deploying Azure App Service Environment (ASE), which is also available in the US Government Cloud. In this blog post, I will explore another deployment option - Kubernetes. Specifically, I will discuss how to deploy a Kubernetes cluster in Azure Government and host a web application in it.
There is already some documentation on deploying Kubernetes to Azure Government. You can also watch this video on how to all of this works. The main purpose of this blog post is to provide a bit more detail on the instructions and provide some tools and examples to get you started.
Kubernetes is an open source orchestration engine for automated deployment, scaling, and management of containerized applications. If you are not familiar with containers, I recommend running through an introduction to containers before proceeding. It is easy to host Kubernetes in Azure. There is a tool called ACS Engine (Azure Container Service Engine), which you can use to generate templates for deploying clusters into Azure. ACS Engine actually supports several different orchestrators (Kubernetes, OpenShift, DC/OS, etc.), but we will focus on Kubernetes here.
In Azure Commercial, there is now a fully managed offering for Kubernetes called Azure Kubernetes Service (AKS). If you are using AKS, the platform will handle several operations and management tasks for you, e.g. patching of the OS on the nodes, etc. AKS is a great option for many types of workloads, but it is not yet available in Azure Government and it does not offer the configuration flexibility that ACS Engine does. Consequently, we will be taking a closer look at deploying with ACS Engine. By the end of this blog post, I will have demonstrated:
- How to deploy a Kubernetes cluster to Azure Government.
- How to interact with the cluster (using
- How to view the web dashboard for the cluster.
- How to deploy a web application in the cluster.
Let's get started. I have created a GitHub repository with some notes and utilities for ACS Engine. All the code samples, etc. that we will be using can be found there. I am assuming that you have the following tools installed:
You need a few additional tools to be able to deploy and manage Kubernetes clusters.
Install ACS Engine:
#Download binary, replace with latest version wget https://github.com/Azure/acs-engine/releases/download/v0.17.0/acs-engine-v0.17.0-linux-amd64.zip #Unzip and copy to location in path unzip acs-engine-v0.17.0-linux-amd64.zip mkdir -p ~/bin cp acs-engine-v0.17.0-linux-amd64/acs-engine ~/bin/
To be able to interact with your cluster, you will need
#Download the latest version curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl #Make it executable: chmod +x ./kubectl #Copy to bin, you could also move it to /usr/bin mv kubectl ~/bin/
Deploying a Kubernetes Cluster in Azure Government
You are now ready to follow the instructions on the ACS Engine site, and I will give a few specific hints to make it a smooth operation in Azure Government.
Clone the scripts and notes to have a local copy:
git clone https://github.com/hansenms/acs-utils.git
Make sure you are logged into Azure Government using the Azure CLI:
az cloud set --name AzureUSGovernment az login
This will take you through the device login procedure. If you have multiple subscriptions, make sure you pick the right one with:
az account set -s SUBSCRIPTION-ID-OR-NAME
Then we can create a resource group and a service principal for the Kubernetes cluster. We will grant the service principal Contributor rights on the resource group to allow it to create and manage resources in the cluster:
subscription=$(az account show | jq -r .id) #Name and Location rgname=mykubernetes loc=usgovvirginia #Create Group rg=$(az group create --name $rgname --location $loc) rgid=$(echo $rg | jq -r .id) #Create the Service Principal and assign as contributor on group sp=$(az ad sp create-for-rbac --role contributor --scopes $rgid)
Next step is to define the cluster. ACS Engine uses an cluster definition, a.k.a. api model, file (json format) to define the cluster options. There are a number of example definitions (api models) available in the ACS Engine examples, and there are a few included in the acs-utils GitHub project. The api model file needs to be populated with the DNS name for the cluster and the details for the service principal created above. It is pretty easy to do that manually with some copy and paste, but I have also created a utility script (convert-api.sh) that will take an api model file and populate it with the details. You can use the script like this:
./scripts/convert-api.sh -c $(echo $sp | jq -r .appId) \ -s $(echo $sp | jq -r .password) -f kubernetes.json \ -d myClusterDns | jq -M . > converted.json
After this, the cluster definition is in
converted.json and ready to be deployed:
acs-engine deploy --api-model converted.json \ --subscription-id $subscription --resource-group $rgname \ --location $loc --azure-env AzureUSGovernmentCloud
You may be prompted to go through the device login procedure. After deployment (which will take 10 minutes or so), you will find a folder called
_output/myClusterDns (replace your DNS name). This folder will contain a bunch of files including the actual Resource Manager template, that was used for deployment. For the purposes of this demo, we will need the
_output/myClusterDns/kubeconfig/kubeconfig.REGIONNAME.json file, which contains your credentials, etc. for interacting with the cluster.
Your available Kubernetes configurations are usually stored in
~/.kube/config. You can also specify a particular kubeconfig by setting the
KUBECONFIG environment variable. If you would like to merge the configuration of your new cluster into your existing
~/.kube/config, you could do something like this:
#Merge configuration with your current kube config KUBECONFIG=~/.kube/config:_output/myClusterDns/kubeconfig/kubeconfig.usgovvirginia.json \ kubectl config view --flatten > new-config #Always good to back up cp ~/.kube/config ~/config-backup #Put the new merged config in place cp new-config ~/.kube/config
Now you should be able to list your "contexts" and see that your new cluster is in the list:
kubectl config get-contexts
You can select the new cluster with:
kubectl config use-context CONTEXT-NAME
Finally validate that you can interact with the cluster by sending a command, e.g.:
kubectl get nodes
which will give you a list of the current nodes in the cluster (there should be a master and 2 agents if you are following the tutorial here):
NAME STATUS ROLES AGE VERSION k8s-agentpool1-15861492-0 Ready agent 12m v1.10.2 k8s-agentpool1-15861492-1 Ready agent 13m v1.10.2 k8s-master-15861492-0 Ready master 13m v1.10.2
Accessing the Web Dashboard
Kubernetes has a nice web dashboard for monitoring and managing the cluster. You can access it by opening up a proxy:
Then point your browser to
http://localhost:PROXY-PORT/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/. You will need a token to sign in. I have made a convenience script:
This will display a token that you can copy and paste to log in. You should see the dashboard:
Deploying a Workload
Workloads are deployed in Kubernetes clusters using manifest files. If you are completely new to Kubernetes, have a look at one of the introductory tutorials. I have included some simple manifest files in the GitHub repository. As an example, you can deploy an ASP.NET Core Web App with the web-service.yaml manifest:
kubectl apply -f manifests/web-service.yaml
This will deploy the web application itself in a pod. You can see the list of pods running with:
kubectl get pods
You should see something like:
NAME READY STATUS RESTARTS AGE web-service-674f64575b-xzflq 1/1 Running 0 1m
There is also a frontend service (a load balancer) that exposes the web application outside the cluster. You can list the services:
kubectl get svc
And you should see the running services:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE frontend-service LoadBalancer 10.0.189.218 13.XX.XX.X 80:30353/TCP 3m kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 1h
Copy the EXTERNAL-IP of the frontend-service and point your browser to
http://EXTERNAL-IP and you should see the web application:
Conclusions and Next Steps
In this blog post, we have looked at how easy it is to set up a Kubernetes cluster in Azure Government and deploy the first workload to it.
There is still work to be done. If you try to access the web application with HTTPS, you will notice that we need a valid certificate. To make that work well, we will need to set up an Ingress Controller and Ingress rules. I cover that in a this blog post on Ingress in Kubernetes. It may also be a requirement to keep Government workloads on internal/private IP space. I will also cover that scenario in a later blog post.
For now, have some fun setting up Kubernetes in Azure Government (or Commercial) and let me know if you have questions/comments/suggestions.