We're currently evolving the .NET microservices guidance and eShopOnContainers reference application. One of the most important topics is about the API Gateway pattern, why it is interesting for many microservice-based applications but also, how you can implement it in a .NET Core based microservice application with a deployment based on Docker containers.
Why should you use API Gateways instead of direct communication?
In a microservices architecture, the client apps usually need to consume functionality from more than one microservice. If that consumption is performed directly, the client will need to handle multiple microservice endpoints to call. What happens when the application evolves and new microservices are introduced or existing microservices are updated? If your application has many microservices, handling so many endpoints from the client apps can be a nightmare and since the client app would be coupled to those internal endpoints, evolving the microservices in the future can cause high impact for the client apps.
Therefore, having an intermediate level or tier of indirection (Gateway) can be very convenient for microservice-based applications. If you don't have API Gateways, the client apps must send requests directly to the microservices and that raises problems, such as the following issues.
- Coupling: Without the API Gateway pattern the client apps are coupled to the internal microservices. The client apps need to know how the multiple areas of the application is decomposed in microservices. When evolving and refactoring the internal microservices that will impact the client apps and it will be hard to maintain as the client apps need to keep track of the multiple microservice endpoints.
- Too many round-trips: A single page/screen in the client app might require several calls to multiple services. That can result in multiple network round trips between the client and the server, adding significant latency. Aggregation handled in an intermediate level could improve the performance and user experience for the client app.
- Security issues: Without a gateway, all the microservices must be exposed to the "external world", making larger the attack surface than if you hide internal microservices not directly used by the client apps. The smaller is the attack surface, the more secure your application can be.
- Cross-cutting concerns: Each publicly published microservice must handle concerns such as authorization, SSL, etc. In many situations those concerns could be handled in a single tier so the internal microservices are simplified.
Ocelot: A lightweight and Open Source API Gateway. Great fit to start learning this pattern with .NET Core microservices
Ocelot is an Open Source .NET Core based API Gateway especially made for microservices architecture that need unified points of entry into their system. It is lightweight, fast, scalable and provides routing and authentication among many other features.
The main reason why Ocelot was chosen to be used in eShopOnContainers reference application is because it is a .NET Core lightweight API Gateway that you can deploy into the same application deployment environment where you are deploying your microservices/containers, such as a Docker Host, Kubernetes, Service Fabric, etc. and since it is based on .NET Core it is cross-platform allowing you to deploy on Linux or Windows.
After the initial architecture and patterns explanation sections, the next sections explain how to implement API Gateways with Ocelot.
What is the API Gateway pattern
When you design and build large or complex microservice-based applications with multiple client apps, a good approach to consider can be an API Gateway. This is a service that provides a single entry-point for certain groups of microservices. It is similar to the Facade pattern from object‑oriented design, but in this case, it is part of a distributed system. A variation of the API Gateway pattern is also known as the "backend for frontend" (BFF) because you might create multiple API Gateways depending on the different needs from each client app.
Therefore, the API gateway sits between the client apps and the microservices. It acts as a reverse proxy, routing requests from clients to services. It can also provide additional cross-cutting features such as authentication, SSL termination, and cache.
The next figure shows how a custom API Gateway can fit into a microservice-based architecture.
Using an API Gateway implemented as a custom Web API service
In the previous example, the API Gateway would be implemented as a custom Web API or ASP.NET WebHost service running as a container.
It is important to highlight that in that diagram, you would be using a single custom API Gateway service facing multiple and different client apps. That fact can be an important risk because your API Gateway service will be growing and evolving based on many different requirements from the client apps. Eventually, it will be bloated because of those different needs and effectively it could be pretty similar to a monolithic application or monolithic service. That is why it is very much recommended to split the API Gateway in multiple services or multiple smaller API Gateways, one per form-factor type, for instance.
You need to be careful with the API Gateway pattern. Usually it isn't a good idea to have a single API Gateway aggregating all the internal microservices of your application. If it does, it acts as a monolithic aggregator or orchestrator and violates microservice autonomy by coupling all the microservices.
Therefore, the API Gateways should be segregated based on business boundaries and the client apps and not act as a single aggregator for all the internal microservices.
When splitting the API Gateway tier into multiple API Gateways, if your application has multiple client apps, that can be a primary pivot when identifying the multiple API Gateways types, so that you can have a different façade for the needs of each client app. This case is a pattern named "Backend for Frontend" (BFF) where each API Gateway can provide a different API tailored for each client app type, possibly even based on the client form factor by implementing specific adapter code which underneath calls multiple internal microservices, as shown in the following image.
The previous image shows a simplified architecture with multiple fine-grained API Gateways. In this case the boundaries identified for each API Gateway are based purely on the "Backend for Frontend" (BFF) pattern, hence based just on the API needed per client app. But in larger applications you should also go further and create additional API Gateways based on business boundaries as a second design pivot.
Main features in the API Gateway pattern
An API Gateway can offer multiple features. Depending on the product it might offer richer or simpler features, however, the most important and foundational features for any API Gateway are the following design patterns.
Reverse proxy or gateway routing. The API Gateway offers a reverse proxy to re-direct or route requests (layer 7 routing, usually Http requests) to the endpoints of the internal microservices. The gateway provides a single endpoint or URL for the client apps and then internally maps the requests to a group of internal microservices. This routing feature helps to decouple the client apps from the microservices but it is also pretty convenient when modernizing a monolithic API by sitting the API Gateway in between the monolithic API and the client apps, then you can add new APIs as new microservices while still using the legacy monolithic API until it is split into many microservices in the future. Because of the API Gateway, the client apps won't notice if the APIs being used are implemented as internal microservices or a monolithic API and more importantly, when evolving and refactoring the monolithic API into microservices, thanks to the API Gateway routing, client apps won't be impacted with any URI change.
For more information check the Gateway routing pattern information.
Depending on the API Gateway product you use, it might be able to perform this aggregation. However, in many cases it is more flexible to create aggregation microservices under the scope of the API Gateway, so you define the aggregation in code (i.e. C# code).
For more information check the Gateway aggregation pattern information.
Cross-cutting concerns or gateway offloading. Depending on the features offered by each API Gateway product, you can offload functionality from individual microservices to the gateway which will simplify the implementation of each microservice by consolidating cross-cutting concerns into one tier. This is especially convenient for specialized features that can be complex to implement properly in every internal microservice such as the following functionality.
- Authentication and authorization
- Service discovery integration
- Response caching
- Retry policies, circuit breaker and QoS
- Rate limiting and throttling
- Load balancing
- Logging, tracing, correlation
- Headers, query strings and claims transformation
- IP whitelisting
There can be many more cross-cutting concerns offered by the API Gateways products depending on each implementation, but those are the most common features. For instance, Azure API Management offers most of those features plus many more advanced features very useful for commercial APIs. However, for simpler approaches a lightweight API Gateway like Ocelot is pretty flexible as you can deploy it to your selected environment (any orchestrator) along with your microservices.
For more information check the Gateway offloading pattern information.
Aggregation and composition pattern
Implementing API Gateways with Ocelot
In the reference microservice application eShopOnContainers, it is using Ocelot because it is a simple and lightweight API Gateway that you can deploy anywhere along with your microservices/containers such as in the following environments used by eShopOnContainers.
- Docker host, in your local dev PC, on-premises or in the cloud
- Kubernetes cluster, on-premises or in managed cloud such as Azure Kubernetes Service (AKS)
- Service Fabric cluster, on-premises or in the cloud
- Service Fabric mesh, as PaaS/Serverless in Azure
Architecting and designing your API Gateways
The following architecture diagram shows how API Gateways are implemented with Ocelot in eShopOnContainers.
That diagram shows how the whole application is deployed into a single Docker host or development PC with "Docker for Windows" or "Docker for Mac". However, deploying into any orchestrator would be pretty similar but any container in the diagram could be scaled-out in the orchestrator and the infrastructure assets such as databases, cache and message brokers should be offloaded from the orchestrator and deployed into high available systems for infrastructure, like Azure SQL Database, Azure Cosmos DB, Azure Redis, Azure Service Bus, or any HA clustering solution on-premises.
As you can also notice in the diagram, having several API Gateways allows the multiple development teams to be autonomous (in this case Marketing vs. Shopping) when developing and deploying their microservices plus their own related API Gateways. If you had a single monolithic API Gateway that would mean a single point to be updated by multiple development teams which could couple all the microservices with a single part of the application.
Going much further in the design, sometimes a fine-grained API Gateway can also be limited to a single business microservice depending on the chosen architecture. Having the API Gateway's boundaries dictated by the business or domain will help you to get a better design. For instance, fine granularity in the API Gateway tier can be especially useful for more advanced composite UI applications based on microservices, because the concept of a fine-grained API Gateway is similar to a UI composition service. We discuss this topic in the section Creating composite UI based on microservices.
Therefore, for many medium- and large-size applications, using a custom-built API Gateway product is usually a good approach, but not as a single monolithic aggregator or unique central custom API Gateway unless that API Gateway allows multiple independent configuration areas for the multiple development teams with autonomous microservices.
Sample microservices/containers to re-route through the API Gateways
As an example, eShopOnContainers has around 6 internal microservice-types that have to be published through the API Gateways, as shown in the following image.
In the case of the Identity service, in our design we left it out of the API Gateway routing, but with Ocelot it is also possible to include it as part of the re-routing lists.
All those services are currently implemented as ASP.NET Core Web API services, as you can tell because of the code. Let's focus on one of the microservices like the Catalog microservice code.
You can see this is a pretty typical ASP.NET Core Web API project with several controllers and methods like in the following code.
The Http request will end up running that kind of C# code accessing the microservice database, etc.
In regards the microservice URL, when the containers are deployed in your local development PC (local Docker host), each microservice's container has always an internal port (usually port 80) specified in its dockerfile, as in the following partial dockerfile.
|FROM microsoft/aspnetcore:2.0.5 AS base
But that port is internal within the Docker host, so it cannot be reached by the client apps, only to the external ports (if any) provided when deploying with docker-compose.
Those external ports should not be published when deploying into production environment because that's why we're using the API Gateway, to hide the direct communication to the microservices.
However, when developing, it might be useful to be able to access the microservice/container directly and run it through Swagger. That's why in eShopOnContainers the external ports are still specified even when those won't be used by the API Gateway or the client apps.
Here's an example of a partial section of the docker-compose.override.yml file for the Catalog microservice.
You can see how in the docker-compose.override.yml configuration the internal port for the Catalog container is port 80, but the port for external access is 5101. But this port shouldn't be used by the application when using an API Gateway, only to debug run and test just the Catalog microservice.
Note that you usually won't be deploying with docker-compose into a production environment because the right production deployment environment for microservices is an orchestrator like Kubernetes or Service Fabric and when deploying to those environments you will use different configuration files where you won't publish directly any external port for the microservices but you will always use the reverse proxy from the API Gateway.
Let's run the catalog microservice in your local Docker host either by running the full eShopOnContainers solution from Visual Studio (it'll run all the services in the docker-compose files) or just starting the Catalog microservice with the following docker-compose command in CMD or PowerShell positioned at the folder where the docker-compose.yml and docker-compose.override.yml are placed.
|docker-compose run --service-ports catalog.api|
This command will only run the catalog.api service container plus its dependencies specified in the docker-compose.yml, in this case, the SQL Server container and RabbitMQ container.
Then you could directly access the Catalog microservice and see its methods through the Swagger UI accessing directly through that "external" port, in this case http://localhost:5101
At this point you could set a breakpoint in C# code in VS, test the microservice with the methods exposed in Swagger UI, etc. and finally clean-up everything with the "docker-compose down" command.
But this direct access communication to the microservice (in this case through the external port 5101) is precisely what you want to avoid in your application by setting the additional level of indirection of the API Gateway, Ocelot, in this case. Therefore, the client app won't directly access the microservice.
Implementing your API Gateways with Ocelot
Ocelot is basically a set of middlewares that you can apply in a specific order.
Ocelot is designed to work with ASP.NET Core only and it targets netstandard2.0. This means it can be used anywhere .NET Standard 2.0 is supported, including .NET Core 2.0 runtime and .NET Framework 4.6.1 runtime and up.
You install Ocelot and its dependencies in your ASP.NET Core project with Ocelot NuGet package.
In the case of eShopOnContainers, its API Gateway implementation is a very simple ASP.NET Core WebHost project, and Ocelot's middlewares handle all the API Gateway features, as shown in the following image.
This ASP.NET Core WebHost project is basically made with two simple files, the Program.cs and Startup.cs.
The Program.cs just needs to create and configure the typical ASP.NET Core BuildWebHost.
The important point here for Ocelot is the configuration.json file that you must provide to the builder through the AddJsonFile() method. That configuration.json is where you specify all the API Gateway ReRoutes, meaning the external endpoints and ports and the correlated internal endpoints and internal ports.
There are two sections to the configuration. An array of ReRoutes and a GlobalConfiguration. The ReRoutes are the objects that tell Ocelot how to treat an upstream request. The Global configuration allows overrides of ReRoute specific settings. It's useful if you don't want to manage lots of ReRoute specific settings.
Here's a simplified example of ReRoute configuration file from one of the API Gateways from eShopOnContainers (web-bff-shopping).
The main functionality of an Ocelot API Gateway is to take incoming http requests and forward them on to a downstream service, currently as another http request. Ocelot's describes the routing of one request to another as a ReRoute.
For instance, let's focus on one of the ReRoutes in the configuration.json from above, the configuration for the Basket microservice.
The DownstreamPathTemplate, Scheme and DownstreamHostAndPorts make the internal microservice URL that this request will be forwarded to.
The port is the internal port used by the service. When using containers, the port specified at its dockerfile.
The Host will be a service name that will depend on the service name resolution you are using. When using docker-compose, the services names are provided by the Docker Host which is using the service names provided in the docker-compose files. If using an orchestrator like Kubernetes or Service Fabric, that name should be resolved by the DNS or name resolution provided by each orchestrator.
DownstreamHostAndPorts is an array that contains the host and port of any downstream services that you wish to forward requests to. Usually this will just contain one entry but sometimes you might want to load balance requests to your downstream services and Ocelot lets you add more than one entry and then select a load balancer. But if using Azure and any orchestrator it is probably a better idea to load balance with the cloud and orchestrator infrastructure.
The UpstreamPathTemplate is the URL that Ocelot will use to identify which DownstreamPathTemplate to use for a given request from the client. Finally, the UpstreamHttpMethod is used so Ocelot can distinguish between different requests (GET, POST, PUT) to the same URL and is obviously needed to work.
But as introduced in the architecture and design section, if you really want to have autonomous microservices it might be better to split that single monolithic API Gateway into multiple API Gateways and/or BFF (Backend for Frontend). For that purpose, let's see how to implement that approach with Docker containers.
Using a single Docker container image to run multiple different API Gateway / BFF containers
In eShopOnContainers we're taking advantage of a single Docker container image with the Ocelot API Gateway but then, at run time, we create different services/containers for each type of API-Gateway/BFF by providing a different configuration.json file for each container, in runtime.
Therefore, in eShopOnContainers, the "Generic Ocelot API Gateway Docker Image" is created with the project named "OcelotApiGw" and the image name "eshop/ocelotapigw" that is specified in the docker-compose.yml file. Then, when deploying to Docker, there will be four API-Gateway containers created from that same Docker image, as shown in the following extract from the docker-compose.yml file.
Additionally, and as you can see in the docker-compose.override.yml file, the only difference between those API Gateway containers is the Ocelot configuration file which is different for each service container and specified at runtime through a Docker volume, as shown in the following docker-compose.override.yml file.
By splitting the API Gateway into multiple API Gateways, different development teams focusing on different subsets of microservices can manage their own API Gateways by using independent Ocelot configuration files while re-using the same Ocelot Docker image.
Now, if you run eShopOnContainers with the API Gateways (included by default in VS when opening eShopOnContainers-ServicesAndWebApps.sln solution or if running "docker-compose up"), the following sample routes will be performed.
For instance, when visiting the upstream URL http://localhost:5202/api/v1/c/catalog/items/2/ served by the webshoppingapigw API Gateway, you get the result from the internal Downstream URL http://catalog.api/api/v1/2 within the Docker host, as in the following browser.
Because of testing or debugging reasons, if you wanted to directly access to the Catalog Docker container (only at the development environment) without passing through the API Gateway, since "catalog.api" is a DNS resolution internal to the Docker host (service discovery handled by docker-compose service names), the only way to directly access the container is through the external port published in the docker-compose.override.yml, which is provided only for development tests, such as http://localhost:5101/api/v1/Catalog/items/1 in the following browser.
But the application is configured so it accesses all the microservices through the API Gateways, not though the direct port "shortcuts".
The Gateway aggregation pattern in eShopOnContainers
As introduced previously, a very flexible way to implement requests aggregation is with custom services, by code. You could also implement request aggregation with the Request Aggregation feature in Ocelot, but it might not be as flexible as you need. Therefore, the selected way to implement aggregation in eShopOnContainers is with an explicit ASP.NET Core Web API services for each aggregator. So, the API Gateway composition diagram is in reality a bit more extended when taking into account the aggregator services that are not shown in the simplified global architecture diagram shown previously. In the following diagram you can also see how the aggregator services work with their related API Gateways.
Zooming in into the diagram in the following image, you can notice how for the "Shopping" business area, the client apps could be improved by reducing chattiness with microservices by implementing those aggregator services under the realm of the API Gateways.
You can notice how when the diagram shows the possible requests coming from the API Gateways it can get pretty complex. Although you can see how the arrows in blue would be simplified, from a client apps perspective, when using the aggregator pattern by reducing chattiness and latency in the communication, ultimately significantly improving the user experience for the remote apps (mobile and SPA apps), especially.
In the case of the "Marketing" business area and microservices, it is a very simple use case so there was no need to use aggregators, but it could also be possible, if needed.
Authentication and authorization in Ocelot API Gateways
In an Ocelot API Gateway you can sit the authentication service, such as an ASP.NET Core Web API service using IdentityServer providing the auth token, either out or inside the API Gateway.
Since in eShopOnContainers is using multiple API Gateways with boundaries based on BFF and business areas, the Identity/Auth service is left out of the API Gateways, as highlighted in yellow in the following diagram.
However, Ocelot also supports to sit the Identity/Auth microservice within the API Gateway boundary, as in this other diagram.
Since in eShopOnContainers we have split the API Gateway into multiple BFF (Backend for Frontend) and business areas API Gateways, another option would had been to create an additional API Gateway for cross-cutting concerns. That choice would be fair in a more complex microservice based architecture with multiple cross-cutting concerns microservices. But sin we have just one cross-cutting concerns in eShopOnContainers it was decided to just handle the security service out of the API Gateway realm, for simplicity's sake.
In any case, the authentication module of Ocelot API Gateway will be visited at first when trying to use any secured microservice (if secured at the API Gateway level). That will re-direct to visit the Identity or auth microservice to get the access token so so you can visit the protected services with the access_token.
The way you secure with authentication any service at the API Gateway level is by setting the AuthenticationProviderKey in its related settings at the configuration.json
When Ocelot runs, it will look at the ReRoutes AuthenticationOptions.AuthenticationProviderKey and check that there is an Authentication Provider registered with the given key. If there isn't, then Ocelot will not start up. If there is, then the ReRoute will use that provider when it executes.
Because the Ocelot WebHost is configured with the authenticationProviderKey = "IdentityApiKey", that will require authentication whenever that service has any requests without any auth token.
Then, you also need to set authorization with the [Authorize] attribute on any resource to be accessed, like a microservice, such as in the following Basket microservice controller.
The ValidAudiences such as "basket" are correlated with the audience defined in each microservice with AddJwtBearer() at the ConfigureServices()of the Startup class, such as the code below.
Now, if you try to access any secured microservice like the Basket microservice with a ReRoute URL based on the API Gateway like http://localhost:5202/api/v1/b/basket/1
then you'll get a 401 Unauthorized unless you provide a valid token. On the other hand, if a ReRoute URL is authenticated, Ocelot will invoke whatever downstream scheme is associated with it (the internal microservice URL).
Authorization at Ocelot's ReRoutes tier. Ocelot supports claims-based authorization evaluated after the authentication. You set the authorization at a route level by adding the following to you ReRoute configuration.
In that example, when the authorization middleware is called, Ocelot will find if the user has the claim type "UserType" in the token and if the value of that claim is "employee". If it isn't then the user will not be authorized and the response will be 403 forbidden.
Additional cross-cutting features in an Ocelot API Gateway
There are other important features to research and use when using an Ocelot API Gateway which are described in the following links.
Service discovery in the client side integrating Ocelot with Consul or Eureka
Caching at the API Gateway tier
Logging at the API Gateway tier
Quality of Service (Retries and Circuit breakers) at the API Gateway tier
Usage from customers in real life
A final comment about Ocelot. It is not just a very didactic API Gateway, it also has large customers like Tencent (owns WeChat in China) using .NET Core services with high scalability needs.
Wow! This was a very large post, but I wanted to have the most important topics (patterns + implementation) related to API Gateways within a single blog post so anyone can provide feedback so this content can be improved.
In the official guidance documentation, I might split it in two, first the API Gateway pattern and theory, then a second part on the implementation with Ocelot.
I hope it is interesting and useful for the reader and please, feel free to post your feedback! 🙂