Service Mesh

Hello, I hope you are doing well.

Today I would like to tell you about service mesh. To better understand it, let’s start with a simple architecture: the monolith. In its simplest form, the monolith architecture involves clients accessing the backend directly through a user interface.

As your application and team grow, some organizations choose to change their application architecture to microservices. The services will be separated based on their domains. Here is a simplified example of the architecture.

Instead of directly accessing the backend, the client needs to access the reverse proxy, which then routes the request to the appropriate service. The reverse proxy typically also handles the role of a load balancer, distributing traffic across multiple nodes for a service.

In an ideal scenario, each service is not supposed to communicate directly with others. However, in my experience, this was inevitable due to various reasons such as accessing legacy services or poorly defined service boundaries.

To communicate with another service, such as the order and payment service needing to perform a product lookup, the flow would be as follows:

1. The request goes to the reverse proxy.
2. The reverse proxy routes and load balances the request to the service.

As you can imagine, this increases the load on the reverse proxy, which is responsible for serving both north-south and east-west traffic. If your application reaches this point, there are signs indicating you should consider using a service mesh:

1. There are private services that do not need to be exposed to the reverse proxy.
2. East-west traffic is overwhelming the reverse proxy.
3. You need to improve service-to-service communication latency.
4. Updating the reverse proxy becomes inefficient.
5. You need to enhance the security of your east-west traffic.

Even though you can separate the load balancer and the reverse proxy, as shown in the following figure, you will only address problems 1 and 2.

The following depicts the same architecture with a service mesh.

As you can observe, instead of routing through the reverse proxy/load balancer, the product lookup goes directly to the service node. What’s even better, the request is secured using mTLS. With this setup, there’s no longer a need to expose private services to the reverse proxy, east-west traffic only occurs behind the reverse proxy, service-to-service latency is reduced due to direct calls to the service node, and the traffic is more secure because a service mesh like Consul uses mTLS by default.

How is this achieved? First, you need to set up a service mesh agent for each service that will utilize the service mesh. Below is an example of the Consul configuration. Whenever a new node comes online with the service mesh agent as a sidecar, the agent registers itself with the service mesh server. This communication occurs in the control plane. Consequently, if another service needs to call yours, it will inquire with the service mesh agent and can then directly call the service. However, even though the service-to-service communication occurs directly with the server node, the request actually passes through the service mesh agent before being forwarded to the service. This is where the agents generate the mTLS certificate to secure the request. This process takes place in the data plane.

Here’s an example of a Consul configuration:

{
  "node_name": "order-uuid",
  "retry_join": ["10.0.0.10"],
  "service": {
    "name": "order",
    "port": 80
  },
  "connect": {
    "sidecar_service": {
      "proxy": {
        "upstreams": [
          {
            "destination_name": "product",
            "local_bind_port": 5000
          }
        ]
      }
    }
  }
}

Service Mesh Open Source

Here are some open-source service mesh options for you to explore.

That’s all for now. I hope this gives you a better understanding of what a service mesh is and when it’s beneficial to use.