What Is a Service Mesh?

Quick answer
A service mesh is a dedicated infrastructure layer that manages service-to-service communication.
It adds features like mutual TLS, retries, and traffic observability without changing application code.
For a small app it is usually overkill; its value appears with many services and strict security needs.

What is a service mesh?

A service mesh is a layer that controls and observes the traffic between services in a distributed system. Instead of each service implementing encryption, retries, and telemetry itself, the mesh handles these concerns in a proxy — traditionally a “sidecar” running next to each service — plus a control plane that configures those proxies. Istio and Linkerd are the best-known implementations.

How does it work?

Each service instance runs alongside a proxy (the data plane) that intercepts its inbound and outbound traffic. The control plane pushes policy — which services may talk, whether traffic is encrypted, how retries behave — to the proxies. This gives you mutual TLS between services, traffic splitting for canaries, and per-request metrics, all without modifying the services themselves.

Why does it matter?

A mesh centralizes cross-cutting networking concerns that would otherwise be re-implemented per service. That is especially valuable for zero trust postures: encrypting and authorizing every service-to-service call is hard to retrofit, and a mesh makes it the default.

When is it overkill?

A mesh adds a proxy to every pod, a control plane to operate, and meaningful latency and complexity. For one or two services, that is a poor trade. Most small teams should start without a mesh and adopt one only when they need uniform mTLS or fine-grained traffic control across many services.

Where this bites vibecoders
The vibecoder failure is adopting trendy infrastructure for its own sake: an AI assistant suggests Istio, and suddenly a three-service app runs a mesh “for security.” The discipline is to name the specific problem first — “I need mTLS between services” — and then pick the smallest tool that solves it, which is often not a mesh at all.

Where AI coding assistants get this wrong

Adding a mesh to a small app where plain Kubernetes networking suffices.
Generating mesh config that enables features (mTLS, retries) without the services being ready for them.
Introducing sidecar timeouts that break long-running requests silently.
Underestimating the operational burden of running the control plane.

Checklist

Name the specific problem before adopting a mesh.
Start with built-in platform features (NetworkPolicies, Ingress) for small apps.
Consider a mesh only for many services or uniform mTLS requirements.
Monitor the added latency the proxies introduce.
Operate the control plane deliberately — it is now production infrastructure.

FAQ

What is a sidecar proxy?

A sidecar is a proxy container deployed in the same pod as a service, intercepting its network traffic. The mesh control plane configures the sidecars. This pattern keeps networking logic out of application code, at the cost of an extra container per pod.

What is the difference between Istio and Linkerd?

Both are service meshes. Istio is feature-rich (Envoy-based, strong traffic management and policy) but heavier; Linkerd prioritizes simplicity and low resource overhead. For a first mesh, many teams find Linkerd easier to operate.

Do I need a service mesh for mTLS?

Not necessarily. You can encrypt service traffic with application-level TLS or platform features. A mesh is the right tool when you want encryption and policy uniformly across many services without changing each one.