Secure Microservices with Consul Connect: A Practical Guide

Problem: Insecure Inter‑Service Traffic

In a typical microservice environment, services discover each other via DNS or a service registry, but the traffic between them is often left unencrypted. Once an attacker gains network access, they can sniff or tamper with traffic, compromising data integrity and confidentiality. Adding a separate service‑mesh tool adds operational complexity and duplicate configuration.

Takeaway

Consul Connect embeds Envoy sidecars and mutual TLS (mTLS) directly into the Consul service registry, giving you unified discovery, health checks, and secure communication without a separate mesh stack.

Why Consul Connect Works

Consul Connect automates sidecar injection and policy enforcement:

• Service discovery and health checks are already managed by Consul.
• Envoy sidecars are injected via the connect block, eliminating code changes.
• Policy‑based ACLs control which services may talk to each other.
• Zero‑trust is the default – all traffic must go through a sidecar.

Section 1: Unified Discovery & mTLS

When you enable Connect for a service, Consul automatically generates a short‑lived TLS certificate for the Envoy sidecar. The sidecar negotiates mTLS with peers before forwarding traffic to the actual application process. This removes the need for external TLS termination or manual certificate management.

Section 2: Fine‑Grained Policy Control

Connect policies are written in HCL and attached to services. A policy can allow or deny traffic to specific services or ports. Example policy:

# allow-service-b.hcl
service "service-b" {
  policy = "read"
}

Apply it with:

consul acl policy write -name allow-service-b -rules @allow-service-b.hcl
consul acl token create -description "Token for Service A" -policy-name allow-service-b

Use the token in Service A’s Connect block to enforce the policy.

Section 3: Zero‑Trust Default & Network Nuances

All traffic must go through a sidecar. This requires that the underlying network allows pods or containers to reach the sidecar on the same host. In Kubernetes, the default pod networking satisfies this. In VM‑based workloads, you must manually configure Envoy and expose the sidecar port.

Concrete Example: Deploying a Minimal Connect‑Enabled Service

Below is a step‑by‑step illustration of enabling Connect for two services in a Docker environment. Replace placeholders with your actual names.

1. Start a dev Consul cluster (>=1.10)

consul agent -dev -config-dir ./config
# Wait for the agent to be ready

2. Register Service A with Connect

consul services register -service service-a -port 8080 \
  -connect { sidecar_service: service-a }

3. Inject Envoy sidecar for Service A

docker run --name service-a-envoy \
  -e CONSUL_HTTP_ADDR=consul.service.consul:8500 \
  -v $(pwd)/envoy-config.yaml:/etc/envoy/envoy.yaml \
  consul/connect-envoy

Check that the sidecar is running:

consul connect envoy --status
# Expect output indicating "Envoy is listening on 127.0.0.1:15000"

4. Register Service B and apply an ACL that denies Service A

# Policy that denies any traffic from service-a
policy deny-service-a.hcl:
service "service-a" {
  policy = "deny"
}
consul acl policy write -name deny-service-a -rules @deny-service-a.hcl
consul acl token create -description "Token for Service B" -policy-name deny-service-a

5. Test communication

# From Service A attempt to call Service B
curl http://service-b.consul:9090/health
# Expect HTTP 403 or connection reset

Verify the failure is due to the ACL by checking Consul logs for "policy denied" messages.

Trade‑offs & Limitations

• Workload Support: Sidecar injection is automated for Docker and Kubernetes only. VM workloads require manual Envoy setup.
• TLS Rotation: Certificates rotate every 24 hours by default. If the rotation process fails, services may experience brief downtime.
• Routing Loops: Misconfiguring sidecar ports or network policies can create loops where traffic keeps bouncing between Envoy instances.
• Operational Overhead: While Connect reduces the need for a separate mesh, you still need to monitor Envoy sidecars and Consul health.

Actionable Checklist for Adopting Consul Connect

1. Run a test cluster with Consul Connect enabled for a subset of services.
2. Verify Envoy sidecar creation with consul connect envoy --status.
3. Write and attach Connect policies for each service.
4. Monitor TLS handshakes: consul connect envoy --stats or Envoy’s stats/prometheus endpoint.
5. Set up alerts for policy violations and sidecar failures.
6. Gradually roll out Connect to production workloads, ensuring network paths allow sidecar traffic.

By following these steps, you can secure inter‑service communication with minimal additional tooling while keeping discovery and health checks in a single place.