Load Balancing: From L4/L7 to eBPF and Service Meshes

A survey of the layers, algorithms, and operational patterns of traffic distribution, from L4/L7 through eBPF and service meshes.

Overview

Load balancing distributes incoming network traffic evenly across multiple servers to secure availability, performance, and scalability. It is what makes horizontal scaling possible beyond the limits of a single server, and it is standard practice in any service that takes significant traffic.

eBPF (extended Berkeley Packet Filter) based load balancing, as implemented by Cilium, replaces the linear rule scan of iptables with a hash table lookup. The gap widens as the number of services grows. Application-level traffic management is increasingly handled by a service mesh.

L4 vs L7 load balancing

L4 load balancer (transport layer)

Distributes traffic at the TCP/UDP level using only IP addresses and port numbers.

Characteristics:

• Inspects packet headers only, so processing is very fast
• Cannot see request content (HTTP headers, URLs, and so on)
• Operates on NAT (Network Address Translation)
• Cannot perform TLS termination (encrypted content is unreadable at L4)

Common implementations:

• AWS NLB (Network Load Balancer)
• HAProxy (TCP mode)
• IPVS (Linux Virtual Server)
• Cilium (eBPF based)

L7 load balancer (application layer)

Distributes traffic at the HTTP/HTTPS level by inspecting request content such as URLs, headers, and cookies.

Characteristics:

• Supports fine-grained routing based on request content
• Handles TLS termination, compression, caching, authentication, and similar functions
• Slower than L4 but distributes more intelligently
• Can handle protocol-specific traffic such as WebSocket and gRPC

Common implementations:

• AWS ALB (Application Load Balancer)
• NGINX
• HAProxy (HTTP mode)
• Envoy Proxy
• Traefik

L4 vs L7 comparison

Load balancing algorithms

1. Round robin

Distributes requests sequentially across the server list.

• Advantage: simple to implement, distributes evenly
• Drawback: ignores differences in server capacity

2. Weighted round robin

Assigns a weight to each server so that more capable servers receive more requests.

3. Least connections

Sends the request to the server with the fewest active connections.

• Advantage: reflects server load in real time
• Drawback: connections can pile up on a slow server

4. IP hash

Hashes the client IP so that the same client always reaches the same server.

• Advantage: guarantees session persistence
• Drawback: rebalancing problems when servers are added or removed

5. Consistent hashing

Uses a hash ring so that adding or removing a server relocates only a minimal set of keys.

Servers A, B, C, D and keys 1 through 4 sit clockwise on the hash ring. When server B is removed, only keys 2 and 3 move to server C and the remaining keys are unaffected.

Virtual nodes: A single physical server maps to several virtual nodes on the hash ring, which produces a more even distribution.

Health checks and circuit breaking

Health checks

The load balancer checks backend server status periodically so that traffic never reaches an unhealthy server.

A well-formed health check endpoint:

// /health or /healthz
app.get('/health', async (req, res) => {
  const checks = {
    database: await checkDatabase(),
    redis: await checkRedis(),
    externalApi: await checkExternalApi(),
  };
 
  const isHealthy = Object.values(checks).every(c => c.status === 'up');
 
  res.status(isHealthy ? 200 : 503).json({
    status: isHealthy ? 'healthy' : 'unhealthy',
    checks,
    timestamp: new Date().toISOString(),
  });
});

checks collects the status of the database, Redis, and the external API. If any one of them is not up, isHealthy becomes false and the endpoint returns 503, so the load balancer pulls the instance with a failing dependency out of rotation.

Circuit breaker

Temporarily blocks calls to a failing service to prevent cascading failure.

Service mesh

Concept

A dedicated layer that manages network communication between microservices at the infrastructure level. It provides load balancing, service discovery, traffic management, security (mTLS), and observability without changing application code.

Comparing the major service meshes

Envoy Proxy

A CNCF graduated project and the high-performance proxy used as the data plane in most service meshes.

Main capabilities:

• L3/L4 filters: TCP proxying, rate limiting
• L7 filters: HTTP routing, gRPC, WebSocket
• Service discovery: xDS API
• Load balancing: round robin, least request, ring hash
• Observability: distributed tracing, metrics, access logs

eBPF-based load balancing

Concept

eBPF (extended Berkeley Packet Filter) performs load balancing directly in the Linux kernel. It replaces iptables or sidecar proxies, shortening the packet path and turning rule lookup from a linear scan into a hash table lookup.

Load balancing in Cilium

Cilium replaces kube-proxy with eBPF to load balance Kubernetes services.

Capabilities:

• North-south LB: external traffic into the cluster, using XDP
• East-west LB: communication between services inside the cluster
• Direct server return (DSR): response packets bypass the load balancer and go straight to the client
• Maglev consistent hashing: consistent hashing based on the Google Maglev paper
• L7 load balancing: L7 routing through TPROXY in combination with Envoy

Performance comparison:

How much improves depends heavily on the workload and the number of rules. In CNCF benchmarks, P99 latency drops 30 to 60% once the cluster passes 1,000 services. There are also reports of a widening throughput gap in environments with 100 network policies. The source is the CNCF/Cilium benchmark. In small environments with few services, kube-proxy can be the faster option.

Global Server Load Balancing (GSLB)

Concept

Distributes traffic across geographically separated data centers. It works through DNS, routing users to the nearest or healthiest data center.

Routing policies:

• Geo-proximity: the closest data center
• Latency-based: the data center with the fastest response
• Weighted: a fixed traffic ratio per data center
• Failover: switches to a backup data center when the primary fails

Tools:

• AWS Route 53: global DNS-based GSLB
• Cloudflare Load Balancing: edge network based
• F5 BIG-IP DNS: enterprise GSLB
• NS1: intelligent DNS traffic management

Combining autoscaling with load balancing

Kubernetes HPA (Horizontal Pod Autoscaler)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: payment-service-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: payment-service
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"

minReplicas and maxReplicas are the lower and upper bounds on pod count, and averageUtilization: 70 is the target that triggers scale-out once average CPU usage passes 70%. The second metric adds pods when HTTP requests per second per pod exceed 100.

Scaling strategies

WebSocket load balancing

WebSocket uses long-lived connections, which calls for different handling than ordinary HTTP load balancing.

Challenges

1. Session affinity: the same client must always connect to the same server
2. Connection draining: existing WebSocket connections must close safely when a server is removed
3. Uneven distribution: long-lived connections mean new servers receive no connections

Approaches

• Sticky sessions: affinity based on IP hash or cookies
• Redis pub/sub: broadcast messages between servers so behavior does not depend on session placement
• Connection redistribution: periodically prompt clients to reconnect

CDN and edge load balancing

CDN (Content Delivery Network)

Serves static and dynamic content from a location near the user through edge servers distributed worldwide.

Edge computing integration:

• Cloudflare Workers: serverless compute on the V8 engine
• AWS CloudFront Functions: lightweight function execution at the edge
• Vercel Edge Functions: edge computing integrated with Next.js

Running load balancing, authentication, caching, and A/B testing at the edge keeps origin server load to a minimum.

Summary

Load balancing is the work of combining layers (L4/L7), algorithms, and operational patterns to fit the workload. L4 distributes quickly using only IP and port, while L7 reads URLs and headers to handle content-based routing and TLS termination. Where servers change often, consistent hashing reduces relocation, and health checks plus circuit breakers isolate failing servers from traffic. Communication between microservices belongs to a service mesh and kernel-level distribution to eBPF, though the benefit of the latter grows with the number of services and rules. The next post covers monitoring and observability for tracking the state of distributed systems.

References

• Designing Data-Intensive Applications - Martin Kleppmann (Chapter 6)
• Envoy Proxy Documentation: https://www.envoyproxy.io/docs
• Cilium Documentation: https://docs.cilium.io
• NGINX Load Balancing Guide: https://docs.nginx.com/nginx/admin-guide/load-balancer