Why StatefulSets for Trading?

Trading systems are stateful:

• Order management systems track open orders
• Market makers maintain inventory positions
• Matching engines preserve order book state

Deployments are for stateless apps. StatefulSets are for stateful apps.

---

What You’ll Learn

By the end of this lesson, you’ll understand:

1. StatefulSets vs Deployments - When to use each
2. Pod identity - Stable hostnames and network IDs
3. Graceful shutdown - Draining orders before pod termination
4. Persistent storage - Surviving pod restarts

---

The Foundation: StatefulSets vs Deployments

Feature	Deployment	StatefulSet
Pod naming	Random (app-xyz123)	Ordered (app-0, app-1)
Scaling	All pods equal	Ordered startup/shutdown
Storage	Shared or none	Per-pod PersistentVolumes
Network	Random IPs	Stable DNS names
Use case	Stateless web apps	Databases, trading systems

---

The “Aha!” Moment

Here’s what makes StatefulSets essential for trading:

When a Deployment pod restarts, it gets a new identity. Your order manager loses track of which orders belong to which pod. With StatefulSets, trading-0 is always trading-0-even after restarts.

This stable identity enables:

• Consistent order routing
• Per-pod state persistence
• Predictable failover patterns

---

Let’s See It In Action: StatefulSet Definition

# trading-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: order-manager
spec:
  serviceName: "order-manager"
  replicas: 3
  selector:
    matchLabels:
      app: order-manager
  template:
    metadata:
      labels:
        app: order-manager
    spec:
      terminationGracePeriodSeconds: 60  # Time to drain orders
      containers:
      - name: order-manager
        image: trading/order-manager:v1.2
        ports:
        - containerPort: 8080
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "kill -SIGTERM 1 && sleep 30"]
        volumeMounts:
        - name: order-state
          mountPath: /data/orders
  volumeClaimTemplates:
  - metadata:
      name: order-state
    spec:
      accessModes: ["ReadWriteOnce"]
      storageClassName: "fast-ssd"
      resources:
        requests:
          storage: 50Gi

This creates:

• order-manager-0, order-manager-1, order-manager-2
• Each pod gets its own 50Gi persistent volume
• 60s grace period for order draining

---

Stable Network Identity

StatefulSets create DNS entries:

# Each pod gets a predictable DNS name
order-manager-0.order-manager.trading.svc.cluster.local
order-manager-1.order-manager.trading.svc.cluster.local
order-manager-2.order-manager.trading.svc.cluster.local

Your order routing can use these stable names:

# Route orders by symbol partition
def get_order_manager(symbol):
    partition = hash(symbol) % 3
    return f"order-manager-{partition}.order-manager:8080"

---

Graceful Shutdown: Draining Orders

Trading pods can’t just terminate. They must:

1. Stop accepting new orders
2. Complete pending orders
3. Persist state to disk
4. Terminate

spec:
  terminationGracePeriodSeconds: 120  # 2 minutes to drain
  containers:
  - name: order-manager
    lifecycle:
      preStop:
        exec:
          command:
          - /bin/sh
          - -c
          - |
            # 1. Signal app to stop accepting new orders
            kill -SIGUSR1 1
            # 2. Wait for pending orders to complete
            /app/wait-for-drain.sh
            # 3. Persist final state
            /app/checkpoint-state.sh

---

Common Misconceptions

Myth: “I can use Deployments and just save state to Redis.”
Reality: External state stores add latency and failure modes. For hot trading state (open orders, positions), local SSD with StatefulSet is faster and simpler.

Myth: “StatefulSets are slow to scale.”
Reality: StatefulSets scale one pod at a time by default (ordered). For trading, this is a feature-you don’t want 3 new order managers stealing positions simultaneously.

Myth: “PersistentVolumes survive node failures.”
Reality: Depends on storage class. EBS-backed volumes survive node failures. Local NVMe does not. Choose based on your durability requirements.

---

Headless Service for DNS

StatefulSets require a headless service:

apiVersion: v1
kind: Service
metadata:
  name: order-manager
  labels:
    app: order-manager
spec:
  clusterIP: None  # Headless!
  ports:
  - port: 8080
    name: http
  selector:
    app: order-manager

clusterIP: None tells Kubernetes not to load-balance-instead, DNS returns individual pod IPs.

---

AWS EKS Configuration

For trading on AWS:

# Use io2 EBS for low-latency persistent storage
storageClassName: io2

# Node affinity for dedicated trading nodes
affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: node-type
          operator: In
          values:
          - trading

# Pod anti-affinity: spread across AZs
podAntiAffinity:
  requiredDuringSchedulingIgnoredDuringExecution:
  - labelSelector:
      matchLabels:
        app: order-manager
    topologyKey: topology.kubernetes.io/zone

---

Practice Exercises

Exercise 1: Create a StatefulSet

kubectl apply -f trading-statefulset.yaml
kubectl get pods -w  # Watch ordered startup

Exercise 2: Test Pod Identity

# Delete a pod, watch it come back with same name
kubectl delete pod order-manager-1
kubectl get pods -w
# order-manager-1 returns (not random name)

Exercise 3: Graceful Shutdown

# Scale down and watch drain
kubectl scale statefulset order-manager --replicas=2
# order-manager-2 gets preStop signal first

---

Key Takeaways

1. StatefulSets = stable identity - Pod names survive restarts
2. Ordered scaling - One at a time, predictable
3. Per-pod storage - Each pod gets its own PersistentVolume
4. Graceful shutdown is mandatory - Use preStop hooks

---

What’s Next?

🎯 Continue learning: Trading System Metrics

🔬 Expert version: Kubernetes StatefulSets: Why Trading Systems Need State

Now you know how to run stateful trading workloads on Kubernetes. ⚓