---
title: "Performance, capacity, scalability, and efficiency"
chapter: "21"
---

# Performance, capacity, scalability, and efficiency

Performance architecture connects workload shape to latency, throughput, concurrency, resource use, queues, and cost. Scalability means maintaining objectives as a dimension grows—not merely adding servers.

## Core ideas and patterns

| Idea | What it solves | Cost, limit, or warning |
|---|---|---|
| **Scale up/out** | Adds resources to one node or more nodes | State and coordination can limit horizontal scale |
| **Caching** | Trades freshness and memory for lower latency/load | Invalidation, stampede, and authorization are hard |
| **Read replica** | Scales read workloads | Replication lag changes observed consistency |
| **Partitioning** | Spreads state and processing | Skew and cross-partition work dominate |
| **Queue-based load leveling** | Buffers bursts for bounded consumers | Adds delay and backlog risk |
| **Backpressure** | Signals producers to slow or bound work | Requires propagation across the whole path |
| **Batching** | Amortizes overhead across items | Increases wait time and partial-failure complexity |
| **CDN** | Caches content near consumers | Purge, personalization, and security constraints apply |
| **Precomputation** | Moves work before the request | Freshness and storage cost increase |

## How to apply it

Define workload model: arrival rate, burst, payload, mix, growth, locality, concurrency, and dependency limits. Budget end-to-end latency. Apply Little’s Law to queues. Find the bottleneck with measurement, then change one constraint at a time. Preserve load-test realism and production safeguards.

## Evidence, not opinion

Track p50/p95/p99, throughput, utilization, saturation, queue depth/age, hit rate, eviction, skew, garbage collection, cost, and quality under overload. Test beyond the knee to verify graceful failure.

## Small example

At 500 requests/s and 200 ms average in-system time, average concurrency is about 100. A 50-connection database pool cannot support every request holding a connection for the full path.

## Feynman check

Explain the design to a new engineer without using the pattern names. State the
problem, the forces that conflict, the chosen boundary or mechanism, what can
fail, and the evidence that would prove the choice still works.