Resilience

Overview Retries mask transient failures by having a client re-attempt a request that timed out or errored, trading latency for reliability. Load amplification is what happens when those same retries compound: a struggling downstream service receives multiplied traffic from many clients retrying at once, turning a partial slowdown into a full outage. The design challenge is keeping the first from causing the second. Comparison Diagram RetriesLoad AmplificationClientServicereq 1 (timeout)retry (backoff+jitter)1 extra request, delayedC1C2C3Service6x request volume, queue growing Comparison Table Aspect Retries Load Amplification Trigger A single failed or timed-out request at the client Many clients (or one client’s retries) hitting an already degraded service Nature Deliberate resilience mechanism Emergent side effect of that mechanism under stress Scope Per-request, client-local decision Fleet-wide, system-level consequence Timing pattern Delayed re-attempt, ideally with exponential backoff and jitter Requests pile up faster than the service can drain them Effect on downstream load Small, bounded increase of one extra attempt Multiplicative increase, often several times baseline traffic Worst-case outcome Slightly higher latency for the caller Cascading failure or full outage from a retry storm Mitigation Retry budgets, idempotency keys, capped attempt counts Circuit breakers, load shedding, backpressure, rate limiting Observability signal Retry count and retry rate per endpoint Request rate vs baseline, queue depth, error rate spike Key Differences Retries are a client-side decision; load amplification is a system-wide consequence that emerges when many retries overlap A single retry adds one extra attempt, but a retry storm can multiply traffic several times over in seconds Exponential backoff with jitter reduces retry-driven amplification by spreading re-attempts over time instead of synchronizing them Amplification is contained with circuit breakers and load shedding at the service, not by removing retries entirely When to Use Each Retries ...

Overview Timeouts define how long a caller waits for a response before giving up, and the duration you pick trades off resource protection against tolerance for slow-but-valid work. Short timeouts fail fast and protect callers from cascading slowness, while long timeouts give operations more room to complete under load or over slow links at the cost of holding resources longer. Comparison Diagram Short TimeoutLong TimeoutCallServerwait window: 200msTimeout errorfails fast, retries quicklyCallServerwait window: 30sResponse arrivestolerates slow workfrees threads/connections soonerrisk: false failures under loadholds resources longer per callrisk: cascading pileup/exhaustion Comparison Table Aspect Short Timeout Long Timeout Request initiation Caller sets an aggressive deadline immediately on send Caller allows a generous window before send returns control Behavior under normal latency Succeeds well within budget, negligible overhead Succeeds with unused slack, no functional difference Behavior under slow dependency Aborts before slow-but-valid work finishes, causing false failures Waits out transient slowness, letting valid work complete Resource holding Frees threads, sockets, and connection pool slots quickly Ties up threads, sockets, and pool slots for the full wait Failure propagation Fails fast, enabling quick retry or fallback logic Delays failure detection, slowing retries and fallback triggers System behavior under overload Sheds load quickly, protecting upstream and downstream services Risks thread/connection exhaustion and cascading backpressure Retry and circuit breaker interaction Pairs well with fast retries and quick breaker tripping Delays breaker tripping, masking degradation until timeout expires Tuning basis Set near p99 latency of a healthy, fast dependency Set to cover legitimate worst-case work like batch jobs or large payloads Key Differences A short timeout favors quick failure detection over completing genuinely slow requests A long timeout risks resource exhaustion when many calls stall simultaneously Short timeouts pair naturally with fast retries, while long timeouts delay circuit breaker activation Choosing either wrong direction turns normal latency variance into either false failures or cascading pileups The right value depends on the dependency’s actual p99 latency, not a guessed constant When to Use Each Short Timeout ...

Overview Failover and fallback both describe what a system does when something breaks, but they differ in what changes. Failover swaps a failed component for an identical redundant one so behavior stays the same, while fallback switches to a different, usually simpler or lower-fidelity path when the preferred one is unavailable. Confusing the two leads to designs that promise seamless continuity but actually degrade functionality, or vice versa. Comparison Diagram FailoverFallbackClientPrimary (Active)same behaviorStandby (identical)takes over, same outputredundant component, unchanged functionClientPrimary Pathfull behaviorFallback (degraded/default)reduced or cached responsealternate path, changed function Comparison Table Aspect Failover Fallback Core action Switch to a redundant, identical component Switch to a different, usually simpler alternative Functional parity Preserves full functionality and quality Often reduced functionality, accuracy, or freshness Typical scope Infrastructure/system level (servers, nodes, DCs) Application/logic level (methods, values, services) Trigger Health check or heartbeat failure detection Exception, timeout, cache miss, or unmet condition Example Active database node dies; standby replica takes over queries transparently Live pricing API call fails; app falls back to last cached price Recovery expectation Usually paired with failback once primary recovers Often stays on fallback until explicitly retried or root cause fixed User-visible impact Ideally none, if failover is seamless Often visible as a lower-quality or generic result Design goal High availability / continuity of service Graceful degradation / resilience of a single call or feature Key Differences Failover replaces a broken component with an equivalent one; fallback replaces a preferred behavior with a lesser one. Failover targets infrastructure-level continuity (nodes, clusters, regions); fallback targets code-level resilience (a single function or request). Failover implies redundancy of identical capability; fallback implies acceptance of reduced capability. Failover is often followed by ‘failback’ to the restored primary; fallback usually persists until the underlying issue is resolved or retried. A system can use both together: infrastructure fails over to a standby, while an individual call within that system falls back to cached data. When to Use Each Failover ...