Performance

Overview Enabling compression shrinks data before it’s stored or sent, but that reduction is paid for with extra CPU cycles spent encoding and decoding it. The right choice depends on which resource is actually scarce in your system — disk/network bandwidth, or processor headroom. Comparison Diagram One resource saved, one resource spentCompressionNo CompressionCPU UsageCompressionNo CompressionhighlowData Size / BandwidthCompressionNo CompressionlowhighCompression converts spare CPU cycles into saved bytes — and vice versa Comparison Table Aspect Compression No Compression Data footprint at rest Reduced, often 30-90% smaller depending on algorithm and data Full raw size, no reduction CPU cost on write Extra cycles spent encoding data before it’s stored or sent None — data written or sent as-is Network/bandwidth usage Lower — fewer bytes cross the wire Higher — full payload transmitted every time CPU cost on read Extra cycles spent decoding data before use None — data read directly, no decode step Latency on small or frequent operations Can add overhead that outweighs the I/O time saved Lowest possible latency, nothing to encode/decode Behavior under CPU-bound load Competes with application logic for cores, can become the bottleneck Frees all cores for application work Behavior under I/O- or bandwidth-limited conditions Shines — spends cheap CPU cycles to relieve a scarce resource Becomes the bottleneck since every byte must move uncompressed Tuning and control Adjustable via algorithm choice and compression level No knob to turn — behavior is fixed Key Differences Compression is fundamentally a trade of spare CPU cycles for reduced data size, not a free optimization. The right choice depends on which resource is the actual bottleneck — bandwidth/disk or the processor. Compression level lets you dial how much CPU you spend for how much size reduction. Compressing already-dense data like video or ciphertext yields little size benefit while still paying the full encoding cost. When to Use Each Compression ...

Overview Cache freshness measures whether the data returned by a cache still matches the current state of its source of truth, while hit rate measures how often requests are answered directly from the cache instead of falling through to the origin. The two metrics pull in opposite directions: optimizing for freshness means shorter TTLs and more origin traffic, while optimizing for hit rate means longer TTLs and a higher chance of serving stale data. Tuning a cache well means choosing the right balance point for that specific data’s tolerance for staleness. ...

Overview A local cache stores data in the memory of a single application process, giving the fastest possible reads but no visibility into what other instances hold. A shared cache lives in a separate service that every instance queries over the network, trading a bit of latency for one consistent view of cached data across the whole fleet. The choice shapes how you handle invalidation, scaling, and failure in a multi-instance deployment. ...

Overview Latency and throughput are two orthogonal measures of system performance: latency is the time a single request takes to complete, while throughput is the volume of work a system finishes per unit of time. The distinction matters because architectures optimized for one can quietly degrade the other. Comparison Diagram Latency Client Server Time for ONE request to complete Throughput Client Server Total requests completed per second Comparison Table Aspect Latency Throughput Definition Time elapsed for one request to travel and complete Amount of work completed across all requests per unit time What is measured A single request’s round trip or processing delay Aggregate output of the system over an observation window Unit of measurement Milliseconds, microseconds, or seconds Requests/sec, transactions/sec, or Mbps Primary driver Network round-trip time, serialization, and processing delay Available bandwidth, parallel capacity, and resource pool size Effect of concurrency Individual request latency can rise as queueing builds up Throughput rises with more parallel workers, up to a capacity limit Behavior under overload Tail latency spikes as queues grow (p95/p99 degrade) Throughput plateaus or drops once the system saturates Typical optimization Reduce round trips, cache results, shorten the critical path Batch requests, add parallel workers, scale out capacity Measurement method Ping, request timers, percentile latency (p50/p95/p99) Requests-per-second counters, load testing, capacity benchmarks Key Differences Latency measures the time for one request; throughput measures the volume processed per unit time. Batching to raise throughput can increase tail latency for individual requests. Latency is bounded by physical round-trip time; throughput is bounded by system capacity. Under heavy load, latency spikes from queueing while throughput plateaus at a ceiling. Little’s Law links the two: average latency times concurrency roughly equals throughput. When to Use Each Latency ...

Overview In serverless and containerized systems, a cold start happens when a request must wait for a new execution environment to be provisioned and initialized before it can run, while a warm start reuses an already-running instance and skips straight to execution. The gap between the two explains why the same function can respond in 5ms or 2 seconds depending on whether an idle instance was standing by. Comparison Diagram Cold StartReqProvisioncontainerInit runtime+ codeExecutelatency: tens of ms - several secondsWarm Startidle, pre-initialized container waitingReqExecutelatency: sub-ms - low tens of ms Comparison Table Aspect Cold Start Warm Start Trigger condition No idle instance available (scale-to-zero, scale-out, or fresh deploy) Idle, already-initialized instance is available to handle the request Environment state at invocation No running process; container or sandbox must be created from scratch Process is already running in memory from a prior invocation Steps performed Provision compute, load code, initialize runtime and dependencies, run init code, then handle request Skip provisioning and init; execute the handler directly on the existing process Typical latency added Tens of milliseconds to several seconds depending on runtime and package size Sub-millisecond to low tens of milliseconds Resource cost to provider Higher; allocates new compute, memory, and network setup Lower; reuses resources already allocated Frequency of occurrence Rare relative to total traffic but concentrated after idle periods, deploys, or scale-out Common; most requests during steady, active traffic Primary mitigation Provisioned concurrency, smaller packages, lighter runtimes, scheduled pings Sustained traffic, minimum instance counts, connection reuse Key Differences Cold start pays full provisioning overhead; warm start reuses an already-initialized process. The latency gap can span orders of magnitude — low milliseconds versus multiple seconds. Cold starts are triggered by scale-to-zero or scale-out events, not by what the request contains. Whether a start is warm depends on the platform’s idle timeout before it reclaims the instance. Avoiding cold starts usually means paying for reserved capacity to keep instances standing by. When to Use Each Cold Start ...

Overview Caching strategies differ mainly in who updates the cache and when. Cache-Aside leaves the application responsible for loading data into the cache on a miss and writing straight to the database, while write-through/write-behind caches push writes through the cache itself — synchronously for durability or asynchronously for speed. The choice shapes consistency guarantees, crash-safety, and how much write latency the app absorbs. Comparison Diagram Cache-AsideAppCacheDBreadon miss: fetch + populateWrite-ThroughAppCacheDBwritesync writeWrite-BehindAppCacheDBwrite (fast ack)async flush (delayed) Comparison Table Aspect Cache-Aside Write-Through / Write-Behind Read path App checks the cache first; on a miss, it reads from the DB itself and populates the cache Cache is always kept current on writes, so reads simply hit the cache without app-managed fallback logic Write path App writes directly to the DB; the cache entry is invalidated or left stale until the next read App writes only to the cache; the cache layer propagates the write to the DB itself Write latency perceived by app Only DB write latency, since the cache isn’t touched on write Write-Through waits for both cache and DB commit; Write-Behind returns after the cache write only Consistency between cache and DB Brief staleness window possible between invalidation and the next read Write-Through stays consistent immediately; Write-Behind lags until the queued flush completes Data loss risk on crash None — the DB is always the write target, so a cache crash loses nothing Write-Through has no loss; Write-Behind can lose unflushed writes if the cache crashes before flush Implementation complexity App owns miss handling and invalidation logic explicitly Cache layer owns persistence logic; Write-Behind adds a queue/flush scheduler Typical use case Read-heavy workloads with unpredictable key access, e.g. Redis in front of a relational DB Write-Through suits systems needing instant durability; Write-Behind suits high write-throughput systems tolerant of brief loss, e.g. metrics buffers Key Differences Cache-Aside puts the application in charge of both fetch-on-miss and invalidation, while write-through/write-behind push that responsibility into the cache layer itself Only Write-Behind delivers real write latency savings by acknowledging before the DB commit completes Write-Through guarantees immediate durability at write time, while Write-Behind trades some of that durability for throughput Cache-Aside is the only strategy where a cache outage never risks data loss, since writes never pass through it When to Use Each Cache-Aside ...

Overview The stack and heap are two distinct memory regions used for different allocation strategies at runtime. The stack manages function call frames automatically via a single pointer increment/decrement, while the heap handles dynamic allocations with flexible lifetimes through an allocator. The distinction directly affects allocation speed, data lifetime, size constraints, and thread safety — core considerations in systems, embedded, and performance-sensitive programming. Comparison Diagram STACKgrows downward, LIFOhighlowframe: main()frame: compute(n)frame: factorial(3)SPunusedauto-managed · O(1) · fast~1-8 MB per threadHEAPunordered, explicit lifetimeObjectVec<T>freeHashMapBoxfreeString dataArcmanual/GC · flexible · fragmentslimited by OS / RAM Comparison Table Aspect Stack Heap Allocation mechanism Pointer decrement — O(1), no bookkeeping Allocator call (malloc/new) — higher constant, free-list bookkeeping Deallocation Automatic on scope/frame exit Explicit (free/delete) or garbage collector Data lifetime Bound to the declaring scope or call frame Arbitrary; can outlive any function call Size constraint Fixed at thread creation (typically 1–8 MB) Limited only by available virtual memory / RAM Size known at compile time Required — compiler must know the type’s layout Not required — length/capacity decided at runtime Fragmentation None — LIFO order keeps allocation contiguous Yes — internal and external fragmentation accumulate over time Thread ownership Each thread has its own stack; no synchronization needed Shared across threads; allocator must serialize internally Failure mode Stack overflow → immediate crash (SIGSEGV/signal) OOM → null / exception / OOM-killer; potentially recoverable Key Differences Stack allocation is a single SP register decrement; heap allocation invokes an allocator with metadata updates, free-list traversal, and possible OS syscalls. Stack lifetime is strictly scoped to the function frame — data cannot be returned by pointer from the stack safely; heap memory can be returned, stored globally, or transferred across threads. Each thread has its own stack and needs no locking; the heap is process-wide and requires allocator-level synchronization on every alloc/free. Stack size is fixed and small (set by the OS or linker script); heap can grow dynamically, making it the only viable region for large buffers or runtime-sized collections. Heap fragmentation is a real operational concern in long-lived or allocation-heavy processes; the stack never fragments because it always grows and shrinks from one end in LIFO order. When to Use Each Stack ...

Overview Latency and bandwidth both describe network performance, but they measure completely different things: latency is how long a single piece of data takes to travel from source to destination, while bandwidth is how much data can move through the connection per second. A link can have huge bandwidth and still feel laggy, or tiny bandwidth and still respond instantly — understanding which one is limiting you determines whether the fix is a faster link or a shorter path. ...