Concurrency

Overview A synchronous call blocks the caller until the server returns a result, tying up a thread or connection for the full round trip. An asynchronous call returns immediately with an acknowledgment and delivers the actual result later via a callback, event, or poll, letting the caller do other work in the meantime. Comparison Diagram Synchronous APIClientblocked - thread waitsrequest sentresponse receivedserver processingAsynchronous APIClientclient free: other workrequest sentcallback receivedserver working Comparison Table Aspect Sync API Async API Request initiation Caller invokes and immediately awaits the result on the same call Caller invokes and gets an immediate acknowledgment or handle (future, promise, message ID), not the result Response delivery Result returned in-line over the same connection/thread that made the call Result delivered later via callback, event, webhook, or by polling Caller behavior while waiting Thread or connection is blocked and cannot do other work Caller is free to continue other work or serve other requests Concurrency model Needs roughly one thread or connection per in-flight call A single thread or event loop can multiplex many in-flight calls Failure handling Errors surface immediately as exceptions or status codes at the call site Errors arrive out-of-band later and must be matched back to the original request Ordering and sequencing Strict: caller code executes in the exact order calls complete Responses can arrive out of order, requiring correlation IDs to reassemble sequence Latency impact on caller Caller’s total latency equals the full round trip Caller’s perceived latency is just the time to ack; real work overlaps with other tasks Implementation complexity Simpler code: straightforward call and return More complex: needs callback/promise/event handling and explicit state tracking Key Differences Sync calls block the caller until the response arrives, while async calls return control immediately. Async APIs scale better under load because they avoid thread-per-request limits inherent to blocking calls. Sync errors surface in-line at the call site; async errors require correlation back to the original request. Async responses can arrive out of order, adding sequencing complexity that sync calls never face. Sync code is easier to trace and debug since execution follows a single linear call stack. When to Use Each Sync API ...

Overview Synchronous and asynchronous describe whether a caller waits for an operation to finish before moving on. Synchronous execution blocks the calling thread until a result returns, while asynchronous execution lets the caller continue and gets notified or polls for completion later. The choice shapes throughput, resource usage, and how errors and ordering are handled throughout a system. Comparison Diagram SynchronousAsynchronousCallerCall fn()Work runsCallerblockedResult usedone blocking timelineCallerCall fn()Work runsContinues freeOther workCallback firesinterleaved timelines Comparison Table Aspect Synchronous Asynchronous Call initiation Caller invokes and immediately waits Caller invokes and registers a continuation, then moves on Thread/resource occupancy Calling thread stays occupied for the full duration Calling thread is freed while work happens elsewhere Execution order Strictly sequential, one step completes before the next starts Interleaved; multiple operations can be in flight concurrently Result delivery Return value comes directly back from the call Result delivered via callback, promise/future, or event Error handling Exceptions propagate up the same call stack Errors surface in the callback or rejection handler, separate from the call site Code structure Linear, easy to read top-to-bottom Requires callbacks, promises, or async/await to manage flow Debugging Stack traces map directly to the logical call path Stack traces are fragmented across event loop turns, harder to trace Scalability under load Threads block on I/O, limiting concurrent connections per resource Single thread or few threads can handle many pending operations at once Key Differences Blocking is the defining trait of synchronous calls; the caller cannot proceed until the operation resolves Asynchronous code relies on an event loop or scheduler to resume work when results arrive Synchronous flow gives simpler stack traces, while async flow gives better resource utilization Async introduces race conditions and ordering complexity that synchronous code avoids by construction Choosing async trades readability for the ability to handle many concurrent I/O-bound operations efficiently When to Use Each Synchronous ...

Overview Concurrency and parallelism are often used interchangeably, but they describe different things: concurrency is about interleaving multiple tasks so a program can make progress on all of them, while parallelism is about simultaneous execution of tasks on separate hardware. A single-core CPU can be concurrent but never truly parallel; a multi-core CPU can be both. Comparison Diagram ConcurrencyParallelism1 CPU CoreABACBtime (single lane, tasks interleave)only one task runs at any instantCore 1Core 2Core 3ABCtime (all run at once)tasks execute simultaneously Comparison Table Aspect Concurrency Parallelism Core definition Structuring a program to make progress on multiple tasks by interleaving them Executing multiple tasks or subtasks at the literal same instant Hardware requirement Works on a single core via context switching Requires multiple cores, processors, or SIMD units Execution pattern Tasks take turns; interleaved progress, order not guaranteed Tasks run simultaneously on separate execution units Task independence Tasks often share state and need coordination to interleave safely Tasks are usually split to run independently, minimizing interference Primary goal Improve responsiveness and structure for handling many things at once Improve throughput by doing more work in the same time Typical workload I/O-bound: network calls, file access, user events CPU-bound: numerical computation, data processing Common pitfalls Race conditions, deadlocks, callback/coordination complexity Synchronization overhead, diminishing returns (Amdahl’s law) Language/runtime constructs Event loops, coroutines, async/await, green threads OS threads, multiprocessing, GPU kernels, SIMD Key Differences Concurrency is a way of structuring code to deal with multiple tasks; it doesn’t require them to run at the same instant. Parallelism requires multiple cores executing work simultaneously, while concurrency runs fine on a single core. Concurrent code can run without being parallel, and parallel code (like SIMD data processing) can run without concurrent structure. Concurrency’s main hazard is a race condition from shared state; parallelism’s main cost is synchronization overhead. Concurrency optimizes for responsiveness; parallelism optimizes for raw computational throughput. When to Use Each Concurrency ...

Overview Blocking and non-blocking I/O differ in what happens to the calling thread when a read or write can’t complete immediately. Blocking I/O suspends the thread until data is ready; non-blocking I/O returns at once and lets the caller check back later. The choice shapes how a system scales to many simultaneous connections. Comparison Diagram Blocking I/ONon-blocking I/OThreadcall read()thread blockedno other work possibleuntil data arrivesdata ready, resumes1 thread ≈ 1 in-flight callthread / event loopread() returns at oncepoll / epoll checkretrymeanwhile: serves otherconnectionsready → callback fires1 thread ≈ many in-flight calls Comparison Table Aspect Blocking I/O Non-blocking I/O Call behavior Call halts the calling thread until the operation completes Call returns immediately, with data or an EWOULDBLOCK/EAGAIN error Thread state while I/O is pending Thread is suspended off the run queue — no CPU used, but unavailable for other work Thread stays runnable and can be reused to serve other requests How readiness is discovered OS wakes the thread automatically once data is ready Caller polls or registers with select/poll/epoll/kqueue Concurrency model One thread (or process) per concurrent connection Single or few threads multiplex many connections via an event loop Resource overhead at scale Grows linearly with connections — thread stacks, context switches Stays flat, bounded by CPU cores rather than connection count Code / control-flow complexity Simple, sequential, top-to-bottom logic Callback, promise, or async/await structure; state tracked across suspensions Failure / edge-case handling A slow or hung peer blocks the thread indefinitely without a timeout A slow peer only delays its own event; a stalled callback can starve the whole loop Key Differences Blocking I/O ties up a thread for the full call; non-blocking frees it immediately. Non-blocking servers rely on an event loop to learn when data is finally ready. Blocking scales concurrency with more threads; non-blocking scales with more callbacks on fewer threads. Blocking code reads sequentially; non-blocking code needs explicit state management across suspensions. At high connection counts, blocking hits a C10K wall that non-blocking avoids. When to Use Each Blocking I/O ...

Overview A process is an independently executing program instance with its own private address space, while a thread is a lightweight unit of execution that runs inside a process and shares that process’s memory with its sibling threads. The distinction matters because it determines how much isolation, communication overhead, and crash-safety you get versus how cheap context switching and data sharing are. Comparison Diagram Process Thread Process A Code Data Heap Stack Process B Code Data Heap Stack no shared memory (IPC only) single process address space Shared Code / Data / Heap Thread 1 Stack Registers Thread 2 Stack Registers Thread 3 Stack Registers shared heap/code, private stack per thread Comparison Table Aspect Process Thread Memory space Own isolated virtual address space Shares address space with sibling threads in the same process Creation cost Expensive (fork/CreateProcess, new page tables) Cheap (allocate stack + TCB, reuse existing address space) Context switch cost Higher (flush TLB, swap page tables) Lower (same address space, just swap registers/stack pointer) Communication Requires IPC: pipes, sockets, shared memory, message queues Direct via shared variables/heap; needs locks/mutexes for safety Fault isolation A crash typically stays contained to that process A crash (e.g. bad pointer, unhandled exception) can take down the whole process Scheduling unit OS schedules processes (which contain ≥ 1 thread) OS (or runtime) schedules threads independently within a process Concurrency primitives needed Rarely needed within a single process Mutexes, semaphores, atomics to guard shared state Typical use case Running separate, independently-failing programs (browser tabs as processes, microservices) Parallelizing work within one program (web server handling many requests, UI thread + workers) Key Differences A thread lives inside a process and shares its code, heap, and open file handles; a process owns its own private address space. Threads communicate by directly reading/writing shared memory (needing synchronization); processes must use explicit IPC mechanisms. Creating and context-switching a thread is much cheaper than doing the same for a process, since no new address space or page table is involved. A crashing thread can corrupt or kill its entire parent process; a crashing process is generally isolated from other processes by the OS. Multiple threads share one process’s resource limits (file descriptors, memory quota); each process gets its own. When to Use Each Process ...