Event-Driven

Overview Both patterns coordinate a multi-step business process across independent services, but they differ in where the coordination logic lives. In choreography, each service reacts to events and decides its own next move with no central brain; in orchestration, a dedicated controller tells every service what to do and in what order. Comparison Diagram ChoreographyOrchestrationOrder SvcPayment SvcShipping SvceventeventeventOrchestratorPayment SvcOrder SvcShipping Svcno central controllercommands out, responses back Comparison Table Aspect Choreography Orchestration Trigger Any service publishes an event when something happens A client or event calls the orchestrator to start the process Coordination logic Distributed across each service’s event handlers Centralized in one orchestrator component Communication style Asynchronous events broadcast to whoever is listening Explicit commands and replies directed at specific services Step sequencing Emergent from chained event subscriptions Explicitly defined as a workflow or state machine Failure handling Each service listens for failure events and compensates locally Orchestrator detects failure and drives compensating transactions Adding a new step Add a listener; no existing service needs to change Update the orchestrator’s workflow definition Observability Hard to see the full process; requires distributed tracing Process state is visible in one place, easy to audit Coupling Low coupling between services, higher coupling to event schema Services decoupled from each other, but coupled to the orchestrator Key Differences Choreography spreads decision-making across services via events; orchestration centralizes it in a single controller. Choreography scales extensibility easily but makes the overall process hard to trace. Orchestration makes the workflow explicit and easy to audit, at the cost of a single point of coordination. Compensation logic lives in each service under choreography, but is driven centrally under orchestration. Orchestration introduces a dependency on the orchestrator itself as new coupling, even as it decouples the services from each other. When to Use Each Choreography ...

Overview A message queue and an event log both move data from producers to consumers, but they differ in what happens after a message is read. A queue treats delivery as a one-time handoff where each message is consumed once and then removed, while an event log keeps every event in an ordered, replayable sequence that multiple independent readers can consume at their own pace. This distinction drives how each handles multiple consumers, failure recovery, and historical reprocessing. ...

Overview Lambda Architecture and Kappa Architecture are two patterns for building big-data pipelines that need both real-time and historical results. Lambda runs parallel batch and speed layers that get merged at query time, while Kappa pushes everything through a single, replayable stream pipeline. The choice determines how much duplicate logic you maintain and how reprocessing actually works. Comparison Diagram Lambda ArchitectureKappa ArchitectureData SourceBatch Layerfull recomputeSpeed Layerrecent, approximateServingLayerreprocess = rerun batch jobEvent LogStream ProcessingLayersingle codebasereprocess = replay the logqueryquery Comparison Table Aspect Lambda Architecture Kappa Architecture Ingestion path Raw events are forked to both a batch store and a stream processor at once Raw events are written once to an immutable, replayable log (e.g. Kafka) Processing model Two independent codebases — a batch job and a stream job — implement the same logic twice One stream-processing codebase handles both real-time and historical computation Historical reprocessing The batch layer periodically recomputes results over the entire raw dataset Reprocessing means replaying the log from an earlier offset through the same stream job State and storage Separate batch views and speed views are maintained independently, often in different stores A single serving store is continuously updated by the stream processor Result merging The query layer merges or reconciles batch and speed views at read time No merge step — the stream processor’s output is the only view Consistency behavior Speed layer results are approximate until the batch layer overwrites them, so the two can disagree temporarily One computation path avoids batch/speed drift, but correctness depends entirely on the stream engine’s guarantees Operational overhead Higher — two parallel pipelines to build, deploy, and monitor, with duplicated logic Lower pipeline count, but requires a log system with long enough retention to support full replays Best-fit scenario Batch and speed logic genuinely differ, or the org already has mature batch infrastructure Team wants one canonical pipeline and has a stream engine that can absorb both live and replay traffic Key Differences Lambda splits ingestion across a batch layer and a speed layer, while Kappa sends everything through one stream. Reprocessing history in Lambda means rerunning a full batch job; in Kappa it means replaying the log through the same stream code. Lambda’s query layer must reconcile two separate views; Kappa exposes a single serving store with no merge step. Kappa’s design hinges on a durable, long-retention event log capable of full replays, which Lambda does not require. When to Use Each Lambda Architecture ...