Log-based streaming versus point-to-point queues for microservice backbones

Log-based streaming. Arslan Ahmad argues that log-based systems, such as Apache Kafka or Redpanda, provide the necessary backbone for microservices by retaining events regardless of consumption (Arslan Ahmad from System Design Nuggets). Companies like Slack and Discord use these partitioned commit logs to maintain causal ordering for threaded conversations and audit trails, ensuring that replies never render before the original message (Hands On System Design). Because events are retained in the log, new services like fraud or loyalty modules can start from the beginning to build their own state without requiring changes to the original producer.

Point-to-point queues. Arslan Ahmad notes that simpler point-to-point work queues offer a different model where brokers delete a message immediately once a consumer acknowledges it (Arslan Ahmad from System Design Nuggets). While log-based systems provide massive throughput via sequential disk I/O, they introduce operational weight through partition planning and head-of-line blocking if a single message fails (Arslan Ahmad from System Design Nuggets). Teams that choose simple queues avoid the complexity of managing offsets and rebalancing partitions, though they lose the ability to replay history or support multiple independent consumer groups.

Systemic trade-offs. The choice between these models depends on whether the system requires durable event sourcing or simple task handoff. Log-based architectures enable massive fan-out and temporal decoupling, but they force developers to handle eventual consistency and correlation IDs across asynchronous hops.