The Alpakka Kafka connector (formerly known as reactive-kafka) is a component of the Alpakka project. It provides a diverse streaming toolkit, but sometimes it can be challenging to design these systems without a lot of experience with Akka Streams and Akka. By combining Akka Streams with Kafka using Alpakka Kafka, we can build rich domain, low latency, and stateful streaming applications with very little infrastructure.
This talk will discuss solutions to common Kafka and streaming problems such as consumer group partition rebalancing, effectively-once/transactional message delivery, stateful stages, state durability/persistence, and common production concerns like job failover and deployment.
The Alpakka project is an open source initiative to implement stream-aware, reactive, integration pipelines for Java and Scala. It is built on top of Akka Streams, and has been designed from the ground up to understand streaming natively and provide a DSL for reactive and stream-oriented programming, with built-in support for backpressure.
SHARE THIS TALK
Lightbend, Software Engineer, Fast Data Platform Team
Sean is a Senior Software Engineer on the Fast Data Platform team at Lightbend. He specializes in Apache Kafka and its ecosystem. He has experience consulting with Global 5000 companies on how to build Fast Data platforms using technologies such as Scala, Kafka, Spark, and Akka on Kubernetes and Mesos. Sean organizes the Scala Toronto meetup and enjoys speaking about and contributing to open source projects.