Kafka is a cornerstone of LinkedIn’s data infrastructure. It is the replication stream for Espresso; the message transport for Brooklin (our change capture system), Samza and Venice (our derived data serving store). We describe Kafka’s fundamental roles: data storage, movement, processing and analysis; and discuss the requirements to serve these data systems, issues that we hit and how we addressed them.
![]() |
Joel Koshy, Senior Staff Engineer, LinkedIn |
![]() |
Kartik Paramasivam, Director of Engineering, Streams Infrastructure, LinkedIn |