Data Processing at LinkedIn with Apache Kafka

Kafka Summit NYC 2017 | Systems Track

Kafka is a cornerstone of LinkedIn’s data infrastructure. It is the replication stream for Espresso; the message transport for Brooklin (our change capture system), Samza and Venice (our derived data serving store). We describe Kafka’s fundamental roles: data storage, movement, processing and analysis; and discuss the requirements to serve these data systems, issues that we hit and how we addressed them.

	Joel Koshy, Senior Staff Engineer, LinkedIn
	Kartik Paramasivam, Director of Engineering, Streams Infrastructure, LinkedIn