Downloads: 16 | Views: 357 | Weekly Hits: ⮙1 | Monthly Hits: ⮙2

Informative Article | Data & Knowledge Engineering | India | Volume 7 Issue 11, November 2018 | Popularity: 5.6 / 10

Rate This Article!      

Design and Implementation of High-Throughput Data Streams using Apache Kafka for Real-Time Data Pipelines

Preyaa Atri

Abstract: In an era dominated by the need for real-time data processing, Apache Kafka emerges as a crucial technology for constructing high-throughput data pipelines capable of handling extensive data streams with minimal latency. This paper provides an in-depth exploration into the design and implementation of Kafka-based data pipelines, discussing their architectural patterns, performance optimization techniques, and practical applications across various domains. Through detailed analysis and expert recommendations, this study addresses current challenges and maps out future research directions, underlining Kafka's pivotal role in advancing real-time data processing systems. The insights presented aim to guide professionals in enhancing the efficiency and reliability of their real-time data solutions.

Keywords: Apache Kafka, data streams, real-time processing, data pipelines, high- throughput, distributed systems, stream processing, big data

Edition: Volume 7 Issue 11, November 2018

Pages: 1988 - 1991

DOI: https://www.doi.org/10.21275/SR24422184316

Make Sure to Disable the Pop-Up Blocker of Web Browser

Click to Cite this Article

Similar Articles