Downloads: 5 | Views: 380 | Weekly Hits: ⮙1 | Monthly Hits: ⮙2
Informative Article | Engineering Science | India | Volume 12 Issue 10, October 2023 | Popularity: 4.7 / 10
Evaluating and Selecting Appropriate Technologies and Tools for Data Ingestion: Apache Kafka and Custom - Built Solutions
Fasihuddin Mirza
Abstract: The landscape of data ingestion technologies is rapidly evolving to meet the growing demands of big data analytics. As organizations strive to gain real - time insights from their data, the need for robust, scalable, and efficient data ingestion platforms has never been more critical. This paper presents a comprehensive evaluation of Apache Kafka, a prominent open - source streaming platform, against custom - built data ingestion solutions. We examine the scalability, performance, fault tolerance, and ease of integration of these technologies. A systematic approach is employed to evaluate their suitability for various use cases, including high - throughput systems, IoT environments, and real - time analytics.
Keywords: Apache Kafka, Data ingestion technologies, Stream processing, Big data analytics, Real - time data processing, Scalability, Fault tolerance, Performance benchmarking, System integration, Custom - built data solutions, Messaging systems, Distributed systems, Internet of Things (IoT), High - throughput systems, Data pipeline architecture, Publish - subscribe model, Data integrity, Data volume, Cloud computing, Data engineering.
Edition: Volume 12 Issue 10, October 2023
Pages: 2140 - 2143
DOI: https://www.doi.org/10.21275/SR24418102327
Please Disable the Pop-Up Blocker of Web Browser
Verification Code will appear in 2 Seconds ... Wait