Downloads: 5 | Views: 168 | Weekly Hits: ⮙1 | Monthly Hits: ⮙1
Informative Article | Science and Technology | India | Volume 10 Issue 9, September 2021 | Popularity: 5 / 10
Autonomous Scheduling for Recurring Tasks to Manage Ingestion and Stream Processing
Mahidhar Mullapudi, Satish Kathiriya, Siva Karthik Devineni
Abstract: The need for effective large-scale heterogeneous distributed data ingestion pipelines, crucial for transforming and processing data essential to advanced analytics and machine learning models, has seen a significant surge in importance. Modern services increasingly depend on near-real-time signals to precisely identify or predict customer behavior, sentiments, and anomalies, thereby facilitating informed, data-driven decision-making.In the rapidly evolving landscape of large-scale enterprise data applications, the demand for efficient data ingestion and stream processing solutions has never been more critical. This technical paper introduces a groundbreaking autonomous self-schedulable library designed for recurring jobs, addressing the challenges faced by enterprises in orchestrating complex data workflows seamlessly. Leveraging authoritative expertise in building robust enterprise applications, this library provides a paradigm shift in how organizations manage and execute recurring tasks within data pipelines.
Keywords: Modern Ingestion Platform, Autonomous Scheduling Library, Parallel Stream Processing
Edition: Volume 10 Issue 9, September 2021
Pages: 1731 - 1736
DOI: https://www.doi.org/10.21275/SR24203215713
Make Sure to Disable the Pop-Up Blocker of Web Browser