Downloads: 2 | Views: 129 | Weekly Hits: ⮙1 | Monthly Hits: ⮙1
Informative Article | Information Technology | India | Volume 10 Issue 5, May 2021 | Rating: 5 / 10
Implementing Cloud-Native Technologies for Big Data Processing: A Case Study with Kubernetes and Airflow
Abstract: This paper presents a case study on the architecture design and implementation details of cloud-native technologies for big data processing. Cloud-native technologies, such as Kubernetes and Airflow, are modern solutions intricately connected as essential components within IT infrastructures. They are designed specifically for processing and managing data in cloud environments. These technologies leverage the scalability and flexibility of cloud computing to enable efficient and reliable data storage, analysis, and retrieval., focusing on Apache Airflow and Kubernetes. Acting as a container orchestrator, Kubernetes efficiently manages a vast number of containers, eliminating the necessity to explicitly outline the configuration for executing specific tasks. Meanwhile, Airflow is the orchestration layer for managing data processing workflows within the Kubernetes environments. The findings of this paper underscore the potential of Kubernetes and Airflow in enabling seamless orchestration and management of big data workflows in cloud environments.
Keywords: Cloud-native, Apache Airflow, Kubernetes, Big Data Processing, Orchestration Stability, Efficiency
Edition: Volume 10 Issue 5, May 2021,
Pages: 1335 - 1340