A Novel Hybrid Cloud Edge Resource Allocation Algorithm to Optimize Real Time Big Data Stream Processing in Distributed Computing Environments
DOI:
https://doi.org/10.62951/ijcts.v1i2.378Keywords:
Cloud Computing, Container Orchestration, Decentralized Scheduling, Resource Allocation, Virtual MachinesAbstract
Cloud-based resource allocation and VM/container orchestration play a crucial role in ensuring performance, scalability, and energy efficiency in modern distributed computing environments. This study investigates the effectiveness of centralized and decentralized scheduling models combined with heuristic and optimization-based allocation strategies in container-based cloud infrastructures. A quantitative experimental approach was employed to evaluate system performance under varying workload intensities. Key evaluation metrics included response time, throughput, resource utilization, SLA violation rate, and energy consumption. The experimental results indicate that centralized scheduling mechanisms experience scalability limitations and increased latency under high workload conditions. Although optimization-based allocation improves performance within centralized architectures, coordination bottlenecks remain significant. In contrast, decentralized scheduling models demonstrate superior adaptability, reduced response time, and improved throughput due to distributed decision-making and reduced control overhead. The integration of intelligent optimization techniques further enhances resource utilization and energy efficiency, achieving the lowest SLA violation rates and highest system stability. Overall, the findings confirm that combining decentralized scheduling with optimization-driven resource allocation provides a more scalable and sustainable orchestration strategy for modern cloud environments. This approach is particularly suitable for dynamic, large-scale, and latency-sensitive applications in container-based and edge-integrated cloud systems.
References
A. Mohanty, A. G. Mohapatra, S. K. Mohanty, and S. Nayak, “Harnessing the power of IoT and big data: Advancements and applications in smart environments,” in Internet of Things and Big Data Analytics-Based Manufacturing, CRC Press, 2024, pp. 19–58. doi: 10.1201/9781032673479-3.
S. Liu, “Case studies of big data applications for IoT,” in Empowering IoT with Big Data Analytics: A volume in intelligent data-centric systems, Elsevier, 2024, pp. 265–311. doi: 10.1016/B978-0-443-21640-4.00010-7.
H. Zhang et al., “How far have edge clouds gone? A spatial-temporal analysis of edge network latency in the wild,” in IEEE International Workshop on Quality of Service (IWQoS), IEEE, 2023. doi: 10.1109/IWQoS57198.2023.10188741.
H. Zhang et al., “Large-scale measurements and optimizations on latency in edge clouds,” IEEE Trans. Cloud Comput., vol. 12, no. 4, pp. 1218–1231, 2024, doi: 10.1109/TCC.2024.3452094.
N. Tabassum and C. R. K. Reddyy, “Review on QoS and security challenges associated with the internet of vehicles in cloud computing,” Meas. Sensors, vol. 27, p. 100562, 2023, doi: 10.1016/j.measen.2022.100562.
R. R. Nikam and D. Motwani, “Towards decentralized fog computing: A comprehensive review of models, architectures, and services,” in Lecture Notes in Networks and Systems, vol. 818, 2024, pp. 135–147. doi: 10.1007/978-981-99-7862-5_11.
X. Merino, C. Otero, D. Nieves-Acaron, and B. Luchterhand, “Towards orchestration in the cloud-fog continuum,” in IEEE SoutheastCon 2021, IEEE, 2021. doi: 10.1109/SoutheastCon45413.2021.9401822.
C.-X. Wang, Y.-Z. Shan, P.-F. Zuo, and H.-M. Cui, “Reinvent cloud software stacks for resource disaggregation,” J. Comput. Sci. Technol., vol. 38, no. 5, pp. 949–969, 2023, doi: 10.1007/s11390-023-3272-0.
A. Bandi and J. A. Hurtado, “Edge computing as an architectural solution: An umbrella review,” in Lecture Notes in Electrical Engineering, vol. 869, 2022, pp. 601–616. doi: 10.1007/978-981-19-0019-8_45.
D. Panda, B. K. Mishra, and C. R. Panigrahi, “A study of computation offloading techniques used by mobile edge computing in an IoT environment,” in The role of IoT and blockchain: Techniques and applications, 2022, pp. 73–86.
P. K. Sharma, J. H. Ryu, K. Y. Park, J. Y. Park, and J. H. Park, “Li-Fi based on security cloud framework for future IT environment,” Human-Centric Comput. Inf. Sci., vol. 8, no. 1, p. Article 23, 2018, doi: 10.1186/s13673-018-0146-5.
K. Sampath Kini and K. Pai, “Exploring real-time data processing using big data frameworks,” Commun. Appl. Nonlinear Anal., vol. 31, no. 8S, pp. 620–634, 2024, doi: 10.52783/cana.v31.1561.
M. Hanif, E. Kim, S. Helal, and C. Lee, “SLA-based adaptation schemes in distributed stream processing engines,” Appl. Sci., vol. 9, no. 6, p. Article 1045, 2019, doi: 10.3390/app9061045.
P. S. Rawat and P. K. Soni, “Efficient virtual machine allocation technique based on hybrid approach,” in Advanced computing techniques for optimization in cloud, 2024, pp. 87–109. doi: 10.1201/9781003457152-5.
K. Netaji Vhatkar and G. P. Bhole, “Self-improved moth flame for optimal container resource allocation in cloud,” Concurr. Comput. Pract. Exp., vol. 34, no. 23, p. e7200, 2022, doi: 10.1002/cpe.7200.
B. Tan, H. Ma, Y. Mei, and M. Zhang, “A cooperative coevolution genetic programming hyper-heuristics approach for on-line resource allocation in container-based clouds,” IEEE Trans. Cloud Comput., vol. 10, no. 3, pp. 1500–1514, 2022, doi: 10.1109/TCC.2020.3026338.
Z. Fang, H. Ma, G. Chen, and S. Hartmann, “A group genetic algorithm for energy-efficient resource allocation in container-based clouds with heterogeneous physical machines,” in Lecture Notes in Computer Science, vol. 14472, 2024, pp. 453–465. doi: 10.1007/978-981-99-8391-9_36.
J.-Y. Roh, S.-H. Choi, and K.-W. Park, “CO-TRIS: Container orchestration transforming container using resource inspection system,” in Proceedings of the 2023 IEEE International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), IEEE, 2023, pp. 121–124. doi: 10.1109/AIKE59827.2023.00027.
J. A. Murali and T. Brindha, “An advanced hierarchical virtual resource management model in cloud data centers,” AIP Conf. Proc., vol. 2587, no. 1, p. Article 050005, 2023, doi: 10.1063/5.0150551.
Z. Becvar, P. Mach, M. Elfiky, and M. Sakamoto, “Hierarchical scheduling for suppression of fronthaul delay in C-RAN with dynamic functional split,” IEEE Commun. Mag., vol. 59, no. 4, pp. 95–101, 2021, doi: 10.1109/MCOM.001.2000697.
J. Qi, X. Su, and R. Wang, “Toward distributively build time-sensitive-service coverage in compute first networking,” IEEE/ACM Trans. Netw., vol. 32, no. 1, pp. 582–597, 2024, doi: 10.1109/TNET.2023.3289830.
R. Kiruthiga and D. Akila, “Prediction-based cost-efficient resource allocation scheme for big data streams in cloud systems,” in Advances in Intelligent Systems and Computing, vol. 1292, 2021, pp. 233–242. doi: 10.1007/978-981-33-4389-4_22.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 International Journal of Computer Technology and Science
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
