Efficient Artificial Intelligence Workload Orchestration in Edge-Cloud Infrastructure using Deep Reinforcement Learning

Abstract

The emergence of Artificial Intelligence into everyday applications has introduced unprecedented demands on modern computing systems, ranging from heavy training to real time inference tasks. Although these workloads are increasingly deployed across distributed Edge-Cloud infrastructures, their efficient orchestration under limited resources remains a highly complex challenge.

This thesis proposes an intelligent orchestration framework, specifically designed for AI workloads that utilizes Deep Reinforcement Learning (DRL). The core architecture integrates a DRL agent that continuously monitors system resources in order to find the optimal placement and select the most efficient parallelization strategy for each workload. The main objectives are to minimize execution costs and reduce system idle time.

To validate its performance, the proposed framework is benchmarked against a greedy baseline across varying levels of infrastructure congestion. Furthermore, a detailed study is conducted to examine how different system parameters affect the agent’s performance, such as the reward function weights and the structure of the neural network model.

Experimental results reveal that in environments with resource abundance, the heuristic algorithm is more efficient due to its high execution speed. However, as the infrastructure transitions into a scenario of mild contention, the DRL agent achieves 94.1% of the optimal service rate provided by the corresponding MILP formulation, whereas the greedy algorithm is restricted to 88.2%. Overall, to leverage the strengths of both methods, a hybrid framework is proposed that utilizes the fast heuristic algorithm under low-traffic conditions and switches to the intelligent agent as congestion increases.