Authors - Aleksander Karastoyanov Abstract - Reinforcement learning (RL) has been widely proposed for adaptive virtual machine (VM) right-sizing in cloud environments, yet most published work reports results from a single random seed with a fixed reward formulation, conditions that may not reflect genuine generalization. This paper addresses both limitations through a systematic multi-seed, multi-reward evaluation of a Proximal Policy Optimization (PPO) agent applied to VM right-sizing on the Alibaba Cluster Trace 2018. Fifteen independent training runs (three reward configurations × five seeds) are conducted on a ten-VM simulation environment. The key finding is that reward formulation, not the RL algorithm per se, is the dominant determinant of SLA compliance: the unified reward variant (Config A) achieves a mean SLA violation rate of 1.1% (±2.11) across five seeds, statistically comparable to a Threshold baseline (2.43%), while dimension-aware variants expose a critical instability in memory-saturated environments. A one-sample t-test yields t = −1.40, p = 0.23, confirming that neither superiority nor inferiority relative to Threshold can be claimed, and motivating the need for larger seed sets and alternative reward designs in future work. Three empirically grounded reward design principles are derived for practitioners deploying RL-based resource managers in high-memory-pressure infrastructure.
Open Zoom