Q-LSHADE-PS: An Individual-Level Adaptive Differential Evolution with Q-Learning and History-Based Parameter Adaptation

Abstract

Differential Evolution (DE) is widely used for continuous optimization due to its simple structure and strong global search ability. However, classical and many adaptive DE variants often suffer from premature convergence and diversity loss on complex problems, where suitable operators may vary across individuals and search stages. To address this issue, this paper proposes Q-LSHADE-PS, an Linear Population Size Reduction Success History based Adaptive Differential Evolution (LSHADE) variant that equips each individual with state-conditioned, tabular Q-learning for mutation strategy selection, while preserving LSHADE’s success-history parameter adaptation, external archive, and linear population size reduction (LPSR). Each individual maintains a compact Q-table to adaptively select mutation strategies according to its stagnation state and the global search phase. In addition, a population-size-aware Q-table decay mechanism is introduced to prevent outdated strategy preferences from dominating after population reduction, thereby maintaining exploration capability under non-stationary search dynamics. Experimental results on standard benchmark suites demonstrate that the proposed algorithm achieves superior or highly competitive performance compared with several state-of-the-art DE variants, while introducing only negligible computational overhead. These results indicate that individual-level reinforcement learning provides an effective and practical mechanism for adaptive strategy control in modern DE frameworks.