Application of Genetic Algorithms in Hyperparameter Tuning of Large-Scale Machine Learning Models

Abstract

Hyperparameter tuning is a system problem that requires multiple accelerators to be used for an extended period to perform single model evaluation, and the search space includes continuous, discrete, and conditional choices. This paper presents a budget-aware genetic algorithm for large-scale machine learning models and integrates typed chromosome repair, asynchronous multi-fidelity evaluation, uncertainty-adjusted fitness, adaptive mutation, and stagnation-triggered diversity recovery. Random search, Bayesian optimization, Hyperband and a traditional genetic algorithm were used as baselines to compare the results obtained by gradient-boosted trees, a transformer text classifier and a graph neural network under the same compute budget. The mean normalised validation quality of the three workloads in the proposed method reached 0.913, compared with 0.889 for Bayesian optimization and 0.881 for Hyperband. Reduced accelerator-hour consumption by 31.6% compared with the conventional genetic algorithm and shortened wall-clock search time by 27.4% through asynchronous scheduling. The probability of an infeasible configuration after typed repair dropped to 1.7%, and the median quality loss caused by low-fidelity screening was still less than 0.6 percentage points. Based on ablation results, multi-fidelity promotion achieved the largest cost reduction, and adaptive diversity control prevented early convergence in conditional subspaces. Therefore, under the assumption that evolutionary operators, fidelity allocation and cluster scheduling are designed as a single optimization system, genetic search can still be applied to optimize expensive-model tuning.