Large system analysis of SURE based hyper- parameter optimizing in sparse Bayesian learning

This paper studies stationary points arising from outputdomain Stein’s unbiased risk estimator (SURE) based hyperparameter optimization in sparse Bayesian learning from a large-system perspective. By analyzing the coordinatewise stationary conditions and leveraging tools from random matrix theory, we show that the high-dimensional problem admits a deterministic decoupling, leading to a distributional fixed-point characterization of stationary solutions. This framework reveals that multiple stationary points may naturally coexist. We further provide a local stability interpretation, clarifying which stationary solutions are observable under coordinatewise optimization. The analysis focuses on stationary-point structure rather than specific algorithms or finite-dimensional performance, and provides a principled theoretical understanding of the stationary landscape of SURE-based hyper-parameter learning in sparse Bayesian learning.