An Adaptive Riemannian Spectral Gradient Method Without Cost Function Evaluations
DOI:
https://doi.org/10.5540/03.2026.012.01.0244Palabras clave:
Riemannian Optimization, Gradient Method, Steplength Selection, Adaptive MethodsResumen
This work addresses a spectral correction for the adaptive weighted normalized gradient method in the minimization of smooth functions on a Riemannian manifold. The proposed algorithm incorporates a Barzilai-Borwein-type stepsize in order to accelerate the Riemannian gradient procedure. The new approach is globally convergent and does not require evaluating the objective function throughout the iterative process. Our preliminary numerical experiments show that the use of spectral correction improves the performance of the adaptive Riemannian gradient method.
Descargas
Citas
P-A. Absil, R. Mahony, and R. Sepulchre. Optimization algorithms on matrix manifolds. Princeton University Press, 2009. isbn: 9780691132983.
J. Barzilai and J. M. Borwein. “Two-point step size gradient methods”. In: IMA journal of numerical analysis 8.1 (1988), pp. 141–148. doi: 10.1093/imanum/8.1.141.
G. C. Bento, O. P. Ferreira, and J. G. Melo. “Iteration-complexity of gradient, subgradient and proximal point methods on Riemannian manifolds”. In: Journal of Optimization Theory and Applications 173.2 (2017), pp. 548–562. doi: 10.1007/s10957-017-1093-4.
E. G. Birgin, J. M. Martínez, and M. Raydan. “Spectral projected gradient methods: review and perspectives”. In: Journal of Statistical Software 60 (2014), pp. 1–21. doi: 10.18637/ jss.v060.i03.
J-P. Chehab, H. Oviedo, and M. Raydan. “Optimization schemes on manifolds for structured matrices with fixed eigenvalues”. In: Computational Optimization and Applications (2024), pp. 1–26. doi: 10.1007/s10589-024-00630-3.
D. Di Serafino, V. Ruggiero, G. Toraldo, and L. Zanni. “On the steplength selection in gradient methods for unconstrained optimization”. In: Applied Mathematics and Computation 318 (2018), pp. 176–195. doi: 10.1016/j.amc.2017.07.037.
O. P. Ferreira, M. S. Louzeiro, and L. Prudente. “Gradient method for optimization on Riemannian manifolds with lower bounded curvature”. In: SIAM Journal on Optimization 29.4 (2019), pp. 2517–2541. doi: 10.1137/18M1180633.
G. N. Grapiglia and G. Stella. “An Adaptive Riemannian Gradient Method Without Function Evaluations”. In: Journal of Optimization Theory and Applications 197.3 (2023), pp. 1140–1160. doi: 10.1007/s10957-023-02227-y.
J. Hu, X. Liu, Z. Wen, and Y. Yuan. “A brief introduction to manifold optimization”. In: Journal of the Operations Research Society of China 8 (2020), pp. 199–248. doi: 10.1007/s40305-020-00295-9.
B. Iannazzo and M. Porcelli. “The Riemannian Barzilai–Borwein method with nonmonotone line search and the matrix geometric mean computation”. In: IMA Journal of Numerical Analysis 38.1 (2018), pp. 495–517. doi: 10.1093/imanum/drx015.
E. Kokiopoulou, J. Chen, and Y. Saad. “Trace optimization and eigenproblems in dimension reduction methods”. In: Numerical Linear Algebra with Applications 18.3 (2011), pp. 565–602. doi: 10.1002/nla.743.
J. Nocedal and S. J. Wright. Numerical optimization. New York: Springer, 1999.
H. Oviedo. “A cyclic delayed weighted steplength for the gradient method”. In: Ricerche di Matematica 73.2 (2024), pp. 873–885. doi: 10.1007/s11587-021-00646-5.
H. Oviedo. “Global convergence of Riemannian line search methods with a Zhang-Hager-type condition”. In: Numerical Algorithms 91.3 (2022), pp. 1183–1203. doi: 10.1007/s11075- 022-01298-8.
H. Oviedo. “Implicit steepest descent algorithm for optimization with orthogonality constraints”. In: Optimization Letters 16.6 (2022), pp. 1773–1797. doi: 10.1007/s11590- 021-01801-5.
H. Oviedo, O. Dalmau, and H. Lara. “Two adaptive scaled gradient projection methods for Stiefel manifold constrained optimization”. In: Numerical Algorithms 87 (2021), pp. 1107– 1127. doi: 10.1007/s11075-020-01001-9.
H. Oviedo and H. Lara. “Spectral residual method for nonlinear equations on Riemannian manifolds”. In: Computational and Applied Mathematics 40.7 (2021), p. 238. doi: 10.1007/s40314-021-01630-3.
M. Raydan. “The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem”. In: SIAM Journal on Optimization 7.1 (1997), pp. 26–33. doi: 10.1137/S1052623494266365.
U. Rodriguez and O. Dalmau. “Symmetric nonnegative matrix factorization with elastic-net regularized block-wise weighted representation for clustering”. In: Pattern Analysis and Applications 25.4 (2022), pp. 807–817. doi: 10.1007/s10044-022-01062-7.
U. Rodríguez and O. Dalmau. “Block-wise weighted sparse representation-based classification”. In: Signal, Image and Video Processing 14.8 (2020), pp. 1647–1654. doi: 10 . 1007/s11760-020-01700-9.
S. T. Smith. Optimization techniques on Riemannian manifolds. Online. Accessed on 16/04/2025, arXiv preprint arXiv:1407.5965. 2014.
X. Wu, R. Ward, and L. Bottou. Wngrad: Learn the learning rate in gradient descent. Online. Accessed on 16/04/2025, arXiv preprint arXiv:1803.02865. 2018.
