Structure-exploiting interior-point solver for high-dimensional entropy-sparsified regression learning

Edoardo Vecchi*, Juraj Kardoš, Malik Lechekhab, Andreas Wächter, Illia Horenko, Olaf Schenk

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

The solution of high-dimensional nonlinear regression problems through standard machine learning approaches often relies on first-order information, due to the numerical and memory challenges arising from the computation of the Hessian matrix and of the higher-order derivatives. While this scenario seems not favorable to second-order methods, here we show that an efficient and modular structure-exploiting interior-point solver can be successfully applied to the recently introduced class of entropy-based methods for regression learning. Specifically, by exploiting the favorable structure of the problem and of the Hessian matrix, we suggest a robust solution strategy based on explicit low-rank updates combined with an iterative Symmetric Quasi-Minimal Residual (SQMR) algorithm to solve the underlying system of linear equations. The results show that the proposed structure-exploiting solver – which relies on the hybrid parallelism and distributed-memory computing paradigm – allows a significant solution time speed-up with respect to a naive solution strategy. Furthermore, through an adequate use of the Message Passing Interface (MPI) and of Open Multi-Processing (OpenMP), the proposed solver enables the solution of large-scale problems on high-performance computing architectures consisting of thousands of compute nodes. The accompanying detailed convergence and performance analyses demonstrate both numerical robustness and high-performance capabilities for increasingly high-dimensional problems.

Original languageEnglish (US)
Article number102208
JournalJournal of Computational Science
Volume76
DOIs
StatePublished - Mar 2024

Funding

The authors gratefully acknowledge the great scientific support and HPC resources provided by the Erlangen National High Performance Computing Center(NHR@FAU) of the Friedrich-Alexander-Universitaät Erlangen Nürnberg (FAU) under the NHR project 286745. NHR funding is provided by federal and Bavarian state authorities. NHR@FAU hardware is partially funded by the German Research Foundation (DFG) – 440719683. This work was supported by a grant from the Swiss National Supercomputing Centre (CSCS) under project ID d120. The authors gratefully acknowledge the great scientific support and HPC resources provided by the Erlangen National High Performance Computing Center(NHR@FAU) of the Friedrich-Alexander-Universitaät Erlangen Nürnberg (FAU) under the NHR project 286745. NHR funding is provided by federal and Bavarian state authorities. NHR@FAU hardware is partially funded by the German Research Foundation (DFG) – 440719683. This work was supported by a grant from the Swiss National Supercomputing Centre (CSCS) under project ID d120.

Keywords

  • Regression learning
  • Structure-exploiting interior-point solver

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science
  • Modeling and Simulation

Fingerprint

Dive into the research topics of 'Structure-exploiting interior-point solver for high-dimensional entropy-sparsified regression learning'. Together they form a unique fingerprint.

Cite this