Hierarchical deep-learning neural networks: finite elements and beyond

Lei Zhang, Lin Cheng, Hengyang Li, Jiaying Gao, Cheng Yu, Reno Domel, Yang Yang, Shaoqiang Tang*, Wing Kam Liu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

26 Scopus citations


The hierarchical deep-learning neural network (HiDeNN) is systematically developed through the construction of structured deep neural networks (DNNs) in a hierarchical manner, and a special case of HiDeNN for representing Finite Element Method (or HiDeNN-FEM in short) is established. In HiDeNN-FEM, weights and biases are functions of the nodal positions, hence the training process in HiDeNN-FEM includes the optimization of the nodal coordinates. This is the spirit of r-adaptivity, and it increases both the local and global accuracy of the interpolants. By fixing the number of hidden layers and increasing the number of neurons by training the DNNs, rh-adaptivity can be achieved, which leads to further improvement of the accuracy for the solutions. The generalization of rational functions is achieved by the development of three fundamental building blocks of constructing deep hierarchical neural networks. The three building blocks are linear functions, multiplication, and inversion. With these building blocks, the class of deep learning interpolation functions are demonstrated for interpolation theories such as Lagrange polynomials, NURBS, isogeometric, reproducing kernel particle method, and others. In HiDeNN-FEM, enrichment functions through the multiplication of neurons is equivalent to the enrichment in standard finite element methods, that is, generalized, extended, and partition of unity finite element methods. Numerical examples performed by HiDeNN-FEM exhibit reduced approximation error compared with the standard FEM. Finally, an outlook for the generalized HiDeNN to high-order continuity for multiple dimensions and topology optimizations are illustrated through the hierarchy of the proposed DNNs.

Original languageEnglish (US)
Pages (from-to)207-230
Number of pages24
JournalComputational Mechanics
Issue number1
StatePublished - Jan 2021


  • Data-driven
  • Fundamental building block
  • Neural network interpolation functions
  • Rational functions (i.e. RKPM, NURBS and IGA)
  • r- and rh-adaptivity

ASJC Scopus subject areas

  • Computational Mechanics
  • Ocean Engineering
  • Mechanical Engineering
  • Computational Theory and Mathematics
  • Computational Mathematics
  • Applied Mathematics


Dive into the research topics of 'Hierarchical deep-learning neural networks: finite elements and beyond'. Together they form a unique fingerprint.

Cite this