Convergence analysis of physics-informed neural networks and comparison with finite difference methods of the two-dimensional heat equation
DOI:
https://doi.org/10.5540/03.2025.011.01.0343Keywords:
PINN, Convergence Analysis, Comparative Study, FDM, Two-Dimensional Heat EquationAbstract
Partial differential equations (PDEs) are used in mathematics to describe physical phenomena. However, analytical solutions are often unavailable, requiring approximations. The physics-informed neural network (PINN) is a recent technique that combines deep neural networks with physical knowledge leveraging automatic differentiation techniques, to provide accurate approximations. This work analyzes the convergence of the PINN method, considering the discretizations and architecture of the networks. PINN approximations were compared with Finite Difference Methods (FDM) in the case of the two-dimensional heat equation. The experiments carried out suggested that there is an optimal number of points and architectural parameters to be used, which can lead to an improvement in the generalization and estimation error. It was possible to see an advantage over FDM as it approximates the solution of all the time steps at once, without propagating the error from one step to the next.
Downloads
References
I. E. Lagaris, A. Likas, and D. I. Fotiadis, “Artificial neural networks for solving ordinary and partial differential equations,” IEEE transactions on neural networks, vol. 9, no. 5, pp. 987–1000, 1998. doi: 10.1109/72.712178.
M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” Journal of Computational physics, vol. 378, pp. 686–707, 2019. doi: 10.1007/s10915-022-01939-z.
A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind, “Automatic differentiation in machine learning: A survey,” Journal of Machine Learning Research, vol. 18, pp. 1–43, 2018. [Online]. Available: http://jmlr.org/papers/v18/17-468.html.
V. Souza, “Convergence analysis of physics-informed neural networks and comparison with finite difference methods,” Master dissertation, UERJ, 2024.
V. Souza, C. Faria, and K. Leite, “Pinn convergence with regular discretizations and comparison with fdm,” in Anais do Encontro Nacional de Modelagem Computacional e Encontro de Ciência e Tecnologia dos Materiais, 2023. [Online]. Available: https://www.even3.com.br/anais/xxvi-encontro-nacional-de-modelagem-computacional-xiv-encontro-de-ciencia-e-tecnologia-dos-materiais-338941/705486-pinn-convergence-with-regular-discretizations-and-comparison-with-fdm.
Z. Hao, S. Liu, Y. Zhang, C. Ying, Y. Feng, H. Su, and J. Zhu, “Physics-informed machine learning: A survey on problems, methods and applications,” arXiv preprint:2211.08064, 2022.
M. A. Nabian, R. J. Gladstone, and H. Meidani, “Efficient training of physics-informed neural networks via importance sampling,” Computer-Aided Civil and Infrastructure Engineering, vol. 36, no. 8, pp. 962–977, 2021. doi: 10.1111/mice.12685.
Y. Shin, J. Darbon, and G. E. Karniadakis, “On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type pdes,” Communications in Computational Physics, vol. 28, no. 5, pp. 2042–2074, 2020. doi: 10.4208/cicp.OA-2020-0193.
M. A. et al, TensorFlow: Large-scale machine learning on heterogeneous systems, Software available from tensorflow.org, 2015. [Online]. Available: https://www.tensorflow.org.
K. A. Hoffmann and S. T. Chiang, Computational Fluid Dynamics Volume 1. Engineering Education System, 1998, isbn: 978-0962373107.
R. J. LeVeque, Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems. SIAM, 2007, isbn: 978-0898716290.
D. Hendrycks and K. Gimpel, “Gaussian error linear units (GELUs),” arXiv preprint:1606.08415, 2016.
