Optimization Method for an Improved Training of Physics Informed Neural Networks

Physics Informed Neural Networks (PINNs) are promising methodologies to improve accuracy and extend applicability of Deep Learning in engineering applications, providing a hybrid modeling that combine physics domain knowledge with data-driven methods. Nevertheless, the adoption of PINNs in practice is still limited by the complexity of the training process, where the unbalanced loss gradients in the back propagation step can lead to inaccurate training. This paper discusses a method to improve PINNs training by optimizing the hyperparameters of the model in a standard NN scenario, before applying physics-based constraints, and then balancing the backpropagation by weighting the PINN loss gradients to target the simplified NN loss decay. The result is a faster and improved PINN training that can enable hybrid modeling in industrial applications where both data and domain knowledge are available. A case study is a fluid-dynamics problem taken from literature, described by the Navier Stokes equations that drive thermosenergy processes.