Next Article in Journal
Multilevel Diffractive Lenses: Recent Advances and Applications
Previous Article in Journal
Diffusion Augmented Complex Inverse Square Root for Adaptive Frequency Estimation over Distributed Networks
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Symmetry
Volume 16
Issue 10
10.3390/sym16101376
symmetry-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Improved PINN Algorithm for Shallow Water Equations Driven by Deep Learning

by
Yanling Li
,
Qianxing Sun
*,
Junfang Wei
and
Chunyan Huang
Department of Mathematics and Statistics, North China University of Water Resources and Electric Power, Zhengzhou 450046, China
*
Author to whom correspondence should be addressed.
Symmetry 2024, 16(10), 1376; https://doi.org/10.3390/sym16101376
Submission received: 6 September 2024 / Revised: 8 October 2024 / Accepted: 11 October 2024 / Published: 16 October 2024
(This article belongs to the Section Computer)
Browse Figures
Review Reports Versions Notes

Abstract

:
Solving shallow water equations is crucial in science and engineering for understanding and predicting natural phenomena. To address the limitations of Physics-Informed Neural Network (PINN) in solving shallow water equations, we propose an improved PINN algorithm integrated with a deep learning framework. This algorithm introduces a regularization term as a penalty in the loss function, based on the PINN and Long Short-Term Memory (LSTM) models, and incorporates an attention mechanism to solve the original equation across the entire domain. Simulation experiments were conducted on one-dimensional and two-dimensional shallow water equations. The results indicate that, compared to the classical PINN algorithm, the improved algorithm shows significant advantages in handling discontinuities, such as sparse waves, in one-dimensional problems. It accurately captures sparse waves and avoids smoothing effects. In two-dimensional problems, the improved algorithm demonstrates good symmetry and effectively reduces non-physical oscillations. It also shows significant advantages in capturing details and handling complex phenomena, offering higher reliability and accuracy. The improved PINNs algorithm, which combines neural networks with physical mechanisms, can provide robust solutions and effectively avoid some of the shortcomings of classical PINNs methods. It also possesses high resolution and strong generalization capabilities, enabling accurate predictions at any given moment.

1. Introduction

In the fields of science and engineering, solving partial differential equations (PDEs) is a critical and complex task. The shallow water equations [1], a common type of PDE, are often used to describe the dynamics of fluid flow in environments such as rivers, oceans, and lakes. These equations hold significant application value in fields such as hydrology, water resource management, natural disaster prediction, and coastal engineering. Currently, various traditional numerical algorithms [2,3,4,5,6] have been applied to solve shallow water equations. However, these algorithms rely on complex grid discretization, incur high computational costs, and are unable to predict solutions at arbitrary time points. Therefore, a more efficient and flexible solution is of great importance for research in the scientific and engineering domains. In recent years, machine learning techniques such as deep learning have achieved revolutionary progress in multiple fields [7]. However, their application is limited in certain complex domains, especially in scenarios with scarce data. Before the significant advancements in data-driven machine learning, the fields of science and engineering primarily relied on using partial differential forms [8,9] to describe or characterize physical models for research and prediction. While directly using physical models can lead to accurate predictions, several issues persist, such as significant errors arising from simplistic physical models and solution errors caused by missing or inaccurate initial boundary data. Furthermore, traditional numerical methods for solving partial differential equations face significant challenges in inverse [10] problem solving, handling complex geometric domains, and addressing high-dimensional spaces. In these scientific and engineering fields, training data often encompasses substantial prior knowledge. For example, in fluid dynamics problems, flow field data must adhere to the physical laws of mass conservation and momentum conservation.
Traditional machine learning algorithms rely on feeding large amounts of data into the model, allowing it to learn autonomously and form an automated model. However, the use of prior knowledge is not fully leveraged, which, to some extent, results in lower generalization capabilities of the algorithms. To address this shortcoming, Raisi et al. [11] proposed the Physics-Informed Neural Networks (PINN) algorithm, which combines the strengths of data-driven machine learning and physical models. PINNs leverage the powerful representational capabilities of neural networks along with the physical knowledge of PDEs to approximate solutions to PDEs through neural networks, eliminating the need for complex grid generation or discretization. This approach offers a new paradigm for efficiently solving PDE problems and has achieved significant success in various fields, including fluid mechanics, solid mechanics, and heat conduction. However, it still faces challenges such as limited generalization performance and issues related to numerical stability [12,13]. Particularly when solving discontinuous problems in hyperbolic conservation equations, PINNs may encounter issues similar to those in traditional numerical algorithms, such as solution smoothing and spurious oscillations. Therefore, further optimizing PINNs to address these challenges is crucial for expanding their application to a broader range of complex problems. In recent years, the rise of deep learning methods has brought new opportunities to this field. For example, Sirignano et al. [14] introduced the Deep Galerkin Method (DGM) network, similar to an artificial neural network, and proposed a method for calculating second-order differential operators based on the Galerkin approach. Mao et al. [15], when using Physics-Informed Neural Networks (PINNs) to solve the Euler equations for inviscid compressible flow, enhanced the training sample weights at discontinuity locations through clustering, successfully capturing the discontinuities. However, this method still requires prior knowledge of the shockwave locations. Minbashian et al. [16] used PINNs to solve hyperbolic conservation laws with non-convex flux by approximating the solution with small-scale diffusion and dispersion regularization. However, the results were highly sensitive to the choice of diffusion and dispersion coefficients. Jagtap et al. [17] proposed the cPINN (conservative PINN), which decomposes the domain into multiple subdomains, with a separate PINN applied to each subdomain. This method was applied to solve the Burgers equation and the Euler equations. The results show that cPINN is more effective at accurately capturing discontinuities than the classical PINN, although the decomposition of subdomains undoubtedly increases computational complexity. Lu et al. [18] proposed a new residual-based adaptive refinement method based on PINN, which adaptively increases training points near discontinuities to effectively capture them. Liu et al. [19] proposed the concept of an adaptive minimum energy path (AtPINN), enabling the parameters to be adaptively updated during the PDE-solving process and guiding the optimization tasks. This approach effectively addresses the issue of sharp local gradients in wide domains. Gyouho Cho et al. [20] proposed an LSTM-PINN model to estimate the temperature of lithium-ion batteries, and the results showed that LSTM-PINN can provide better accuracy as an input method. Xu and Zhang [21] integrated deep learning with genetic algorithms, enhancing the robustness of the algorithm. Yan et al. [22] proposed MultiInNet PINNs, which utilize multi-input residual networks combined with multi-step training examples to enhance unsupervised training methods, improving the speed stability of PINNs and achieving faster convergence. In addition, ensemble methods also demonstrate unique advantages, For example, Supei Zheng et al. [23] introduced a viscous dissipation regularization PINN algorithm, incorporating a viscosity term as a penalty in the loss function to address the shortcomings in solving discontinuous problems in shallow water equations. Fang Jin et al. [24] constructed a network structure composed of two parallel neural networks. The results indicate that the new network structure can be used for solving shallow water wave equations, and it allows for the relatively accurate estimation of water depth using velocity data. Psaros [25] and Liu [26] proposed improved meta-learning techniques to enhance the generalization and adaptability of models. LIN et al. [27] introduced a diffusion term regularization into the equations and incorporated the reference solution into the loss function as a training coefficient. The results demonstrate that the new algorithm better handles discontinuities without generating parasitic oscillations.
To overcome the limitations of deep learning methods and traditional finite element (mesh) methods, this paper proposes a new algorithm that integrates L 1 regularization as a penalty term in the loss function based on the PINN and LSTM neural network models. Additionally, an attention mechanism is incorporated into the LSTM network structure. By combining L 1 regularization, PINN, and Long Short-Term Memory (LSTM) networks, the algorithm aims to improve the accuracy of numerical solutions for shallow water equations. This paper presents an algorithm with strong generalization capabilities that effectively avoids numerical smoothing and spurious oscillations, while offering more precise handling of complex phenomena in later stages. It provides a more comprehensive solution for solving PDEs and explores the synergistic application of deep learning and numerical methods. Moreover, by leveraging the advantages of modern computational resources, the improved PINN algorithm can play a crucial role in real-time prediction and simulation, thereby holding significant potential applications in natural disaster warning and emergency response. This research not only advances scientific inquiry but also contributes to the sustainable development of society.

2. Shallow Water Equations

The shallow water equations are derived from the three-dimensional Navier–Stokes equations for fluid motion in two-dimensional or one-dimensional space. They are suitable for scenarios where the water depth is much smaller compared to the wavelength. These equations describe the variations of water depth and velocity with respect to time and space, enabling the prediction of water wave propagation and flow behavior. Widely applied in hydrology, oceanography, and engineering, they find extensive use in predicting the dynamics of water in various settings.
This study focuses on the one-dimensional conservation form of the shallow water equations, given by Equation (1):
∂ t U + ∂ x f ( U ) = 0
where U = h h u represents the conserved variables, and f ( U ) = h u h u 2 + g h 2 2 denotes the flux. Here, h represents the total water depth, u is the average flow velocity in the x -direction, and g is the constant acceleration.
This study focuses on the two-dimensional conservation form of the shallow water equations, given by Equation (2):
∂ t U + ∂ x f ( U ) + ∂ y g ( U ) = 0
where U = h h u h v represents the conserved variables; f ( U ) = h u h u 2 + g h 2 2 h u v and g ( U ) = h v h u v h v 2 + g h 2 2 represent the fluxes, where h is the total water depth, and u and v are the average flow velocities in the x -direction and y -direction, respectively; and g is the constant acceleration due to gravity.

3. Methods

First, the classical PINN algorithm is introduced, followed by the presentation of the improved algorithm proposed in this paper for solving shallow water equations. The improved algorithm integrates L 1 regularization as a penalty term in the loss function based on the PINN and LSTM neural network models, while incorporating an attention mechanism into the LSTM network. The improved algorithm is validated to demonstrate significant advantages in handling discontinuities, spurious oscillations, and complex scenarios in shallow water equations.

3.1. Classical PINN Algorithm

PINNs combine neural networks with the physical knowledge of PDEs. They incorporate the physical equations as constraints within the neural network, enabling the direct approximation of solutions to PDEs. Typically, PINNs establish the functional relationship between the input ( x , t ) (where x and t represent the coordinates of time and space, respectively) and output U θ (where θ represents the parameter weights w and biases b in the neural network) of neural networks by employing multi-hidden-layer fully connected neural networks and nonlinear activation functions. Employing automatic differentiation techniques, the partial derivatives of the output variables of the fully connected network with respect to the input variables are calculated to obtain the residual of the equation F ( x , t ) . Subsequently, the loss function is constructed using the residual of the equation. Taking the one-dimensional case as an example, the structure of the PINN is illustrated in Figure 1, and the residual of the equation is defined as follows:
F ( x , t ) = ∂ t U θ + ∂ x ∂ x f ( U θ )
In this context, the parameter θ is a shared parameter between the fully connected neural network U θ ( x , t ) and the equation residual F ( x , t ) . The output of the neural network U θ ( x , t ) includes h θ ( x , t ) and U θ ( x , t ) . The loss function of PINN consists of two parts: data fitting loss and equation residual. The formula is as follows:
L O S S ( θ ) = ω h L h ( θ ) + ω u L u ( θ ) + ω F L F ( θ )
In the equation, L h ( θ ) represents the data fitting part for the total water depth h , L u ( θ ) denotes the data fitting part for the velocity u , and L f ( θ ) represents the equation residual part. The parameters ω h , ω u , and ω F are used to adjust the proportions of the data-driven part and the equation residual part in the loss function. In this paper, L h , L u , and L f are each defined by the mean squared error as follows:
L h = 1 N ∑ n = 1 N [ h ( x n , t n ) − h θ ( x n , t n ) ] 2
L u = 1 N ∑ n = 1 N [ u ( x n , t n ) − u θ ( x n , t n ) ] 2
L F = 1 N ∑ n = 1 N F ( x n , t n ) 2
Here, x n , t n n = 1 N refers to training data, which are configuration points randomly selected within the computational domain. The loss function is continuously adjusted using optimization algorithms such as Adam or L-BFGS to update the parameters θ . When the optimization of the loss function reaches its minimum, the neural network U θ ( x , t ) can accurately predict the solution to the equation. Consequently, the process of solving the equation is transformed into optimizing the loss function using the backpropagation mechanism of neural networks and optimization algorithms.

3.2. LSTM

Long Short-Term Memory (LSTM) represent a specialized type of recurrent neural network capable of efficiently conveying and expressing information in long-term sequences without causing the forgetting of valuable information from distant past. The key element of LSTM is the cell state C t , and it employs “gates” to introduce or remove information from C t . The control of information retention and propagation in LSTM primarily relies on input gates, forget gates, and output gates, ultimately manifesting in the cell state C t and the output signal h t . The network structure diagram of LSTM is illustrated in Figure 2.
The input gate determines which input information is to be remembered or stored in the neuron as C t . It consists of two components: firstly, calculating the content to be input through a sigmoid layer, and secondly, computing the candidate value C ˜ t for C t through a Tanh layer. The computation formula is as follows [28].
i t = σ ( W i x t + U i h t − 1 + b i )
C ˜ t = tanh ( W c x t + U c h t − 1 + b c )
where σ represents the sigmoid function, σ ∈ [ 0 , 1 ] , 0 represents complete discard, and 1 represents complete retention. W and U are weight matrices; b is the bias matrix.
The forget gate takes the output from the previous time step and the input from the current time step as input. It uses a sigmoid layer to compute whether to forget information, and the calculation formula is as follows.
f t = σ ( W f x t + U f h t − 1 + b f )
In the input gate, we obtain the input activation value i t and the candidate value for C t denoted as C ˜ t . The calculation for the new state value C t at the current time step t is as follows.
C t = i t ⊗ C ˜ t + f t ⊗ C t − 1
where ⊗ denotes the Hadamard product.
Similarly, the output gate, based on the output from the previous time step and the input from the current time step, calculates which information should be output. The calculation formula is as follows:
ο t = σ ( W ο x t + U ο h t − 1 + b ο )
h t = ο t ⊗ tanh h ( C t )
where tanh ( ⋅ ) represents the hyperbolic tangent activation function.

3.3. Attention Mechanism

The attention mechanism in deep learning mimics the human visual and cognitive systems. By incorporating this mechanism, neural networks can automatically learn to selectively focus on important information within the input, thereby enhancing the model’s performance and generalization capabilities [29]. Self-attention is a form of the attention mechanism where the model can establish relationships across different positions within the same sequence, rather than focusing solely on other positions in the input sequence. In this paper, a self-attention mechanism is introduced into the LSTM network structure to enhance the model’s performance and representational capacity.
In this study, the exceptional time series modeling capabilities of LSTM, along with its ability to capture long-term dependencies, are particularly important for addressing time-dependent dynamic systems. These characteristics make LSTM an ideal choice for improving the PINN algorithm in this research. Furthermore, by incorporating attention mechanisms, LSTM can further enhance model performance by focusing on key time steps, thereby improving the efficiency of information processing and allowing for more accurate simulations of physical phenomena in shallow water equations.

3.4. Improved Algorithm

The classical PINN algorithm is generally applicable for solving ordinary partial differential equations. However, when solving initial value problems for hyperbolic partial differential equations, such as shallow water equations, the unique properties of these equations—such as shocks, discontinuities, and other non-smooth characteristics—can lead to entropy dissipation and numerical stability issues. In traditional numerical methods, artificial viscosity is introduced to reduce numerical oscillations and enhance stability. Additionally, by controlling the time step, smaller time steps are used around discontinuities to better capture their evolution. References [30,31] incorporated LSTM/RNN into PINN to enhance the robustness of the algorithm. References [32,33] emphasize the importance of generalization in machine learning, leading to the introduction of regularization terms in networks to improve the algorithm’s adaptability. Inspired by this, this paper introduces the concepts of LSTM and L 1 regularization into PINN for solving shallow water equations, and incorporates an attention mechanism into the LSTM network structure to avoid over-processing irrelevant or secondary information, thereby improving the model’s efficiency and performance.
In the improved algorithm, the L 1 regularization term is introduced as part of the loss function to penalize the weights in the neural network, helping the model learn more sparse representations. Sparse weights also make the model easier to interpret. Additionally, the LSTM neural network architecture is integrated to improve the numerical solution of shallow water equations. This combination leverages the respective advantages of L 1 regularization and LSTM, complementing each other to achieve better numerical results. On the one hand, the L 1 regularization term makes the model more concise and interpretable, while also helping to avoid overfitting; on the other hand, LSTM effectively handles sequential data and long-term dependencies, enhancing the model’s predictive capabilities and generalization performance.

Regularization Term

Regularization is a commonly used technique in machine learning, primarily aimed at controlling model complexity and reducing overfitting. The most basic regularization method involves adding a penalty term to the original objective (cost) function, penalizing models with higher complexity [34]. Its mathematical expression is given by
J ˜ ( w ; X , y ) = J ( w ; X , y ) + α Ω ( w )
where X , y represent training samples and their corresponding labels, and w is the weight coefficient. J ( ⋅ ) denotes the objective function, Ω ( w ) is the penalty term, and the parameter α controls the strength of regularization. Common choices for the Ω function include the L 1 norm and L 2 norm, commonly referred to as L 1 regularization and L 2 regularization, The computation formula is as follows.
L 1 : Ω ( w ) = w 1 = ∑ i w i
L 2 : Ω ( w ) = w 2 2 = ∑ i w i 2
The introduction of L 1 and L 2 into the algorithm in this paper reveals that L 1 regularization yields superior results. This implies that the L 1 regularization term aids the network in identifying and utilizing crucial physical features, while compressing the weights of irrelevant or noisy features to zero. This enhances the model’s adaptation to the temporal characteristics of shallow water equations and improves model interpretability. The structure of the improved algorithm is illustrated in Figure 3:
Figure 4 illustrates the changes in the loss values during the numerical simulations of one-dimensional and two-dimensional shallow water equations when L 1 and L 2 regularization terms are introduced, respectively. The results show that in both models, L 1 regularization demonstrates faster convergence and lower final loss values, especially in the early stages of iteration. This suggests that L 1 regularization, by promoting sparsity, can effectively simplify the model and enhance its noise resistance and generalization capabilities. Therefore, the introduction of L 1 regularization offers significant advantages in the numerical simulation of shallow water equations.

4. Results

This paper uses two specific examples of one-dimensional and two-dimensional shallow water equations to validate the performance of the newly developed algorithm and compares the results with those of the classical PINN. During the simulations, the network was trained on a platform built using PyTorch version 2.1.0. The classical PINN neural network architecture was fully connected, with the hyperbolic tangent (Tanh) function selected as the activation function due to its continuity in higher-order derivatives. In the one-dimensional shallow water equation, the Adam optimizer was used for 10,000 steps, with a fourth-order Runge–Kutta method employed as the reference solution. Similarly, in the two-dimensional shallow water equation, Adam was used for 10,000 steps to achieve convergence. Additionally, the selection of hyperparameters during the training process is based on recommended values from the previous literature [23]. We also utilize the Adam optimizer to dynamically adjust the learning rate, which accelerates convergence and improves model stability. Through this approach, we ensure that the chosen hyperparameters exhibit good robustness and adaptability across different datasets and experimental setups.
1. One-Dimensional Dam-Break Problem
Equation (1) satisfies the initial conditions:
h ( x , 0 ) = 2 , x < 0 1 , x > 0 , u ( x , 0 ) = 0
The computation is performed over the interval [ − 1 , 1 ] with t = 0.2 . The total water depth h is composed of left-traveling rarefaction waves and right-traveling shock waves. In the classical PINN, a fully connected neural network is constructed with seven hidden layers, each containing 64 neurons. The model is trained to obtain solutions across the entire time and space. The new algorithm incorporates two layers of LSTM and introduces L 1 regularization in the loss function. Each hidden layer also employs 64 neurons with Tanh activation function, and the model is trained for 10,000 steps using Adam. Figure 5 provides the results for both classical PINN and improved PINN, illustrating the solutions at t = 0.08 , t = 0.12 , t = 0.16 .
In the spatiotemporal plots of Figure 5, the variation of water depth at different positions and times, and the propagation and interaction of shock waves, sparse waves, and other wave phenomena, can be visualized. In (a), non-physical numerical oscillations can be observed near t = 0 in the numerical simulation, resulting in shock waves and sparse waves that do not conform to actual physical behavior. In (b), after improvements, the propagation of shock waves and sparse waves is clearly visible, particularly at the edges, where the waves are not excessively affected by numerical dissipation. Since LSTM captures temporal dependencies and L 1 regularization prevents overfitting by reducing model complexity, the improved algorithm demonstrates accurate wave propagation capture. In the solutions at t = 0.08 , t = 0.12 , t = 0.16 , it can be observed that the improved algorithm’s solution closely matches the reference solution, with no smoothing occurring at the corners of shock waves and sparse waves. The improved algorithm is more sensitive to capturing finer details.
The overall error relative to the reference solution is quantified using the Root Mean Square Error (RMSE), where a smaller RMSE indicates better numerical simulation performance. As shown in Table 1, the RMSE values clearly demonstrate that the overall error decreases as time progresses and that the improved PINN is closer to the reference solution compared to the classical PINN. Therefore, the improved PINN exhibits better numerical stability and provides greater accuracy in describing the true dynamic behavior of the system.
Figure 6 shows a comparison of the loss function convergence between the classical PINN and the improved PINN during the training process. By comparing the trends in the loss functions over training iterations, the superiority of the improved PINN in terms of convergence speed and final fitting accuracy can be intuitively evaluated. This comparison provides important insights into the performance differences between different neural network models when dealing with physical problems. The improved PINN, compared to the classical PINN, demonstrates faster convergence and higher fitting accuracy. Additionally, the improved PINN exhibits smaller loss fluctuations during training, reflecting its stability throughout the process. These characteristics make the improved PINN more efficient and reliable when solving complex physical problems.
2. Two-Dimensional Circular Dam-Break Problem
Equation (2) satisfies the initial conditions:
0.1 , x 2 + y 2 ≤ 0.35 1.0 , o t h e r , u ( x , y , 0 ) = v ( x , y , 0 ) = 0
The computation is performed over the interval [ − 1 , 1 ] × [ − 1 , 1 ] with t = 0.4 , and gravity acceleration g is taken as 0.98. The total water depth h consists of outward-propagating sparse waves and inward-propagating shock waves. The classical PINN is set up similarly to Case 1, using seven hidden layers, each containing 64 neurons to construct a fully connected neural network. The Adam optimizer is used for 10,000 iterations, and after training, the solution across the entire time and spatial domain can be obtained. The improved algorithm incorporates two layers of LSTM and introduces an L 1 regularization term in the loss function. Each hidden layer also uses 64 neurons and the Tanh (hyperbolic tangent) activation function. Figure 7 and Figure 8, respectively, show the water depth surface plots and contour plots of the classical PINN and the improved PINN at t = 0.03 and t = 0.21 , and at t = 0.30 and t = 0.37 . The numerical results indicate that the improved algorithm exhibits good symmetry and overall consistency in the early stages. As time progresses, the initial symmetric structure is influenced by the nonlinear effects within the fluid, initial disturbances, and boundary conditions, making the fluid dynamics more complex. In the contour plots of the improved algorithm, some deviations appear in the finer details, revealing more intricate changes, which indicates an enhancement in capturing finer details.
The 3D surface plots in Figure 7 and Figure 8 show smooth and continuous surfaces, with no obvious signs of numerical instability. The shape of the wave peaks is regular, consistent with the expected physical phenomena of the circular dam-break problem, indicating good stability in the simulation. In Figure 7, the 3D surface plot at t = 0.03 shows a deep concave structure in the central region, with few complex details. The concave region represents areas in the fluid dynamics where rapid decreases or disappearances occur. Additionally, in Figure 8, at t = 0.30 and t = 0.37 , the improved algorithm’s 3D surface plots show more detailed variations at the top and edges, indicating that the method captures more complex fluid dynamics characteristics.
The contour plot comparison in Figure 7 shows that the improved algorithm exhibits good symmetry, especially with the water depth contours displaying a clear circular shape, without unnatural or excessive oscillations. This indicates that the improved algorithm successfully eliminates spurious oscillations, further enhancing the stability and accuracy of the simulation. The denser and more complex contour distribution suggests that the improved algorithm produces numerical results with a higher resolution. Although the shock wave is not yet fully developed at the early time of t = 0.03 , by t = 0.21 , the shock wave begins to emerge. The denser contours and more complex surface morphology indicate that the improved algorithm is able to more accurately capture the shape and propagation characteristics of the shock wave. As time progresses, nonlinear effects in fluid dynamics, such as turbulence and vortices, begin to appear, leading to more complex and asymmetric solutions. Although the symmetry is still somewhat maintained, the comparison of the contour plots in Figure 8 shows that the improved algorithm captures more complex details, with some asymmetric features emerging. This indicates that the improved method performs better when handling complex flows and significantly enhances the ability to capture finer details of the fluid dynamics.
As shown clearly in Figure 9, the loss curve of the classical PINN exhibits several noticeable fluctuations and peaks after 8000 epochs of training, indicating potential instability in the later stages of training. In contrast, the loss curve of the improved PINN remains stable, with almost no similar fluctuations, demonstrating that the improved method is more stable during long-term training.

5. Discussion

The treatment of the one-dimensional shallow water equation demonstrates that the improved algorithm has the following capabilities:
  • Sparse Wave Capturing Ability: In the simulation experiments of the one-dimensional shallow water equation, we observed that the improved PINN algorithm shows significant advantages in handling discontinuous problems such as sparse waves and shock waves. This is mainly due to the integration of the LSTM network model, L 1 regularization, and the self-attention mechanism. This combination enhances the model’s ability to handle partial differential equations involving time-series data and improves the model’s generalization capabilities, thereby increasing its ability to identify and manage discontinuities.
  • Avoiding Smoothing Effects: The traditional PINN algorithm often exhibits solution smoothing when handling discontinuous problems. However, the improved algorithm in this study successfully avoids this issue, which can likely be attributed to the introduction of the L 1 regularization term. This enhances the model’s ability to capture fine details, allowing for a more accurate simulation of real-world physical phenomena.
From the treatment of the two-dimensional shallow water equation, it can be seen that the improved algorithm has the following capabilities:
3.
Symmetry and Numerical Stability: In the handling of the two-dimensional shallow water equation, the improved PINN algorithm demonstrated good symmetry and reduced non-physical oscillations, which is crucial for maintaining numerical stability. This finding highlights the potential and applicability of the improved algorithm in solving more complex and higher-dimensional problems. The reduction in non-physical oscillations showcases the superiority of the improved algorithm in numerical simulations. This characteristic is especially important for engineering applications, as it provides more reliable predictions, helping to avoid erroneous decisions based on inaccurate simulations.
4.
Shock Wave Handling and High Resolution: In two-dimensional problems, shock waves are a crucial phenomenon in shallow water equations, and numerical simulations must accurately capture the propagation and interaction of shock waves. In the improved algorithm, we observe that the contour lines are denser and more complex, with a significantly narrower shock wave filtering zone, while still maintaining physical accuracy and numerical stability. This indicates that the improved algorithm offers more refined simulation results, better reproducing the details and characteristics of shock waves in fluid dynamics. The high-resolution performance of this algorithm demonstrates its ability to capture subtle physical changes, which is critical for accurately simulating and predicting hydrodynamic phenomena.
5.
In this study, under the same RTX 4090 GPU environment, during 10,000 iterations, we observed that in the one-dimensional problem, the running time for the classical PINN was 253 s, while the running time for the improved PINN was 432 s. In the two-dimensional problem, the classical PINN took 568 s to run, whereas the improved PINN took 1987 s. This result indicates that while the introduction of LSTM and attention mechanisms enhances the model’s performance, it also significantly increases the computational burden. Considering the significant advantages of the proposed improved PINN algorithm in handling one-dimensional and two-dimensional shallow water equations, particularly in accurately capturing complex flow phenomena in later stages, the improved algorithm is better suited for applications requiring precise details and complex flow simulations. In contrast, the classical PINN is more appropriate for predicting and analyzing overall trends, especially in scenarios where detail is not as critical. Future work could further explore how this algorithm can be applied to real fluid dynamics problems, such as flood simulations and coastal engineering, and encourage collaboration with practical fields to validate the algorithm’s effectiveness and potential impact.

6. Conclusions

Building on the classical PINN, this paper introduces the architecture of Long Short-Term Memory (LSTM) networks and an attention mechanism, along with the addition of an L 1 regularization term as a penalty. This paper aimed at addressing the pseudo-oscillations and smoothing phenomena that occur near sparse waves and shock waves, while avoiding some of the deficiencies of classical PINNs methods. The key conclusions of this study are summarized as follows.
In the simulation experiments of the one-dimensional shallow water equation, the improved PINN algorithm demonstrated clear advantages in handling discontinuous problems such as shock waves and sparse waves. The experimental results showed that as time progressed, the improved PINN algorithm’s ability to capture shock waves and sparse waves gradually increased. In the simulation experiments of the two-dimensional shallow water equation, the numerical results indicated that, compared to the classical PINN algorithm, the improved PINN algorithm exhibited a higher resolution and a better ability to capture the details of complex phenomena. It also significantly reduced the occurrence of oscillation relaxation, with this advantage becoming more pronounced as time progressed.
During the network training process, it was found that the weights of the loss function are quite sensitive to the training results. Currently, the selection of weights is primarily based on the experimental approach regarding the impact of each loss term on model performance and the convergence behavior observed during training. Therefore, future research should consider the introduction of adaptive weight mechanisms, which can not only accelerate convergence speed but also improve model accuracy, such as adaptive weight adjustment methods based on loss variations and Bayesian optimization techniques. The improved PINN algorithm proposed in this study demonstrates significant advantages in handling shallow water equations, especially in capturing discontinuities and fine details, which is of great importance for improving the accuracy and efficiency of numerical solutions for shallow water equations.

Author Contributions

Conceptualization was completed by Y.L. and Q.S. Methodology was developed by Q.S., Y.L. and J.W. Formal analysis was conducted by Q.S., Y.L., J.W. and C.H. The original manuscript was jointly prepared by Y.L., Q.S., J.W. and C.H. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by Key Scientific Research Projects Plan of Henan Higher Education Institutions (24A120009).

Data Availability Statement

Data are contained within the article.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Kurganov, A. Finite-volume schemes for shallow-water equations. Acta Numer. 2018, 27, 289–351. [Google Scholar] [CrossRef]
  2. Dorodnitsyn, V.A.; Kaptsov, E.I. Discrete shallow water equations preserving symmetries and conservation laws. J. Math. Phys. 2021, 62, 083508. [Google Scholar] [CrossRef]
  3. Noelle, S.; Pankratz, N.; Puppo, G.; Natvig, J.R. Well-balanced finite volume schemes of arbitrary order of accuracy for shallow water flows. J. Comput. Phys. 2006, 213, 474–499. [Google Scholar] [CrossRef]
  4. Shamkhalchian, A.; De Almeida, G.A.M. Upscaling the shallow water equations for fast flood modelling. J. Hydraul. Res. 2020, 59, 739–756. [Google Scholar] [CrossRef]
  5. Valseth, E.; Dawson, C. A stable space-time FE method for the shallow water equations. Comput. Geosci. 2022, 26, 53–70. [Google Scholar] [CrossRef]
  6. Zhang, J.; Han, R.; Ni, G. High-order curvilinear Lagrangian finite element methods for shallow water hydrodynamics. Int. J. Numer. Methods Fluids 2023, 95, 1846–1869. [Google Scholar] [CrossRef]
  7. Cao, L. Deep Learning Applications. IEEE Intell. Syst. 2022, 37, 3–5. [Google Scholar] [CrossRef]
  8. Taflove, A.; Hagness, S.C.; Piket-May, M. Computational Electromagnetics: The Finite-Difference Time-Domain Method. In The Electrical Engineering Handbook; Elsevier: Amsterdam, The Netherlands, 2005; pp. 629–670. [Google Scholar]
  9. Temam, R. Navier Stokes Equations: Theory and Numerical Analysis. J. Appl. Mech. 1978, 45, 456. [Google Scholar] [CrossRef]
  10. Ames, W.F. Numerical Methods for Partial Differential Equations; Academic Press: Cambridge, MA, USA, 2014. [Google Scholar]
  11. Raissi, M.; Perdikaris, P.; Karniadakis, G.E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 2019, 378, 686–707. [Google Scholar] [CrossRef]
  12. Cha, W.; Li, D.; Shen, L.; Liu, X. A review of solving methods of partial differential equations based on neural networks. Chin. J. Theor. Appl. Mech. 2022, 54, 543–556. [Google Scholar]
  13. Li, Y.; Chen, S. Neural Networks Based on Physical Information: Recent Advances and Prospects. J. Comput. Phys. 2022, 49, 254–262. [Google Scholar]
  14. Sirignano, J.; Spiliopoulos, K. DGM: A deep learning algorithm for solving partial differential equations. J. Comput. Phys. 2018, 375, 1339–1364. [Google Scholar] [CrossRef]
  15. Mao, Z.; Jagtap, A.D.; Karniadakis, G.E. Physics-informed neural networks for high-speed flows. Comput. Methods Appl. Mech. Eng. 2020, 360, 112789. [Google Scholar] [CrossRef]
  16. Minbashian, H.; Giesselmann, J. Deep Learning for Hyperbolic Conservation Laws with Non-convex Flux. Proc. Appl. Math. Mech. 2021, 20, e202000347. [Google Scholar] [CrossRef]
  17. Jagtap, A.D.; Kharazmi, E.; Karniadakis, G.E. Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Comput. Methods Appl. Mech. Eng. 2020, 365, 113028. [Google Scholar] [CrossRef]
  18. Lu, L.; Meng, X.; Mao, Z.; Karniadakis, G.E. DeepXDE: A Deep Learning Library for Solving Differential Equations. SIAM Rev. 2021, 63, 208–228. [Google Scholar] [CrossRef]
  19. Liu, Y.; Wen, L.; Yan, X.; Guo, S.; Zhang, C.-A. Adaptive transfer learning for PINN. J. Comput. Phys. 2023, 490, 112291. [Google Scholar] [CrossRef]
  20. Cho, G.; Zhu, D.; Campbell, J.J.; Wang, M. An LSTM-PINN Hybrid Method to Estimate Lithium-Ion Battery Pack Temperature. IEEE Access 2022, 10, 100594–100604. [Google Scholar] [CrossRef]
  21. Xu, H.; Zhang, D. Robust discovery of partial differential equations in complex situations. Phys. Rev. Res. 2021, 3, 033270. [Google Scholar] [CrossRef]
  22. Liangliang, Y.; You, Z.; Liu, H.; Laijun, L. An improved method for Physics-Informed Neural Networks that accelerates convergence. IEEE Access 2024, 12, 23943–23953. [Google Scholar]
  23. Zheng, S.; Lin, Y.; Feng, J.; Jin, F. Viscous Regularization of the Shallow Water Wave Equation through PINN Algorithm. J. Comput. Phys. 2023, 40, 314–324. [Google Scholar]
  24. Jin, F.; Zheng, S.; Feng, J.; Lin, Y. Parallel physics-informed neural network algorithm for solving shallow water wave equations. Chin. J. Comput. Mech. 2024, 352–358. [Google Scholar]
  25. Psaros, A.F.; Kawaguchi, K.; Karniadakis, G.E. Meta-learning PINN loss functions. J. Comput. Phys. 2022, 458, 111121. [Google Scholar] [CrossRef]
  26. Liu, X.; Zhang, X.; Peng, W.; Zhou, W.; Yao, W. A novel meta-learning initialization method for physics-informed neural networks. Neural Comput. Appl. 2022, 34, 14511–14534. [Google Scholar] [CrossRef]
  27. Yunyun, L.; Zheng, S.; Feng, J.; Fang, J. Diffusive Regularization Inverse PINN Solutions to Discontinuous Problems. Appl. Math. Mech. 2023, 44, 112–122. [Google Scholar] [CrossRef]
  28. Sepp, H.; Jürgen, S. Long Short-Term Memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar]
  29. Atif, K.; Abrar, A.; Salman, J.; Muhammad, B.; Megat, F.Z. Abusive Language Detection in Urdu Text: Leveraging Deep Learning and Attention Mechanism. IEEE Access 2024, 12, 37418–37431. [Google Scholar]
  30. Ren, P.; Rao, C.; Liu, Y.; Wang, J.; Sun, H. PhyCRNet: Physics-informed convolutional-recurrent network for solving spatiotemporal PDEs. Comput. Methods Appl. Mech. Eng. 2022, 389, 114399. [Google Scholar] [CrossRef]
  31. Zhang, R.; Liu, Y.; Sun, H. Physics-informed multi-LSTM networks for metamodeling of nonlinear structures. Comput. Methods Appl. Mech. Eng. 2020, 369, 113226. [Google Scholar] [CrossRef]
  32. Hinton, G.E.; Srivastava, N.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv 2012, arXiv:1207.0580. [Google Scholar]
  33. Moradi, R.; Berangi, R.; Minaei, B. A survey of regularization strategies for deep models. Artif. Intell. Rev. 2019, 53, 3947–3986. [Google Scholar] [CrossRef]
  34. Sarah, F.; Andreas, G.; Katja, I.; Thomas, K.; Markus, P.; Jörg, R.; Tim, F. Regularization approaches in clinical biostatistics: A review of methods and their applications. Stat. Methods Med. Res. 2022, 32, 425–440. [Google Scholar]
Symmetry 16 01376 g001
Figure 1. Diagrammatic sketch of PINN.
Figure 1. Diagrammatic sketch of PINN.
Symmetry 16 01376 g001
Symmetry 16 01376 g002
Figure 2. LSTM network architecture diagram.
Figure 2. LSTM network architecture diagram.
Symmetry 16 01376 g002
Symmetry 16 01376 g003
Figure 3. Depicts a schematic diagram of the improved PINN structure.
Figure 3. Depicts a schematic diagram of the improved PINN structure.
Symmetry 16 01376 g003
Symmetry 16 01376 g004
Figure 4. Comparative analysis of L 1 and L 2 regularization loss in one-dimensional and two-dimensional shallow water equations.
Figure 4. Comparative analysis of L 1 and L 2 regularization loss in one-dimensional and two-dimensional shallow water equations.
Symmetry 16 01376 g004
Symmetry 16 01376 g005
Figure 5. Comparison of results on one dimension: (a) results of the classical PINN algorithm; (b) results of the improved PINN algorithm.
Figure 5. Comparison of results on one dimension: (a) results of the classical PINN algorithm; (b) results of the improved PINN algorithm.
Symmetry 16 01376 g005
Symmetry 16 01376 g006
Figure 6. One-dimensional loss function variation curve.
Figure 6. One-dimensional loss function variation curve.
Symmetry 16 01376 g006
Symmetry 16 01376 g007
Figure 7. Comparison of algorithm results at t = 0.03 and t = 0.21.
Figure 7. Comparison of algorithm results at t = 0.03 and t = 0.21.
Symmetry 16 01376 g007
Symmetry 16 01376 g008
Figure 8. Comparison of algorithm results at t = 0.30 and t = 0.37.
Figure 8. Comparison of algorithm results at t = 0.30 and t = 0.37.
Symmetry 16 01376 g008
Symmetry 16 01376 g009
Figure 9. Two-dimensional loss function variation curve.
Figure 9. Two-dimensional loss function variation curve.
Symmetry 16 01376 g009
Table 1. Value of RMSE.
Table 1. Value of RMSE.
Time0.080.120.16
Classical PINN0.05030.04490.0382
Improved PINN0.03360.03180.0312
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Li, Y.; Sun, Q.; Wei, J.; Huang, C. An Improved PINN Algorithm for Shallow Water Equations Driven by Deep Learning. Symmetry 2024, 16, 1376. https://doi.org/10.3390/sym16101376

AMA Style

Li Y, Sun Q, Wei J, Huang C. An Improved PINN Algorithm for Shallow Water Equations Driven by Deep Learning. Symmetry. 2024; 16(10):1376. https://doi.org/10.3390/sym16101376

Chicago/Turabian Style

Li, Yanling, Qianxing Sun, Junfang Wei, and Chunyan Huang. 2024. "An Improved PINN Algorithm for Shallow Water Equations Driven by Deep Learning" Symmetry 16, no. 10: 1376. https://doi.org/10.3390/sym16101376

APA Style

Li, Y., Sun, Q., Wei, J., & Huang, C. (2024). An Improved PINN Algorithm for Shallow Water Equations Driven by Deep Learning. Symmetry, 16(10), 1376. https://doi.org/10.3390/sym16101376

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop
  NODES
3d 3
Association 2
coding 2
Experiments 4
games 2
games 2
Idea 4
idea 4
innovation 2
Interesting 2
Intern 30
iOS 9
Javascript 2
languages 2
mac 31
Note 14
os 172
text 7
Training 19
twitter 1
visual 2
web 2