Inverse Heat Transfer with PINNs

What it does

A re-implementation, with improvements, of a research paper that solves a 1D inverse heat transfer problem with Physics-Informed Neural Networks (Billah et al., 2023), done with Keshav Kumar.

• Synthetic training data generated with Green's functions.
• A PINN with a custom learning-rate scheduler that co-optimises the network weights and the thermophysical parameters.
• 99% accuracy for time-invariant flux and 97% for time-variant flux.
• Converges ~10× faster than the original paper.
• The repo has the data generation (GT.ipynb), the training (iht_with_pinn.ipynb) and a full report (report.pdf).

The cell below computes the problem's exact solution live: a slab heating up, and the points a PINN would be trained on. Change the heat-flux case, or α to see how fast heat spreads.

The problem

A slab of length $L = 1$ m starts at $T = 0$ (temperatures are measured from the starting temperature). A heat flux $q(t)$ enters at the left face and the right face is insulated:

$$\frac{\partial T}{\partial t} = \alpha\,\frac{\partial^{2} T}{\partial x^{2}}, \qquad -k\left.\frac{\partial T}{\partial x}\right|_{x=0} = q(t), \qquad -k\left.\frac{\partial T}{\partial x}\right|_{x=L} = 0, \qquad T(x, 0) = 0$$

The forward problem is to find $T(x, t)$ when $q$ and the thermal diffusivity $\alpha$ are known. The inverse problem goes the other way: from measured temperatures, find $q(t)$ and $\alpha$. It is ill-posed (many parameter sets fit noisy data almost equally well), and it is what a PINN can do that an ordinary neural network can't: the unknown parameters sit inside the loss, next to the network's weights, and are learned with them.

Three cases, as in the paper:

1. Constant flux: $q = 10{,}000$ W/m², for $t \in [0, 3600]$ s. Here both $q$ and $\alpha$ are unknown.
2. Linear flux: $q(t) = 10{,}000 + 5t$ W/m², for $t \in [0, 2000]$ s.
3. Sinusoidal flux: $q(t) = 15{,}000 + 10{,}000\sin(\omega t)$ W/m², one period in 1600 s ($\omega = 2\pi/1600$).

In the time-varying cases $\alpha$ is known and $q(t)$ is recovered at 9 times.

Ground truth from Green's functions

The training data comes from the exact solution, found with Green's functions. For the constant flux:

$$T(x, t) = \frac{\alpha q_0 t}{kL} + \frac{2 q_0}{kL} \sum_{n=1}^{\infty} \frac{\cos(\gamma_n x)\left[1 - e^{-\alpha\gamma_n^{2} t}\right]}{\gamma_n^{2}}, \qquad \gamma_n = \frac{n\pi}{L}$$

The time-varying fluxes add their own terms. For $q_1 t$:

$$\frac{\alpha q_1 t^{2}}{2kL} + \frac{2 q_1}{\alpha kL} \sum_{n=1}^{\infty} \frac{\cos(\gamma_n x)\left[\alpha\gamma_n^{2} t - 1 + e^{-\alpha\gamma_n^{2} t}\right]}{\gamma_n^{4}}$$

and for $q_1 \sin(\omega t)$:

$$\frac{\alpha q_1\left(1 - \cos\omega t\right)}{\omega kL} + \frac{2\alpha q_1}{kL} \sum_{n=1}^{\infty} \frac{\cos(\gamma_n x)\left[\alpha\gamma_n^{2}\sin\omega t - \omega\cos\omega t + \omega e^{-\alpha\gamma_n^{2} t}\right]}{\left(\alpha\gamma_n^{2}\right)^{2} + \omega^{2}}$$

Each sum is cut at $m = 500$ terms. The first part is the heat that has come in, spread evenly; the series describes how it is distributed along the slab.

The paper gives $\alpha = 9.868 \times 10^{-5}$ m²/s but not the conductivity $k$. It can be recovered from the paper's plot of $T$ at $t = 3300$ s, where $T(0) \approx 27.45$ °C: at that time $e^{-\alpha\gamma_n^{2} t}$ is negligible from $n = 3$ on, and with $\sum 1/n^{2} = \pi^{2}/6$ the series at $x = 0$ can be summed by hand. That gives $k \approx 237.1$ W/(m·K), so $k = 237$ throughout.

Training data

The solution is sampled every $\Delta x = 0.005$ m (201 points) and every 15 s (constant case, a $241 \times 201$ grid) or 1 s (the other two).

• Constant flux: 2,200 points chosen at random out of the 48,441.
• Time-varying flux: all 201 points at each of 9 times, so 1,809 points in all.
• Noisy variants: each temperature gets $0.10\,\sigma_x\,\mathcal{N}(0, 1)$ added, where $\sigma_x$ is the spread of $T$ over time at that position: 10% noise.

The PINN

A fully connected network takes $(x, t)$ and predicts $T$. The thermophysical parameters $q$ and $\alpha$ are two extra trainable numbers. The loss has three parts:

$$\mathcal{L} = \lambda_{\text{data}}\,\overline{\left(T_\theta - T\right)^{2}} + \lambda_{\text{gov}}\,\overline{\left(\frac{\partial T_\theta}{\partial t} - \alpha\,\frac{\partial^{2} T_\theta}{\partial x^{2}}\right)^{2}} + \lambda_{\text{bc}}\,\overline{\left(\left.\frac{\partial T_\theta}{\partial x}\right|_{x=0} + \frac{q}{k}\right)^{2}}$$

The data term fits the measurements; the governing-equation term makes the network obey the heat equation at the training points (this is where $\alpha$ is learned); the boundary term ties the slope at $x = 0$ to the flux (this is where $q$ is learned). The derivatives come from automatic differentiation in PyTorch.

Time is scaled to $t' = t/t_{\max}$, and for the time-varying cases the temperature to $T' = T/T_{\max}$. The equations then hold with $\alpha' = \alpha\,t_{\max}$ and $q' = q/T_{\max}$, so all the learned numbers are of order 1.

For the time-varying cases, one network is trained per time (on that time's 201 points) and the governing-equation term is dropped, as in the paper: each network gives $q$ at its time.

Faster convergence

The paper uses one learning rate, 0.001, for everything. Its training took about 11.2 hours (an estimate from the paper's table for 2,000 points). Re-created exactly, it took 12 hours here. The improved training gives each part its own learning rate and schedule:

Case	Network	Schedule
Constant	8 × 20 tanh	$\eta_q = 5$ (warming up from 0.1 over 2,000 epochs), $\eta_{NN} = \eta_\alpha = 0.001$; $\lambda_{\text{data}} = 20$, $\lambda_{\text{gov}} = 10$, $\lambda_{\text{bc}} = 1$
Time-varying	3 × 30 SiLU	epochs 0–20k: fit the data only ($\lambda_{\text{bc}} = 0$); 20k–30k: network frozen ($\eta_{NN} = 0$) while $q$ learns from its slope; then both fine-tune ($\eta_{NN} = 5 \times 10^{-5}$, $\eta_q$ 6.5 → 5.2 at 40k)

Every run starts from $q = 0$ (and $\alpha = 0.001$ in the constant case).

Results

Constant flux, true values $q = 10{,}000$ W/m² and $\alpha = 9.868 \times 10^{-5}$ m²/s:

Data	Paper: $\alpha$	Paper: $q$	This work: $\alpha$	This work: $q$	Training time
No noise	$9.895 \times 10^{-5}$	9,989	$9.862 \times 10^{-5}$	9,999.7	0.53 h (paper ~11.2)
10% noise	$9.854 \times 10^{-5}$	9,968	$9.790 \times 10^{-5}$	10,042	0.48 h (paper ~11.2)

For the linear and sinusoidal fluxes (no noise), the recovered $q$ at the 9 times is within 1.2% of the truth on average, after about 1.5 hours of training each.

The cell below replays those training runs from the values the notebook logged every 1,000 epochs: $q$ and $\alpha$ walking towards the truth, and, for the time-varying cases, the 9 recovered fluxes rising onto the true $q(t)$ once the boundary term is switched on.

References

1. M. M. Billah, A. I. Khan, J. Liu and P. Dutta, Physics-informed deep neural network for inverse heat transfer problems in materials, Materials Today Communications 35, 106336, 2023.