2026 PINNs Workshop

Published

26/06/2026

Last updated: 26/06/2026

Welcome to 2026 PINNs Workshop — a hands-on course building toward a real project: physics-informed inference of unknown drivers in a 1D vertical ocean column with shallow-water coupling, motivated by AIMS mooring data.

Course material is 75% Julia, 25% Python, developed in collaboration with the Australian Institute of Marine Science (AIMS).

Created by Yoni Nazarathy at Accumulation Point, developed in collaboration with the AIMS. AIMS contact: Dr Takuya Iwanaga.

Prerequisite / Julia refresher — participants who want a gentler ramp into Julia before this workshop should work through the earlier AIMS workshop: Julia ML training. That course covers Julia basics, classical ML, and the SciML stack at a slower pace; this PINNs workshop assumes that level of comfort with Julia.

Workshop schedule

The material is more than fits in a week, so it is built to be paced. For a four-day workshop, a suggested route:

  • Day 1 — System setup, and Units 1, 2, and 3.
  • Day 2 — Follow-ups from Unit 3, then Units 4 and 5.
  • Day 3 — Skimming Unit 6 (the PDE bootcamp), then Units 7 and 8, and starting the project (Unit 9).
  • Day 4 — Continuation of the project (Units 9 and 10).
  • Extra day — Finalisation, specific questions, and further work on the project.

The unit-by-unit outline below has the full detail; this is just a guide to pacing.

Unit 1: Introduction

A worked end-to-end PINN inverse problem — recover the unknown Brisbane River freshwater surge from sparse Moreton Bay tide gauges.

  • A hand-written finite-difference shallow-water reference solve produces a slider-driven animation.
  • Two recovery methods (adjoint Tikhonov inverse vs naive PINN) trained against the same noisy gauge data, to make the ill-posedness visible.
  • Closes with a “How to get these files” callout and a preview of the AIMS thermistor-column capstone that Units 9–10 build properly.
  • Treat the whole unit as motivation: nothing here is re-derived until Units 5–7.

Unit 2: ML Foundations: From Classical to Deep

The supervised-learning foundations PINNs rest on, walked at depth on the MNIST 60k/10k benchmark throughout.

  • Random forest baseline in DecisionTree.jl then scikit-learn.
  • Softmax regression in Lux.jl then scikit-learn.
  • Full MLP training with hand-written mini-batch SGD in Lux, plus PyTorch and sklearn MLP parallels.
  • Optimisation deep-dive derives gradient descent, mini-batch SGD, momentum, Adam (with explicit bias correction), and L-BFGS at the depth of Mathematical Engineering of Deep Learning Ch. 4.
  • Closes with an introduction to Kolmogorov-Arnold Networks (KANs) with KolmogorovArnold.jl and pykan examples.

Unit 3: Scientific Machine Learning and Physics-Informed Machine Learning — an overview

A broad overview — a guided tour of the physics-aware ML landscape (the deep dive on Neural ODEs is Unit 4) — with Julia and Python implementations side by side throughout.

  • ODE refresher: three problems of increasing complexity (1-D exponential decay, undamped harmonic oscillator, damped harmonic oscillator with under/critical/over regimes) in OrdinaryDiffEq.jl and scipy.integrate.solve_ivp.
  • Hybrid physics+ML, surrogates, conservation inductive biases, Hamiltonian / Lagrangian networks.
  • SINDy with a hand-rolled STLSQ Lorenz recovery in Julia and the PySINDy parallel.
  • Proper Orthogonal Decomposition on a 50-mass damped chain (Julia + Python).
  • Closes with the conceptual PINN setup that Units 5–9 implement.

Unit 4: Learning Dynamics with Neural Differential Equations

ODEs as models of dynamics, framed through universal approximation rather than classical numerics.

  • The ResNet-to-Neural-ODE bridge from Unit 2, and the discretise-then-differentiate / differentiate-then-discretise / checkpointing trichotomy for backprop through ODE solvers.
  • Universal Differential Equations (UDEs) as the default “known physics + learned closure” pattern for scientific ML.
  • A Lotka-Volterra phase portrait (Julia + Python).
  • A damped-pendulum UDE with unknown friction.
  • The real-world AIMS domain case — a Crown-of-Thorns starfish UDE for the Great Barrier Reef (after Morello et al. 2014, MEPS), where COTS mortality is the learned closure on top of known logistic coral growth and Holling-II grazing.

Unit 5: PINNs on Basic Models

The first hands-on PINN — it frames the residual loss and collocation-point picture.

  • autodiff deep-dive: the forward / reverse / forward-over-forward composition PINNs depend on, with side-by-side Julia (Lux+ForwardDiff+Zygote) and JAX (jacfwd / grad) implementations and the “inner forward, outer reverse” rule of thumb.
  • Trains \dot x = -x from scratch in pure Lux + Zygote (one page of code).
  • Re-derives the same problem in NeuralPDE.jl.
  • Lifts to 1-D diffusion as the first true PDE-PINN.
  • Catalogues three failure modes (spectral bias, loss imbalance, causal violation) that Unit 7 later fixes.

Unit 6: PDE Bootcamp

The PDE-theory bedrock the ML-only reader needs, from classification through to classical discretisation.

  • Classification (elliptic / parabolic / hyperbolic), boundary conditions (Dirichlet / Neumann / Robin / periodic), Hadamard well-posedness, and a weak-vs-strong solutions subsection that lands the FEM weak form later.
  • Three explicit solution techniques at depth — separation of variables on the heat rod (worked end-to-end), Fourier series as the eigenmode toolbox, Laplace transforms with the half-line erfc heat solution.
  • A tour of the five canonical PDEs (heat, Laplace, Poisson, wave, advection) with physical pedagogy beyond the formula.
  • FD, FV, FE with a comparison table.
  • Revisits the Unit 1 shallow-water solver in classical PDE terms — hyperbolic classification, CFL, the Arakawa-C grid, the radiation-BC sponge.

Unit 7: When PINNs Meet PDEs

The applied core of the course — vanilla PINNs, their failure modes, the modern fixes, and the surrounding ecosystem.

  • A vanilla PINN on the heat equation and a Laplace solve on a disk (a polar heatmap of the closed-form r^3 \sin(3\theta) + collocation cloud).
  • The three failure modes against the Krishnapriyan / Wang literature — not the de Wolff benchmark — with a worked 1-D linearised SWE example and a three-panel failure-signature figure.
  • The modern fixes (causal training, Fourier features, hard BC, adaptive loss weighting).
  • A taxonomy of four PINN workflows practitioners actually use (forward solve, parameter identification, source recovery, hybrid data assimilation).
  • An honest “when inverse PINNs work and when they don’t” discussion.
  • The software ecosystem (Julia NeuralPDE.jl + SciML stack, Python DeepXDE / PhysicsNeMo / jinns, commercial offerings), seven literature application areas, and named industrial deployments (NVIDIA PhysicsNeMo at Siemens Energy, Shell, SimScale, Ansys SeaScape; Altair PhysicsAI; Pasteur Labs).

Unit 8: PDE Modelling in Key AIMS Domains

The ocean physics the capstone column rests on, for readers fluent in PDEs but new to oceanography.

  • Derives the 1-D column equation from 3-D advection-diffusion by horizontal homogeneity, with a labelled SVG schematic of the thermistor column.
  • Vertical advection and three eddy-diffusivity closure schemes (KPP, Mellor-Yamada, Pacanowski-Philander), plus where w(z, t) actually comes from on the GBR (tides, Ekman pumping, mean circulation).
  • The Beer-Lambert two-band penetrating-shortwave model and the COARE 3.0 bulk formulas for sensible / latent / longwave surface fluxes.
  • Stratification, dimensionless regimes, and pointers to the GBR / KPP / DHW literature this kind of column modelling builds on.

Unit 9: Project Specification

The capstone, in two parallel versions chosen by hardware and ambition.

  • Hardware/audience comparison table — pick A or B before reading either spec.
  • Task A — single-site CPU-friendly inverse. Cleveland Bay column (H = 15 m), small MLP, ~30 min on a laptop CPU. Full Spec / Workflow / Success criteria / Runtime / Deliverables / “what you don’t do here” sub-sections.
  • Task B — three-site joint GPU inverse. All three moorings (H up to 100 m), modern PINN toolkit, GPU-class. Same sub-numbering as Task A, plus a CPU sub-scale prototype ladder and an “open questions for the full GPU run” closer.
  • The shared scenario, coupled column + SWE model, boundary / initial conditions, forcing functions, and reference parameter values both tasks build on, plus a shared toy-task ladder.

Unit 10: Project Solution

The worked solutions to both capstone variants — read at your own risk. Each task solution sits behind its own triple-click reveal gate, so reading the shared infrastructure (or one task) needn’t spoil the other.

  • Shared infrastructure (open): the MethodOfLines reference solver, the four toy scenarios, and the SWE-driver design (the scenario-5 forcing bridge).
  • Task A walk-through: the source-recovery framing, the inverse loss with an ablation table on the data / regularisation weights, the ≈15% recovery, and a closed-form-validated forward sanity check.
  • Task B walk-through: maps each modern-toolkit ingredient (Fourier features, hard BC, adaptive weights, causal training) to the failure it addresses, then writes out the joint loss with a shared \tau_\phi across sites, the two-site CPU sub-scale prototype (where the shared-\tau joint measurably beats either single site), the mechanism partition, and a measured A10G full-scale performance table.
  • Sketches the DeepXDE Python parallel for the cross-ecosystem story.

Appendices

  • The @pinn Julia environment — the shared, pinned package environment every Julia cell runs against (identical on the hubs and the take-home Docker image): the exact packages, what’s precompiled and why the first run is the slow one, how the depot and stacked environments work, and why adding a package can trigger a one-time recompile.
  • Running it on your laptop — how to run the same environment on your own computer with the take-home Docker image: as a backup during the workshop, or to keep working on your project after the cloud servers are switched off.
  • Exercise Solutions — worked solutions to the hands-on “section exercise” boxes that close most sections of Units 1–10. Julia throughout, with JAX / Python variants where the comparison is instructive.
  • References — the consolidated bibliography. In-text citations across all units link here.
  • Software Resources — Julia packages (first) then Python packages (second), with one-line descriptions and canonical links.
  • Mathematical Notation — glossary of notation used throughout the course: partial derivatives, gradients, dot products, PINN-specific symbols.
  • Physical / Oceanography Review — glossary of physical and ocean-science terms: Coriolis, eddies, thermocline, mixing-layer closures, bulk flux formulas.

Supporting files

All the source code, data, and configuration the participants need to reproduce the workshop, with direct GitHub links. (If your team doesn’t have access to the GitHub repo, ask the workshop maintainer for a tarball — the file list below is the manifest.)

Course-wide configuration

Unit 1 — Brisbane River surge

  • Notebook: unit_01.ipynb — the runnable Jupyter version of the unit
  • Scripts: scripts/ — build_bay.jl, build_mask_from_tiles.py, generate_site_map.py, generate_surge_data.jl, generate_surge_frames.jl, generate_surge_animation.jl, surge_gpu.jl, replot_bathymetry.jl, render_recovery_plot.jl, train_inverse_pinn.jl, train_inverse_adjoint.jl
  • Data: data/ — bay_bathymetry.csv, bay_gauges.csv, bay_mask.csv, bay_meta.json, river_source.csv, gauges_clean.csv, gauges_observed.csv, gauges_adjoint_pred.csv, gauges_pinn_pred.csv, psi_truth.csv, psi_recovered.csv, psi_recovered_adjoint.csv, snapshots.bin, snapshots_meta.json

Unit 2 — ML foundations

  • Scripts: scripts/ — mnist_rf_julia.jl, mnist_rf_sklearn.py, mnist_display.jl, mnist_linear_lux.jl, mnist_linear_sklearn.py, mnist_mlp_sklearn.py, mnist_mlp_pytorch.py, mnist_gpu_lux.jl, conv_mnist_lux.jl, activations.jl, kan_julia.jl, kan_splines.jl, kan_pykan.py

Unit 3 — Sci-ML and PIML

  • Scripts: scripts/ — decay_python.py, damped_python.py, hnn_pendulum.jl, sindy_lorenz.jl, sindy_lorenz_python.py, pod_chain.jl, pod_chain_python.py

Unit 4 — Neural differential equations

  • Scripts: scripts/ — resnet_block.jl, neural_ode_train.jl, lotka_volterra_python.py

Unit 5 — First PINN

  • Scripts: scripts/ — pinn_neuralpde_ode.jl, pinn_neuralpde_heat.jl, pinn_poisson_disk.jl, pinn_2d_diffusion_gpu.jl

Unit 6 — PDE bootcamp

Unit 7 — When PINNs meet PDEs

  • Scripts: scripts/ — generate_laplace_disk_figure.py, pinn_throughput_gpu.jl

Unit 9 — Project specification

  • Scripts: scripts/ — generate_site_map.jl, generate_site_map.py

Unit 10 — Capstone solution

  • Scripts: scripts/ — column_fd.jl, column_pinn_gpu.jl, generate_mooring_csvs.jl, task_a_inverse_pinn.jl, task_a_forward_deepxde.py, task_b_subscale_prototype.jl, task_b_joint_inverse.jl, task_b_gpu_launch.md
  • Data: data/ — mooring_A.csv, mooring_B.csv, mooring_C.csv, sites_metadata.csv