|
Lecturer(s)
|
-
Bukovský Ivo, doc. Ing. Ph.D.
|
|
Course content
|
COMMON TOPICS T1 ? Introduction to Data-Driven Modelling in Physics Data-driven, model-driven, and hybrid approaches; role of machine learning; explainability, convergence, and stability of neural architectures; black-box vs. interpretable models; in-parameter-linear architectures. T2 ? LNU, HONU, and In-Parameter-Linear Neural Models LNU and HONU models; polynomial input expansion; incremental learning and backpropagation; weight interpretation; approximation of dynamical systems and state-space representation. T3 ? State-Space Models and Their Neural Representations Discrete-time state-space models; modelling systems with known states; derivation from HONU models; latent states; transition to attention-based architectures. T4 ? MLP, GRU, and LSTM Architectures MLP, GRU, and LSTM networks; Backpropagation Through Time; modelling long-term dependencies; interpretability; comparison with shallow neural models. T5 ? Optimization Algorithms for Neural Learning Gradient-based methods, conjugate gradient, and Levenberg?Marquardt optimization; convergence; loss minimization; regularization and hyperparameter tuning. T6 ? Dimensionality Reduction and Latent Space Projections PCA, t-SNE, and UMAP; visualization and compression of multivariate data; feature preparation for modelling and clustering. T7 ? Clustering and Self-Organizing Representations K-means, DBSCAN, and SOM; unsupervised state recognition; topological data representation; applications to physical signals. T8 ? Model Validation, Interpretability, and Generalization Model validation; sensitivity and collinearity analysis; Learning Entropy and AISLE metrics; robustness, novelty detection, interpretability; semester project. FOCUS TOPICS (At least four topics will be selected according to the students' specialization.) F1 ? Applied Fractal-Based Features in Prediction of Complex Dynamical Systems Fractal dimension, recurrence analysis, and entropy-based features; modelling turbulence, chaotic systems, and physiological signals. F2 ? Learning Entropy and Weight Dynamics Monitoring Learning Entropy and AISLE; monitoring weight updates; change detection and novelty assessment. F3 ? Deriving State-Space Representations from HONU Models State-space models derived from HONU; systems with known states; interpretability, convergence, and stability. F4 ? Attention Mechanisms in Systems with Unknown States Attention, self-attention, and multi-head attention; modelling latent states; anomaly detection and behavior segmentation. F5 ? Latent Space Analysis of Neural Networks Latent data representations; geometry and interpretation of latent spaces; clustering, visualization, and system identification. F6 ? Physics-Informed Neural Networks (PINNs) PINNs; PDE-constrained loss functions; automatic differentiation; collocation points; DeepONets and Fourier Neural Operators. F7 ? Autoencoders and Variational Models for Latent Feature Extraction Autoencoders and VAEs; data compression, denoising, and latent feature extraction; system identification. F8 ? Predictive Control with Neural Models Model predictive control; HONU-based predictors; optimization, feedback control, and stability analysis. F9 ? Modern State-Space Neural Architectures: Mamba and Beyond Mamba and state-space sequence models; linear-time memory; comparison with RNNs and Transformers; modelling physical processes. F10 ? Spiking Neural Networks and Event-Based Preprocessing Spiking neural networks; temporal spike coding; STDP and surrogate gradients; event-based data processing; neuromorphic hardware.
|
|
Learning activities and teaching methods
|
|
unspecified
|
|
Learning outcomes
|
The course focuses on data-driven modelling methods of physical systems using neural networks and advanced machine learning algorithms. The lectures consist of 8 common topics (T1-T8) and at least 4 specialized topics (F1?F10) according to the students? area of focus.
|
|
Prerequisites
|
linear algebra, ordinary differential equations, probability fundamentals, basic use of Python
|
|
Assessment methods and criteria
|
unspecified
Credit is given for active participation in classes and consultations and for semestral project , combined exam
|
|
Recommended literature
|
-
G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, ?Physics-informed machine learning,? Nat Rev Phys, vol. 3, no. 6, pp. 422?440, May 2021, doi: 10.1038/s42254-021-00314-5..
-
I. Bukovsky, G. Dohnal, et al, ?Letter on Convergence of In-Parameter-Linear Nonlinear Neural Architectures with Gradient Learnings,? IEEE Trans. Neural Networks and Learning Systems, vol. 34, no. 8, pp. 5189?5192, Aug. 2023, doi: 10.1109/TNNLS.2021.3123533..
-
I. Bukovsky, Interaktivní výukové podklady v Jupyter Notebook, Dpt. of Computer Science, Faculty of Science, Univ. of South Bohemia in Ceske Budejovice (TEAMs, MOODLE).
-
M. Raissi, P. Perdikaris, and G. E. Karniadakis, ?Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,? Journal of Computational Physics, vol. 378, pp. 686?707, Feb. 2019, doi: 10.1016/j.jcp.2018.10.045..
-
Physics-Informed Machine Learning,? in Fundamentals of Pattern Recognition and Machine Learning, Cham: Springer International Publishing, 2024, pp. 293?324. doi: 10.1007/978-3-031-60950-3_12..
-
S. L. Brunton and J. N. Kutz, Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control, 2nd ed. Cambridge: Cambridge University Press, 2022. https://doi.org/10.1017/9781009089517 (volně dostupné, free book: https://databookuw.com/databookV2.pdf).
|