|
Lecturer(s)
|
-
Bukovský Ivo, doc. Ing. Ph.D.
|
|
Course content
|
1. Types and principles of applications, adaptation principles, quantity and quality of training data, process non-stationarity, overtraining, generalization. 2. Concept of continuous-time and discrete-time mathematical-physical models of dynamic systems, data-driven modeling (prediction, approximation identification). 3. Basic neuron model (perceptron), static vs. dynamic neuron, incremental gradient learning algorithm of a static neuron. 4. Convergence of incremental learning of a static neuron (data normalization and learning rate optimization methods, RLS). 5. Formulation of the dynamics of step gradient learning of a static neuron and the influence of learning parameters on its stability (convergence of weights). 6. Dynamic neuron: incremental learning algorithm, weight convergence, dynamic stability of a neuron, adaptive filter analogy. 7. In-Parameter-Linear Nonlinear Neural Architectures (IPLNA), Taylor's series analogy. 8. Batch learning of static and dynamic neurons, Levenberg-Marquardt (LM) algorithm, learning rate, and its effects. 9. Conjugate gradient batch learning, a comparison. 10. Architectures of shallow neural networks: IPLNA, ELM, FRVL, properties, incremental and batch learning, weight convergence. 11. Shallow vs. deep neural networks, ADAM adaptive algorithm. 12. Adaptive anomaly detection, information content according to Shannon, Learning Entropy. 13. Explainability and trustworthiness of neural architectures (IPLNA, shallow networks, deep networks) in industrial applications. Obsah cvičení/semináře: Implementation and testing of applied adaptive algorithms and neural architectures on time series and on a dynamic system model, examples of applications (classification, approximation, prediction, detection, adaptive control) according to lecture topics.
|
|
Learning activities and teaching methods
|
- Preparation for credit
- 25 hours per semester
- Preparation for exam
- 25 hours per semester
- Preparation for classes
- 25 hours per semester
|
|
Learning outcomes
|
Students will learn the theoretical foundations of supervised learning algorithms and acquire practical skills with their implementation via simpler architectures of learning systems (shallow neural networks). They will learn about the tasks of adaptive classification, filtering, prediction, and adaptive monitoring of anomalies in the behavior of, e.g., dynamical biological system and adaptive controller. The subject thus introduces students to mathematical principles and their applications, where deep neural networks are not suitable, e.g., due to lack of data, the need for implementation on specific HW, and the need for explainability and trustworthiness of neural networks in industrial applications.
|
|
Prerequisites
|
Undergraduate mathematics, programming fundamentals, at minimum, Python or Matlab
|
|
Assessment methods and criteria
|
unspecified
Semestral work includes students' own design, implementation, and analysis of the functionality of adaptive neural architectures for assigned tasks with documentation.
|
|
Recommended literature
|
-
G. C. Layek, An Introduction to Dynamical Systems and Chaos, 1st ed. 2015. New Delhi: Springer India?: Imprint: Springer, 2015. doi: 10.1007/978-81-322-2556-0., https://link.springer.com/book/10.1007/978-81-322-2556-0.
-
I. Bukovsky and N. Homma, "An Approach to Stable Gradient-Descent Adaptation of Higher Order Neural Units," IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 9, pp. 2022-2034, Sep. 2017, doi: 10.1109/TNNLS.2016.2572310. (autorská verze na ArXive).
-
M. Kubat, An Introduction to Machine Learning. Cham: Springer Internatio-nal Publishing, 2017. doi: 10.1007/978-3-319-63913-0. https://link.springer.com/book/10.1007/978-3-319-20010-1.
-
Theoretical Biology and Medical Modelling (TBioMed), BioMed Central , (http://www.tbiomed.com/, Open Access).
|