BA Seminar 1 WF-FI-KGN-SLPE1

Graduate seminar/teaching methods: verbal and visual presentation of program content in the form of a PowerPoint presentation and a demonstration of network operation.

1. Introduction to the subject: Neural Networks and Cognitive Science; Machine Eyes; Neocognitron; Berkeley Institute;

2. Introduction to neural networks: the popularity of neural networks, the effectiveness of neural networks as nonlinear models of phenomena and processes, and side-aspects of the popularity of neural networks; Biological inspirations for neurocomputing; The basic model of a neuron and a neural network; Neural network operation and training;

3. Data processing methods: raw data acquisition, filtering, two-dimensional image processing, time series processing, feature extraction from two-dimensional images, and feature extraction from time series; Formatting image sets: creating and coding images describing objects, formatting tabular image sets, clustering; Processing training data sets;

4. Error backpropagation and its modifications: error growth rate algorithm, Park, Yun, and Kim's algorithm, Lee and Park's algorithm, Chan and Falside's algorithm, Jacobs' delta-delta algorithm, Silva and Almeida's algorithm, Sanossian and Evans's algorithm, Jacobs' delta-bar-delta algorithm, Fahlman's Quickprop algorithm, Ridmiller and Braun's RPROP algorithm, Karayiannis's algorithm;

5. Application of the Recursive Last Squares (RLS) algorithm to neural network training: single-layer network, multi-layer network;

6. Neural modular networks: general assumptions; Combining responses for a multi-network classifier: 1) at the abstraction level, 2) at the ranking level, 3) at the metric level (Bayesian probability measures); Architectures and training of modular networks; Selected application examples, e.g., handwritten digit recognition;

7. Self-organizing neural networks: Kohonen network - the concept of neighborhood and activation of neurons from the winner's neighborhood, the dead neuron problem; Self-organizing networks with the minimum entropy criterion; Neural gas algorithm;

8. Static and dynamic networks of the Group Method of Data Handing (GMDH): assumptions, neuron model, and network synthesis; Generalization of the GMDH algorithm: multi-input, multi-output system; Dynamic systems

9. Ontogenous neural networks: Structure-reducing models; Weight removal using Optimal Brain Damage (OBD) and Optimal Brain Surgeon (OBS) methods; Statistical methods for reducing neural networks; Models with expanding structures; Resource Allocation Network (RAN) with resource allocation;

10. Design strategies for neural networks: Pairable datasets, formal neuron layers; Rank layer design; Dipole layer design; Percepton Criterion Function; Dipole Criterion Function; Algorithms for Replacing Basic Solutions; Designing Hierarchical Structures;

11. Neural Network Architecture Optimization: Selecting a Neural Network Architecture: Selecting a Multilayer Perceptron (MLP) Network Architecture; Growth Methods: Tiling Algorithm, Frean Upstart Algorithm, Backpropagation Algorithm with Variable Number of Neurons, Dynamic Neuron Creation, Canonical Form Methods, Cascade Correlation Algorithm; Reduction Methods; Discrete Optimization Methods;

12. Example of Creating and Operating a Neural Network in Excel;

13. Introduction to Neural Network Applications: Function Approximation Using One-Way Neural Networks; Approximation Capabilities of Signoid Networks; Radial Basis Networks;

14. Invited Seminar on Artificial Neural Networks and Artificial Intelligence in its Broadest Sense;

15. Summary.

Syllabus prepared by: Paweł Pęczkowski