CZ4042: Neural networks


Year 4 elective offered in semester 1

Instructor: Professor Jagath Rajapakse





1. Introduction to neural network

Overview, history, biological neural networks, artificial neuron, analysis & characteristics of neural networks, architectures, limitations


2. Perceptron

Principles of simple perceptron, pattern recognition, mathematical modeling, learning algorithms and limitations


3. Regression

Linear regression, linear neuron, least mean square algorithm, least squares estimator, logistic regression, softmax regression


4. Multi-layer Perceptron

Delta learning rule, generalized delta learning rule, backpropagation learning algorithm, mathematical modeling, stochastic gradient descent algorithm


5. Performance estimation and model selection

Motivation, holdout, re-sampling techniques, bootstrap, three-way data splits, optimization


6. Radial basis function networks

Basis function networks, basis function, radial basis function networks, approximation and interpolation approach, learning radial basis function networks,  k-means algorithm


7. Convolution neural networks and deep learning

Feature extraction by convolution, pooling, convolutional neural networks, deep learning


8. Associative Learning and autoencoders

Associative memory networks, bidirectional associative networks, correlation networks, autoencoders, canonical correlation analysis


9. Self-Organizing Neural Network

Self-organization in the human brain, Kohonen networks, Kohonen learning rule, self-organizing feature maps, vector quantization


10. Component neural networks

Principal component analysis, principal component neural networks, whitening, independent component analysis, independent component neural networks




1.     Neural Networks and Learning Machines, 3rd edition, Simon Haykin, Pearson Prentice Hall, 2009

2.  Deep learning tutorial:




Tutorial presentation (5%)

Project 1 (20%)

Project 2 (20%)

Final exam (55%)