CZ4042: Neural networks

 

Year 4 elective offered in semester 1

Instructor: Professor Jagath Rajapakse

 

 

SYLLABUS

 

1. Introduction to neural network

Overview, history, biological neural networks, artificial neuron, analysis & characteristics of neural networks, architectures, limitations

 

2. Perceptron

Principles of simple perceptron, pattern recognition, mathematical modeling, learning algorithms and limitations

 

3. Regression

Linear regression, linear neuron, least mean square algorithm, least squares estimator, logistic regression, softmax regression

 

4. Multi-layer Perceptron

Delta learning rule, generalized delta learning rule, backpropagation learning algorithm, mathematical modeling, stochastic gradient descent algorithm

 

5. Performance estimation and model selection

Motivation, holdout, re-sampling techniques, bootstrap, three-way data splits, optimization

 

6. Radial basis function networks

Basis function networks, basis function, radial basis function networks, approximation and interpolation approach, learning radial basis function networks,  k-means algorithm

 

7. Convolution neural networks and deep learning

Feature extraction by convolution, pooling, convolutional neural networks, deep learning

 

8. Associative Learning and autoencoders

Associative memory networks, bidirectional associative networks, correlation networks, autoencoders, canonical correlation analysis

 

9. Self-Organizing Neural Network

Self-organization in the human brain, Kohonen networks, Kohonen learning rule, self-organizing feature maps, vector quantization

 

10. Component neural networks

Principal component analysis, principal component neural networks, whitening, independent component analysis, independent component neural networks

 

 

TEXTS

1.     Neural Networks and Learning Machines, 3rd edition, Simon Haykin, Pearson Prentice Hall, 2009

http://www.mif.vu.lt/~valdas/DNT/Literatura/Haykin09/Haykin09.pdf

2.  Deep learning tutorial:

     http://ufldl.stanford.edu/tutorial/

 

 

ASSESSMENT

Tutorial presentation (5%)

Project 1 (20%)

Project 2 (20%)

Final exam (55%)