**CZ4042: Neural networks**

Year 4
elective offered in semester 1

Instructor:
Professor Jagath Rajapakse

**SYLLABUS**

1.
Introduction to neural network

Overview,
history, biological neural networks, artificial neuron, analysis & characteristics
of neural networks, architectures, limitations

2.
Perceptron

Principles
of simple perceptron, pattern recognition, mathematical modeling, learning
algorithms and limitations

3.
Regression

Linear
regression, linear neuron, least mean square algorithm, least squares
estimator, logistic regression, softmax regression

4.
Multi-layer Perceptron

Delta
learning rule, generalized delta learning rule, backpropagation
learning algorithm, mathematical modeling, stochastic gradient descent
algorithm

5.
Performance estimation and model selection

Motivation,
holdout, re-sampling techniques, bootstrap, three-way data splits, optimization

6. Radial
basis function networks

Basis
function networks, basis function, radial basis function networks, approximation
and interpolation approach, learning radial basis function networks, k-means
algorithm

7.
Convolution neural networks and deep learning

Feature
extraction by convolution, pooling, convolutional neural networks, deep
learning

8.
Associative Learning and autoencoders

Associative
memory networks, bidirectional associative networks, correlation networks, autoencoders, canonical correlation analysis

9.
Self-Organizing Neural Network

Self-organization
in the human brain, Kohonen networks, Kohonen learning rule, self-organizing feature maps, vector
quantization

10.
Component neural networks

Principal
component analysis, principal component neural networks, whitening, independent
component analysis, independent component neural networks

**TEXTS**

1.
Neural
Networks and Learning Machines, 3rd edition, Simon Haykin, Pearson Prentice
Hall, 2009

http://www.mif.vu.lt/~valdas/DNT/Literatura/Haykin09/Haykin09.pdf

2. Deep learning tutorial:

http://ufldl.stanford.edu/tutorial/

**ASSESSMENT**

Tutorial presentation (5%)

Project 1 (20%)

Project 2 (20%)

Final exam (55%)