Course Outline
DAY 1 - ARTIFICIAL NEURAL NETWORKS
Introduction and ANN Structure.
- Comparison of biological and artificial neurons.
- The modeling of ANNs.
- Activation functions employed in ANNs.
- Common classes of network architectures.
Mathematical Foundations and Learning mechanisms.
- Review of vector and matrix algebra.
- State-space concepts.
- Optimization principles.
- Error-correction learning methods.
- Memory-based learning approaches.
- Hebbian learning.
- Competitive learning.
Single layer perceptrons.
- Structure and learning process of perceptrons.
- Introduction to pattern classifiers and Bayes' classifiers.
- Using the perceptron as a pattern classifier.
- Perceptron convergence properties.
- Limitations inherent in perceptrons.
Feedforward ANN.
- Structures of Multi-layer feedforward networks.
- The Back propagation algorithm.
- Back propagation: training and convergence dynamics.
- Functional approximation via back propagation.
- Practical and design considerations for back propagation learning.
Radial Basis Function Networks.
- Pattern separability and interpolation concepts.
- Regularization Theory.
- Application of Regularization to RBF networks.
- RBF network design and training procedures.
- Approximation capabilities of RBFs.
Competitive Learning and Self organizing ANN.
- General clustering procedures.
- Learning Vector Quantization (LVQ).
- Competitive learning algorithms and their architectures.
- Self organizing feature maps.
- Characteristics of feature maps.
Fuzzy Neural Networks.
- Neuro-fuzzy systems.
- Foundations of fuzzy sets and logic.
- Design of fuzzy stems.
- Design of fuzzy ANNs.
Applications
- A discussion of several Neural Network application examples, highlighting their benefits and associated challenges.
DAY -2 MACHINE LEARNING
- The PAC Learning Framework
- Guarantees for finite hypothesis sets – consistent case
- Guarantees for finite hypothesis sets – inconsistent case
- Generalities
- Deterministic vs. Stochastic scenarios
- Bayes error noise
- Estimation and approximation errors
- Model selection
- Rademacher Complexity and VC – Dimension
- Bias - Variance tradeoff
- Regularisation
- Over-fitting
- Validation
- Support Vector Machines
- Kriging (Gaussian Process regression)
- PCA and Kernel PCA
- Self Organisation Maps (SOM)
- Kernel induced vector space
- Mercer Kernels and Kernel - induced similarity metrics
- Reinforcement Learning
DAY 3 - DEEP LEARNING
This will be taught in relation to the topics covered on Day 1 and Day 2
- Logistic and Softmax Regression
- Sparse Autoencoders
- Vectorization, PCA and Whitening
- Self-Taught Learning
- Deep Networks
- Linear Decoders
- Convolution and Pooling
- Sparse Coding
- Independent Component Analysis
- Canonical Correlation Analysis
- Demos and Applications
Requirements
A solid grasp of mathematics is essential.
A strong understanding of basic statistics is required.
Basic programming skills are not mandatory but are recommended for a smoother learning experience.
Testimonials (2)
Working from first principles in a focused way, and moving to applying case studies within the same day
Maggie Webb - Department of Jobs, Regions, and Precincts
Course - Artificial Neural Networks, Machine Learning, Deep Thinking
It was very interactive and more relaxed and informal than expected. We covered lots of topics in the time and the trainer was always receptive to talking more in detail or more generally about the topics and how they were related. I feel the training has given me the tools to continue learning as opposed to it being a one off session where learning stops once you've finished which is very important given the scale and complexity of the topic.
