Artificial Intelligence Engines
A Tutorial Introduction to the Mathematics of Deep Learning
Preface
List of Pseudocode Examples
Online Code Examples
1 Artificial Neural Networks
1.1 Introduction
1.2 What is an Artificial Neural Network?
1.3 The Origins of Neural Networks
1.4 From Backprop to Deep Learning
1.5 An Overview of Chapters
2 Linear Associative Networks
2.1 Introduction
2.2 Setting One Connection Weight
2.3 Learning One Association
2.4 Gradient Descent
2.5 Learning Two Associations
2.6 Learning Many Associations
2.7 Learning Photographs
2.8 Summary
3 Perceptrons
3.1 Introduction
3.2 The Perceptron Learning Algorithm
3.3 The Exclusive OR Problem
3.4 Why Exclusive OR Matters
3.5. Summary
4 The Backpropagation Algorithm
4.1 Introduction
4.2 The Backpropagation Algorithm
4.3 Why Use Sigmoidal Hidden Units?
4.4 Generalisation and Overfitting
4.5 Vanishing Gradients
4.6 Speeding Up Backprop
4.7 Local and Global Mimima
4.8 Temporal Backprop
4.9 Early Backprop Achievements
4.10 Summary
5 Hopfield Nets
5.1 Introduction
5.2 The Hopfield Net
5.3 Learning One Network State
5.4 Content Addressable Memory
5.5 Tolerance to Damage
5.6 The Energy Function
5.7 Summary
6 Boltzmann Machines 6.1 Introduction
6.2 Learning in Generative Models
6.3 The Boltzmann Machine Energy Function
6.4 Simulated Annealing
6.5 Learning by Sculpting Distributions
6.6 Learning in Boltzmann Machines
6.7 Learning by Maximising Likelihood
6.8 Autoencoder Networks
6.9 Summary
7 Deep RBMs 7.1 Introduction
7.2 Restricted Boltzmann Machines
7.3 Training Restricted Boltzmann Machines
7.4 Deep Autoencoder Networks
7.5 Summary
8 Variational Autoencoders 8.1 Introduction
8.2 Why Favour Independent Features?
8.3 Overview of Variational Autoencoders
8.4 Latent Variables and Manifolds
8.5 Key Quantities
8.6 How Variational Autoencoders Work
8.7 The Evidence Lower Bound
8.8 An Alternative Derivation
8.9 Maximising the Lower Bound 8.10 Conditional Variational Autoencoders 8.11 Summary
9 Deep Backprop Networks 9.1 Introduction
9.2 Convolutional Neural Networks
9.3 LeNet1
9.4 LeNet5
9.5 AlexNet
9.6 GoogLeNet
9.7 ResNet
9.8 Ladder Autoencoder Networks
9.9 Denoising Autoencoder Networks
9.10 Fooling Neural Networks 9.11 Generative Adversarial Networks 9.12 Temporal Deep Neural Networks 9.13 Capsule Neural Networks 9.14 Summary
10 Reinforcement Learning 10.1 Introduction
10.2 What's the Problem?
10.3 Key Quantities
10.4 Markov Decision Processes
10.5 Formalising the Problem
10.6 The Bellman Equation
10.7 Learning State-Value Functions
10.8 Eligibility Traces
10.9 Learning Action-Value Functions 10.10 Balancing a Pole 10.11 Applications 10.12 Summary
11 The Emperor’s New AI? 11.1 Artificial Intelligence
11.2 Yet Another Revolution?
Further Reading
A Glossary
B Mathematical Symbols
C A Vector Matrix Tutorial
D Maximum Likelihood Estimation
E Bayes’ Theorem
Appendices
References
Index