Deep Learning
Deep Learning - from Neural Networks, Activation Functions, to Back Propagation
Chapters
- 01Introduction
Introduction to Deep Learning
- 02XOR Problem
XOR Problem - Why Linear/Logistic Regression Can't Solve it ?
- 03Activation Function
Activation Functions - Sigmoid, TanH, ReLU, Softmax
- 04Optimization Method
Optimization Methods - Momentum, AdaGrad, RMSProp & Adam
- 05Regularization
Regularization Methods - Early Stopping, Dropout & L2 Regularization
- 06Batch Normalization
Batch Normalization
- 07Back Propagation
Back Propagation
This course includes
7 articles
37 mins reading time
11 videos • 4 hr 19 mins
Course content
7 sections • 39 topics • Read: 37 mins • Video: 4 hr 19 mins
Introduction
Deep Learning
History of Deep Learning
Why did Deep Learning take-off now ?
Hardware
Data
Algorithmic Advances
What is Deep Learning | Why is Deep Learning Taking Off Now?
XOR Problem
Perceptron (1958)
XOR Problem Solution
XOR Problem Solution Code
What is XOR Problem? | Why Deep Learning is Required To Solve XOR Problem ?
Activation Function
Universal Approximation Theorem
Common Activation Functions
Sigmoid
Hyperbolic Tangent (TanH)
Rectified Linear Unit (ReLU)
Leaky Rectified Linear Unit (Leaky ReLU)
Softmax
Activation Function | Why Non Linearity is important in Deep Learning ? | Sigmoid | ReLU | Softmax
Optimization Method
Non-Convex Loss Surface
Issues with Non-Convex Surface
Stochastic Gradient Descent
Learning Rate Decay
Terminologies
Optimization Methods
Momentum Based Optimizer
Adaptive Gradient (AdaGrad)
Root Mean Square Propagation (RMSProp)
Adaptive Moment Estimation (Adam)
How to Accelerate Optimization in Deep Learning | Issues with Non Convex Loss | Learning Rate Decay
Why we should use Momentum Based Gradient Descent | Accelerate Deep Learning Training Process
Adaptive Gradient (AdaGrad) | Parameter Specific Optimization | Vanishing Learning Rate Problem
Root Mean Square Propagation (RMSProp) Optimization | Deep Learning | Vanishing Learning Rate
Adaptive Moment Estimation (Adam) Optimizer | Why is it the Industry Standard ? | Fast Convergence
Regularization
Start with Overfitting
L2 Regularization
Early Stopping
Dropout
How Dropout Works
Co-Adaptation
Weight Scaling Inference Rule
Code: Regularization Methods
Regularization in Deep Learning | Dropout | Early Stopping | L2 Regularization | Explained with Code
Batch Normalization
Batch Normalization
Batch Normalization Benefits
Code: Batch Normalization
Batch Normalization | Deep Learning | Internal Covariate Shift | Detailed Explanation
Back Propagation
Back Propagation
Back Propagation | Deep Learning | Explained with Gear Analogy Example | Chain Rule