Training through Backpropagation

1.1: Course Overview
1.2: Unlock Discord Channels

2.1: Neural Networks: The Foundation of Deep Learning
2.2: Deep Learning vs Statistical ML: When to Use What?
2.3: Neural Network Architectures
2.4: Real World Applications of Deep Learning
2.5: Tooling: PyTorch vs Tensorflow
2.6: Tooling: GPU, TPU
2.7: Quiz
2.8: Chapter Summary

3.1: What is Neuron?
3.2: Perceptron and Multilayer Perceptron (MLP)
3.3: Neural Network Intuition: Insurance Prediction
3.4: The Purpose of Activation Function
3.5: Activation Functions: Sigmoid, ReLU, Tanh, SoftMax
3.6: Quiz
3.7: Chapter Summary

4.1: PyTorch Installation
4.2: Matrix Fundamentals
4.3: PyTorch Tensor Basics 1
4.4: PyTorch Tensor Basics 2
4.5: Derivative and Partial Derivative
4.6: Chain Rule
4.7: Autograd in PyTorch
4.8: Numpy Arrays vs PyTorch Tensors
4.9: Quiz
4.10: Exercise
4.11: Chapter Summary

5.1: Training through Backpropagation
5.2: Gradient Descent: Theoretical Foundation
5.3: Gradient Descent: PyTorch Implementation
5.4: Batch GD vs Mini Batch GD vs SGD
5.5: Mini Batch GD and SGD: PyTorch Implementation
5.6: Quiz
5.7: Exercise
5.8: Chapter Summary

6.1: Employee Bonus Prediction
6.2: Datasets and DataLoaders
6.3: Handwritten Digits Classification
6.4: Classification Report and Confusion Matrix
6.5: Cost Function: Binary Cross Entropy (a.k.a. Log Loss)
6.6: Cost Function: Cross Entropy
6.7: Quiz
6.8: Exercise
6.9: Chapter Summary

7.1: Model Optimization Overview
7.2: Exponentially Weighted Moving Average
7.3: Gradient Descent with Momentum
7.4: RMSprop
7.5: Adam
7.6: Optimizers in Action with Fashion MNIST
7.7: Quiz
7.8: Exercise
7.9: Chapter Summary

8.1: What is Regularization?
8.2: Dropout Regularization
8.3: L1, L2 Regularization
8.4: Batch Normalization
8.5: Early Stopping
8.6: Quiz
8.7: Exercise
8.8: Chapter Summary

9.1: What is Hyperparameter Tuning?
9.2: GridSearchCV, RandomSearchCV
9.3: Optuna
9.4: Quiz
9.5: Exercise
9.6: Chapter Summary

10.1: What is CNN? Convolution, Kernels, Pooling and Beyond
10.2: Padding and Strides
10.3: CIFAR10 Image Classification using CNN
10.4: Data Augmentation
10.5: Transfer Learning
10.6: Pre-trained Models – ResNet, EfficientNet, MobileNet etc.
10.7: Caltech101 Classification Using Transfer Learning
10.8: Quiz
10.9: Exercise
10.10: Chapter Summary

11.1: Sequence Models
11.2: RNN in Depth
11.3: Types of RNN
11.4: Vanishing Gradient Problem
11.5: LSTM
11.6: GRU
11.7: Quiz
11.8: Chapter Summary

12.1: Introduction to Transformer Architecture
12.2: Word Embeddings
12.3: Contextual Embeddings
12.4: Overview of Encoder and Decoder
12.5: Tokenization, Positional Embeddings
12.6: Attention Mechanism
12.7: Multi Headed Attention
12.8: Decoder
12.9: How Transformers are Trained?
12.10: Hugging Face: BERT Basics
12.11: Hugging Face: Spam Classification Using BERT
12.12: Hugging Face: Next Word Prediction Using GPT2
12.13: Quiz
12.14: Exercise
12.15: Chapter Summary

13.1: AtliQ AI’s First Big Client: Vroom Cars
13.2: Problem Statement & SOW
13.3: Data Load and Transformation
13.4: Model Training with CNN
13.5: Model Training with CNN and Regularization
13.6: Model Training using Transfer Learning
13.7: Hyperparameter Tunning using Optuna
13.8: Model Evaluation and Export
13.9: Streamlit App
13.10: FastAPI Server

14.1: Quiz

15.1: Transition to NLP

Training through Backpropagation

Prev Next

Training through Backpropagation

In this lecture, we will discuss the algorithms used for the training of Neural Networks.

Contents

1: Welcome to Our Deep Learning Experience

1.1: Course Overview

1.2: Unlock Discord Channels

2: Getting Started

2.1: Neural Networks: The Foundation of Deep Learning

2.2: Deep Learning vs Statistical ML: When to Use What?

2.3: Neural Network Architectures

2.4: Real World Applications of Deep Learning

2.5: Tooling: PyTorch vs Tensorflow

2.6: Tooling: GPU, TPU

2.7: Quiz

2.8: Chapter Summary

3: Neural Networks: Fundamentals

3.1: What is Neuron?

3.2: Perceptron and Multilayer Perceptron (MLP)

3.3: Neural Network Intuition: Insurance Prediction

3.4: The Purpose of Activation Function

3.5: Activation Functions: Sigmoid, ReLU, Tanh, SoftMax

3.6: Quiz

3.7: Chapter Summary

4: Getting Started with PyTorch

4.1: PyTorch Installation

4.2: Matrix Fundamentals

4.3: PyTorch Tensor Basics 1

4.4: PyTorch Tensor Basics 2

4.5: Derivative and Partial Derivative

4.6: Chain Rule

4.7: Autograd in PyTorch

4.8: Numpy Arrays vs PyTorch Tensors

4.9: Quiz

4.10: Exercise

4.11: Chapter Summary

5: Neural Networks: Training

5.1: Training through Backpropagation

5.2: Gradient Descent: Theoretical Foundation

5.3: Gradient Descent: PyTorch Implementation

5.4: Batch GD vs Mini Batch GD vs SGD

5.5: Mini Batch GD and SGD: PyTorch Implementation

5.6: Quiz

5.7: Exercise

5.8: Chapter Summary

6: Neural Networks in PyTorch

6.1: Employee Bonus Prediction

6.2: Datasets and DataLoaders

6.3: Handwritten Digits Classification

6.4: Classification Report and Confusion Matrix

6.5: Cost Function: Binary Cross Entropy (a.k.a. Log Loss)

6.6: Cost Function: Cross Entropy

6.7: Quiz

6.8: Exercise

6.9: Chapter Summary

7: Model Optimization: Training Algorithms

7.1: Model Optimization Overview

7.2: Exponentially Weighted Moving Average

7.3: Gradient Descent with Momentum

7.4: RMSprop

7.5: Adam

7.6: Optimizers in Action with Fashion MNIST

7.7: Quiz

7.8: Exercise

7.9: Chapter Summary

8: Model Optimization: Regularization Techniques

8.1: What is Regularization?

8.2: Dropout Regularization

8.3: L1, L2 Regularization

8.4: Batch Normalization

8.5: Early Stopping

8.6: Quiz

8.7: Exercise

8.8: Chapter Summary

9: Model Optimization: Hyperparameter Tuning

9.1: What is Hyperparameter Tuning?

9.2: GridSearchCV, RandomSearchCV

9.3: Optuna

9.4: Quiz

9.5: Exercise

9.6: Chapter Summary

10: Convolutional Neural Networks (CNN)

10.1: What is CNN? Convolution, Kernels, Pooling and Beyond