Training through Backpropagation

1.1: Deep Learning and Your Career
1.2: The AI family tree
1.3: Pre-requisites
1.4: System Requirements
1.5: Course Overview

2.1: Join Discord for Doubt Clearing Support & Networking
2.2: Setting Video Resolution and Video Speed Globally

3.1: Neural Networks: The Foundation of Deep Learning
3.2: Deep Learning vs Statistical ML: When to Use What?
3.3: Neural Network Architectures
3.4: Real World Applications of Deep Learning
3.5: Tooling: PyTorch vs Tensorflow
3.6: Tooling: GPU, TPU
3.7: Quiz
3.8: Chapter Summary

4.1: What is Neuron?
4.2: Perceptron and Multilayer Perceptron (MLP)
4.3: Neural Network Intuition: Insurance Prediction
4.4: The Purpose of Activation Function
4.5: Activation Functions: Sigmoid, ReLU, Tanh, SoftMax
4.6: Quiz
4.7: Chapter Summary

5.1: PyTorch Installation
5.2: Matrix Fundamentals
5.3: PyTorch Tensor Basics 1
5.4: PyTorch Tensor Basics 2
5.5: Derivative and Partial Derivative
5.6: Chain Rule
5.7: Autograd in PyTorch
5.8: Numpy Arrays vs PyTorch Tensors
5.9: Quiz
5.10: Exercise
5.11: Chapter Summary

6.1: Training through Backpropagation
6.2: Gradient Descent: Theoretical Foundation
6.3: Gradient Descent: PyTorch Implementation
6.4: Batch GD vs Mini Batch GD vs SGD
6.5: Mini Batch GD and SGD: PyTorch Implementation
6.6: Quiz
6.7: Exercise
6.8: Chapter Summary

7.1: Employee Bonus Prediction
7.2: Datasets and DataLoaders
7.3: Handwritten Digits Classification
7.4: Classification Report and Confusion Matrix
7.5: Cost Function: Binary Cross Entropy (a.k.a. Log Loss)
7.6: Cost Function: Cross Entropy
7.7: Quiz
7.8: Exercise
7.9: Chapter Summary

8.1: Model Optimization Overview
8.2: Exponentially Weighted Moving Average
8.3: Gradient Descent with Momentum
8.4: RMSprop
8.5: Adam
8.6: Optimizers in Action with Fashion MNIST
8.7: Quiz
8.8: Exercise
8.9: Chapter Summary

9.1: What is Regularization?
9.2: Dropout Regularization
9.3: L1, L2 Regularization
9.4: Batch Normalization
9.5: Early Stopping
9.6: Quiz
9.7: Exercise
9.8: Chapter Summary

10.1: What is Hyperparameter Tuning?
10.2: GridSearchCV, RandomSearchCV
10.3: Optuna
10.4: Quiz
10.5: Exercise
10.6: Chapter Summary

11.1: What is CNN? Convolution, Kernels, Pooling and Beyond
11.2: Padding and Strides
11.3: CIFAR10 Image Classification using CNN
11.4: Data Augmentation
11.5: Transfer Learning
11.6: Pre-trained Models – ResNet, EfficientNet, MobileNet etc.
11.7: Caltech101 Classification Using Transfer Learning
11.8: Quiz
11.9: Exercise
11.10: Chapter Summary

12.1: Sequence Models
12.2: RNN in Depth
12.3: Types of RNN
12.4: Vanishing Gradient Problem
12.5: LSTM
12.6: GRU
12.7: Quiz
12.8: Chapter Summary

13.1: Introduction to Transformer Architecture
13.2: Word Embeddings
13.3: Contextual Embeddings
13.4: Overview of Encoder and Decoder
13.5: Tokenization, Positional Embeddings
13.6: Attention Mechanism
13.7: Multi Headed Attention
13.8: Decoder
13.9: How Transformers are Trained?
13.10: Hugging Face: BERT Basics
13.11: Hugging Face: Spam Classification Using BERT
13.12: Hugging Face: Next Word Prediction Using GPT2
13.13: Quiz
13.14: Exercise
13.15: Chapter Summary

14.1: AtliQ AI’s First Big Client: Vroom Cars
14.2: Problem Statement & SOW
14.3: Data Load and Transformation
14.4: Model Training with CNN
14.5: Model Training with CNN and Regularization
14.6: Model Training using Transfer Learning
14.7: Hyperparameter Tunning using Optuna
14.8: Model Evaluation and Export
14.9: Streamlit App
14.10: FastAPI Server

15.1: Quiz

Training through Backpropagation

Prev Next

Training through Backpropagation

In this lecture, we will discuss the algorithms used for the training of Neural Networks.

Contents

1: Welcome to Our Deep Learning Experience

1.1: Deep Learning and Your Career

1.2: The AI family tree

1.3: Pre-requisites

1.4: System Requirements

1.5: Course Overview

2: Enhance your Learning Experience

2.1: Join Discord for Doubt Clearing Support & Networking

2.2: Setting Video Resolution and Video Speed Globally

3: Getting Started

3.1: Neural Networks: The Foundation of Deep Learning

3.2: Deep Learning vs Statistical ML: When to Use What?

3.3: Neural Network Architectures

3.4: Real World Applications of Deep Learning

3.5: Tooling: PyTorch vs Tensorflow

3.6: Tooling: GPU, TPU

3.7: Quiz

3.8: Chapter Summary

4: Neural Networks: Fundamentals

4.1: What is Neuron?

4.2: Perceptron and Multilayer Perceptron (MLP)

4.3: Neural Network Intuition: Insurance Prediction

4.4: The Purpose of Activation Function

4.5: Activation Functions: Sigmoid, ReLU, Tanh, SoftMax

4.6: Quiz

4.7: Chapter Summary

5: Getting Started with PyTorch

5.1: PyTorch Installation

5.2: Matrix Fundamentals

5.3: PyTorch Tensor Basics 1

5.4: PyTorch Tensor Basics 2

5.5: Derivative and Partial Derivative

5.6: Chain Rule

5.7: Autograd in PyTorch

5.8: Numpy Arrays vs PyTorch Tensors

5.9: Quiz

5.10: Exercise

5.11: Chapter Summary

6: Neural Networks: Training

6.1: Training through Backpropagation

6.2: Gradient Descent: Theoretical Foundation

6.3: Gradient Descent: PyTorch Implementation

6.4: Batch GD vs Mini Batch GD vs SGD

6.5: Mini Batch GD and SGD: PyTorch Implementation

6.6: Quiz

6.7: Exercise

6.8: Chapter Summary

7: Neural Networks in PyTorch

7.1: Employee Bonus Prediction

7.2: Datasets and DataLoaders

7.3: Handwritten Digits Classification

7.4: Classification Report and Confusion Matrix

7.5: Cost Function: Binary Cross Entropy (a.k.a. Log Loss)

7.6: Cost Function: Cross Entropy

7.7: Quiz

7.8: Exercise

7.9: Chapter Summary

8: Model Optimization: Training Algorithms

8.1: Model Optimization Overview

8.2: Exponentially Weighted Moving Average

8.3: Gradient Descent with Momentum

8.4: RMSprop

8.5: Adam

8.6: Optimizers in Action with Fashion MNIST

8.7: Quiz

8.8: Exercise

8.9: Chapter Summary

9: Model Optimization: Regularization Techniques

9.1: What is Regularization?

9.2: Dropout Regularization

9.3: L1, L2 Regularization

9.4: Batch Normalization

9.5: Early Stopping

9.6: Quiz

9.7: Exercise

9.8: Chapter Summary

10: Model Optimization: Hyperparameter Tuning

10.1: What is Hyperparameter Tuning?

10.2: GridSearchCV, RandomSearchCV