Autoloader and Structured Streaming | Codebasics

Go to Home

0% Completed

- Login
- Register

back

- Login
- Register

0% Completed

Contents

1.1: Project Overview
1.2: Scope of Work and Tech Architecture
1.3: Data Understanding & Schema Exploration

2.1: Create a Free Azure Account
2.2: Databricks Setup in Azure
2.3: Upload Data in ADLS
2.4: Setup Catalog and Azure Connectors
2.5: Define Raw Schema

3.1: Ingest Dimension Data into Bronze Layer
3.2: Bronze to Silver: Data Cleansing and Transformation
3.3: Silver to Gold: Building BI Ready Tables

4.1: Batch vs Stream Processing
4.2: Autoloader and Structured Streaming
4.3: Ingest Fact Data into Bronze
4.4: Bronze to Silver: Data Cleansing and Transformation
4.5: Silver to Gold: Building BI Ready Tables
4.6: Build Daily Summary Table

5.1: Dimensional Pipeline
5.2: Fact Pipeline
5.3: Daily Refresh Job

6.1: Governance through Unity Catalog
6.2: Build BI Dashboard in Power BI

7.1: Exercise Walkthrough

Instructor

Dhaval Patel

Dhaval Patel

Data Entrepreneur | YouTuber | Ex - Bloomberg, Nvidia

I have 17 years of experience in Programming and Data Science working for big tech companies like NVIDIA and Bloomberg. I also run a famous YouTube channel called Codebasics where I pursue my passion for teaching. Codebasics is one of the top channels on YouTube when it comes to data science, machine learning, data structures, etc. I firmly believe that “Anyone can code” and I use analogies, simple explanations, and step-by-step storytelling to explain difficult concepts in such a way that even a high school student can understand them easily.

Autoloader and Structured Streaming

Prev

Lecture Content Locked

Get Notified for Next Cohort →

Login

Log in with Google

Log in with Linkedin

E-Mail Address

Enter your Password

Forgot Password?

Don't have an account? Register Now!

By signing up, you agree to
our Terms and Conditions and Privacy Policy.