Daily File Download through Airflow | Codebasics

Go to Home

0% Completed

- Login
- Register

back

- Login
- Register

0% Completed

Contents

1.1: Problem Statement
1.2: Tech Architecture and SOW

2.1: Massive/Polygon Setup
2.2: Extract Historical Data using Polygon
2.3: Backfill Raw Historical Data in Snowflake
2.4: Transform and Load into Core
2.5: Populate Dim Tables
2.6: Populate Fact Tables

3.1: Daily File Download through Airflow
3.2: Setup Connection with S3
3.3: Upload Data to S3 via DAG
3.4: Create S3 Stage in Snowflake
3.5: Connect Airflow with Snowflake
3.6: Ingest Data in Snowflake RAW
3.7: Transform from RAW to CORE
3.8: Ingest Invalid Records in Reject Table

4.1: Connect Airflow with Slack
4.2: Add Slack Integration in DAG
4.3: What is SA Layer?
4.4: Build Views for SA Layer
4.5: Power BI Dashboard

5.1: Step Into the Practice Room (Snowflake & Airflow)

Instructor

Dhaval Patel

Dhaval Patel

Data Entrepreneur | YouTuber | Ex - Bloomberg, Nvidia

I have 17 years of experience in Programming and Data Science working for big tech companies like NVIDIA and Bloomberg. I also run a famous YouTube channel called Codebasics where I pursue my passion for teaching. Codebasics is one of the top channels on YouTube when it comes to data science, machine learning, data structures, etc. I firmly believe that “Anyone can code” and I use analogies, simple explanations, and step-by-step storytelling to explain difficult concepts in such a way that even a high school student can understand them easily.

Daily File Download through Airflow

Prev

Lecture Content Locked

Get Notified for Next Cohort →

Login

Log in with Google

Log in with Linkedin

E-Mail Address

Enter your Password

Forgot Password?

Don't have an account? Register Now!

By signing up, you agree to
our Terms and Conditions and Privacy Policy.