Upload Data to S3 via DAG | Codebasics

Go to Home

0% Completed

- Login
- Register

back

- Login
- Register

0% Completed

Contents

1.1: Problem Statement
1.2: Tech Architecture and SOW

2.1: Massive/Polygon Setup
2.2: Extract Historical Data using Polygon
2.3: Backfill Raw Historical Data in Snowflake
2.4: Transform and Load into Core
2.5: Populate Dim Tables
2.6: Populate Fact Tables

3.1: Daily File Download through Airflow
3.2: Setup Connection with S3
3.3: Upload Data to S3 via DAG
3.4: Create S3 Stage in Snowflake
3.5: Connect Airflow with Snowflake
3.6: Ingest Data in Snowflake RAW
3.7: Transform from RAW to CORE
3.8: Ingest Invalid Records in Reject Table

4.1: Connect Airflow with Slack
4.2: Add Slack Integration in DAG
4.3: What is SA Layer?
4.4: Build Views for SA Layer
4.5: Power BI Dashboard

Instructor

Dhaval Patel

Dhaval Patel

Data Entrepreneur | YouTuber | Ex - Bloomberg, Nvidia

I have 17 years of experience in Programming and Data Science working for big tech companies like NVIDIA and Bloomberg. I also run a famous YouTube channel called Codebasics where I pursue my passion for teaching. Codebasics is one of the top channels on YouTube when it comes to data science, machine learning, data structures, etc. I firmly believe that “Anyone can code” and I use analogies, simple explanations, and step-by-step storytelling to explain difficult concepts in such a way that even a high school student can understand them easily.

Upload Data to S3 via DAG

Prev

Next

Lecture content Locked

Enroll in Bootcamp to Unlock

Congratulations!

The courses you picked are already included in Data Engineering Bootcamp 1.0: Job Assistance, Live Problem Solving & Virtual Internship.

Go to Cart Replace with Bootcamp

Login

Log in with Google

Log in with Linkedin

E-Mail Address

Enter your Password

Forgot Password?

Don't have an account? Register Now!

By signing up, you agree to
our Terms and Conditions and Privacy Policy.