Spark Architecture | Codebasics

Go to Home

0% Completed

- Login
- Register

back

- Login
- Register

0% Completed

Contents

1.1: Big Data and Distributed Computing
1.2: Apache Spark and Hadoop
1.3: What is Databricks?
1.4: Databricks Setup

2.1: Basic Operations
2.2: Reading Data from CSV
2.3: Handle Missing Values
2.4: SQL in pyspark
2.5: Joins

3.1: Reading Spark Plans with explain()
3.2: Spark Architecture
3.3: Transformations vs Actions and Lazy Evaluation
3.4: Narrows vs Wide Transformations
3.5: Partitions & Parallelism: Repartition
3.6: Partitions & Parallelism: Coalesce
3.7: Catalyst Optimizer, Predicate Pushdown, Column Pruning
3.8: Joins at Scale: Broadcast, Shuffle Hash and Hints
3.9: Data Skew and Mitigation Techniques
3.10: Adaptive Query Execution (AQE)
3.11: RDD, Dataframes and Datasets
3.12: Managed vs External Tables
3.13: Unity Catalog: Governance & Access Control
3.14: Understanding ACID
3.15: Time Travel Explained
3.16: What is Delta Lake and Delta Format

4.1: Final Quiz

Instructor

Dhaval Patel

Dhaval Patel

Data Entrepreneur | YouTuber | Ex - Bloomberg, Nvidia

I have 17 years of experience in Programming and Data Science working for big tech companies like NVIDIA and Bloomberg. I also run a famous YouTube channel called Codebasics where I pursue my passion for teaching. Codebasics is one of the top channels on YouTube when it comes to data science, machine learning, data structures, etc. I firmly believe that “Anyone can code” and I use analogies, simple explanations, and step-by-step storytelling to explain difficult concepts in such a way that even a high school student can understand them easily.

Spark Architecture

Prev

Next

Lecture content Locked

Enroll in Bootcamp to Unlock

Congratulations!

The courses you picked are already included in Data Engineering Bootcamp 1.0: Job Assistance, Live Problem Solving & Virtual Internship.

Go to Cart Replace with Bootcamp

Login

Log in with Google

Log in with Linkedin

E-Mail Address

Enter your Password

Forgot Password?

Don't have an account? Register Now!

By signing up, you agree to
our Terms and Conditions and Privacy Policy.