Spark Architecture | Codebasics

Go to Home

0% Completed

- Login
- Register

back

- Login
- Register

0% Completed

Contents

1.1: Big Data and Distributed Computing
1.2: Apache Spark and Hadoop
1.3: What is Databricks?
1.4: Databricks Setup

2.1: Basic Operations
2.2: Reading Data from CSV
2.3: Handle Missing Values
2.4: SQL in pyspark
2.5: Joins

3.1: Reading Spark Plans with explain()
3.2: Spark Architecture
3.3: Transformations vs Actions and Lazy Evaluation
3.4: Narrows vs Wide Transformations
3.5: Partitions & Parallelism: Repartition
3.6: Partitions & Parallelism: Coalesce
3.7: Catalyst Optimizer, Predicate Pushdown, Column Pruning
3.8: Joins at Scale: Broadcast, Shuffle Hash and Hints
3.9: Data Skew and Mitigation Techniques
3.10: Adaptive Query Execution (AQE)
3.11: RDD, Dataframes and Datasets
3.12: Managed vs External Tables
3.13: Unity Catalog: Governance & Access Control
3.14: Understanding ACID
3.15: Time Travel Explained
3.16: What is Delta Lake and Delta Format

4.1: Final Quiz

Instructor

Dhaval Patel

Dhaval Patel

Data Entrepreneur | YouTuber | Ex - Bloomberg, Nvidia

I have 17 years of experience in Programming and Data Science working for big tech companies like NVIDIA and Bloomberg. I also run a famous YouTube channel called Codebasics where I pursue my passion for teaching. Codebasics is one of the top channels on YouTube when it comes to data science, machine learning, data structures, etc. I firmly believe that “Anyone can code” and I use analogies, simple explanations, and step-by-step storytelling to explain difficult concepts in such a way that even a high school student can understand them easily.

Spark Architecture

Prev

Lecture Content Locked

Enroll in Bootcamp to Unlock

Login

Log in with Google

Log in with Linkedin

E-Mail Address

Enter your Password

Forgot Password?

Don't have an account? Register Now!

By signing up, you agree to
our Terms and Conditions and Privacy Policy.