Build a Production Pipeline Live: Raw Files to a Medallion Lakehouse in 60 Minutes

Hosted by: Harun Raseed & Kirandeep Marala

Free

Thu, 05 Nov 2026
07:00 PM - 08:00 PM IST

Online (Zoom)

Hosted by

Harun Raseed

Kirandeep Marala

Share With Friends

Most tutorials stop at a working notebook. Real pipelines need more than that. They need to run again tomorrow without breaking, and someone has to trust the numbers at the end.

In this session you will watch a pipeline built from nothing to a working medallion lakehouse in Databricks, in one hour, with every decision explained as it happens.

Part 1: The medallion idea, in plain words

✅ What bronze, silver and gold actually mean, and why the split exists

✅ The job each layer does, and what should never happen in the wrong one

✅ Why this structure is the one most data teams settle on

Part 2: Bronze and silver, built live

✅ Landing raw files exactly as they arrive, and why you keep them untouched

✅ Cleaning into silver: duplicates, missing values, types and schema

✅ Writing Delta tables you can query, roll back and trust

Part 3: Gold, and making it hold up

✅ Building the gold tables that reporting actually reads

✅ Making a re-run safe, so yesterday's data is not counted twice

✅ Scheduling the flow and checking the results came out right

Who Should Join

Data analysts and aspiring data engineers who have written some Python or SQL and want to see how the pieces fit into a real pipeline. No Databricks experience needed.

Prerequisites

None required. You can just watch and learn.

Nice to have, to try things later on your own:

• A Databricks Free Edition account with a workspace ready to use