dbt for Data Analytics & Engineering
My full course to help you
build production data pipelines with dbt
All video lessons are free on YouTube
Supporter Access unlocks structure, guided practice, and community
Unlock Full Access
Pricing
Course Outline
0️⃣ Course Intro▾
0️⃣ Course Intro
▾
Course Intro▾
Course Intro
▾
Welcome
⏱️ 3 mins
Introduction to the course, how it is structured, the two portfolio projects you will build, and why dbt is a growing skill for data analysts and engineers.
What You'll Build
⏱️ 2 mins
Who the course is for, the SQL and terminal prerequisites, and a tour of the two halves: dbt Basics on DuckDB and Advanced dbt on Databricks.
Support & Help
⏱️ 2 mins
What is free in the course, what the supporter perks add, and where to get help when you get stuck, including the community, chatbots, and the public answer-key repos.
What is dbt▾
What is dbt
▾
The Problem
⏱️ 2 mins
A quick SQL refresher and why a folder of scripts breaks down at scale: wrong run order, bad data reaching stakeholders, and no documentation.
How dbt Works
⏱️ 2 mins
How dbt compiles the SELECT in a model file into a CREATE TABLE AS for the warehouse, and where it fits as the T in ELT.
How dbt Runs
⏱️ 2 mins
dbt as a command-line tool run from the terminal on Windows and Mac, powered by Rust in v2, and where the course runs it locally and in the cloud.
The dbt Family
⏱️ 1 min
The dbt v2 product lineup:
dbt versus dbt OSS, the hosted dbt platform, the VS Code extension, and how the old product names map to the new ones.Data & Project Intro▾
Data & Project Intro
▾
Project Overview
⏱️ 2 mins
What a data pipeline is and what each project builds: Project #1 on your laptop with DuckDB, Project #2 in production on Databricks.
The Dataset
⏱️ 2 mins
A year of real job postings from the datanerd.tech scraper, the two raw tables, the four star-schema tables you will build, and why the data is messy on purpose.
Types of Data
⏱️ 1 min
Tour of the columns you will query most, roles, platforms, locations, and skills, with example DuckDB queries against the raw parquet files.
1️⃣ dbt Basics▾
1️⃣ dbt Basics
▾
Tool Setup▾
Tool Setup
▾
VS Code install
⏱️ 3 mins
Install VS Code, tour the explorer, editor, and integrated terminal, and add the Microsoft Python extension the course depends on.
Terminal setup
⏱️ 3 mins
Terminal basics inside VS Code, navigating with
pwd, ls, and cd, and the command pattern used throughout the course.Git install
⏱️ 3 mins
What git is and why the course uses it, verifying the install and your identity, and switching Windows users to Git Bash as the default terminal.
What is an environment
⏱️ 3 mins
Why every project needs its own isolated Python and packages, and how
uv replaces venv, pyenv, and pip in one tool.uv install
⏱️ 3 mins
Install uv, create the course project with
uv init and uv sync, activate the environment, and understand pyproject.toml and uv.lock.DuckDB CLI install
⏱️ 3 mins
Install the DuckDB CLI, run a first query in its shell, take a peek at the browser UI, and complete the five-tool setup checklist.
dbt Setup▾
dbt Setup
▾
Setup Project
⏱️ 6 mins
Add dbt to the project with
uv add dbt, learn what an adapter is, and scaffold the analytics/ project from scratch with dbt init.Project Anatomy
⏱️ 5 mins
Tour of the repo layout and the six dbt folders, what
dbt_project.yml controls, what a YAML file is, and a read through dbt's sample project.Profiles YAML
⏱️ 5 mins
Write
profiles.yml by hand to connect dbt to DuckDB with dev and prod outputs, then prove the connection with dbt debug.Inspection Commands
⏱️ 5 mins
The two read-only commands:
dbt ls to list every node in the project and dbt debug to smoke test the config and connection.Execution Commands
⏱️ 5 mins
Run
dbt seed, dbt run, and dbt test on the sample project, then inspect the resulting tables with the DuckDB CLI.Compile & Clean Commands
⏱️ 5 mins
Read the compiled SQL in
target/ to see exactly what the warehouse ran, then use dbt clean and dbt compile.dbt show
⏱️ 5 mins
Preview a model's rows in the terminal with
dbt show, its limit flag, and a recap of every core dbt command run in the lesson.Teardown
⏱️ 5 mins
Remove the sample project, trim
dbt_project.yml, and wipe the demo database so the real project starts from a blank canvas.Models▾
Models
▾
Get the data
⏱️ 5 mins
Download the course dataset, 13 parquet files of job postings, into
data/raw/ with a script run from the repo root.First model
⏱️ 4 mins
What a model is, one SELECT in a
.sql file, then write, break, and fix your first model and preview its rows.DuckDB UI
⏱️ 4 mins
Browse what dbt built in the DuckDB UI, learn the one-writer lock rule, rebuild the disposable database, and refine the model with named columns and a filter.
First commit
⏱️ 4 mins
Add a
.gitignore that keeps data out of the repo, make the first commit, and start the ritual of ending every lesson with a commit.Dependencies & Jinja▾
Dependencies & Jinja
▾
Hardcoding & ref()
⏱️ 7 mins
Why a hardcoded table name breaks on a fresh prod database, and how
ref() both points at a model and records the dependency so dbt builds in order.Jinja intro
⏱️ 7 mins
Jinja's three markers, expressions, statements, and comments, proven with
dbt compile to show the warehouse only ever receives plain SQL.dbt --select
⏱️ 7 mins
Build a subset of the project with
--select, reading name, +name, and name+ to pick upstream or downstream models.Properties▾
Properties
▾
Sources
⏱️ 7 mins
Declare raw parquet inputs once in YAML and read them with
source(), using external_location to tell dbt where the files live.Models
⏱️ 7 mins
Describe every model and column in YAML, persist the descriptions into the database with
persist_docs, and split the file into sources.yml and schema.yml.Docs & the DAG▾
Docs & the DAG
▾
dbt docs
⏱️ 5 mins
Generate and serve the documentation site with
dbt docs generate and dbt docs serve, then tour a model page: description, lineage, code, and columns.The DAG
⏱️ 5 mins
What Directed, Acyclic, and Graph each mean, how dbt derives build order from the DAG, and reading your own pipeline's graph in the docs site.
Multiple ref()
⏱️ 5 mins
Build
monthly_summary from two ref() calls so the graph converges into a diamond, then regenerate the docs and watch it grow.dbt Extension
⏱️ 5 mins
Install the official dbt VS Code extension to preview rows and CTEs, compile files, jump between models, catch Jinja errors early, and view live lineage.
Materializations▾
Materializations
▾
View vs Table
⏱️ 3 mins
The trade-off between a view that recomputes on every read and a table that stores rows, measured live with the DuckDB timer.
Model config()
⏱️ 3 mins
Set one model's materialization with
config(materialized='table') at the top of the file and watch the database file grow with real rows.Project Defaults
⏱️ 3 mins
Set a project-wide materialization default in
dbt_project.yml, then prove that a file-level config() overrides it.Materialization Options
⏱️ 3 mins
The full menu of view, table, ephemeral, incremental, and materialized_view, with a demo of an ephemeral model vanishing into a CTE.
Tests - pt 1▾
Tests - pt 1
▾
Why Tests
⏱️ 3 mins
Why SQL that runs is not data that is right, and how dbt tests catch bad data at the model before it reaches a stakeholder.
Generic Test - not_null
⏱️ 3 mins
Declare a
not_null test in YAML, read the compiled query behind it, and see both a passing test and a failure with a row count.dbt build
⏱️ 3 mins
Run models and tests together in DAG order with
dbt build, and watch a failing upstream test skip every downstream model.Types of Tests
⏱️ 2 mins
The three types of dbt tests: generic tests that reuse built-in rules, singular tests written as SQL, and unit tests that check model logic.
Generic Test - unique
⏱️ 2 mins
A checklist for which columns deserve tests, then enforce the grain of
monthly_summary with unique and not_null on its month column.🚀 Ship & Automate — Project 1▾
🚀 Ship & Automate — Project 1
▾
Ship & Automate▾
Ship & Automate
▾
Project Goals
⏱️ 4 mins
What Project #1 adds to the pipeline you built: a public repo, a showcase README, docs on GitHub Pages, and automation that runs it without you.
Push to GitHub
⏱️ 4 mins
Run a pre-push guard to confirm no data files are tracked, then publish the repo to GitHub with its full one-commit-per-lesson history.
Prod Target
⏱️ 3 mins
Build the shipping copy with
dbt build --target prod and commit a second ci/profiles.yml so the CI runner knows where to write.GitHub Actions
⏱️ 3 mins
Read a workflow file line by line that checks out the repo, downloads data, runs
dbt build, and publishes the warehouse on every push and on a weekly schedule.Query It Live
⏱️ 3 mins
Query the published warehouse from any DuckDB session with a single
ATTACH over https, plus a map of every YAML file in the pipeline.Showcase▾
Showcase
▾
Draft the README
⏱️ 5 mins
Draft a showcase README from a skeleton with your architecture diagram, DAG screenshot, live badges, and a query block anyone can run.
Docs & Pages
⏱️ 5 mins
Copy the generated docs site into a
docs/ folder and serve it live on GitHub Pages, then wire the README badge to it.Share on LinkedIn
⏱️ 4 mins
Write a LinkedIn post for Project #1 with a hook, two or three concrete receipts, your biggest lesson, and the repo and docs links.
2️⃣ Advanced dbt▾
2️⃣ Advanced dbt
▾
To the Cloud▾
To the Cloud
▾
Advanced Chapter Topics
⏱️ 3 mins
A map of the nine topic areas the Advanced chapter covers across twelve lessons, from a real cloud warehouse to tests, snapshots, and incremental models.
Project Kickoff
⏱️ 3 mins
Project #2 starts fresh with a new repo on Databricks: the staging, intermediate, and marts layers you will build, and why dbt has no compute engine of its own.
Databricks Signup
⏱️ 3 mins
Why the course uses Databricks Free Edition, a permanently free serverless workspace with no credit card, and how to sign up.
Databricks Tour
⏱️ 3 mins
Tour of the six workspace areas the course uses, the
catalog.schema.object address format, and waking the SQL warehouse.Ingest Data
⏱️ 2 mins
Run the course's one notebook inside Databricks to download the raw parquet into a Volume and land
raw.jobs tables, with no laptop in the loop.dev & prod Catalogs
⏱️ 2 mins
Create
dev and prod catalogs as environments beside the raw landing zone, so both read the same raw data while builds land separately.Local Setup▾
Local Setup
▾
Setup Repo
⏱️ 4 mins
Stand up the Project #2 repo and environment at speed with
uv init, uv sync, and uv add dbt, and see which work runs on your laptop versus Databricks.Databricks Profile
⏱️ 4 mins
Write the
databricks profile with your warehouse hostname, HTTP path, and a personal access token, plus the optional env_var() alternative.Create Project
⏱️ 3 mins
Make the dbt project by hand instead of
dbt init: six folders, a three-key dbt_project.yml pointing at the profile, and a .gitignore.Test the Setup
⏱️ 3 mins
Run the first
dbt debug against the cloud and read it as a checklist, then round-trip an ad-hoc query with dbt show --inline and find it in Query History.SQLTools (Optional)
⏱️ 3 mins
Optionally install SQLTools and the Databricks driver in VS Code and set up
raw and dev connections to browse the warehouse from the editor.Commit the Setup
⏱️ 3 mins
Review the ignore files, close the stray
logs/ gap, confirm no token-shaped file is tracked, and make Project #2's first commit.Data Modeling▾
Data Modeling
▾
The Three Layers
⏱️ 6 mins
Data modeling as staging, intermediate, and marts layers with one job each, the medallion vocabulary, and the layer config in
dbt_project.yml.Sources
⏱️ 6 mins
Declare
raw.jobs in sources.yml with a catalog and schema so source() compiles to the right address, with nothing run on the warehouse.Jobs Model
⏱️ 6 mins
Build
stg_job_postings end to end: discover columns with DESCRIBE, write the source and cleaned CTEs, drop the scraper's error rows, and polish.Skills Model
⏱️ 5 mins
Build
stg_job_skills as the deliberately boring staging model: rename, order, trim, and stop, with no joins or business logic.Properties YAML
⏱️ 5 mins
Describe every column of both staging models in
staging.yml and turn on persist_docs project-wide so the warehouse carries the comments.Jinja Pt.2▾
Jinja Pt.2
▾
Analyses Folder
⏱️ 6 mins
The
analyses/ folder holds SQL that dbt compiles but never builds, a zero-risk sandbox you run with dbt show and that dbt build skips.Loops
⏱️ 6 mins
Write a
{% set %} list and a {% for %} loop in the sandbox and read the compiled output to see Jinja writing three SELECT statements from one template.Filters
⏱️ 6 mins
Transform values at compile time with filters like
| title, | lower, and | replace, chaining them to fix casing and build column names.Conditionals
⏱️ 5 mins
Use Jinja's inline
if expression and loop.last to glue looped statements into one query with UNION ALL, then run it with dbt show.Clean Benefits Column
⏱️ 5 mins
Turn the JSON extensions column into four
has_* boolean flags in staging with one loop, parse the JSON once in its own CTE, and read the compiled SQL.Macros Pt.1▾
Macros Pt.1
▾
Macros Intro
⏱️ 6 mins
What a macro is and the three places macros come from: dbt's built-ins, installed packages, and your own
macros/ folder, explored through the docs site.Custom Macros
⏱️ 6 mins
Define a
slugify() macro from the filter chain, test it with dbt compile --inline and the sandbox, then swap it into the flags loop.Install a Package
⏱️ 6 mins
Install
dbt_utils with packages.yml and dbt deps, then use get_column_values so the loop reads roles from the data instead of a typed list.Override Built-in Macros
⏱️ 5 mins
Trace the
default_staging schema name to dbt's own generate_schema_name macro, then override it so staging lands in dev.staging.Macros Pt.2▾
Macros Pt.2
▾
Salary Cleanup w/ SQL
⏱️ 9 mins
Parse salary text into
salary_min, salary_max, salary_period, and salary_currency columns, with the TRY_CAST and USD-stamp decisions explained.Macro - parse_salary()
⏱️ 9 mins
Spot the duplicated parse expression in staging and extract it into a
parse_salary(part) macro, then add salary_avg as a one-liner and finish staging.The Star Schema▾
The Star Schema
▾
What We're Building
⏱️ 4 mins
A map of the five tables this lesson adds to the right of staging, the intermediate model, fact, seed, dimensions, and bridge, and why they build in that order.
Intermediate Model
⏱️ 4 mins
Build
int_skill_observations to re-grain 3.3 million skill pairs into one row per skill, and learn the test for when an aggregation belongs in intermediate.Fact Table
⏱️ 4 mins
Build
fct_job_postings as the curated center of the star with one row per posting, check its actual grain, and start marts.yml.Seeds
⏱️ 4 mins
Load the
skill_categories taxonomy as a version-controlled seed CSV with dbt seed, give it a schema, and learn when a seed is the right choice.Dimension Tables
⏱️ 4 mins
Build
dim_company by aggregating the fact and dim_skill by joining the seed to intermediate counts, two different ways to build a dimension.Bridge Table
⏱️ 3 mins
Build
bridge_job_skills to resolve the many-to-many between jobs and skills, completing the star schema in dev.marts.Analyses▾
Analyses
▾
Analyses vs Models
⏱️ 5 mins
When a question belongs in
analyses/ instead of models/, and the two run paths: compile then run the SQL, or dbt show in the terminal.In-Demand Skills
⏱️ 4 mins
Write an analysis joining the fact, bridge, and skill dimension to rank the most in-demand skills, then add a
var() knob to filter by role.Highest-Paying Skills (BONUS)
⏱️ 4 mins
Rank skills by average salary with the USD and yearly unit filters stated explicitly, run via
dbt show and again from the compiled file in SQLTools.Tests Pt.2▾
Tests Pt.2
▾
Tests Refresher
⏱️ 7 mins
Recap of the three test types, generic, singular, and unit, and a map of which topic in this lesson exercises each one on the real scraped data.
Generic Tests pt.2
⏱️ 7 mins
Pin categorical columns with
accepted_values, make the ERD's foreign-key lines executable with relationships, and choose error versus warn severity.Custom Generic Tests
⏱️ 6 mins
Write a
non_negative test as a {% test %} block in tests/generic/ and call it from YAML on the salary columns like a built-in.Test-Driven Cleaning
⏱️ 6 mins
Put
not_null and unique on job_id, cut the id-less rows in staging, dedupe the fact with QUALIFY, and move the grain tests to where the promise lives.Tests Pt.3▾
Tests Pt.3
▾
Tests Refresher
⏱️ 6 mins
Recap of the three test types and which two this lesson writes: a singular test on the real data and a unit test on the salary parser's logic.
Singular Tests
⏱️ 5 mins
Turn a QA query into a singular test by moving it from
analyses/ to tests/, interrogate its failures in two layers, and ship it at warn severity.Unit Tests
⏱️ 5 mins
Write a unit test that runs
stg_job_postings against fake rows to check the salary parser before any build, and see when it runs in dbt build.Snapshots▾
Snapshots
▾
Slowly Changing Dimensions
⏱️ 12 mins
The SCD menu of Types 0 through 3, a quick look at 4 through 6, and why dbt snapshots implement Type 2 with new rows and validity dates.
Walkthrough Manual “Snapshot”
⏱️ 12 mins
Build an ephemeral time-machine model with
var('as_of') and QUALIFY, and watch one posting's salary change between two dates to specify what a snapshot must do.Build the Snapshot
⏱️ 12 mins
Write
snapshots.yml with the check strategy, run dbt snapshot night by night, backfill the year, then query who edits their postings.Incremental Models▾
Incremental Models
▾
Incremental Intro
⏱️ 9 mins
The incremental materialization and its five strategies, append, merge, delete+insert, insert_overwrite, and microbatch, and why the fact table uses merge on
job_id.Incremental Build
⏱️ 8 mins
Convert
fct_job_postings to incremental with a merge config and an is_incremental() filter, reading each run's receipt with DESCRIBE HISTORY.Full Refresh
⏱️ 8 mins
When an incremental model needs a
--full-refresh rebuild, then run the full project build and commit the end of the Advanced chapter.🏗️ Ship & Orchestrate — Project 2▾
🏗️ Ship & Orchestrate — Project 2
▾
Ship & Orchestrate▾
Ship & Orchestrate
▾
Project 2 Intro
⏱️ 5 mins
Inventory of everything Project #2 built, from the warehouse and star schema to macros, tests, and snapshots, and why orchestration is needed to fill
prod.Push to GitHub
⏱️ 5 mins
Run a pre-push guard for tracked secrets, publish the Project #2 repo to GitHub with its full history, and link it into Databricks as a Git folder.
Databricks Job
⏱️ 4 mins
Create a scheduled Databricks Job that runs the
load_raw notebook, then dbt deps, dbt debug, and dbt build into the prod catalog with temporary credentials and zero secrets.Showcase▾
Showcase
▾
Draft the README
⏱️ 3 mins
Draft the Project #2 README from a skeleton with your architecture poster, lineage screenshot, and real numbers pulled from your own warehouse.
Docs & Pages
⏱️ 3 mins
Copy the generated docs site into
docs/, publish it on GitHub Pages, wire up the badge and About link, and see the same descriptions in Unity Catalog.Share on LinkedIn
⏱️ 3 mins
Write a LinkedIn post for Project #2 with a hook, two or three concrete receipts, your biggest lesson, the repo link, and a visual.
What to Learn Next
⏱️ 3 mins
A map of what dbt offers beyond the course: semantic models, dimensions, metrics, MetricFlow, and the dbt Semantic Layer that serves governed numbers to BI tools and AI agents.
Course Resources
💽 Course Dataset — Raw Job Postings
This is the primary dataset used throughout the entire course. It contains a year of real-world data job postings (July 2025 to June 2026) scraped by my datanerd.tech pipeline, delivered as raw parquet files that you transform with dbt.
🔗 Step 1 — Grab the download script
From your project root, run this in your terminal (we do this together in Lesson 1.03):
💻 Step 2 — Download the data
Run the script once and 13 parquet files (~81MB) land in data/raw/:
Prefer to browse the files? Dataset release on GitHub 👉 https://github.com/lukebarousse/dbt_Analytics_Engineering_Course/releases/tag/dataset-v1
📊 Project 1 — Local dbt Pipeline
Build a full dbt pipeline on your laptop with DuckDB as the warehouse: raw parquet files in, one staging view and three mart tables out. Then push it to GitHub, where GitHub Actions rebuilds and tests it on a schedule and publishes the warehouse as a file anyone can query.
🔗 Project #1 Answer Key
👉 https://github.com/lukebarousse/dbt_Analytics_Engineering_Course/tree/main/project_1
🏗️ Project 2 — Production Cloud Pipeline
Build the pipeline again in the cloud on Databricks: models split into staging, intermediate, and marts layers ending in a star schema, snapshots that track history, and incremental models that only process new rows. A nightly Databricks job loads the raw data and builds prod with nobody at the keyboard.
🔗 Project #2 Answer Key
👉 https://github.com/lukebarousse/dbt_Analytics_Engineering_Course/tree/main/project_2
Supporter Resources
📝 Practice Problems
🧩 100+ Interview-Level Problems: Learn dbt faster with meticulously designed exercises spanning a range from easy to challenging
🔍 Detailed Solutions and Results: Every problem is accompanied by a comprehensive solution and your expected results
📺 Structured Video Lessons
🚢 Navigate with Ease: Jump instantly to any lesson or specific topic within the course – no more wasting time scrubbing through hours of video to find what you need
🧠 Focused Learning: Master concepts more effectively with dedicated, bite-sized videos for each distinct lesson, allowing for better concentration and easier review
🗒️ Lesson Notes & Cheat Sheets
📖 Structured Lesson Notes: Step-by-step walkthroughs for every topic, helping you follow along with each lesson and understand why models, tests, and pipelines are built the way they are
📋 Practical Cheat Sheets: Quick-reference guides for core dbt commands, Jinja and macros, testing, and data modeling concepts you’ll reuse across projects
✨ Certificate of Completion
🎖️ Certificate of Completion: Receive a certificate to validate your new skills and enhance your LinkedIn profile
🧑💻 Showcase Experience: Share how you used real-world data to help solve a problem for data professionals
About the Instructor
Luke Barousse - Course Instructor
🌎 Real-world Experience with dbt
Builds data pipelines with tools like dbt, from a Global Fortune 500 company to MrBeast's team, and today for datanerd.tech, the job postings pipeline behind this course's dataset.
💡🤖 Sharing Knowledge about Data & AI
Guides a community of +600,000 data nerds in harnessing analytical tools to revolutionize their professional workflows.
🎓 Trusted Course Developer
Imparted wisdom to +30,000 learners on DataCamp in leveraging analytical tools to elevate their career efficiency.
100% Satisfaction Guarantee or Your Money Back
⏱️ If you don’t feel the course problems and notes help you learn this tool as it has for countless others, I’ll refund your money!
📫 Email me within 30 days of purchasing the course on why you are unsatisfied, and I’ll return the full purchase price to you ASAP.
FAQ
What is “dbt for Data Analytics & Engineering”?
Who is this course for?
Should I take this course or "SQL for Data Engineering"?
Are the course problems and notes required?
What are the computer requirements?
How long should this course take to complete?
📣 Enrollment Paused 📣
ChatGPT for Data Analytics
🤚 I've paused signups to the course to revise it for ChatGPT updates 🤓
Signup to get notified when it's available (and get my newsletter as a bonus 📩)