Course prerequisites

This course is geared towards Dagster beginners.

If you’re interested in learning to build robust, scalable data pipelines using software engineering best practices, then you’re in the right place.

  • Python knowledge

    While you don’t need to be a Python expert to get started, you do need some Python familiarity to complete this course and use Dagster. In Lesson 2, we’ll cover Dagster’s specific installation requirements. Here are some Pythonic skills that you’ll be using, along with resources to learn about them: Functions, Packages and Modules, Decorators, and Type Hints.

  • SQL knowledge

    You won’t be writing complex SQL, but you will need to understand the concept of SELECT statements, what tables are, and how to make them. If you’d like a 5-minute crash course, here’s a short article and cheatsheet on using SQL.

  • Command line familiarity

    Like the Python requirement, you don’t need to be an expert in the command line - just familiar with it. This course provides copy-and-pasteable commands so you won’t have to write your own, but you should know how to open and use the terminal. If you need some help getting started, check out Code Academy’s course.

Course curriculum

    1. What's data engineering?

    2. What's an orchestrator?

    3. Orchestration approaches

    4. Why is asset-centric orchestration good for data engineering?

    5. Project preview

    6. Knowledge check

    1. Prerequisites and installation

    2. Create the Dagster project

    3. Project files

    4. Knowledge check

    1. Overview

    2. What's an asset?

    3. Knowledge check

    4. Defining your first asset

    5. Knowledge check

    6. Asset materialization

    7. Viewing run details

    8. Troubleshooting failed runs

    9. Practice: Create a taxi_zones_file asset

    10. Lesson recap

    1. Overview

    2. What's a dependency?

    3. Assets and database execution

    4. Loading data into a database

    5. Practice: Create a taxi_zones asset

    6. Assets with in-memory computations

    7. Practice: Create a trips_by_week asset

    1. Overview

    2. What's the Definitions object?

    3. What's a code location?

    4. Code locations in the Dagster UI

    5. Knowledge check

    1. Overview

    2. What's a resource?

    3. Knowledge check

    4. Setting up a database resource

    5. Using resources in assets

    6. Practice: Refactoring assets to use resources

    7. Analyzing resource usage using the Dagster UI

    8. Recap

    9. Knowledge check

Skills you'll gain in this course

  • Confidence in writing a data pipeline in Dagster
  • Viewing your data pipelines as the data assets that they produce
  • Understanding of essential Dagster concepts and benefits

Course reviews

5 star rating

Filling the Gap: ‘Dagster Essentials’ Delivers a Comprehensive Masterclass

Juan Domenech

The dedication and effort the team put into creating comprehensive content are evident. Every module resonates with quality, making it an enriching learning experience. Highly recommended for anyone diving into Dagster.

The dedication and effort the team put into creating comprehensive content are evident. Every module resonates with quality, making it an enriching learning experience. Highly recommended for anyone diving into Dagster.

Read Less
5 star rating

Great explanation

nick vazquez

Found this very helpful in refreshing myself on what the new abstractions around “definition” are and a basic structure to follow for a project. Would love to see more about the toml/profile setup of a local environment

Found this very helpful in refreshing myself on what the new abstractions around “definition” are and a basic structure to follow for a project. Would love to see more about the toml/profile setup of a local environment

Read Less
5 star rating

The best way to start your journey

Sven Sporer

I played with Dagster before but wasn't able to wrap my head around all the concepts. This course however tied all these essential Dagster components (assets, jobs, sensors, partitions, ...) together for me, based on a nice project. Great job Team...

Read More

I played with Dagster before but wasn't able to wrap my head around all the concepts. This course however tied all these essential Dagster components (assets, jobs, sensors, partitions, ...) together for me, based on a nice project. Great job Team Dagster!

Read Less
5 star rating

Very fun and well done tutorial !

Zyad Benameur

This is a must do for every dagster beginner. Would love to see more tutorials setup this way and covering other parts of dagster (Jobs, Graphs etc..). Great job putting this together : )

This is a must do for every dagster beginner. Would love to see more tutorials setup this way and covering other parts of dagster (Jobs, Graphs etc..). Great job putting this together : )

Read Less