Course prerequisites

This course is geared towards users who are unfamiliar with Dagster and looking to get started.

Beginner level

  • Python knowledge

    While you don’t need to be a Python expert to get started, you do need some Python familiarity to complete this course and use Dagster. In Lesson 2, we’ll cover Dagster’s specific installation requirements. Here are some Pythonic concepts used, along with resources to learn about them:

  • SQL knowledge

    You won’t be writing complex SQL, but you will need to understand the concept of select statements, what tables are, and how to make them. If you’d like a 5-minute crash course, here’s a short article and cheatsheet on using SQL.

  • Command line familiarity

    Like the Python requirement, you don’t need to be an expert in the command line - just familiar with it. This course provides copy-and-pasteable commands so you won’t have to write your own, but you should know how to open and use the terminal. If you need some help getting started, check out Code Academy’s course.

Course curriculum

    1. What's data engineering?

    2. What's an orchestrator?

    3. Orchestration approaches

    4. Why is asset-centric orchestration good for data engineering?

    5. Project preview

    6. Knowledge check

    1. Prerequisites and installation

    2. Set up local

    3. Set up Codespaces

    4. Project files

    5. Knowledge check

    1. Overview

    2. What's an asset?

    3. Knowledge check

    4. Defining your first asset

    5. Knowledge check

    6. Asset materialization

    7. Viewing run details

    8. Troubleshooting failed runs

    9. Practice: Create a taxi_zones_file asset

    10. Lesson recap

    1. Overview

    2. What's a dependency?

    3. Assets and database execution

    4. Loading data into a database

    5. Practice: Create a taxi_zones asset

    6. Assets with in-memory computations

    7. Practice: Create a trips_by_week asset

    1. Overview

    2. What's the Definitions object?

    3. What's a code location?

    4. Code locations in the Dagster UI

    5. Knowledge check

    1. Overview

    2. What's a resource?

    3. Knowledge check

    4. Setting up a database resource

    5. Using resources in assets

    6. Practice: Refactoring assets to use resources

    7. Analyzing resource usage using the Dagster UI

    8. Recap

    9. Knowledge check

Skills you'll gain in this course

  • Confidence in writing a data pipeline in Dagster
  • Viewing your data pipelines as the data assets that they produce
  • Understanding of essential Dagster concepts and benefits