Every dashboard, report, and machine learning model depends on data that someone had to collect, clean, and deliver. Data engineering is the work of building the systems that do exactly that, and this module gives you the map before you start learning the tools.
You'll start with what a data engineer actually does and how the role differs from a data analyst or data scientist. From there you'll follow data through its lifecycle: where it is generated, how it is ingested and stored, how it is transformed, and how it is finally served to the people and models that use it. Along the way you'll meet the core building blocks, including pipelines, data warehouses, and the idea of big data.
The module closes with one of the most important design choices in the field: batch vs stream processing. You'll see how each approach handles data, when real-time results are worth the extra complexity, and why many systems use both.
With the big picture in place, you'll be ready for the next module, Python & SQL for Data, where you start writing the code that moves and shapes data.
An animated walkthrough of data infrastructure, from basic setups to ETL pipelines, data warehouses, big data, streaming, and distributed computing, plus how data engineers and data scientists work together.
A beginner-friendly crash course covering the data engineering lifecycle (ingestion, storage, processing, and serving), the core skills the role needs, and the tools used in practice.
A quick comparison of the two ways to process data: what each one is, when streaming is worth choosing over batch, and real-world use cases for both.
A conversation with a data processing practitioner that goes deeper into the trade-offs between batch and real-time pipelines and how teams decide between them.