You can now move and process data in batches and in real time. This module is about where all of that data lives, and why almost every modern data platform runs in the cloud. Cloud storage and compute let you scale up when you need to and pay only for what you use, without buying and maintaining your own servers.
You'll start with the three storage patterns you'll hear about constantly. A data warehouse holds clean, structured data for fast reporting. A data lake is cheap, flexible storage for raw data of any kind. A lakehouse combines the two, bringing warehouse-style reliability and speed to data sitting in a lake.
Lakehouses rely on open table formats such as Apache Iceberg and Delta Lake. These add features like transactions, schema changes, and consistent reads on top of plain files in cloud storage. You'll learn what they do and how they differ.
Finally, you'll tour the data services offered by the big three clouds, AWS, Azure, and Google Cloud, and see how to think about choosing one to learn. With the platform in place, the last module covers keeping the data trustworthy, in Data Quality & Observability.
A simple explanation of what a data warehouse, a data lake, and a data lakehouse are, and how they differ.
The architecture and core concepts of Apache Iceberg, and how it brings database features such as SQL querying and ACID compliance to data lakes.
A session on the latest features of Delta Lake and Apache Iceberg, showing how open table formats are evolving.
A beginner-friendly overview of the big data services offered by the leading cloud providers, AWS, Azure, and Google Cloud.
A comparison of AWS, Azure, and Google Cloud from a data engineering point of view, to help you decide where to focus your learning.