Domains

Data Engineering

Collecting, moving, storing and shaping data so the rest of the company can trust and use it.

Topics

• Data Engineering Foundations

  What data engineering is, the lifecycle, and the shapes data comes in.

  14 concepts

• Collection & Instrumentation

  Capturing data at the source: events, logs, devices and external data.

  15 concepts

• Ingestion

  Getting data from source systems into your platform reliably.

  20 concepts

• Storage, Formats & Lakehouse

  Warehouses, lakes, file formats and table formats.

  32 concepts

• Data Modeling for Analytics

  Shaping data so it's easy and correct to query.

  26 concepts

• Batch & Distributed Processing

  Processing large datasets across many machines.

  22 concepts

• Stream Processing

  Processing data continuously as it arrives.

  14 concepts

• Transformation & Analytics SQL

  Turning raw data into clean, modeled, trustworthy tables.

  19 concepts

• Orchestration & Pipelines

  Scheduling and coordinating pipeline steps so they run reliably.

  14 concepts

• Data Quality & Observability

  Knowing whether data is correct, complete and on time.

  13 concepts

• Data Governance & Privacy

  Knowing what data you have, who owns it, and who may use it.

  20 concepts

• Serving & Analytics

  Getting data to the people and systems that use it.

  23 concepts

• DataOps & Platform

  Engineering practices that keep a data platform reliable and affordable.

  9 concepts

• Working as a Data Engineer

  The everyday tasks of the job.

  10 concepts