Working as a Data Engineer

The everyday tasks of the job.

Data Analyst track

Junior

Write correct SQL, build trusted dashboards, ask good questions.

Nothing here yet.

Own an analysis end to end, from vague question to recommendation.

Core: start here

Investigating Metric DiscrepanciesExplaining why two numbers that should match don't.

1 more mid-level concepts

Documenting DatasetsWriting what a table contains, its grain, owner and caveats.

Own experimentation and metrics design; call out bad numbers.

Nothing here yet.

Shape how the organization measures and decides.

Nothing here yet.

Set measurement strategy across the company.

Nothing here yet.

Data Engineer track

Build and fix pipelines from clear specs; write correct SQL.

Debugging a Failed PipelineFinding which task broke, why, and what data it affected.
Documenting DatasetsWriting what a table contains, its grain, owner and caveats.
Handling Data RequestsClarifying what an analyst or stakeholder actually needs before building.

Own pipelines and models end to end, including their quality.

Core: start here

Handling Upstream Schema ChangesResponding when a source's structure changes under you.
Investigating Metric DiscrepanciesExplaining why two numbers that should match don't.
Onboarding a New Data SourceUnderstanding, ingesting, testing and documenting a new source.
Reprocessing HistoryRecomputing past data after a logic fix, safely and affordably.

3 more mid-level concepts

Data On-CallBeing responsible for pipeline failures and late data.
Deprecating TablesRetiring datasets without breaking hidden downstream users.
Tuning a Slow PipelineFinding the slow step and fixing it: partitions, joins, skew, file sizes.

Design the platform's storage, processing and modeling choices.

Nothing here yet.

Shape how the whole organization produces and uses data.

Nothing here yet.

Set data strategy and architecture across the company.

Nothing here yet.