Events & Integration
Connecting systems through events, data pipelines and shared contracts.
Backend Engineer
Mid-level
Own a feature end to end without hand-holding.
- ETL / ELTExtracting, transforming and loading data between systems.
- EventA record that something happened.
- Publish-SubscribeSenders publish to topics; receivers subscribe without knowing each other.
- Unix PhilosophySmall tools that do one thing well and compose through pipes.
Senior
Own a system, its failure modes, and its trade-offs.
Core: start here
- CQRSSeparating the write model from the read model.
11 more senior concepts
- Batch vs Stream ProcessingProcessing data in periodic chunks vs continuously.
- Apache KafkaA distributed, durable log for high-volume event streaming.
- Change Data CaptureStreaming database changes out as events.
- Distributed Batch ProcessingEngines like Spark that split big jobs across many machines.
- Schema RegistryA central store of event schemas and their compatibility rules.
- Stream WindowingGrouping events by time windows: tumbling, sliding and session windows.
- Event Schema VersioningEvolving event formats without breaking consumers.
- MapReduceProcessing huge datasets by mapping in parallel and then reducing the results.
- Event BusA channel through which events reach their subscribers.
- Event SourcingStoring every change as an event and deriving state from them.
- Event vs Command"This happened" vs "please do this".
Staff
Shape how many teams build, across systems.
- Enterprise Integration PatternsA catalog of messaging patterns: routers, translators, aggregators.
Data Engineer
Junior
Build and fix pipelines from clear specs; write correct SQL.
- ETL / ELTExtracting, transforming and loading data between systems.
- Batch vs Stream ProcessingProcessing data in periodic chunks vs continuously.
Mid-level
Own pipelines and models end to end, including their quality.
Core: start here
- Apache KafkaA distributed, durable log for high-volume event streaming.
- Change Data CaptureStreaming database changes out as events.
7 more mid-level concepts
- EventA record that something happened.
- Unix PhilosophySmall tools that do one thing well and compose through pipes.
- Distributed Batch ProcessingEngines like Spark that split big jobs across many machines.
- Schema RegistryA central store of event schemas and their compatibility rules.
- Stream WindowingGrouping events by time windows: tumbling, sliding and session windows.
- Event Schema VersioningEvolving event formats without breaking consumers.
- MapReduceProcessing huge datasets by mapping in parallel and then reducing the results.
Senior
Design the platform's storage, processing and modeling choices.
Core: start here
- Event SourcingStoring every change as an event and deriving state from them.
- CQRSSeparating the write model from the read model.
2 more senior concepts
- Event BusA channel through which events reach their subscribers.
- Event vs Command"This happened" vs "please do this".
Staff
Shape how the whole organization produces and uses data.
- Enterprise Integration PatternsA catalog of messaging patterns: routers, translators, aggregators.
Frontend Engineer
Mid-level
Own a feature end to end without hand-holding.
- EventA record that something happened.
- Publish-SubscribeSenders publish to topics; receivers subscribe without knowing each other.
- Unix PhilosophySmall tools that do one thing well and compose through pipes.
Senior
Own an app's architecture, performance, and failure modes.