Data Engineer
Every concept a data engineer meets, from collecting events to serving trusted tables, in the order you actually need it.
Junior
Build and fix pipelines from clear specs; write correct SQL.
Core: start here
- .gitignoreFiles Git should never track, like build output and .env.
- AmendChanging the most recent commit's content or message.
- BranchA movable pointer to a line of commits, for isolated work.
- CloneCopying a remote repository, with its full history, to your machine.
- CommitA snapshot of changes with a message explaining them.
- Commit MessageA summary line plus a body explaining why a change was made.
- DiffSeeing exactly what changed between commits, branches or your working copy.
- GitThe distributed version control system almost everyone uses.
- Log and HistoryBrowsing and filtering commit history.
- Merge ConflictTwo branches changed the same lines and Git needs you to decide.
- RebaseReplaying commits on top of another base for a linear history.
- Code ReviewTeammates reading your change to catch bugs and share knowledge.
- PR DescriptionExplaining what changed, why, and how to test it.
- Pull RequestA proposal to merge a branch, with discussion and review.
- Receiving Code ReviewTaking feedback without defensiveness and resolving it clearly.
- Self-ReviewReading your own diff before asking others to.
- Small Pull RequestsKeeping changes small so they're reviewed faster and better.
- DebuggerA tool to pause code, inspect variables and step through execution.
- DebuggingSystematically finding why code doesn't do what you expect.
- Minimal Reproducible ExampleThe smallest code that still shows the bug.
- Reading Error MessagesActually reading the error: the first and most skipped debugging step.
- Reproducing a BugGetting the bug to happen reliably before trying to fix it.
- Rubber Duck DebuggingExplaining the problem out loud until you spot the bug.
- API TestingCalling endpoints and checking status codes, bodies and side effects.
- Integration TestA test of several components working together.
- MockA test double that verifies how it was called.
- Regression TestA test ensuring a fixed bug doesn't come back.
- Test CoverageThe share of code executed by tests, and why 100% isn't the goal.
- Test Runner / Test FrameworkTools like pytest, JUnit, Jest or Vitest that find, run and report tests.
- Unit TestA fast test of one small piece of code in isolation.
- Clean CodeCode that's easy to read, change and trust.
- Code SmellA surface sign that the design may have a deeper problem.
- DRY (Don't Repeat Yourself)Every piece of knowledge should have one authoritative place.
- KISS (Keep It Simple)Prefer the simplest solution that works.
- ReadabilityCode is read far more often than it's written.
- Technical DebtThe future cost of shortcuts taken now.
- YAGNI (You Aren't Gonna Need It)Don't build things before you need them.
- RefactoringChanging code's structure without changing its behavior.
- Development Environment SetupGetting a project running locally from a fresh machine.
- Environment VariableA key-value setting passed to processes from their environment.
- FormatterA tool that rewrites code to a consistent style automatically.
- LinterA tool that flags likely bugs and style issues.
- LockfileA record of exact dependency versions so every install is identical.
- Package ManagerA tool for installing and versioning dependencies, like npm, pip or cargo.
- Reading DocumentationGoing to the official docs first, and knowing how to navigate them.
- Searching EffectivelyFinding answers fast in error messages, issues, docs and forums.
- SSHSecurely logging into and running commands on remote machines.
- Learning While Using AIUsing AI without skipping the understanding a junior needs to build.
- Reviewing AI-Generated CodeReading and testing AI output as critically as a stranger's PR.
- MarkdownThe plain-text formatting syntax used for READMEs, docs and PRs.
- READMEThe front page of a project: what it is, how to run it, how to contribute.
- Percentiles (p50, p95, p99)The value below which a given share of measurements fall; the right way to read latency.
- Statistics for EngineersMean, median, percentiles and variance.
- LoggingRecording what the application does so you can debug it later.
- CASE ExpressionConditional logic inside a SQL query.
- Common Table Expression (WITH)Named subqueries that make complex SQL readable.
- Foreign KeyA column referencing another table's primary key.
- GROUP BY and AggregatesSummarizing rows with COUNT, SUM and AVG.
- INNER, LEFT, RIGHT and FULL JOINWhich rows each kind of join keeps.
- JOINCombining rows from several tables.
- NULL in SQLThree-valued logic, and why NULL = NULL isn't true.
- Primary KeyA column that uniquely identifies each row.
- SELECT, WHERE, ORDER BYReading, filtering and sorting rows.
- SQLThe language for querying relational databases.
- SubqueryA query nested inside another query.
- UpsertInsert, or update if the row exists, in one statement.
- Window FunctionsCalculations across related rows, like running totals and rankings.
- BackfillFilling in data for existing rows after a change.
- CSV Import / ExportMoving tabular data in and out, with all its edge cases.
- PartitioningDividing data into parts, within one machine or across many.
- Batch vs Stream ProcessingProcessing data in periodic chunks vs continuously.
- ETL / ELTExtracting, transforming and loading data between systems.
- Data Engineer vs Analyst vs Scientist vs ML EngineerWho builds pipelines, who answers questions, who models, and who ships models.
- Data Engineering LifecycleGeneration, ingestion, storage, transformation and serving, plus the undercurrents beneath them.
- Source SystemsWhere data originates: application databases, APIs, logs, files, SaaS tools and devices.
- Structured, Semi-Structured and Unstructured DataTables, JSON-like records, and free-form text, images or audio.
- Event Naming ConventionsConsistent names like order_completed so events are findable and comparable.
- InstrumentationAdding code that records what users and systems do, so the data exists at all.
- Batch vs Streaming IngestionLoading data on a schedule vs continuously as it's produced.
- Data IngestionMoving data from source systems into storage you control.
- DeduplicationRemoving duplicate records that at-least-once delivery and retries create.
- File-Based IngestionLoading CSV, JSON or Parquet files dropped in storage or SFTP.
- Full vs Incremental LoadReloading everything each time vs only what changed.
- Ingesting from APIsPulling data from REST APIs with pagination, rate limits and retries.
- Landing Zone / Raw LayerWhere ingested data is stored untouched before any transformation.
- CSVThe simplest tabular format, and its quoting, encoding and type pitfalls.
- Medallion Architecture (Bronze, Silver, Gold)Layering data from raw to cleaned to business-ready.
- Partitioned Tables (Hive-Style)Organizing files into folders like date=2024-06-01 so queries can skip data.
- Row vs Columnar File FormatsCSV and Avro store rows together; Parquet and ORC store columns together.
- Date Dimension / Date SpineA table with one row per day and its calendar attributes.
- Dimension TableA table describing the who, what, where of facts, like customers or products.
- Fact TableA table of measurable events, like orders or page views, at a fixed grain.
- GrainWhat one row of a table represents; the first decision in any model.
- Star SchemaOne fact table joined directly to its dimension tables.
Batch & Distributed Processing
- Batch ProcessingProcessing a bounded chunk of data in one run.
- DataFrameA table-like data structure with named columns, as in pandas, Polars and Spark.
- Single-Node Engines (DuckDB, Polars)Fast local processing that often makes a cluster unnecessary.
- Real-Time vs Near-Real-TimeMilliseconds vs minutes, and which one the business actually needs.
Transformation & Analytics SQL
- Data CleaningFixing types, formats, missing values and duplicates.
- Data ProfilingSummarizing a dataset's columns to understand what's really in it.
- Data TransformationCleaning, joining and reshaping raw data into useful tables.
- Deduplicating with ROW_NUMBERKeeping one row per key using a window function.
- ETL vs ELTTransforming before loading vs loading raw data and transforming in the warehouse.
- SQL Transformation Models (dbt)Transformations written as versioned SELECT statements that build tables.
- Pipeline DAGA pipeline expressed as tasks and their dependencies.
- Pipeline SchedulingRunning pipelines on a time schedule or when upstream data arrives.
- Retries and Failure HandlingRetrying transient failures and alerting on real ones.
- Data FreshnessHow recently a table was updated, and whether that's recent enough.
- Data Quality DimensionsAccuracy, completeness, consistency, timeliness, validity and uniqueness.
- Data TestsAutomated checks like not-null, unique and accepted values on tables.
- Data Dictionary / Business GlossaryDefinitions of tables, columns and business terms.
- Metric DefinitionsPrecisely defining what a number means, so two dashboards don't disagree.
- Query CostWhy scanning a whole table can cost real money in a cloud warehouse.
- Debugging a Failed PipelineFinding which task broke, why, and what data it affected.
- Documenting DatasetsWriting what a table contains, its grain, owner and caveats.
- Handling Data RequestsClarifying what an analyst or stakeholder actually needs before building.
- Analytics Event TrackingRecording user actions for later analysis.
- Columnar StorageStoring data by column for fast analytics.
- Data LakeCheap storage for raw data in any format.
- Data PipelineA sequence of steps that moves and transforms data.
- Data QualityMaking sure data is accurate, complete and fresh.
- Data WarehouseA database optimized for analytics, like BigQuery or Snowflake.
- dbtTransforming warehouse data with versioned SQL.
- OLTP vs OLAPTransaction processing vs analytical queries.
- ParquetA columnar file format for analytics.
- Pipeline Orchestration (Airflow)Scheduling and managing data pipelines.
- Acceptance CriteriaThe conditions a story must meet to be done.
- Definition of DoneThe team's checklist for when work is truly finished.
- Ticket / IssueA tracked unit of work in Jira, Linear or GitHub Issues.
- Writing a Bug ReportSteps to reproduce, expected vs actual behavior, and environment.
- Breaking Down TasksSplitting work into pieces small enough to finish and estimate.
- EstimationPredicting how long work will take, and communicating the uncertainty.
- Asking Good QuestionsSharing what you tried, what you expected and what happened.
- Status UpdatesProactively telling people where things stand.
- When to Ask for HelpNot too soon, not too late: time-box before asking.
- Clarifying TicketsAsking questions before building the wrong thing.
- Following Team ConventionsMatching the existing style even when you'd do it differently.
- Making Mistakes in ProductionOwning up quickly when you break something.
- Productive StruggleStruggling long enough to learn, but not so long you waste days.
- Reading an Unfamiliar CodebaseStarting from entry points, tests and data flow.
- Testing Your Own WorkChecking that your change works before calling it done.
- Understand Before You ChangeKnowing why code exists before you modify it.
- Your First WeeksSetting up, meeting people, and shipping something small early.
- Brag DocumentA running record of your accomplishments.
426 more junior concepts
- ASCIIThe original 7-bit character set that UTF-8 is backward compatible with.
- Base CaseThe condition that stops recursion.
- Block Scope vs Function ScopeWhether a variable lives until the end of its block or of the whole function.
- BooleanA true/false value.
- Boolean Flag ArgumentsPassing true/false to switch a function's behavior, and why it hurts readability.
- Character EncodingHow characters map to bytes: ASCII, UTF-8, UTF-16.
- Code CommentText for humans in code, best used to explain why rather than what.
- Conditional (if/else)Running different code depending on a condition.
- ConstantA name bound to a value that cannot be reassigned.
- Control FlowThe order in which statements execute: branches, loops, returns.
- Data TypeThe kind of value something is, which determines what operations are valid on it.
- Dates and TimesTime zones, UTC, ISO 8601, and why date bugs are everywhere.
- Default ParameterA parameter value used when the caller omits it.
- Expression vs StatementAn expression produces a value; a statement performs an action.
- Floating-Point NumberA binary approximation of real numbers, and why 0.1 + 0.2 != 0.3.
- FunctionA named, reusable block of code that takes inputs and returns an output.
- Global VariableA variable visible everywhere, and why it makes code hard to reason about.
- IntegerA whole-number type, usually with a fixed size and range.
- ISO 8601The standard text format for dates and times, like 2026-10-10T09:00:00Z.
- IterationStepping through the items of a collection one at a time.
- LoopRepeating code with for, while, do-while or for-each.
- Magic NumberAn unexplained literal in code that should be a named constant.
- Naming ThingsChoosing names that reveal intent; one of the hardest parts of programming.
- Null / None / nilA value meaning "nothing here", and the source of countless crashes.
- Off-by-One ErrorA loop or index that runs one step too many or too few.
- OperatorA symbol that performs an operation on values, like +, == or &&.
- Operator PrecedenceThe rules for which operators bind first in an expression.
- Parameter vs ArgumentParameters are in the definition; arguments are the values passed at the call.
- Primitive TypeA built-in basic type such as integer, float, boolean or character.
- RecursionA function solving a problem by calling itself on smaller inputs.
- Regular ExpressionA pattern language for matching and extracting text.
- Return ValueThe result a function hands back to its caller.
- ScopeThe region of code where a name is visible.
- ShadowingAn inner variable hiding an outer one with the same name.
- Short-Circuit Evaluation&& and || stopping as soon as the result is known.
- Stack OverflowCrashing when the call stack runs out of space, usually from runaway recursion.
- StringA sequence of characters, usually immutable.
- String InterpolationEmbedding values directly inside a string literal.
- Switch / MatchChoosing between many branches based on a value.
- Ternary OperatorA compact inline if/else expression.
- Time ZoneOffsets from UTC that change with location and daylight saving.
- Truthy and FalsyNon-boolean values that act as true or false in conditions.
- Type CoercionImplicit conversion by the language, like "5" + 1 in JavaScript.
- Type ConversionTurning a value of one type into another, explicitly or implicitly.
- UnicodeThe universal character set, and why string length is trickier than it looks.
- Unix TimestampSeconds since 1970-01-01 UTC; a simple, unambiguous point in time.
- UTF-8The dominant variable-length encoding of Unicode.
- VariableA named reference to a value stored in memory.
- Variadic FunctionA function that accepts any number of arguments.
- ArrayAn ordered, indexed collection of elements.
- DestructuringUnpacking values from arrays or objects into variables.
- Dictionary / MapA collection of key-value pairs with fast lookup by key.
- Equality vs IdentitySame value vs same object in memory.
- IndexingAccessing elements by position, usually starting at zero.
- ListAn ordered collection; a dynamic array or a linked list depending on the language.
- Map, Filter, ReduceThe three core operations for transforming collections.
- Mutable vs ImmutableWhether a value can be changed after it's created.
- Pass by Value vs ReferenceWhether a function receives a copy or a reference to the caller's data.
- SetAn unordered collection of unique values.
- Shallow vs Deep CopyCopying only the top level vs copying everything nested inside.
- SlicingTaking a sub-range of a sequence.
- Sorting with a Key or ComparatorSorting by a custom key or comparison function.
- Spread / Rest SyntaxExpanding or collecting elements with ... (or * and ** in Python).
- TupleA fixed-size, ordered group of values, often of different types.
- AbstractionExposing what something does while hiding how it does it.
- Access Modifierspublic, private and protected: who may see a member.
- ClassA blueprint for creating objects.
- ConstructorThe method that initializes a new object.
- EncapsulationHiding internal state behind a public interface.
- Field / PropertyA piece of data stored on an object.
- Getters and SettersMethods controlling access to a field.
- InheritanceA class reusing and extending another class.
- MethodA function that belongs to an object or class.
- Method OverloadingSeveral methods with the same name but different parameters.
- Method OverridingA subclass replacing a parent class's method.
- ObjectAn instance of a class holding its own state.
- Object-Oriented ProgrammingOrganizing code around objects that bundle data and behavior.
- PolymorphismOne interface with many implementations chosen at runtime.
- Static MemberA field or method that belongs to the class rather than an instance.
- this / selfThe reference to the current object inside a method.
- First-Class FunctionsFunctions treated as values that can be stored, passed and returned.
- Higher-Order FunctionA function that takes or returns other functions.
- Lambda / Anonymous FunctionA function without a name, defined inline.
- Pure FunctionSame input, same output, and no side effects.
- Side EffectAnything a function does besides returning a value: I/O, mutation, logging.
- AssertionA check that crashes when an assumption turns out to be false.
- Custom ExceptionA domain-specific error type carrying meaningful context.
- Error HandlingAnticipating, detecting and responding to things going wrong.
- ExceptionAn object signalling an error that unwinds the call stack until caught.
- Guard ClauseReturning early on invalid cases to avoid deep nesting.
- Stack TraceThe chain of function calls that led to an error.
- Swallowing ErrorsCatching an error and ignoring it, which hides bugs.
- Throwing / RaisingSignalling an error explicitly.
- try / catch / finallyCatching exceptions and running cleanup code regardless.
- EnumA type with a fixed set of named values.
- Nullable TypeA type that explicitly allows null, forcing you to handle it.
- Static vs Dynamic TypingTypes checked at compile time vs at runtime.
- Strong vs Weak TypingHow willingly a language converts between types implicitly.
- Type AliasA new name for an existing type.
- Type AnnotationExplicitly declaring the type of a variable or parameter.
- Type SystemThe rules a language uses to assign and check types.
- async / awaitSyntax for writing asynchronous code that reads like synchronous code.
- CallbackA function passed in to be called when work finishes.
- Callback HellDeeply nested callbacks that make async code unreadable.
- Promise / FutureAn object representing a value that will be available later.
- Atomic CommitOne commit doing one logical thing, easy to review and revert.
- BlameSeeing who last changed each line and in which commit.
- Branch Naming ConventionsNames like feat/login-page that tell people what a branch is for.
- Conventional CommitsA commit message convention like "feat:" and "fix:" that tools can parse.
- Detached HEADChecking out a commit directly instead of a branch.
- Force PushOverwriting remote history; use --force-with-lease and never on shared branches.
- ForkYour own copy of someone else's repository, for contributing via pull requests.
- Git ConfigYour name, email, aliases and defaults, at global or repository level.
- HEADGit's pointer to the commit you currently have checked out.
- MergeCombining the histories of two branches.
- Push, Pull, FetchSending commits, downloading and merging, or only downloading.
- RemoteAnother copy of the repository, usually "origin" on GitHub or GitLab.
- RepositoryA project's files plus their complete history.
- ResetMoving a branch pointer back, in soft, mixed or hard mode.
- RevertCreating a new commit that undoes an earlier one.
- SquashCombining several commits into one.
- Staging AreaWhere you choose which changes go into the next commit.
- StashTemporarily shelving uncommitted changes.
- TagA named pointer to a commit, usually marking a release.
- Version ControlTracking every change to code so you can review, revert and collaborate.
- ChangelogA human-readable list of notable changes per release.
- Feature BranchA short-lived branch for one change.
- GitHub FlowBranch from main, open a PR, merge back to main.
- HotfixAn urgent fix shipped outside the normal release cycle.
- Protected BranchA branch that requires reviews and passing checks before merging.
- Semantic VersioningMAJOR.MINOR.PATCH, and what each number promises about compatibility.
- Approve vs Request ChangesThe review states and what each one signals.
- Draft PRA PR opened early for feedback, not yet ready to merge.
- Labels and Issue HygieneLabeling, linking and closing issues so work stays findable.
- NitsMinor, optional review comments, labeled so they don't block.
- Pair ProgrammingTwo people working on the same code at the same time.
- BreakpointA place where the debugger pauses execution.
- Browser DevToolsThe browser's built-in tools for inspecting DOM, network, console and performance.
- Divide and Conquer DebuggingHalving the search space until the cause is isolated.
- Network TabInspecting every HTTP request a page makes.
- Print DebuggingAdding temporary output to see what code is doing.
- Step Over, Into, OutMoving through code one line or one call at a time.
- Arrange, Act, AssertThe standard three-part structure of a test.
- Flaky TestA test that passes and fails without any code change.
- Functional TestingTesting that features do what the requirements say, from the outside.
- Manual / Exploratory TestingA human exploring the product to find what automation misses.
- Smoke TestA quick check that the most basic things work after a deploy.
- Software TestingChecking code behaves as intended, automatically and repeatably.
- StubA test double that returns canned answers.
- Test AssertionThe check that decides whether a test passes.
- Test Case and Test SuiteA single scenario being checked, and a group of them run together.
- Test FixtureSetup data or state that tests rely on.
- Test IsolationTests that don't depend on each other or on shared state.
- Boy Scout RuleLeave the code a little cleaner than you found it.
- ConsistencyDoing similar things the same way across a codebase.
- Dead CodeCode that never runs and should be deleted.
- Deep NestingToo many levels of ifs and loops; flatten them with guard clauses.
- Long MethodA function doing too much; one of the most common smells.
- Premature OptimizationOptimizing before you know where the real bottleneck is.
- Separation of ConcernsEach part of the program handles one distinct concern.
- Structured ProgrammingBuilding programs from sequence, selection and loops instead of goto.
- Extract FunctionPulling a block of code into a well-named function.
- IDE Refactoring ToolsLetting the IDE perform renames and extractions safely.
- Inline Function / VariableReplacing an unnecessary indirection with its body.
- RenameChanging a name everywhere it's used, safely.
- Cargo Cult ProgrammingCopying code or rituals without understanding why they work.
- Spaghetti CodeTangled control flow that's impossible to follow.
- Build ToolAutomating compile, test and package steps, like Make, Gradle or Vite.
- curlThe command-line tool for making HTTP requests.
- DependencyExternal code your project relies on.
- Editor ShortcutsLearning your editor's shortcuts and multi-cursor editing to work faster.
- EditorConfigA shared config for indentation and line endings across editors.
- Getting Help (man, --help, tldr)Reading a command's built-in documentation.
- Go to Definition / Find ReferencesNavigating code by symbol instead of searching text.
- grep / ripgrepSearching text across files from the terminal.
- HTTP Client Toolscurl, HTTPie, Postman or Bruno for calling APIs by hand.
- IDE / Code EditorYour main tool for editing, navigating, refactoring and debugging.
- Language Version ManagerTools like nvm, pyenv or mise for switching language versions.
- PATHThe list of directories the shell searches for commands.
- Pipes and RedirectionChaining commands with a pipe and sending output to files with >.
- Pre-commit HooksRunning linters and formatters automatically before each commit.
- Regex TestersTools for building and testing regular expressions interactively.
- ShellThe program interpreting your commands: bash, zsh, fish.
- SSH KeysKey pairs for passwordless authentication to servers and Git hosts.
- Task Runner / Project ScriptsNamed project commands, like npm scripts or a justfile.
- Terminal / Command LineControlling your computer and tools through text commands.
- Terminal Text Editors (Vim, Nano)Editing files on a server where there's no IDE, and how to quit Vim.
- Virtual EnvironmentAn isolated set of packages per project, as with Python's venv.
- AI Coding AssistantTools like Copilot, Cursor or Claude Code that suggest and write code.
- HallucinationAn AI confidently producing APIs, facts or code that don't exist.
- Prompting for CodeGiving an AI enough context and constraints to produce useful code.
- Vibe CodingAccepting AI code without reading it: fine for prototypes, risky in production.
- Docstrings / JSDocDocumenting functions and modules right next to the code.
- Technical WritingWriting clearly for other engineers.
- Binary TreeA tree where each node has at most two children.
- Data StructureA way of organizing data so certain operations are efficient.
- Dynamic ArrayAn array that grows by reallocating, with amortized O(1) appends.
- Hash FunctionA function mapping data of any size to a fixed-size value.
- Hash TableKey-value storage with average O(1) lookup via a hash function.
- Linked ListNodes pointing to the next node: fast inserts, slow random access.
- QueueA first-in, first-out collection.
- StackA last-in, first-out collection.
- TreeA hierarchy of nodes with one root and no cycles.
- AlgorithmA step-by-step procedure for solving a problem.
- Best, Worst and Average CaseDifferent inputs can make the same algorithm fast or slow.
- Big O NotationDescribing how runtime or memory grows with input size.
- Binary SearchFinding an item in sorted data by halving the range each step.
- Brute ForceTrying every possibility; the baseline before optimizing.
- Coding Interview ProblemsAlgorithm puzzles used in hiring, and how they relate to real work.
- Common Complexity ClassesO(1), O(log n), O(n), O(n log n), O(n²), O(2ⁿ) and what they feel like in practice.
- Linear SearchChecking every element in turn.
- Sorting AlgorithmsBubble, insertion, merge, quick and heap sort, and their trade-offs.
- Space ComplexityHow memory use grows with input size.
- Time ComplexityHow the number of steps grows with input size.
- Binary and HexadecimalBase-2 and base-16 number systems.
- Bits and BytesThe basic units of data, and KB vs KiB.
- Boolean LogicAND, OR, NOT and De Morgan's laws.
- LogarithmsWhy repeatedly halving something gives O(log n).
- ModuloThe remainder after division; used for wraparound and hashing.
- Rounding and MoneyNever store money as a float; use integers or decimals.
- CPUThe processor that executes instructions.
- RAMFast, temporary memory for running programs.
- SSD vs HDDStorage types and their very different speeds.
- CronScheduling commands to run at fixed times.
- Exit CodeThe number a program returns when it ends; 0 means success.
- File PermissionsRead, write and execute rights for user, group and others.
- File SystemHow files and directories are stored and organized.
- Operating SystemSoftware that manages hardware and runs programs.
- Process ID (PID)A number identifying a running process.
- stdin, stdout, stderrThe three standard streams every process starts with.
- Symbolic LinkA file that points to another path.
- DNSThe internet's phone book, turning names into IP addresses.
- Domain NameA human-readable address like example.com.
- IP AddressA numeric address identifying a device on a network.
- Latency vs BandwidthHow long data takes to arrive vs how much can flow at once.
- localhost / 127.0.0.1The address that always means "this machine".
- ping and tracerouteChecking reachability and the path packets take.
- PortA number identifying a specific service on a host.
- 2xx Success Codes200 OK, 201 Created, 204 No Content and friends.
- 3xx Redirects301, 302, 307, 308 and when to use each one.
- 4xx Client Errors400, 404, 409, 422, 429: the client did something wrong.
- 5xx Server Errors500, 502, 503, 504: the server failed.
- Content-TypeThe header declaring a body's format, like application/json.
- HTTPThe request-response protocol of the web.
- HTTP HeadersMetadata attached to requests and responses.
- HTTP MethodsGET, POST, PUT, PATCH, DELETE and what each one means.
- HTTP RequestA method, URL, headers and an optional body.
- HTTP ResponseA status code, headers and a body.
- HTTP Status CodesThree-digit codes grouped 1xx–5xx describing the result.
- HTTPSHTTP encrypted with TLS.
- Path vs Query Parameters/users/42 vs /users?id=42, and when to use each.
- Query StringKey-value parameters after the ? in a URL.
- RedirectSending a client to a different URL.
- URLThe address of a resource: scheme, host, path, query and fragment.
- URL EncodingEscaping special characters in URLs as %XX.
- User-AgentThe header identifying the client software.
- WebhookAn HTTP callback: a service calls your URL when something happens.
- Let's EncryptA free, automated certificate authority.
- TLS CertificateA file proving a server controls a domain, signed by a trusted authority.
- APIAn interface that lets programs talk to each other.
- EndpointA specific URL and method an API exposes, like POST /orders.
- JSONThe text data format most APIs speak.
- Request and Response BodyThe payload sent with a request or returned in a response, usually JSON.
- Resource NamingPlural nouns, nested paths and consistent URL design.
- RESTAn API style built on resources, URLs and HTTP methods.
- SerializationConverting objects to bytes or text for storage or transfer.
- YAMLA human-friendly data format common in configuration files.
- ConfigurationSettings that change between environments, kept out of code.
- Environments (dev, staging, prod)Separate deployments for development, testing and real users.
- Log LevelsDEBUG, INFO, WARN and ERROR, and when to use each.
- PaginationReturning large result sets one page at a time.
- Scheduled JobsTasks that run on a schedule, like nightly cleanups.
- TimeoutsNever waiting forever on a network call.
- Breaking ChangeA change that forces clients to update.
- Audit Columns (created_at, updated_at)Recording when, and by whom, rows changed.
- COALESCE and NULLIFHandling NULLs inside SQL expressions.
- ConstraintsDatabase rules like NOT NULL, UNIQUE and CHECK.
- CRUDCreate, Read, Update, Delete: the four basic data operations.
- DatabaseOrganized, persistent storage for data.
- Database Client / GUITools like psql, DBeaver or TablePlus for exploring databases.
- Database Connection and Connection StringHow an app connects to a database: host, port, credentials and options.
- Database SchemaThe structure of tables, columns and relationships.
- DDL, DML, DCL and DQLSQL's categories: defining structure, changing data, granting access, querying.
- HAVINGFiltering groups after aggregation.
- Junction TableA table linking two others in a many-to-many relationship.
- One-to-Many and Many-to-ManyThe basic kinds of relationship and how to model them.
- PostgreSQL, MySQL and SQLiteThe common relational databases and where each one fits.
- Relational DatabaseData stored in tables with rows, columns and relationships.
- Self JoinJoining a table to itself, e.g. employees and their managers.
- SQL Data TypesChoosing integer, numeric, text, timestamp and other column types.
- SQL String, Date and Math FunctionsBuilt-in functions for transforming values in queries.
- Table, Row, ColumnThe basic structure of relational data.
- UNION, INTERSECT, EXCEPTCombining the results of several queries.
- ViewA saved query that acts like a table.
- Database IndexA lookup structure that speeds up queries at the cost of slower writes.
- N+1 Query ProblemOne query for a list, then one more query for every item in it.
Transactions & Concurrency Control
- TransactionA group of operations that succeed or fail together.
- Document DatabaseStoring JSON-like documents, as in MongoDB.
- Key-Value StoreStoring values by key, as in Redis or DynamoDB.
- NoSQLDatabases not built on the relational table model.
- Object StorageStoring files as objects, as in S3.
- RedisAn in-memory data store used for caching, queues and more.
- Migration ToolsTools like Flyway, Alembic, Prisma Migrate and Rails migrations.
- Schema MigrationA versioned script that changes the database schema.
- Seed DataInitial data loaded for development or tests.
- Bulk Loading (COPY)Loading large amounts of data much faster than row-by-row inserts.
- Cache Hit and MissWhether the requested data was found in the cache.
- CachingKeeping copies of data somewhere faster to avoid repeated work.
- TTL (Time to Live)How long a cached item stays valid.
- Background JobWork done outside the request, like sending an email.
- MessageA self-contained unit of data sent from one component to another through a broker.
- MonolithOne deployable application containing all the features.
- Vertical ScalingScaling up with a bigger machine.
- BackupsCopies of data for restoring, and why untested backups don't count.
- LatencyThe time a single operation takes.
- Base64An encoding of bytes as text; not encryption.
- Don't Roll Your Own CryptoUse vetted libraries; homemade cryptography is almost always broken.
- Encryption in TransitEncrypting data as it travels, with TLS.
- Hashing vs Encryption vs EncodingA one-way fingerprint vs reversible with a key vs just a different format.
- Keeping Sensitive Data Out of LogsNever logging passwords, tokens or personal data.
- Phishing and Social EngineeringAttacks that trick people instead of breaking code.
- Principle of Least PrivilegeGive every user and service only the access it needs.
- PIIPersonally identifiable information that needs special care.
- Archiving and Compression (tar, gzip, zip)Bundling and compressing files.
- Basic Shell Commandsls, cd, cp, mv, rm, cat, less and find.
- Copying Files Remotely (scp, rsync)Moving files between machines.
- Disk Space (df, du)Finding what's filling up the disk.
- LinuxThe operating system most servers run.
- Linux Directory StructureWhat /etc, /var, /usr, /home and /tmp are for.
- Managing Processes (ps, top, kill)Seeing and controlling running programs.
- OS Package Managers (apt, dnf, brew)Installing system software.
- sudo and rootRunning commands with administrator privileges.
- Virtual MachineA fully emulated computer running on shared hardware.
- ContainerA lightweight, isolated package of an app and its dependencies.
- Container LogsReading a container's stdout and stderr.
- Container RegistryWhere images are stored and pulled from.
- Container vs Virtual MachineSharing the host kernel vs emulating a full machine.
- DockerThe most common tool for building and running containers.
- Docker ComposeRunning multi-container setups from one YAML file.
- DockerfileThe recipe for building an image.
- Ephemeral Container FilesystemAnything written inside a container disappears when it's removed.
- ImageA read-only template that containers are created from.
- Port MappingExposing a container's port on the host.
- VolumePersistent storage that outlives a container.
- AWS, GCP and AzureThe three biggest cloud providers.
- Cloud ComputingRenting computing resources on demand.
- Compute Instance (EC2)A virtual server in the cloud.
- IaaS, PaaS, SaaSHow much of the stack the provider manages for you.
- Managed DatabaseA database the cloud provider runs for you.
- Platforms (Vercel, Netlify, Render, Fly)Hosts that deploy straight from Git with little setup.
- S3 / Blob StorageCheap, durable object storage.
- ServerA machine, or a program on one, that provides a service to clients.
- Build ArtifactThe packaged output of a build, deployed as-is.
- CI PipelineThe sequence of automated build, test and deploy steps.
- Continuous IntegrationAutomatically building and testing every change.
- DeploymentPutting a new version into an environment.
- GitHub ActionsGitHub's built-in CI/CD.
- RollbackReturning to the previous version when a deploy goes wrong.
- Staging EnvironmentA production-like environment for final testing.
- DashboardA visual display of key metrics.
- Error TrackingTools like Sentry that group and report exceptions.
- MonitoringWatching known metrics and alerting when they go wrong.
- Uptime MonitoringChecking from the outside that a site responds.
- IncidentAn unplanned event that disrupts or degrades service.
- Reading Production LogsFinding the few lines that matter among millions, by request ID, time and level.
- Data EngineeringBuilding the systems that collect, move, store and prepare data for analysis and ML.
- Data Science Hierarchy of NeedsCollect, move, store, clean, analyze, then learn: why reliable plumbing comes before AI.
- DatasetA named collection of related data, like a table or a set of files.
- Volume, Velocity, VarietyThe three dimensions that make data "big", and which one is actually your problem.
- Managed Ingestion ConnectorsTools like Fivetran or Airbyte that sync SaaS and database sources for you.
- JSON LinesOne JSON object per line, easy to stream and append.
Batch & Distributed Processing
- Notebooks (Jupyter)Interactive documents mixing code, output and notes.
- pandasPython's standard DataFrame library for data analysis.
Transformation & Analytics SQL
- Staging, Intermediate and Mart LayersA conventional way to organize transformation code.
- Standardizing ValuesMaking country codes, casing, units and currencies consistent.
- Task DependenciesWhich steps must finish before others can start.
- BI DashboardA collection of charts answering a recurring business question.
- Business Intelligence (BI)Dashboards and reports that help people make decisions.
- Data ServingMaking prepared data available for analysis, ML and applications.
- Large Language ModelA model trained on huge amounts of text to understand and generate language.
- AgileDelivering in small increments and adapting to feedback.
- BacklogThe prioritized list of work that hasn't started yet.
- Backlog RefinementClarifying and sizing upcoming work.
- Daily StandupA short daily sync on progress and blockers.
- EpicA large body of work split into smaller stories.
- KanbanVisualizing work on a board and limiting work in progress.
- RetrospectiveA regular meeting to reflect on how the team works and improve it.
- ScrumAn agile framework with sprints, roles and ceremonies.
- Software Development LifecycleThe stages from idea to production to maintenance.
- SprintA fixed period, often two weeks, to deliver planned work.
- Sprint PlanningChoosing the work for the next sprint.
- Sprint Review / DemoShowing what was built to stakeholders.
- User Story"As a user, I want… so that…": a requirement from the user's point of view.
- WaterfallSequential phases from requirements to release.
- Planning PokerEstimating as a team by revealing guesses at the same time.
- Story PointsRelative effort estimates instead of hours.
- TimeboxingFixing the time and adjusting the scope.
- 1:1 MeetingsRegular private conversations with your manager.
- Giving a DemoShowing your work clearly to an audience.
- Receiving FeedbackListening, asking for specifics, and acting on it.
- Working with QACollaborating with testers to ship quality.
- Writing ClearlyLead with the point, be specific, keep it short.
- Edge CasesUnusual inputs and situations that break naive code.
- RequirementsWhat the software must do.
- User EmpathyRemembering a real person uses what you build.
- Context Switching CostWhy jumping between tasks destroys productivity.
- Deep Work / Focus TimeLong, uninterrupted blocks for hard problems.
- Done Is Better Than PerfectShipping working software instead of polishing forever.
- Finding a MentorGetting guidance from someone more experienced.
- Fundamentals over FrameworksFrameworks change; the concepts underneath them don't.
- Impostor SyndromeFeeling you don't belong despite the evidence.
- Keeping an Engineering JournalWriting down what you learn, decide and get stuck on.
- Learning How to LearnDeliberate practice, spaced repetition and building things.
- Onboarding YourselfUsing code, docs and history to learn a new codebase.
- Time ManagementPlanning your day, protecting focus time and finishing what you start.
- Tutorial HellEndlessly following tutorials without building anything on your own.
- Behavioral Interview (STAR)Answering with Situation, Task, Action and Result.
- BurnoutChronic work stress, and how to notice and prevent it.
- Career LadderThe levels and expectations from junior to principal.
- Contributing to Open SourceFinding issues, following the guidelines and opening PRs.
- Job SearchResumes, portfolios, applications and referrals.
- Junior EngineerLearning to deliver well-defined tasks with guidance.
- Performance ReviewA periodic assessment of your work.
- Side ProjectsBuilding outside work to learn and to show your skills.
- Sustainable PaceWorking at a pace you can keep up for years.
- Technical InterviewsCoding, system design and behavioral rounds.
Mid-level
Own pipelines and models end to end, including their quality.
Core: start here
- Schema EvolutionChanging data formats so old and new readers and writers keep working.
- DenormalizationDeliberately duplicating data for read performance.
- Materialized ViewA view whose results are stored and refreshed.
- Apache KafkaA distributed, durable log for high-volume event streaming.
- Stream ProcessingComputing over events as they arrive, as with Flink.
- Change Data CaptureStreaming database changes out as events.
- Data RetentionHow long to keep data, and when to delete it.
- Append vs Upsert (MERGE)Adding new rows vs updating existing ones by key.
- High-Water MarkRemembering the last loaded timestamp or ID to know where to resume.
- Push vs Pull IngestionThe source sending data to you vs you fetching it.
- Schema DriftA source adding, renaming or retyping fields without telling you.
- CompactionMerging small files into fewer, larger ones.
- Compression Codecs (Snappy, Zstd, Gzip)Trading CPU for smaller files, and which codecs are splittable.
- LakehouseWarehouse-style tables and transactions on top of cheap lake storage.
- Open Table Formats (Iceberg, Delta, Hudi)Metadata layers that give files in a lake ACID transactions, schemas and time travel.
- Partition PruningThe engine skipping partitions a query doesn't need.
- Predicate PushdownFiltering inside the storage layer before data is read.
- Separation of Storage and ComputeScaling query engines independently from where data lives.
- Small Files ProblemToo many tiny files slowing queries and overloading metadata.
- Table Catalog / MetastoreThe registry of table names, schemas and file locations, like Hive Metastore or Glue.
- Dimensional ModelingKimball's approach: facts surrounded by descriptive dimensions.
- SCD Types 1, 2 and 3Overwrite, add a versioned row, or keep a previous-value column.
- Slowly Changing Dimension (SCD)Handling attributes that change over time, like a customer's address.
- Star vs Snowflake SchemaSimpler queries vs less duplication.
- Surrogate Keys in the WarehouseWarehouse-generated keys that stay stable when source IDs change.
Batch & Distributed Processing
- Apache SparkThe most widely used engine for distributed batch and streaming processing.
- Distributed SQL Query Engines (Trino, Presto)Querying data where it lives, across lakes and databases.
- ShuffleRedistributing data across machines by key, often the slowest step.
- Transformations vs ActionsLazy steps that build a plan vs commands that trigger execution.
- Event Time vs Processing TimeWhen something happened vs when your system saw it.
- Late-Arriving DataEvents that show up after their window was already computed.
Transformation & Analytics SQL
- Incremental ModelsProcessing only new or changed rows instead of rebuilding a whole table.
- Data-Aware / Event-Driven SchedulingTriggering work when data lands instead of at fixed times.
- Idempotent PipelinesPipelines that produce the same result when rerun, so retries and backfills are safe.
- Partitioned Pipeline RunsEach run processing one time slice, like one day of data.
- Reruns and Catch-UpRe-running past intervals after a failure or code change.
- Data IncidentA data quality failure that reaches users, and how to respond.
- Data ObservabilityMonitoring freshness, volume, schema and distributions to catch silent breakage.
- Data ReconciliationChecking that totals match between source and destination.
- Data CatalogA searchable inventory of datasets, their meaning and their owners.
- Data LineageWhere data came from and everything it flows into.
- Data MaskingHiding sensitive values while keeping data usable.
- Aggregate / Summary TablesPrecomputed rollups that make dashboards fast.
- Semantic Layer / Metrics LayerOne place that defines metrics like revenue, so every tool computes them the same way.
- CI/CD for Data PipelinesTesting and deploying pipeline and model changes automatically.
- DataOpsApplying DevOps practices like CI, testing and monitoring to data.
- Development and Production Data EnvironmentsSeparate schemas or warehouses so development never touches production data.
- Warehouse Cost ManagementControlling spend through query patterns, scheduling and sizing.
- Handling Upstream Schema ChangesResponding when a source's structure changes under you.
- Investigating Metric DiscrepanciesExplaining why two numbers that should match don't.
- Onboarding a New Data SourceUnderstanding, ingesting, testing and documenting a new source.
- Reprocessing HistoryRecomputing past data after a logic fix, safely and affordably.
599 more mid-level concepts
- Bitwise OperationOperating on individual bits: AND, OR, XOR and shifts.
- Integer OverflowA value exceeding its type's range and wrapping around or failing.
- Pattern MatchingBranching on the shape of data and destructuring it at the same time.
- Tail RecursionRecursion where the call is the last action, which some languages turn into a loop.
- GeneratorA function that lazily yields a sequence of values.
- IteratorAn object that yields items one at a time from a sequence.
- Lazy EvaluationComputing values only when they're actually needed.
- Abstract ClassA class that can't be instantiated and leaves some methods to subclasses.
- Composition over InheritanceBuilding behavior by combining objects instead of deep class hierarchies.
- InterfaceA contract of methods a type promises to implement.
- Mixin / TraitReusable behavior added to classes without inheritance.
- Special MethodsMethods the language calls implicitly, like __eq__ or toString.
- ClosureA function that remembers variables from the place it was defined.
- Declarative vs ImperativeDescribing what you want vs spelling out how to do it.
- Function CompositionCombining functions so the output of one feeds the next.
- Functional ProgrammingBuilding programs from pure functions and immutable data.
- ImmutabilityNever changing data after creation, producing new values instead.
- MemoizationCaching a function's results for inputs it has already seen.
- Option / Maybe TypeA type that explicitly represents "a value or nothing".
- Partial ApplicationFixing some arguments of a function to get a new function.
- Result / Either TypeA type that represents success or failure without exceptions.
- Checked vs Unchecked ExceptionsWhether the compiler forces you to handle an exception.
- Defensive ProgrammingWriting code that guards against invalid inputs and states.
- Error Values vs ExceptionsReturning errors as values vs throwing them.
- Fail FastStopping loudly on problems instead of continuing in a bad state.
- PanicAn unrecoverable error that aborts the program or thread.
- Discriminated UnionA union whose members are told apart by a tag field.
- Duck TypingIf it has the right methods, it's the right type.
- Finite State MachineA model with a fixed set of states and allowed transitions, e.g. an order going from paid to shipped.
- GenericsCode that works over many types while keeping type safety.
- Runtime ValidationChecking that untrusted data matches a type at runtime, e.g. with Zod or Pydantic.
- Type InferenceThe compiler working out types without annotations.
- Type NarrowingRefining a broad type to a specific one through checks.
- Union TypeA value that can be one of several types.
- Atomic OperationAn operation that completes entirely or not at all, with no interleaving.
- Blocking vs Non-blocking I/OWhether waiting for I/O stops the thread.
- CancellationStopping in-flight async work cleanly, e.g. with AbortController or a context.
- Check-Then-Act Race (TOCTOU)Checking a condition and acting on it as two steps, so something changes in between.
- Concurrency vs ParallelismDealing with many things at once vs doing many things at once.
- CoroutineA function that can pause and resume; the basis of async/await.
- Critical SectionCode that must not run concurrently with itself.
- DeadlockTwo or more tasks waiting on each other forever.
- Event LoopA single thread running tasks from a queue, as in JavaScript and asyncio.
- Global Interpreter Lock (GIL)Python's lock that lets only one thread run bytecode at a time.
- Mutex / LockEnsuring only one thread enters a critical section at a time.
- ProcessA running program with its own memory space.
- Race ConditionA bug where the result depends on unpredictable timing.
- ThreadAn execution path within a process that shares its memory.
- Thread PoolA fixed set of reusable worker threads.
- Thread SafetyCode that behaves correctly when called from many threads.
- BisectBinary-searching history to find the commit that introduced a bug.
- Cherry-PickApplying one specific commit onto another branch.
- Fast-Forward vs Merge CommitMoving a branch pointer ahead vs recording a merge commit.
- Git HooksScripts that run on Git events, like linting before a commit.
- Git LFSStoring large binary files outside normal Git history.
- Interactive RebaseRewriting commits: squash, reorder, edit, drop.
- Merge vs RebasePreserving history as it happened vs rewriting it to be linear.
- MonorepoMany projects living in one repository.
- ReflogGit's log of where HEAD has been; your undo history after disasters.
- Removing Secrets from Git HistoryPurging a committed secret with git filter-repo, and why you must rotate it anyway.
- Rewriting Shared HistoryWhy rebasing or force-pushing shared branches breaks teammates.
- Signed CommitsCryptographically proving who authored a commit.
- SubmoduleA repository embedded inside another at a fixed commit.
- Branching StrategyThe team's rules for how branches are created and merged.
- Git FlowA model with develop, feature, release and hotfix branches.
- Long-Lived BranchA branch kept for weeks, and the painful merges that follow.
- Release BranchA branch stabilized for shipping a specific version.
- Release NotesA user-facing explanation of what changed in a release.
- Trunk-Based DevelopmentEveryone merges small changes to main frequently.
- AI-Assisted Code ReviewUsing AI reviewers as a first pass, not a replacement for human review.
- CODEOWNERSA file that assigns required reviewers by path.
- CONTRIBUTING GuideA file explaining how to propose changes to a project.
- Giving Code ReviewReviewing others' code helpfully: priorities, tone and turnaround.
- Merge Commit vs Squash vs Rebase MergeHow a PR lands on main and what history it leaves behind.
- Mob ProgrammingA whole team working on one problem together.
- Review TurnaroundHow quickly reviews happen, and why it matters to team speed.
- Conditional BreakpointPausing only when a condition is true.
- Five WhysAsking "why" repeatedly to get from a symptom to its root cause.
- HeisenbugA bug that disappears when you try to observe it.
- ProfilerA tool that measures where time or memory goes.
- Remote DebuggingAttaching a debugger to code running somewhere else.
- Root Cause AnalysisFinding the underlying reason, not just the symptom.
- Acceptance TestVerifying a feature meets the agreed requirements.
- Behavior-Driven DevelopmentSpecifying behavior as plain-language scenarios shared with non-developers.
- Deterministic TestsControlling time, randomness and network so tests always behave the same.
- End-to-End TestA test of the whole system through its real interface.
- FakeA working but simplified implementation, like an in-memory database.
- Given / When / ThenDescribing tests as context, action and expected outcome.
- Load TestingMeasuring behavior under expected traffic.
- Mock ServerA fake API that returns canned responses, for testing clients in isolation.
- Over-MockingMocking so much that tests check implementation instead of behavior.
- Performance TestingMeasuring speed and resource use; load, stress, soak and spike tests.
- Red, Green, RefactorThe three-step TDD cycle.
- Snapshot TestComparing output to a saved snapshot.
- SpyA test double that records calls to a real or fake function.
- Test Data Builder / FactoryHelpers that create valid test objects with sensible defaults.
- Test DatabasesUsing a real, isolated database in tests and resetting it between runs.
- Test DoubleAny stand-in for a real dependency in tests.
- Test-Driven DevelopmentWriting a failing test first, then the code to pass it, then refactoring.
- TestcontainersSpinning up real databases and services in Docker for tests.
- Testing PyramidMany unit tests, fewer integration tests, few end-to-end tests.
- Broken Windows TheorySmall neglected messes invite bigger ones.
- Chesterton's FenceDon't remove something until you know why it was put there.
- CohesionHow closely the parts of a module belong together; more is better.
- Command-Query SeparationA method should either change state or return data, not both.
- Convention over ConfigurationSensible defaults so you only configure what's unusual.
- CouplingHow much modules depend on each other's internals; less is better.
- Cyclomatic ComplexityA count of independent paths through code; a rough complexity measure.
- Dependency Inversion Principle (D)Depend on abstractions, not concrete implementations.
- Encapsulate What VariesIsolating the parts that change from the parts that stay the same.
- Feature EnvyA method more interested in another class's data than its own.
- God ObjectA class that knows and does too much.
- IdempotenceDoing something twice has the same effect as doing it once.
- Interface Segregation Principle (I)Clients shouldn't depend on methods they don't use.
- Law of DemeterOnly talk to your immediate neighbors; avoid a.b().c().d().
- Liskov Substitution Principle (L)Subtypes must work anywhere their parent type is expected.
- Open/Closed Principle (O)Open for extension, closed for modification.
- Premature AbstractionGeneralizing before you have enough examples, creating the wrong abstraction.
- Primitive ObsessionUsing raw strings and ints where a small domain type belongs.
- Principle of Least AstonishmentCode and APIs should behave the way people expect.
- Program to an InterfaceDepending on what something does rather than which class it is.
- Rule of ThreeTolerating duplication until the third time, to avoid the wrong abstraction.
- Shotgun SurgeryOne change requiring edits in many scattered places.
- Single Responsibility Principle (S)A module should have one reason to change.
- SOLID PrinciplesFive object-oriented design principles for maintainable code.
- Tell, Don't AskTell objects what to do instead of querying their state and deciding for them.
- Characterization TestA test that captures current behavior before you change legacy code.
- Extract ClassSplitting a class that does too much.
- Introduce Parameter ObjectGrouping parameters that travel together into one object.
- Legacy CodeCode without tests that you're afraid to change.
- Replace Conditional with PolymorphismSwapping a type switch for subclasses or strategies.
- Anti-PatternA common solution that looks right but causes problems.
- Dependency InjectionPassing dependencies in instead of creating them inside.
- Design PatternA named, reusable solution to a recurring design problem.
- Factory MethodLetting subclasses or functions decide which class to instantiate.
- SingletonEnsuring a class has exactly one instance; often an anti-pattern.
- StrategySwapping algorithms behind a common interface.
- Code GenerationGenerating boilerplate from schemas or templates.
- Dependency HellConflicting version requirements that can't all be satisfied.
- Dev ContainersA reproducible development environment defined as a container.
- DotfilesYour personal config files, often kept in Git.
- jqA command-line JSON processor.
- Language Server ProtocolThe protocol that gives editors autocomplete and diagnostics for any language.
- MakefileA classic way to define project tasks and their dependencies.
- Reading Library Source CodeReading a library's code when its docs don't answer your question.
- Shell ScriptingAutomating tasks with shell scripts.
- Static AnalysisFinding bugs by analyzing code without running it.
- Transitive DependencyA dependency of one of your dependencies.
- Agent Instruction FilesRepository files like CLAUDE.md or AGENTS.md that tell AI agents the project's conventions.
- Coding AgentAn AI that plans, edits across files and runs commands on its own.
- Context WindowHow much text a model can consider at once.
- Model Context Protocol (MCP)A standard for connecting AI assistants to tools and data.
- API DocumentationReference docs for an API's endpoints, parameters and errors.
- Architecture DiagramA box-and-arrow picture of a system's components and data flow.
- Class DiagramA diagram of classes, their fields and relationships.
- Diagrams as CodeWriting diagrams as text, with tools like Mermaid or PlantUML.
- Docs as CodeKeeping docs in the repository and reviewing them like code.
- Onboarding DocumentationDocs that get a new teammate productive quickly.
- RunbookStep-by-step instructions for operating or fixing a system.
- Sequence DiagramA diagram of messages passed between components over time.
- State DiagramA diagram of states and the transitions between them.
- UMLA standard visual language for software diagrams.
- Adjacency List vs MatrixTwo ways to store a graph's edges.
- B-TreeA wide, shallow tree optimized for disks; how database indexes work.
- Binary Search TreeA binary tree ordered so each lookup can halve the search.
- DequeA double-ended queue that adds and removes at both ends.
- Directed Acyclic Graph (DAG)A graph with directed edges and no cycles, as in build systems and pipelines.
- Doubly Linked ListA linked list with pointers in both directions.
- GraphNodes connected by edges; models networks, dependencies and maps.
- Hash CollisionTwo keys hashing to the same slot, and how tables handle it.
- HeapA tree that keeps the min or max at the root; backs priority queues.
- LRU CacheA cache evicting the least recently used item, built from a hash map and a linked list.
- Priority QueueA queue that always returns the highest-priority item first.
- Tree TraversalVisiting tree nodes in pre-order, in-order, post-order or level order.
- TrieA tree of characters for fast prefix lookups.
- Amortized AnalysisThe average cost per operation over a sequence of operations.
- BacktrackingTrying choices and undoing them when they lead nowhere.
- Big Omega and Big ThetaLower bounds and tight bounds, alongside Big O's upper bound.
- Breadth-First SearchExploring a graph level by level; finds shortest unweighted paths.
- Depth-First SearchExploring a graph as deep as possible before backtracking.
- Divide and ConquerSplitting a problem into smaller ones and combining the results.
- Dynamic ProgrammingSolving overlapping subproblems once and reusing the answers.
- Greedy AlgorithmTaking the locally best choice at each step.
- Merge SortSplit in half, sort each half, merge: a stable O(n log n) sort.
- QuicksortPartitioning around a pivot; fast in practice, O(n²) in the worst case.
- Sliding WindowMaintaining a moving range over a sequence.
- Stable SortA sort that keeps equal elements in their original order.
- Topological SortOrdering tasks so every dependency comes first.
- Two PointersWalking two indexes through data to avoid nested loops.
- CombinatoricsCounting arrangements and combinations, e.g. how many IDs a format allows.
- Discrete MathLogic, sets, graphs and combinatorics: the math of computing.
- Probability BasicsReasoning about chance, for sampling, A/B tests and failure rates.
- Set TheoryUnions, intersections and differences; the basis of SQL.
- GPUA processor for massively parallel work like graphics and ML.
- Latency Numbers Every Programmer Should KnowRough costs of cache, memory, disk and network access.
- Daemon / ServiceA background process, usually managed by systemd.
- File DescriptorA handle to an open file, socket or pipe.
- Graceful ShutdownFinishing in-flight work before exiting on SIGTERM.
- KernelThe core of the OS with full access to the hardware.
- Out of Memory (OOM)Running out of memory, and the OOM killer that ends processes.
- Process LifecycleHow processes are created, run, wait and exit.
- Resource Limits (ulimit)Per-process caps on open files, memory and processes.
- SignalsMessages like SIGTERM and SIGKILL sent to processes.
- System CallA program asking the kernel to do something, like read a file.
- Virtual MemoryEach process sees its own address space, mapped onto physical memory.
- Connect Timeout vs Read TimeoutWaiting to establish a connection vs waiting for data on it.
- Connection PoolingReusing a set of open connections instead of opening new ones.
- DHCPHow devices get an IP address automatically when they join a network.
- DNS Record TypesA, AAAA, CNAME, MX, TXT, NS and what each one does.
- DNS TTL and PropagationHow long DNS answers are cached, and why changes take time to spread.
- FirewallRules that allow or block network traffic.
- FTP and SFTPOlder file transfer protocols, and the SSH-based secure version.
- ICMPThe protocol behind ping and many network error messages.
- Inspecting Connections (ss, netstat, lsof)Seeing which ports and connections a machine has open.
- IPv4 vs IPv632-bit vs 128-bit addresses, and why IPv6 exists.
- Keep-Alive ConnectionsReusing one connection for many requests.
- NATMany private addresses sharing one public IP.
- OSI ModelA seven-layer model of how network communication is structured.
- PacketA unit of data sent over a network.
- Private IP RangesAddress blocks reserved for internal networks.
- Round-Trip Time (RTT)The time for a message to go out and a reply to come back.
- SocketAn endpoint for sending and receiving data over a network.
- Subnet and CIDRDividing IP ranges, written like 10.0.0.0/16.
- TCPReliable, ordered, connection-based transport.
- TCP Three-Way HandshakeSYN, SYN-ACK, ACK: how a TCP connection starts.
- TCP/IP ModelThe practical four-layer model the internet actually runs on.
- ThroughputHow much work or data gets through per unit of time.
- UDPFast, connectionless transport with no delivery guarantees.
- VPNAn encrypted tunnel into another network.
- Cache-ControlThe header that directs how responses may be cached.
- Content NegotiationClient and server agreeing on format and language through Accept headers.
- ETagA version identifier used for conditional requests and caching.
- HTTP CachingCache-Control, ETag and Last-Modified for reusing responses.
- HTTP Compressiongzip and Brotli shrinking responses on the wire.
- HTTP ProxyAn intermediary that forwards HTTP requests.
- HTTP/1.1The text-based HTTP version with one request at a time per connection.
- HTTP/2Binary framing and many requests multiplexed over one connection.
- multipart/form-dataThe encoding used to upload files from forms.
- Safe and Idempotent MethodsWhich HTTP methods shouldn't change state, and which can be retried safely.
- Synchronous vs Asynchronous APIsReturning the result in the response vs accepting work and reporting later.
- Webhooks vs PollingBeing notified when something changes vs repeatedly asking.
- Certificate AuthorityAn organization trusted to sign certificates.
- Certificate ExpiryExpired certificates as a classic cause of outages, and automating renewal.
- Self-Signed CertificateA certificate not signed by a trusted authority, used in development.
- TLSThe protocol that encrypts and authenticates network connections.
- API Client / SDKA library wrapping an API so callers don't hand-write HTTP.
- AvroA binary serialization format whose schemas are designed to evolve.
- GraphQLA query language that lets clients ask for exactly the data they need.
- gRPCA fast RPC framework built on HTTP/2 and Protocol Buffers.
- JSON SchemaA vocabulary for validating the structure of JSON.
- OpenAPIA standard format for describing REST APIs.
- Protocol BuffersA compact, schema-based binary serialization format.
- Resource ModelingDeciding what your API's resources are and how they relate.
- REST ConstraintsStatelessness, uniform interface, cacheability and REST's other rules.
- RPCCalling a function on another machine as if it were local.
- XMLA verbose markup format still common in enterprise and legacy systems.
- A/B TestingComparing two variants with real users to see which performs better.
- Product AnalyticsTracking how people use features to inform decisions.
- Offset vs Cursor PaginationSimple page numbers vs stable, scalable cursors.
- Serverless FunctionsRunning code per request without managing servers.
- The Twelve-Factor AppTwelve practices for building deployable, scalable web services.
- Third-Party IntegrationsCalling payment, email and other external APIs reliably.
- API VersioningEvolving an API without breaking existing clients.
- Backward CompatibilityNew versions that still work with old clients.
- DeprecationMarking something for removal and giving users time to migrate.
- Idempotency KeyA client-supplied ID that makes retried requests safe.
- Rate LimitingLimiting how many requests a client can make.
- Correlated SubqueryA subquery that runs once per row of the outer query.
- Cross JoinEvery row of one table paired with every row of another.
- Data ModelingDesigning how your data is structured and related.
- Database CursorFetching a large result set in batches instead of all at once.
- Dynamic SQLBuilding SQL strings at runtime, and doing it without injection.
- Enums in the DatabaseStoring fixed sets of values safely.
- ER DiagramA diagram of entities and their relationships.
- EXPLAINShowing how the database plans to run a query.
- Full-Text Search in SQLSearching words in text columns with built-in indexes.
- JSON ColumnsStoring semi-structured data inside a relational database.
- Natural vs Surrogate KeyUsing real-world data as the key vs a generated ID.
- NormalizationOrganizing tables to reduce duplication: 1NF, 2NF, 3NF.
- Prepared StatementA query parsed once and executed many times with different values.
- Recursive CTEQuerying hierarchies and graphs, like org charts or category trees.
- Relational ModelThe theory behind SQL: relations, tuples, attributes and keys.
- SequenceA database object that generates increasing numbers, used for auto-increment IDs.
- Soft DeleteMarking rows as deleted instead of removing them.
- Stored ProcedureLogic that runs inside the database.
- Timestamp With vs Without Time ZoneStoring moments in time correctly in the database.
- TriggerCode the database runs automatically on insert, update or delete.
- UUID vs Auto-Increment IDsSequential integers vs globally unique IDs, and their trade-offs.
- Composite IndexAn index on several columns, where column order matters.
- Eager vs Lazy LoadingLoading related data upfront vs on first access.
- Query OptimizationRewriting queries and adding indexes to make them fast.
- Slow Query LogA record of queries that take too long.
- Unique IndexAn index that also enforces uniqueness.
Transactions & Concurrency Control
- ACIDAtomicity, Consistency, Isolation, Durability.
- Atomic UpdateLetting the database do the change in one statement, like SET stock = stock - 1.
- AtomicityAll or nothing.
- Lost UpdateTwo writers overwriting each other's changes.
- Optimistic LockingDetecting conflicts with a version number at write time.
- Pessimistic LockingLocking rows before changing them, e.g. with SELECT ... FOR UPDATE.
- Read-Modify-Write RaceReading a value, changing it in code and writing it back while someone else does the same.
- SavepointA point inside a transaction you can roll back to.
- Transaction BoundariesDeciding where a transaction should start and end.
- Unique Constraints as a Concurrency GuardLetting the database reject duplicates instead of checking first in code.
- Embedded DatabaseA database that runs inside your app, like SQLite.
- Leaderboards with Sorted SetsRanking scores in real time with Redis sorted sets.
- PipeliningSending many commands without waiting for each reply.
- Redis Data StructuresStrings, hashes, lists, sets, sorted sets and streams, and what each is for.
- Search EngineElasticsearch, OpenSearch and others for full-text search.
- SQL vs NoSQLChoosing a database by data shape, consistency needs and access patterns.
- Vector DatabaseStoring embeddings for similarity search.
- Data MigrationMoving or transforming existing data.
- Rolling Back a MigrationUndoing a migration, and why some can't be undone.
- Database Authentication and TLSHow clients prove who they are to the database, and encrypting that connection.
- Database Version UpgradesMoving to a new major version with little downtime.
- Key-Range vs Hash PartitioningSplitting data by ranges of keys vs by hashed keys.
- Logical vs Physical BackupsSQL dumps vs copies of the data files.
- Roles and Privileges (GRANT, REVOKE)Controlling who can read and change which data.
- Row-Level SecurityThe database itself filtering rows per user or tenant.
- Table PartitioningSplitting one huge table into smaller physical pieces, e.g. by month.
- Cache InvalidationRemoving stale data from a cache; famously one of the hard problems.
- Cache Key DesignChoosing keys so different data never collides.
- Cache-AsideThe app checks the cache, then the database, then fills the cache.
- CDN CachingCaching content on edge servers near users.
- Eviction PolicyLRU, LFU or FIFO: which item to drop when the cache is full.
- In-Process vs Distributed CacheA cache in your app's memory vs a shared one like Redis.
- Request-Scoped CachingCaching within a single request to avoid repeated lookups.
- Acknowledgement (ack and nack)A consumer confirming a message was processed, or asking for it to be redelivered.
- At-Least-Once DeliveryMessages may arrive more than once, so consumers must be idempotent.
- Consumer GroupConsumers sharing the work from a topic.
- Dead Letter QueueWhere messages go after failing repeatedly.
- Delayed JobsJobs scheduled to run at a later time.
- Delivery GuaranteesAt-most-once, at-least-once and exactly-once delivery.
- Event StreamingProcessing a continuous flow of events.
- Idempotent ConsumerA consumer that handles duplicate messages safely.
- Job QueueA queue of background tasks with workers, like Sidekiq, Celery or BullMQ.
- Message BrokerMiddleware like RabbitMQ that routes messages.
- Message Envelope and HeadersMetadata travelling with a message: IDs, timestamps, correlation and type.
- Message QueueA buffer where producers leave messages for consumers to process later.
- Producer and ConsumerThe two sides of a queue.
- Queue Depth and Consumer LagHow far behind consumers are; a key health signal.
- Queue vs TopicPoint-to-point (each message to one consumer) vs pub/sub (each message to every subscriber).
- Retry with Exponential BackoffRetrying with growing delays so you don't hammer a failing service.
- SubscriptionA consumer's registration to receive messages from a topic.
- Topics and PartitionsHow Kafka-style logs split and order messages.
- Visibility TimeoutHow long a received message stays hidden before it's delivered again.
- WorkerA process that pulls jobs from a queue and runs them.
- Data ExportLetting users download their data, often as a background job.
- Presigned URLA temporary URL letting clients upload or download directly from storage.
- Streaming Large FilesProcessing files in chunks instead of loading them into memory.
- Record History and VersioningKeeping previous versions of rows with history tables or temporal tables.
- MicroservicesMany small, independently deployable services.
- Non-Functional RequirementsRequirements about how well a system works rather than what it does.
- Serverless ArchitectureBuilding from managed functions and services without running servers.
- Data LocalityKeeping data close to where it's processed.
- Database ReplicationCopying data to other servers for availability and read scaling.
- Horizontal ScalingScaling out with more machines.
- Hot, Warm and Cold StorageTiering data by how often it's accessed.
- Read ReplicaA copy of the database for serving reads.
- ShardingSplitting data across databases by key.
- BASEBasically Available, Soft state, Eventual consistency: the counterpart to ACID.
- CAP TheoremDuring a network partition, you must choose consistency or availability.
- Eventual ConsistencyReplicas converge once updates stop, but reads may be stale in the meantime.
- AvailabilityThe share of time a system is usable, often measured in "nines".
- Handling Dependency FailuresDeciding what happens when something you call is down.
- Liveness vs Readiness"Am I running?" vs "Can I take traffic?"
- Nines (99.9%, 99.99%)How much downtime each level of availability allows.
- RedundancyExtra components so a single failure isn't fatal.
- ReliabilityA system doing what it should, even when things fail.
- Restore TestingActually restoring backups regularly to prove they work.
- Single Point of FailureOne component whose failure takes everything down.
- AutoscalingAdding and removing capacity automatically as load changes.
- BatchingGrouping work to reduce per-item overhead.
- BenchmarkingMeasuring performance under controlled conditions.
- BottleneckThe component that limits overall throughput.
- Cold StartThe delay when a serverless function or container starts fresh.
- Moving Work Off the Request PathDoing slow work in the background to keep responses fast.
- Payload SizeSending fewer bytes over the wire.
- PerformanceHow fast and efficiently a system does its work.
- Distributed Batch ProcessingEngines like Spark that split big jobs across many machines.
- EventA record that something happened.
- Event Schema VersioningEvolving event formats without breaking consumers.
- MapReduceProcessing huge datasets by mapping in parallel and then reducing the results.
- Schema RegistryA central store of event schemas and their compatibility rules.
- Stream WindowingGrouping events by time windows: tumbling, sliding and session windows.
- Unix PhilosophySmall tools that do one thing well and compose through pipes.
- Cryptographic Hash (SHA-256)A hash where finding collisions is infeasible.
- CryptographyThe math of keeping data secret and verifying it.
- Digital SignatureProving who created data and that it hasn't been changed.
- Encryption at RestEncrypting stored data on disk.
- HMACA keyed hash proving a message came from someone holding the secret.
- Public-Key CryptographyA public key encrypts or verifies; a private key decrypts or signs.
- Secure Random NumbersCryptographically secure randomness for tokens and keys.
- Symmetric EncryptionOne shared key both encrypts and decrypts, as with AES.
- Attack SurfaceEvery point where an attacker could try to get in.
- Audit LoggingRecording who did what and when, for accountability.
- CVEA public identifier for a known vulnerability.
- Defense in DepthSeveral layers of security, so one failure isn't fatal.
- Dependency ScanningFinding known vulnerabilities in your dependencies.
- SASTStatic analysis that looks for security bugs in source code.
- Secrets ManagementKeeping API keys and passwords in a vault, not in code.
- TyposquattingMalicious packages named like popular ones.
- Zero-DayA vulnerability exploited before a fix exists.
- Anonymization vs PseudonymizationIrreversibly vs reversibly removing identity from data.
- CopyleftLicenses that require derivative work to stay open source.
- Data MinimizationCollecting only the data you actually need.
- GDPRThe EU regulation on personal data: consent, access and deletion rights.
- Open Source LicensesMIT, Apache, GPL, and what each one lets you do.
- Bare MetalRunning directly on physical servers.
- Disks, Filesystems and MountsAttaching storage and making it usable.
- Hard Link vs Symbolic LinkTwo names for the same file vs a pointer to a path.
- Load AverageA rough measure of how busy a machine is.
- Reading System Logsjournalctl, /var/log, and finding out what went wrong.
- System Monitoring (top, htop, vmstat, iotop)Seeing CPU, memory and disk activity on a machine.
- systemdThe Linux service manager for starting, stopping and logging services.
- Text Processing (sed, awk, cut, sort)Command-line tools for slicing and transforming text.
- tmux / screenKeeping terminal sessions running after you disconnect.
- Users and GroupsHow Linux controls who can do what.
- Bind Mounts vs VolumesMounting a host directory vs Docker-managed storage.
- Container NetworkingHow containers talk to each other and the outside world.
- ENTRYPOINT and CMDWhat runs when a container starts.
- Hot Reloading in ContainersSeeing code changes instantly while developing in Docker.
- Image Layers and Build CacheHow images are built from cached layers, and how to order steps.
- Image ScanningChecking container images for known vulnerabilities.
- Image Tags and DigestsNaming image versions, and why :latest is risky.
- Multi-Stage BuildBuilding in one stage and shipping a small final image.
- Running as Non-RootNot running container processes as root.
- ConfigMap and SecretConfiguration and sensitive values injected into pods.
- Container OrchestrationAutomating the deployment, scaling and healing of containers.
- CrashLoopBackOffA pod that keeps crashing and restarting.
- DeploymentDeclares how many replicas of a pod should run and how to update them.
- HelmA package manager for Kubernetes.
- IngressRouting external HTTP traffic into the cluster.
- Job and CronJobRunning one-off and scheduled tasks in Kubernetes.
- kubectlThe command-line tool for Kubernetes.
- KubernetesThe dominant container orchestration platform.
- Liveness and Readiness ProbesHow Kubernetes checks whether a container is healthy.
- NamespaceDividing cluster resources between teams or apps.
- NodeA machine in the cluster that runs pods.
- PodThe smallest deployable unit in Kubernetes: one or more containers.
- ReplicaSetKeeps a set number of identical pods running.
- ServiceA stable network address for a set of pods.
- ClusterA group of machines working together as one system.
- Functions as a Service (Lambda)Running code in response to events without servers.
- IAMIdentity and access management for cloud resources.
- IAM PolicyA document granting permissions on resources.
- Managed Queues (SQS, Pub/Sub)Message queues run by the cloud provider.
- Public vs Private SubnetSubnets reachable from the internet vs only from inside.
- Regions and Availability ZonesGeographic locations, and isolated data centers within them.
- Security GroupA virtual firewall around cloud resources.
- Service AccountAn identity for software rather than a person.
- VPCA private network in the cloud.
- Configuration Management (Ansible)Automating the setup of servers.
- Declarative vs Imperative InfrastructureDescribing the end state vs listing the steps.
- Infrastructure as CodeDefining infrastructure in versioned files instead of clicking in consoles.
- Pets vs CattleHand-tended servers vs interchangeable, replaceable ones.
- Terraform / OpenTofuDeclarative infrastructure provisioning across providers.
- Artifact RepositoryStorage for built packages and images.
- Blue-Green DeploymentSwitching traffic between two identical environments.
- Build CachingReusing previous build outputs to make CI fast.
- Canary ReleaseReleasing to a small share of users first.
- Continuous Delivery / DeploymentKeeping every change releasable, or releasing it automatically.
- Deploy vs ReleaseShipping code vs exposing it to users.
- Deployment FreezeA period when deploys are paused, like around holidays.
- Preview EnvironmentA temporary deployment for each pull request.
- Roll ForwardFixing a bad deploy with a new deploy instead of reverting.
- Rolling DeploymentReplacing instances a few at a time.
- Secrets in CIHandling credentials safely in pipelines.
- Alert FatigueSo many alerts that people start ignoring them.
- AlertingNotifying people when something needs attention.
- APMApplication performance monitoring tools.
- Correlation / Request IDAn ID passed through every service to tie one request's logs together.
- Four Golden SignalsLatency, traffic, errors and saturation.
- Log AggregationCollecting logs from every server into one searchable place.
- Logs, Metrics and TracesThe three main kinds of telemetry.
- MetricsNumeric measurements over time.
- ObservabilityUnderstanding a system's internal state from its outputs.
- Prometheus and GrafanaA common open-source stack for metrics and dashboards.
- Structured LoggingLogging key-value fields instead of free text.
- Blameless PostmortemFocusing on systems rather than individuals when things go wrong.
- Emergency Change ProcessShipping urgent fixes safely under pressure.
- EscalationKnowing when and how to pull in more help.
- Incident ResponseHow a team detects, coordinates, fixes and communicates during an incident.
- Incident Severity LevelsSEV1 to SEV4: how bad an incident is.
- Mitigate First, Fix LaterStopping the bleeding before hunting for the root cause.
- On-CallBeing responsible for responding to production issues.
- PagingAlerts that wake someone up.
- PostmortemA written review of an incident: what happened and how to prevent it.
- SLAA service level agreement: a promise to customers, with consequences.
- Status PagePublic communication about outages.
- Ad-Hoc Queries on ProductionQuerying live data without hurting it: read replicas, timeouts and LIMIT.
- Feature Flag CleanupRemoving flags once rollout is done so they don't pile up as debt.
- Fixing Data in ProductionCorrecting bad rows safely: a reviewed script, a backup, and a record of what changed.
- Launch ChecklistMonitoring, rollback, docs and support readiness before something goes live.
- Maintenance WindowA scheduled, announced time for risky changes.
- On-Call HandoverPassing open issues and context to the next person on call.
- One-Off ScriptsScripts run once against production, and why they deserve review and tests too.
- Production ConsolesInteractive shells against live data, like a Rails or Django console, and their dangers.
- Reproducing Production Issues LocallyRecreating a production bug with realistic, anonymized data.
- Bounded vs Unbounded DataA finite dataset vs a stream that never ends.
- Modern Data StackCloud warehouse, managed ingestion, SQL transformations and BI tools wired together.
- Bot and Test Traffic FilteringKeeping crawlers, monitoring checks and internal users out of the numbers.
- Client-Side vs Server-Side TrackingRecording events in the browser or app vs on your servers, and what each misses.
- Consent at CollectionOnly collecting what users agreed to, and recording that they agreed.
- Customer Data Platform (CDP)Tools like Segment that collect events once and route them to many destinations.
- External and Third-Party DataData you buy, scrape or pull from partners, and the contracts and quality risks it brings.
- SessionizationGrouping events into sessions by user and inactivity gaps.
- Tracking PlanA shared spec of which events to track, their names and their properties.
- User-Generated ContentPosts, reviews, uploads and messages as a data source.
- Web ScrapingExtracting data from websites, and its legal and reliability limits.
- MERGE StatementSQL that inserts, updates or deletes rows in one pass based on a match.
- Messages vs StreamsTransient queued messages vs a durable, replayable log of events.
- Data MartA subset of the warehouse focused on one team or subject.
- Data Retention and TieringExpiring or moving old data to cheaper storage on purpose.
- Distributed File System (HDFS)Storing huge files across many machines with replication.
- MPP Data WarehouseWarehouses that split one query across many nodes in parallel.
- Time TravelQuerying a table as it was at an earlier point in time.
- One Big Table (Wide Tables)Denormalizing everything into one wide table for simple, fast queries.
- Snowflake SchemaA star schema whose dimensions are normalized into sub-tables.
Batch & Distributed Processing
- Distributed ComputingSplitting work across many machines that coordinate over a network.
- Driver and ExecutorsThe process that plans a job and the workers that run its tasks.
- Partitions in Distributed ProcessingThe chunks of data that tasks process in parallel.
- User-Defined Function (UDF)Custom code called from SQL or DataFrame operations, and its performance cost.
- Micro-BatchingProcessing a stream as a series of tiny batches.
Transformation & Analytics SQL
- Gaps and IslandsFinding consecutive runs and breaks in sequences with SQL.
- GROUP BY ROLLUP and CUBEComputing subtotals and grand totals in one query.
- QUALIFYFiltering on window function results without a subquery.
- SamplingWorking on a representative subset to go faster.
- SensorsTasks that wait for a condition, like a file arriving.
- Anomaly Detection on DataFlagging unusual volumes or values automatically.
- Data DowntimePeriods when data is missing, late or wrong.
- Data Expectations / AssertionsDeclared rules data must satisfy, checked in the pipeline.
- Column-Level SecurityRestricting access to specific sensitive columns.
- Data ClassificationLabeling data by sensitivity: public, internal, confidential, restricted.
- Data GovernanceThe policies and roles that decide how data is managed and used.
- Reference DataShared code lists like country codes, currencies and status values.
- Technical, Business and Operational MetadataSchemas, meanings, and run history: the three kinds of data about data.
- Data APIsServing data to applications through an API instead of direct database access.
- Reverse ETL Use CasesSyncing warehouse data to CRMs, ad tools and support systems.
- Self-Service AnalyticsLetting non-engineers explore data safely on their own.
- Training DataLabeled examples prepared for machine learning.
- Data On-CallBeing responsible for pipeline failures and late data.
- Deprecating TablesRetiring datasets without breaking hidden downstream users.
- Tuning a Slow PipelineFinding the slow step and fixing it: partitions, joins, skew, file sizes.
- Reverse ETLSyncing warehouse data back into operational tools.
- Deep LearningNeural networks with many layers.
- Machine LearningSoftware that learns patterns from data instead of following explicit rules.
- ModelThe learned function that turns inputs into predictions.
- Neural NetworkLayers of weighted connections that learn from data.
- Supervised LearningLearning from labeled examples.
- Training vs InferenceBuilding a model vs using it to make predictions.
- AI AgentA model that plans and takes actions with tools in a loop.
- EmbeddingsNumeric vectors representing meaning, used for similarity search.
- How LLMs WorkTokens in, next-token prediction out: a mental model of what's happening.
- Multimodal ModelsModels that handle images, audio and text together.
- Prompt EngineeringWriting instructions that get reliable results from a model.
- Prompt InjectionUntrusted text hijacking a model's instructions.
- Retrieval-Augmented Generation (RAG)Giving a model relevant documents so it answers from your data.
- Semantic SearchSearching by meaning instead of keywords.
- Streaming ResponsesShowing model output token by token as it's generated.
- Structured OutputGetting models to return valid JSON matching a schema.
- System PromptInstructions that set a model's behavior for a conversation.
- TemperatureA setting controlling how random a model's output is.
- TokenThe unit of text a model reads and writes, and what you pay for.
- Tool Use / Function CallingLetting a model call your functions and APIs.
- Continuous Improvement (Kaizen)Small, ongoing improvements to how the team works.
- Definition of ReadyWhat a ticket needs before work on it starts.
- Iterative DeliveryShipping a thin working slice first, then improving it.
- MVPThe smallest product that tests whether an idea works.
- TriageSorting incoming bugs and requests by urgency and impact.
- VelocityHow much work a team finishes per sprint, and how it gets misused.
- Vertical SliceA feature built through every layer, end to end.
- WIP LimitCapping how many tasks are in progress at once.
- Buffers and UnknownsAccounting for the work you can't see yet.
- Hofstadter's LawIt always takes longer than you expect, even when you account for this.
- MilestoneA significant checkpoint in a project.
- Ninety-Ninety RuleThe last 10% of the work takes the other 90% of the time.
- PrioritizationDeciding what to do first: impact vs effort, urgent vs important.
- Scope CreepRequirements growing while a project is underway.
- SpikeA time-boxed investigation to reduce uncertainty.
- T-Shirt SizingRough S/M/L/XL estimates for early planning.
- Async CommunicationWriting so people can respond on their own time.
- Effective MeetingsAgendas, notes, clear decisions, and fewer meetings.
- Explaining to Non-EngineersTranslating technical trade-offs into business terms.
- Giving FeedbackFeedback that's specific, timely, kind and actionable.
- Making Work Visible RemotelyOver-communicating when nobody can see you working.
- Saying NoDeclining or pushing back on work constructively.
- Working with DesignersCollaborating on how it should look and behave.
- Working with Product ManagersCollaborating on what to build and why.
- DogfoodingUsing your own product.
- Functional vs Non-Functional RequirementsWhat it does vs how well it does it.
- Handling Support EscalationsInvestigating issues reported by customers.
- Product ThinkingUnderstanding the user problem behind the ticket.
- Career ConversationsDiscussing goals and growth with your manager.
- Compensation and NegotiationSalary, equity and bonuses, and negotiating offers.
- ImpactOutcomes that matter to the business, not just output.
- Mid-Level EngineerDelivering features independently.
- OwnershipTaking responsibility for outcomes, not just tasks.
- PromotionShowing next-level scope before you get the title.
- T-Shaped SkillsBroad knowledge with deep expertise in one area.
Senior
Design the platform's storage, processing and modeling choices.
Core: start here
- Architecture Decision Record (ADR)A short document recording a decision, its context and its consequences.
- Design DocumentA written proposal of how to build something, reviewed before building it.
- Exactly-Once SemanticsThe hard, often misunderstood guarantee of processing each message once.
- Architectural Trade-offsEvery architecture decision gives something up; naming what.
- One-Way vs Two-Way DoorsMoving fast on reversible decisions and carefully on irreversible ones.
- Back-of-the-Envelope EstimationRough math for traffic, storage and bandwidth.
- CQRSSeparating the write model from the read model.
- Event SourcingStoring every change as an event and deriving state from them.
- Data ProductA dataset treated as a product, with an owner, documentation, quality guarantees and users.
- Log-Based CDCCapturing changes from a database's write-ahead log instead of querying tables.
- Clustering and Z-OrderingSorting data inside files so related rows sit together for faster scans.
- Conformed DimensionOne shared dimension used by many fact tables so numbers line up.
- Data VaultModeling with hubs, links and satellites for auditable, change-friendly history.
- Inmon vs KimballA normalized enterprise warehouse first vs dimensional marts first.
Batch & Distributed Processing
- Broadcast JoinSending a small table to every worker to avoid shuffling the big one.
- Data SkewA few keys holding most of the data, so one task runs forever.
- Kappa ArchitectureEverything as a stream, with reprocessing by replaying the log.
- Lambda ArchitectureParallel batch and streaming paths merged at query time.
- Stateful Stream ProcessingKeeping running state like counts or sessions across events.
- Stream-Table DualityA table is the latest state of a stream; a stream is a table's changelog.
- WatermarkA stream's estimate of how far event time has progressed, used to close windows.
- Asset-Based OrchestrationOrchestrating the datasets you want to exist, not just the tasks.
- Choosing an Orchestrator (Airflow, Dagster, Prefect)Task-centric vs asset-centric orchestration and their trade-offs.
- Data SLAs and SLOsPromises about when data will be ready and how correct it will be.
- Write-Audit-PublishWriting to a staging table, validating it, then swapping it into production.
- Data ContractAn agreed, enforced schema and quality promise between data producers and consumers.
- Data Ownership and StewardshipNamed people accountable for a dataset's quality and access.
- Master Data ManagementKeeping one authoritative version of core entities like customers and products.
- Feature StoreA shared store of ML features, consistent between training and serving.
- Data VersioningTracking versions of datasets the way Git tracks code.
- MentoringHelping less experienced engineers grow.
383 more senior concepts
- CurryingTurning a multi-argument function into a chain of single-argument ones.
- MonadA pattern for chaining computations that carry context, like Option or Promise.
- Referential TransparencyAn expression can be replaced by its value without changing behavior.
- Graceful DegradationLosing non-critical features instead of failing completely.
- Algebraic Data TypesComposing types from "and" (products) and "or" (sums).
- Covariance and ContravarianceHow subtyping of generic types follows their type parameters.
- Structural vs Nominal TypingCompatible by shape vs compatible by declared name.
- Actor ModelConcurrency through isolated actors that communicate by messages.
- BackpressureA slow consumer signalling a fast producer to slow down.
- ChannelA typed pipe for passing values between concurrent tasks.
- Green Threads / GoroutinesLightweight threads scheduled by the runtime rather than the OS.
- LivelockTasks keep reacting to each other without making progress.
- SemaphoreA counter limiting how many tasks can use a resource at once.
- StarvationA task never gets the resources it needs to run.
- Git InternalsBlobs, trees, commits and refs: what Git actually stores.
- Monorepo vs PolyrepoOne repository for everything vs one per project.
- Patch FilesSharing changes as diff files instead of branches.
- WorktreeSeveral working directories checked out from one repository.
- Stacked PRsA chain of dependent pull requests, each reviewable on its own.
- Core DumpA snapshot of a crashed process's memory for later analysis.
- Debugging in ProductionFinding issues with logs, traces and metrics when you can't attach a debugger.
- Flame GraphA visualization of where a program spends its time.
- Contract TestingVerifying that services agree on the API contract between them.
- FuzzingFeeding random inputs to find crashes and vulnerabilities.
- Mutation TestingChanging code on purpose to check that the tests notice.
- Property-Based TestingGenerating many inputs to check that properties always hold.
- Soak TestRunning under load for hours to find leaks and slow degradation.
- Stress TestingPushing beyond capacity to find the breaking point.
- Testing in ProductionValidating safely with real traffic using flags, canaries and monitoring.
- Functional Core, Imperative ShellPure logic in the middle, side effects at the edges.
- Gall's LawComplex systems that work evolved from simple systems that worked.
- Hyrum's LawWith enough users, every observable behavior becomes something someone depends on.
- Keep Framework Code at the EdgesKeeping business logic free of framework details so it survives framework changes.
- Postel's LawBe conservative in what you send and liberal in what you accept.
- Big Bang RewriteReplacing a whole system at once, and why it usually fails.
- Parallel Change (Expand-Contract)Add the new way, migrate callers, then remove the old way.
- SeamA place where you can change behavior without editing the code there.
- Strangler Fig PatternGradually replacing a legacy system by routing pieces to new code.
- C4 ModelDiagramming software at four zoom levels: context, containers, components, code.
- DiátaxisOrganizing docs into tutorials, how-to guides, reference and explanation.
- Balanced TreeTrees like AVL or red-black that stay shallow for guaranteed O(log n).
- Bloom FilterA compact structure answering "definitely not" or "probably yes" for set membership.
- HyperLogLogEstimating the number of distinct items with tiny memory.
- K-D TreeA tree for searching points in multi-dimensional space.
- Merkle TreeA tree of hashes that verifies large data efficiently.
- Persistent Data StructureAn immutable structure that shares unchanged parts between versions.
- Ring BufferA fixed-size buffer that wraps around.
- Segment TreeA tree for fast range queries over arrays.
- Skip ListA layered linked list with O(log n) search.
- Union-FindTracking which elements belong to the same group.
- Bellman-Ford AlgorithmShortest paths that also handles negative edge weights.
- Classic Hard ProblemsKnapsack, traveling salesman and others, and how to recognize them in disguise.
- Consistent HashingHashing that moves only a few keys when servers are added or removed.
- Diff AlgorithmsComputing the minimal set of changes between two sequences.
- Dijkstra's AlgorithmFinding shortest paths in a graph with non-negative weights.
- Huffman CodingCompressing data by giving frequent symbols shorter codes.
- Minimum Spanning TreeConnecting all nodes with the least total edge weight (Prim's, Kruskal's).
- P vs NP / NP-CompleteProblems with no known efficient solution, and why some tasks stay hard.
- String SearchingAlgorithms like KMP and Rabin-Karp for finding substrings.
- Amdahl's LawThe speedup from parallelism is limited by the part that stays sequential.
- Linear Algebra BasicsVectors and matrices, the foundation of graphics and ML.
- Little's LawItems in a system = arrival rate × time each spends in it.
- cgroupsLinux control groups that limit resources; one basis of containers.
- Context SwitchThe CPU switching between threads or processes, and what it costs.
- CPU SchedulerHow the OS decides which thread runs next.
- epoll / kqueueOS mechanisms for watching many sockets efficiently.
- fork and execHow Unix creates new processes by copying, then replacing, the current one.
- InodeThe file system record that holds a file's metadata.
- Inter-Process CommunicationPipes, sockets and shared memory for processes to talk to each other.
- Linux NamespacesIsolating what a process can see; the other basis of containers.
- Memory-Mapped FilesMapping a file directly into a process's memory.
- Paging and SwapMoving memory pages to disk when RAM is full.
- Scheduling AlgorithmsRound robin, priority and fair scheduling for sharing the CPU.
- User Space vs Kernel SpaceThe privilege boundary between programs and the OS.
- Head-of-Line BlockingOne slow item holding up everything queued behind it.
- Network PartitionPart of a network becoming unreachable from the rest.
- Packet Capture (tcpdump, Wireshark)Recording and inspecting raw network traffic.
- QUICA UDP-based transport behind HTTP/3, with faster handshakes.
- RoutingHow packets find their way across networks.
- Conditional RequestIf-None-Match and If-Match for caching and concurrency control.
- HTTP/3HTTP running over QUIC.
- Range RequestsFetching part of a resource, for resumable downloads and video.
- Certificate ChainLeaf, intermediate and root certificates linking to a trusted root.
- TLS HandshakeHow client and server agree on keys before sending data.
- MessagePackA compact binary alternative to JSON.
- REST vs GraphQL vs gRPCChoosing an API style for the clients and teams you have.
- Richardson Maturity ModelLevels of REST maturity, from one endpoint to hypermedia.
- LATERAL JoinA join where the right side can reference columns from the left.
- Pivot / UnpivotTurning rows into columns and back.
- Query PlanThe database's chosen strategy of scans, joins and sorts.
- Relational AlgebraThe operations (select, project, join) that SQL queries compile to.
- Sequential Scan vs Index ScanReading the whole table vs jumping in through an index.
- Covering IndexAn index containing every column a query needs.
- Hash IndexAn index for equality lookups only.
- Index SelectivityHow well an index narrows down rows.
- Partial IndexAn index over only the rows matching a condition.
- Specialized Indexes (GIN, GiST, BRIN)Index types for JSON, full-text search, geometry and huge tables.
- Table StatisticsData distributions the planner uses to choose query plans.
- VACUUM and BloatCleaning up dead rows in PostgreSQL.
Transactions & Concurrency Control
- Database DeadlockTransactions waiting on each other's locks.
- Dirty ReadReading another transaction's uncommitted data.
- Distributed TransactionA transaction that spans several databases or services.
- Isolation LevelsRead uncommitted, read committed, repeatable read and serializable.
- MVCCMulti-version concurrency control, where readers don't block writers.
- Non-Repeatable ReadReading the same row twice and getting different values.
- Phantom ReadRows appearing or vanishing between reads in one transaction.
- Row Locks vs Table LocksHow much a lock blocks.
- Two-Phase CommitA coordinator making all participants either commit or abort.
- Write SkewTwo transactions reading the same data and making conflicting writes.
- Write-Ahead LogLogging changes before applying them, for durability and replication.
- B-Tree vs LSM TreeRead-optimized pages vs write-optimized logs, and when each wins.
- Buffer Pool / Page CacheKeeping hot disk pages in memory.
- CheckpointFlushing in-memory changes to disk so recovery doesn't replay the whole log.
- fsync and DurabilityWhat "written to disk" actually means, and when data can still be lost.
- LSM TreeA write-optimized structure that buffers writes in memory and merges sorted files.
- Snapshot IsolationEach transaction reads from a consistent snapshot of the database.
- SSTables and CompactionSorted immutable files and the background process that merges them.
- Storage EngineThe part of a database that reads and writes data on disk.
- System CatalogThe tables where a database describes its own schema.
- Atomic Scripts (Lua in Redis)Running several operations atomically on the server.
- Designing for Access PatternsModeling NoSQL data around the queries you'll run.
- Faceted SearchFiltering search results by categories with counts.
- Fuzzy SearchMatching despite typos and spelling variations.
- Geospatial DataStoring and querying locations: nearby search, distances, geohashes.
- Graph DatabaseDatabases like Neo4j built around relationships.
- Inverted IndexA map from words to the documents that contain them.
- NewSQL / Distributed SQLSQL databases that scale horizontally, like CockroachDB or Spanner.
- Polyglot PersistenceUsing different databases for different needs in one system.
- Redis Persistence (RDB, AOF)Snapshots vs an append-only log for surviving restarts.
- Redis Sentinel and ClusterFailover and sharding for Redis.
- Relevance Scoring (BM25)Ranking search results by how well they match.
- Single-Table DesignStoring many entity types in one DynamoDB table.
- Time-Series DatabaseDatabases optimized for timestamped measurements.
- Tokenizers and AnalyzersHow search engines split and normalize text before indexing.
- Wide-Column StoreDatabases like Cassandra built for massive write volume.
- Expand-Contract MigrationAdd the new, migrate, then remove the old, across several deploys.
- Migration LocksSchema changes that lock tables and take production down.
- Online Schema Change ToolsTools like gh-ost for altering huge tables safely.
- Zero-Downtime MigrationChanging the schema while the app keeps serving traffic.
- Database ExtensionsAdding capabilities to a database, like PostGIS or pgvector.
- Database High AvailabilityAutomatic failover for databases, with tools like Patroni.
- Last Write WinsResolving conflicting writes by timestamp, and the data it silently loses.
- Leaderless ReplicationAny node accepts writes, with quorums to reconcile, as in Cassandra.
- Point-in-Time RecoveryRestoring a database to any moment using a base backup plus the log.
- Query Statistics (pg_stat_statements)Finding which queries use the most time in aggregate.
- Streaming vs Logical ReplicationCopying the raw log vs copying row changes, and what each lets you do.
- Synchronous vs Asynchronous ReplicationWaiting for replicas to confirm vs not, trading safety for latency.
- Using the Database as a QueueJob queues in SQL with SKIP LOCKED, and when that's enough.
- Cache ConsistencyKeeping cached data in line with the source of truth.
- Cache StampedeMany requests rebuilding the same expired item at once.
- Cache WarmingFilling a cache before traffic arrives.
- Negative CachingCaching "not found" results too.
- Read-Through CacheThe cache itself loads missing data from the database.
- Thundering HerdMany clients waking up and hitting a resource at the same moment.
- Write-Behind CacheWriting to the cache now and to the database later.
- Write-Through CacheWriting to the cache and the database together.
- Consumer OffsetA consumer's position in a log, which it commits as it goes.
- Exchanges and Routing KeysHow brokers like RabbitMQ route a message to the right queues.
- Fan-OutSending one message to many consumers.
- Inbox PatternRecording received messages so duplicates can be ignored.
- JitterRandomizing retry delays so clients don't all retry in sync.
- Message OrderingWhen and how messages arrive in order.
- Message TTL and ExpiryDropping or dead-lettering messages that are too old to matter.
- Poison MessageA message that crashes every consumer that tries to process it.
- Push vs Pull ConsumersThe broker delivering messages vs consumers fetching them at their own pace.
- Transactional OutboxSaving events in the same database transaction, then publishing them reliably.
- Workflow EngineDurable orchestration of long multi-step processes, like Temporal.
- Ledger and Double-Entry BookkeepingRecording money movements as balanced, append-only entries.
- Usage MeteringCounting billable usage accurately, without double-counting.
- Event-Driven ArchitectureServices communicating by publishing and reacting to events.
- Quality Attributes (the "-ilities")Scalability, availability, maintainability and other non-functional goals.
- Distributed ID GenerationGenerating unique IDs across machines, e.g. UUIDv7 or Snowflake IDs.
- Replication LagFollowers falling behind the leader.
- Choreography vs OrchestrationServices reacting to events vs a central coordinator directing them.
- Clock SkewMachines' clocks disagreeing, which breaks time-based ordering.
- ConsensusGetting nodes to agree on a value despite failures.
- Consistency ModelsLinearizable, sequential, causal, eventual: what readers are allowed to see.
- Data Consistency Across ServicesKeeping data correct when each service owns its own database.
- Database per ServiceEach microservice owning its data exclusively.
- Distributed LockMutual exclusion across machines, and its pitfalls.
- Distributed SystemMany machines cooperating over an unreliable network.
- Fallacies of Distributed ComputingEight false assumptions, starting with "the network is reliable".
- Gossip ProtocolNodes spreading information by talking to random peers.
- HeartbeatPeriodic "I'm alive" signals used to detect failed nodes.
- Idempotency in Distributed SystemsMaking retries safe when you can't know whether the first attempt worked.
- Leader ElectionChoosing one node to coordinate the others.
- LeaseA lock or role that expires automatically unless renewed.
- PACELCExtends CAP: even without partitions, you trade latency against consistency.
- QuorumRequiring a majority of nodes to agree on reads or writes.
- RaftAn understandable consensus algorithm, used in etcd.
- Request CoalescingCollapsing identical concurrent requests into one.
- Saga PatternA chain of local transactions with compensating actions instead of a distributed transaction.
- Service DiscoveryHow services find each other's addresses.
- Service MeshInfrastructure handling service-to-service traffic, retries and mTLS.
- SidecarA helper process deployed alongside each service.
- Split BrainTwo nodes both believing they're the leader.
- Strong ConsistencyEvery read sees the latest write.
- Two Generals' ProblemWhy agreement over an unreliable link can't be guaranteed.
- Active-Active vs Active-PassiveAll nodes serving traffic vs standbys waiting.
- Blast RadiusHow much breaks when something fails.
- BulkheadIsolating resources so one failure doesn't sink everything.
- Cascading FailureOne failure triggering failures in the systems that depend on it.
- Circuit BreakerStopping calls to a failing service so it can recover.
- Compensating TransactionUndoing a completed step with a new action, since you can't roll it back.
- Deadline PropagationPassing the remaining time budget down a chain of calls.
- Degraded ModesServing reduced functionality when dependencies fail.
- Disaster RecoveryRestoring service after a catastrophic failure.
- FailoverSwitching to a standby when the primary fails.
- Fault ToleranceContinuing to work when components fail.
- Load SheddingRejecting some requests on purpose to protect the rest.
- Retry StormRetries multiplying the load on an already failing system.
- RPO and RTOHow much data you can afford to lose, and how long you can be down.
- Self-Healing SystemsAutomatically restarting and replacing failed parts.
- Thread Pool ExhaustionAll workers stuck waiting on something slow, so nothing else gets served.
- Capacity PlanningPredicting and providing the resources needed for future load.
- Continuous ProfilingProfiling live services to find hot paths.
- PrecomputationComputing results ahead of time.
- Tail LatencyThe slowest requests (p99), which users notice most.
- Event BusA channel through which events reach their subscribers.
- Event vs Command"This happened" vs "please do this".
- End-to-End EncryptionOnly the communicating users can read the data, not the server.
- Envelope EncryptionEncrypting data with a data key, then encrypting that key with a master key.
- Key Management (KMS)Generating, storing, rotating and controlling access to keys.
- Bug BountyPaying outside researchers to report vulnerabilities.
- Container SecurityMinimal images, non-root users and image scanning.
- CVSSA score rating how severe a vulnerability is.
- DASTTesting a running app for vulnerabilities from the outside.
- Penetration TestingAuthorized simulated attacks to find vulnerabilities.
- Responsible DisclosureReporting vulnerabilities privately before going public.
- SBOMA software bill of materials listing every component.
- Secret RotationRegularly replacing credentials.
- Security ReviewReviewing a design or change specifically for security risks.
- Shift-Left SecurityFinding security issues early in development.
- Software Supply Chain SecurityProtecting against compromised dependencies and build pipelines.
- STRIDEA way to categorize threats: spoofing, tampering, repudiation, disclosure, denial of service, elevation.
- Threat ModelingSystematically asking what could go wrong and how to prevent it.
- CCPACalifornia's consumer privacy law.
- HIPAAUS rules for protecting health information.
- PCI DSSRules for handling payment card data.
- Privacy by DesignBuilding privacy in from the start.
- Right to ErasureDeleting a user's data on request, across every system.
- HypervisorSoftware that runs virtual machines.
- iptables / nftablesLinux's built-in firewall.
- Linux Boot ProcessFirmware, bootloader, kernel, init: how a machine starts.
- LVMFlexible logical volumes on top of physical disks.
- Process Priority (nice)Telling the scheduler which processes matter more.
- SELinux and AppArmorMandatory access control that restricts what processes can do.
- Server HardeningReducing a server's attack surface.
- straceWatching the system calls a process makes.
- Minimal Base ImagesAlpine, distroless and scratch images for a smaller attack surface.
- OCIThe open standards for container images and runtimes.
- Union FilesystemStacking read-only image layers with a writable layer on top.
- Control PlaneThe components that manage cluster state.
- DaemonSetRunning one pod on every node.
- Horizontal Pod AutoscalerAdding pods automatically based on load.
- Managed Kubernetes (EKS, GKE, AKS)Kubernetes with a cloud-run control plane.
- Network PoliciesFirewall rules between pods.
- Node Affinity and Topology SpreadControlling which nodes pods run on and how they spread out.
- Persistent VolumeStorage that survives pod restarts.
- Pod Disruption BudgetLimiting how many pods can be down during maintenance.
- Pod Priority and PreemptionEvicting less important pods to make room for important ones.
- Requests and LimitsHow much CPU and memory a pod reserves and is allowed to use.
- Resource QuotasCapping what a namespace can use.
- StatefulSetRunning stateful apps with stable identity and storage.
- Taints and TolerationsKeeping pods off nodes unless they explicitly allow it.
- Vertical Pod AutoscalerAdjusting a pod's CPU and memory requests automatically.
- Cloud Cost Management (FinOps)Understanding and controlling cloud spending.
- Edge ComputingRunning code near users at CDN locations.
- Egress CostsPaying for data leaving the cloud.
- External Secrets in KubernetesSyncing secrets from a vault into the cluster safely.
- Managed vs Self-HostedPaying for convenience vs running it yourself.
- Shared Responsibility ModelWhich security duties belong to the provider and which are yours.
- Storage Lifecycle PoliciesAutomatically moving or deleting old objects.
- Vendor Lock-InDepending on one provider's proprietary services.
- Configuration DriftReal infrastructure diverging from its definition.
- GitOpsGit as the source of truth, with agents syncing the cluster to it.
- Immutable InfrastructureReplacing servers instead of changing them.
- Pulumi and CDKInfrastructure defined in general-purpose programming languages.
- Terraform StateThe record of what infrastructure exists, and why it must be protected.
- Dark LaunchRunning new code in production without users seeing it.
- Deploying Schema and Code TogetherOrdering migrations and code changes so neither breaks the other.
- Deployment FrequencyHow often a team ships; a key DevOps metric.
- Progressive DeliveryGradual rollouts with automatic checks at each step.
- Release TrainShipping on a fixed schedule with whatever is ready.
- Actionable AlertsAlerting only on symptoms that need a human to act.
- CardinalityThe number of unique label combinations, and why high cardinality gets expensive.
- Counter, Gauge, HistogramThe basic metric types.
- Distributed TracingFollowing a single request across many services.
- OpenTelemetryThe vendor-neutral standard for collecting telemetry.
- RED MethodRate, errors and duration for request-driven services.
- SamplingRecording only a fraction of traces or logs to control cost.
- SpanOne timed operation within a trace.
- USE MethodUtilization, saturation and errors for resources.
- Customer Communication in IncidentsTelling users what's happening, honestly and promptly.
- Error BudgetHow much unreliability an SLO allows, used to balance speed and stability.
- Incident CommanderThe person coordinating an incident response.
- MTTR / MTTDMean time to recover, and to detect.
- Runbook AutomationTurning manual runbook steps into scripts.
- Site Reliability EngineeringApplying software engineering to operations.
- SLIA service level indicator: a measured aspect of service quality.
- SLOA service level objective: the target for an SLI.
- ToilManual, repetitive operational work that should be automated.
- Break-Glass AccessEmergency elevated access that is logged, time-limited and reviewed afterwards.
- Data Engineering UndercurrentsSecurity, data management, DataOps, architecture, orchestration and software engineering across every stage.
- Identity ResolutionStitching anonymous visitors, devices and accounts into one person.
- IoT and Sensor DataHigh-frequency readings from devices, with gaps, clock drift and duplicates.
- Telemetry PipelineThe path from emitted logs, metrics and events to where they're stored and queried.
- Apache ArrowAn in-memory columnar format for moving data between tools without conversion.
- BucketingHashing rows into a fixed number of files by key to speed joins.
- Data SharingGiving another team or company live access to data without copying it.
- ORCA columnar format common in the Hadoop ecosystem.
- Zero-Copy CloneCopying a table instantly by sharing its underlying files.
- Bridge TableHandling many-to-many relationships in a dimensional model.
- Degenerate DimensionA dimension attribute, like an order number, stored directly in the fact table.
- Entity-Centric ModelingTables built around business entities with their full history and features.
- Factless Fact TableA fact table recording that something happened, with no measures.
- Junk DimensionGrouping small flags and indicators into one dimension.
- Role-Playing DimensionOne dimension used several ways, like a date as order date and ship date.
- Transaction, Snapshot and Accumulating FactsThree kinds of fact table for events, periodic states and processes.
Batch & Distributed Processing
- Cluster Resource Manager (YARN, Kubernetes)Allocating CPU and memory to jobs on a shared cluster.
- Query FederationOne query reading from several different systems.
- Spill to DiskRunning out of memory and writing intermediate data to disk.
- Apache FlinkA stream-first processing engine with strong event-time support.
- Stream JoinsJoining streams with each other or with tables, within time bounds.
Transformation & Analytics SQL
- Approximate AggregatesFast, nearly exact counts and percentiles on huge data.
- Data Circuit BreakerStopping downstream jobs when upstream data fails checks.
- TokenizationReplacing sensitive values with tokens that map back only through a secure vault.
- Embedded AnalyticsShowing charts and reports inside your product to customers.
- OLAP CubePre-aggregated data for fast slicing by dimensions.
- Classification vs RegressionPredicting categories vs predicting numbers.
- FeatureAn input variable a model uses.
- Model ServingDeploying a model behind an API.
- OverfittingA model memorizing its training data instead of generalizing.
- Recommendation SystemSuggesting items based on behavior and similarity.
- Train / Test SplitHolding out data to evaluate a model honestly.
- Unsupervised LearningFinding structure in unlabeled data.
- Choosing a ModelTrading off quality, speed and cost across model sizes and providers.
- ChunkingSplitting documents into pieces for embedding and retrieval.
- EvalsSystematically measuring the quality of LLM output.
- Fine-TuningFurther training a model on your own examples.
- GuardrailsChecks on model inputs and outputs for safety and correctness.
- LLM Cost and LatencyManaging tokens, model choice and caching.
- LLM-as-JudgeUsing one model to grade another model's output.
- Prompt CachingReusing processed prompt prefixes to save cost and time.
- RerankingRe-ordering retrieved results with a stronger model.
- Cycle Time and Lead TimeHow long work takes from start, or from request, until it's done.
- Lean Software DevelopmentEliminating waste and optimizing the flow of work.
- Shape UpBasecamp's six-week cycles with shaped, fixed-time bets.
- Cone of UncertaintyEstimates get more accurate as work progresses.
- Critical PathThe chain of tasks that determines the earliest finish date.
- Cross-Team DependenciesWork blocked on other teams, and how to manage it.
- OKRsObjectives and key results for setting goals.
- Project KickoffAligning everyone on goals, scope and roles at the start.
- RICE and MoSCoWFrameworks for prioritizing work.
- RoadmapA plan of what to build over the coming months.
- Disagree and CommitVoicing disagreement, then fully backing the decision.
- PresentingCommunicating ideas to a group.
- Psychological SafetyFeeling safe to take risks and admit mistakes on a team.
- Resolving ConflictWorking through disagreements productively.
- Stakeholder ManagementKeeping the people affected by your work informed and aligned.
- Build vs BuyDeciding whether to build something or use an existing product.
- Business MetricsRevenue, retention, conversion: what the business measures.
- Experimentation CultureTesting ideas with data before committing to them.
- Feature AdoptionWhether users actually use what you shipped.
- User ResearchLearning what users actually need.
- Generalist vs SpecialistThe trade-offs between breadth and depth.
- Glue WorkEssential but invisible work that keeps teams running.
- Handling AmbiguityMaking progress when the problem isn't well defined.
- IC vs Management TrackTwo parallel career paths, and how to choose between them.
- ScopeHow big and ambiguous the problems you own are.
- Senior EngineerOwning systems and raising the team around you.
- Tech LeadLeading a team's technical direction while still writing code.
- Writing and Speaking PubliclyBlogs, talks and sharing what you learn.
- Boring Technology / Innovation TokensSpending novelty sparingly and choosing proven tools by default.
- DelegationHanding off work in a way that grows others.
- Developer ExperienceHow easy and pleasant it is for engineers to do their work.
- HiringInterviewing and selecting engineers.
- Interviewing CandidatesRunning fair interviews that produce real signal.
- Managing Technical DebtTracking, prioritizing and paying down debt deliberately.
- Managing UpKeeping your manager informed and aligned.
- Onboarding New EngineersHelping new teammates become productive.
- Technical Decision MakingGathering input, weighing trade-offs, deciding, and recording why.
- Technical LeadershipGuiding technical direction through influence and example.
Staff
Shape how the whole organization produces and uses data.
- Conway's LawSystems mirror the communication structure of the organizations that build them.
- RFCA request for comments: proposing a significant change for wide review.
- Multi-Region ArchitectureRunning in several regions for lower latency and resilience.
- Chaos EngineeringInjecting failures on purpose to find weaknesses.
- Performance vs CostDeciding how much speed is worth paying for.
- Enterprise Integration PatternsA catalog of messaging patterns: routers, translators, aggregators.
- Security ChampionAn engineer on each team who advocates for security.
- Compliance as CodeAutomated checks that systems meet policies.
- Data ResidencyRequirements that data stays within certain countries.
- SOC 2An audit of how a company protects customer data.
- Custom Resource DefinitionTeaching Kubernetes new kinds of objects.
- Do You Need Kubernetes?Weighing its power against its operational cost.
- Multi-Cluster ManagementRunning workloads across several Kubernetes clusters.
- OperatorA custom controller that automates running complex software.
- Multi-CloudUsing several cloud providers, and whether it's worth it.
- DORA MetricsFour delivery metrics: deploy frequency, lead time, change failure rate and recovery time.
- Wide EventsRich, high-cardinality events instead of pre-aggregated metrics.
- Game DayPracticing incident response with simulated failures.
- Production Readiness ReviewA checklist before a service goes live.
- Bus MatrixA planning grid of business processes and the dimensions they share.
- Data FabricAn integration layer that connects data across systems using metadata.
- Data MeshDomain teams owning their data as products, on a self-serve platform.
- EU AI ActEU rules on AI systems and the data used to build them.
- Build vs Buy for Data ToolsChoosing between managed services, open source and building in-house.
- Data Platform TeamA team that builds infrastructure other data teams depend on.
- Self-Serve Data PlatformShared tooling that lets many teams build their own pipelines safely.
- Influence Without AuthorityGetting things done across teams you don't manage.
- Writing CultureTeams that make and record decisions through written documents.
- Cost of DelayWhat it costs to ship something later.
- North Star MetricThe single metric that best captures the value you deliver.
- Engineering ManagerLeading people: hiring, growth, delivery and team health.
- Staff ArchetypesTech lead, architect, solver and right hand.
- Staff EngineerTechnical leadership across teams.
- Building ConsensusBringing people with different views to an agreement.
- Communicating with ExecutivesShort, decision-focused updates for leadership.
- Developer Productivity MetricsSPACE, DORA, and the danger of measuring the wrong thing.
- Engineering CultureThe shared values and habits that shape how a team builds.
- Evaluating New TechnologySeparating hype from what's worth adopting.
- Leading Large MigrationsMoving many teams off an old system.
- Platform ThinkingBuilding shared capabilities that make other teams faster.
- Risk ManagementIdentifying and mitigating what could derail a project.
- Setting Engineering StandardsDefining practices that many teams follow.
- SponsorshipUsing your influence to create opportunities for others.
- Team Cognitive LoadLimiting how much a team must understand to own its systems.
- Team TopologiesStream-aligned, platform, enabling and complicated-subsystem teams.
- Technical VisionA picture of where the technology should be in a few years.
Principal
Set data strategy and architecture across the company.
- Lock-Free Data StructuresConcurrent structures built from atomic operations instead of locks.
- Memory ModelThe rules for when writes by one thread become visible to others.
- Suffix ArrayA sorted array of a string's suffixes for fast substring search.
- Maximum FlowFinding the most that can flow through a network (Ford-Fulkerson).
- Pages, Tuples and File LayoutHow rows are physically laid out on disk.
- Two-Phase LockingAcquire all locks, then release them: the classic way to get serializability.
- Write AmplificationOne logical write turning into many physical ones.
- Byzantine FaultNodes that behave arbitrarily or maliciously.
- Fencing TokenA counter that stops stale lock holders from writing.
- Hedged RequestsSending a duplicate request when the first is slow, to cut tail latency.
- LinearizabilityOperations appear to happen instantly, in real-time order.
- Logical ClocksLamport and vector clocks for ordering events without real time.
- PaxosThe classic, notoriously hard-to-understand consensus algorithm.
- Total Order BroadcastDelivering the same messages in the same order to every node.
- Universal Scalability LawWhy adding nodes eventually makes a system slower.
- eBPFRunning safe, sandboxed programs inside the Linux kernel for tracing and networking.
- Principal EngineerTechnical direction for a whole organization.
- Budget and HeadcountPlanning people and spending.
- Inverse Conway ManeuverDesigning team structure to get the architecture you want.
- Long-Term ThinkingMaking decisions whose payoff comes years later.
- Organization DesignStructuring teams and reporting lines.
- Technical Due DiligenceAssessing another company's technology, as in an acquisition.
- Technical StrategyA plan for reaching the vision under real constraints.