GPU + ML Expert Tutor

The promise

Two altitudes, every single time

Every module explains the idea twice: a plain-language analogy that a non-programmer can follow, then the rigorous framing a Principal Engineer would expect.

A GPU is a stadium of 50,000 diligent students, not a faster CPU. That gets you in the door. Then we map it onto wavefronts, occupancy, and HBM bandwidth.

What you get

30

modules across four tracks — roughly 203 learner-hours if you do the labs and the drills, not just the reading. Every module ships runnable code, exercises, and worked solutions.

Consistency

Every module, the same nine sections

1–2

Layman analogy, then first principles — derived, not asserted

3

Deep dive mapped onto real silicon (CDNA and Hopper)

4–5

Hands-on labs, then performance analysis against the roofline

6

Challenges, drawbacks, tradeoffs — and when not to do this

7–8

Where it appears in production ML, with cited sources

9

Self-assessment and interview drills, with solutions

It is a quality contract, not bureaucracy: a technique with no stated downside means nobody looked hard enough.

Portability

Dual-vendor, and honest about the gaps

Labs ship as hip/ and cuda/ line-for-line counterparts, plus a triton/ path that runs on both.

• A lockstep bundle is 64 lanes on AMD, 32 on NVIDIA — so block sizes that are multiples of 64 are the portable choice.
• NVIDIA-only material (CuTe, CUTLASS, cuTile) is marked optional and names its AMD counterparts.

Module A01 · Foundations

0.06%

of the run was the kernel. A profiled vector-add took 44 microseconds inside a 72 millisecond program. On a GPU, moving the data usually costs more than doing the math — and if you only measure kernel time you will optimize the wrong thing for years.

Module A02 · Memory coalescing

4.8×

faster, same arithmetic. Four transpose kernels, all producing verified-correct output: 700 GB/s naive to 3,400 GB/s tiled-and-padded. One of those steps is a single character — tile[32][32] to tile[32][33].

Module B04 · Precision

"fp8" is two different formats

CDNA 3 implements the FNUZ variants (bias 8, max 240). Hopper implements OCP (bias 7, max 448).

• Bit pattern 0_0111_000 is 1.0 under OCP and 0.5 under FNUZ — load the wrong checkpoint and every weight is 2× too large.
• 0x80 means NaN under FNUZ but -0.0 under OCP, so a genuinely-NaN element silently becomes a harmless zero.

Module B07 · Sharding

Pipelining does not reduce activation memory

Under 1F1B scheduling, each stage holds L/S layers but keeps S micro-batches in flight. The S cancels.

Engineers reach for pipeline parallelism as a memory fix. It is a throughput fix. Deriving why is worth more than a page of guidance.

Module C07 · Capstone

The KV cache was not the binding constraint

Sizing an LLM serving node: HBM allows a 3.56 million token KV pool, but the 30 ms latency SLO caps the scheduler at 567 thousand. Latency binds first.

Quantizing to fp8 then flips decode to compute-bound — so further KV quantization buys nothing. Knowing where an optimization stops paying is the harder judgement.

Start here

Pick a path, then do the labs

Plan

Study planner — tick modules off, watch the hours drop

Test

Recall cards — retrieval practice, every card cited

Look up

Reference sheets — track and concept cheatsheets, printable

Play

Concept players — twelve interactive controls, one per concept cluster

Index

CURRICULUM.md — dependency graph and learning paths

Setup

docs/SETUP.md — get hipcc, nvcc, or Triton working

First

Module A01 — assumes no GPU experience

Reading teaches you the shape of the answer. Running the labs on your own GPU is what makes it yours — so re-measure everything here and quote your own numbers.