What this lesson covers.
This module is one of 42 in the curriculum. Below is the canonical interactive lesson — tabs, cards, and diagrams from the source repo, rendered inside the course shell. An audio narration runs alongside it - the sticky player at the top of the page plays the full Module 19 clip.
If you prefer to read first and play with the demos after, the interactive lesson sits below this section. If you'd rather hear it narrated while you scroll, hit play on the sticky audio bar at the top — or just let it autoplay.
The lesson itself.
Inspector vs. Storyteller
Transformers process sequences differently based on their architecture. Encoders (like BERT) look at the entire sentence at once in both directions, making them great at analyzing text. Decoders (like GPT) are causally masked, meaning they can only see the past and must generate the future one word at a time, making them master storytellers.
The Inspector (Encoder)
Bidirectional Attention. Reads left-to-right AND right-to-left simultaneously.
The Storyteller (Decoder)
Autoregressive Causally Masked Attention. Only sees the past.
Further reading.
The canonical references for this module. External links open in a new tab.
What to read next.
Use the pager below to move sequentially through the curriculum, or jump to any module from the course index. Each track has a "Prereq: ↑ foundation" callout so you can backfill anything that wasn't clear.