Back to Module 1.9: Self-AttentionIn Progress
AI Lesson & Submodule
Attention Weights
Visualize token attention relationships using heatmaps.
Why This Matters
Attention heatmaps show exactly how much weight the model allocates to surrounding context terms.
What You Will Learn
- •Render attention heatmaps
- •Read attention weights
- •Identify semantic ties
Concepts Covered
Attention weight arraysAlignment maps visualizersSemantic connection coordinates
Mapped Foundation Project: Mini Attention Notebook
Self-contained interactive workbook computing step-by-step scaled dot-product attention scores from raw matrices input.
Architecture Preview
Dynamic math workbook running forward projections, scaling matrix outputs, and plotting soft heatmaps.
Character Tokens ArrayQKV Projections WeightsQK Dot Product Score
Tech Stack Planned
ReactTypeScriptMath.js
Technical Interview Value
- ?Explain how to interpret an attention matrix heatmap displaying pronoun coreference ties