<aside> 📖
Date: Sunday, October 4, 2026 · Week 1, Day 7 · Type: Read Day (~30 min) · Paper: Attention Is All You Need (Vaswani et al., 2017)
</aside>
7.1 The Transformer Revolution — understanding why the 2017 paper Attention Is All You Need replaced recurrent networks (RNNs).
7.2 Self-Attention Intuition — how words in a sentence "look at" and update their meaning based on surrounding words.
Read Sections 1, 2, and 3 of Attention Is All You Need, and write a 1-page plain-English summary of how Multi-Head Attention works using a real-world analogy.
[Input Text Tokens] ---> [Linear Embeddings] ---> [Self-Attention: Q, K, V] ---> [Contextualized Representations]
Problem 07. Write a Python function manual_attention_score(query: list[float], key: list[float], dimension: int) -> float that computes the scaled dot product (Q·K)/√d for two small 4-element vectors.
Data Structure: 2D Attention Weight Matrix.
Mechanics: Square N×N matrix tracking the pairwise attention score between all N tokens in a sequence.
Real-World Production Use: Computed in every Transformer block across all modern LLMs (GPT-4, Claude, Llama).
Pattern: Template Method Pattern.
AI Context: Defining the sequential structure of a Transformer layer forward pass (Attention → Add & Norm → Feed Forward → Add & Norm).