<aside> ๐Ÿ“…

Date: Tuesday, October 20, 2026 ยท Week 4, Day 23 ยท Type: Build Day

</aside>

Learning Manual

23.1 The VRAM Footprint โ€” calculating how much GPU memory an LLM needs based on parameter count and precision (FP16, INT8, INT4).

23.2 The KV Cache Formula โ€” why conversation context consumes additional GPU VRAM as users chat.

23.3 The Sizing Formula โ€” VRAM(Weights) = Parameters(Billions) ร— Bits / 8 ร— 1.2 (adds 20% system buffer).

The Daily Build

Build a Python CLI calculator tool that takes a model parameter count (e.g., 8B, 70B), a precision level (FP16, INT8, INT4), and a context length, and prints an exact hardware sizing report stating which GPUs (e.g., RTX 4090, A100 80GB) can run it.

Architecture Diagram

[User Inputs: Model Size + Precision] ---> [VRAM Calculation Logic] ---> [GPU Hardware Matcher]
                                                                                |
                                                                                v
                                                                    [Recommended GPU Setup]

Interview Questions

  1. How much GPU memory (VRAM) do you need to load a 70-billion-parameter model at 4-bit (INT4) quantization?
  2. What is the difference between compute-bound operations (prefill) and memory-bandwidth-bound operations (decoding) during LLM inference?
  3. What is PagedAttention, and how does it prevent GPU memory waste when serving multiple concurrent users?

Problem Statement

Problem 23. Write a Python function calculate_vram_requirement(params_in_billions: float, precision_bits: int, context_tokens: int) -> dict that calculates and returns the base model memory, estimated KV cache memory, and total minimum VRAM in gigabytes.

AI-DSA Primitive

Data Structure: Memory Allocation Table (Bit-Precision Map).

Mechanics: Mathematical mapping converting parameter counts and bit-widths into bytes (1 byte = 8 bits).

Real-World Production Use: Deployed inside inference deployment engines (vLLM, Ollama, Hugging Face Accelerate) to verify GPU fit before loading weights.

Software Design Pattern