Writing
I share what I'm learning, with visuals wherever I can.
KL Divergence — Made Visual
An interactive visual guide to KL divergence: distributions, the formula, live computation, and how it shows up in LLMs and PPO training.
Read the postMoE Routing Calculation (Excel Walkthrough)
Step-by-step Excel breakdown of router logits → top-k selection → normalized expert probabilities. Includes top-k masking, −∞ replacement, softmax, and final expert routing weights.
Read the postBuilding MakeMyDocsBot
Automated multi-language documentation sync across feature branches.
Read the postKV (Key-Value) Cache in Transformers
Reducing inference latency using KV cache.
Read the postHow Does Temperature Change LLM Responses?
Effect of temperature on next-token probability distribution.
Read the postMasked Self-Attention
How masking enforces autoregressive generation in decoder-only transformers.
Read the postSelf-Attention in Transformers
How queries, keys, and values compute attention weights and why it matters.
Read the postTraining the Tokenizer
How tokenizers are trained and why the vocabulary choice shapes model behavior.
Read the postNo posts match this filter.