Notes
I've found I retain things far better when I write them up myself. I've been documenting my learning notes here since 2026, to share ideas I find interesting and to keep a personal reference I can return to when my understanding gets rusty. Inspired by Lil'Log.
- Knowledge Distillation for Language Models Updated Oct 6, 2026
- RL Environments Updated Oct 5, 2026
- Reinforcement Learning for Large Language Models Updated Sep 26, 2026
- Negative Log-Likelihood, Entropy, Cross-Entropy, and KL Divergence Updated Sep 24, 2026
- LLM Pre-training Updated Sep 24, 2026
- Scaling RL Updated Sep 13, 2026
- Actor-Critic Methods Updated Sep 12, 2026
- Instruction and Supervised Finetuning Updated Aug 31, 2026
- Policy Gradients Updated Aug 30, 2026
Up next
- LLM Post-trainingPerplexityMiMo technical report (PDF)
- vLLM
- Continual learningnrehiew
- Reward Model - PRM
- Efficient LLMs - PEFT, LoRA, QLoRAThinking MachinesTensortonic: LoRATensortonic: QLoRA
- Scaling book exercises
- Prefill decode, KV Cache, Paged Attention, Radix Attention, speculative decodingTensortonic: KV cacheTensortonic: speculative decoding
- Distributed training - FSDP, tensor parallelism, pipeline parallelismCS336 Lecture 7CS336 Lecture 8 (PDF)
- Attention alternatives and mixture of expertsCS336 Lecture 4 (PDF)