https://rezag.io/tags/pytorch/ Recent content in Pytorch on Reza Ghafari Hugo en-us Fri, 29 May 2026 00:00:00 +0000 https://rezag.io/posts/2026/self-attention-from-scratch/ Fri, 29 May 2026 00:00:00 +0000 https://rezag.io/posts/2026/self-attention-from-scratch/ I’m writing this post mostly to make sure this stuff actually sticks in my head. Step-by-step explanations of self-attention with working examples are hard to find. If you have used ChatGPT, Claude, or any modern language model, you have interacted with a Transformer. The Transformer is the architecture behind virtually every large language model (LLM) today. It was introduced in the legendary 2017 paper “Attention Is All You Need” by Vaswani et al.