Transformers
Articles tagged Transformers.
Machine Learning A Deep Dive into Attention Mechanisms: The Secret Ingredient of Modern NLP A practical tour of attention mechanisms, from Bahdanau soft attention and hard attention to self-attention and multi-head transformers. Essays Diving Deep: My First Public Project - Training a GPT-2 Level LLM from Scratch! A from-scratch account of training a GPT-2-scale language model: the architecture, data pipeline, compute constraints, and lessons from my first pu...