GPT-2
Articles tagged GPT-2.
Machine Learning Byte Pair Encoding: Explained How Byte Pair Encoding turns text into tokens, how vocabulary size and merge rules affect compression, and the traps to avoid when implementing BPE. Machine Learning Choosing the Right Metrics: How to Actually Know if Your Model is Any Good A guide to choosing evaluation metrics for classification, regression, and language models, and why accuracy alone can hide a failing model. Essays Diving Deep: My First Public Project - Training a GPT-2 Level LLM from Scratch! A from-scratch account of training a GPT-2-scale language model: the architecture, data pipeline, compute constraints, and lessons from my first pu...