From 0.76 to 0.78: A 48-Hour Hackathon Journey in Data Science
How a 48-hour team sprint on the PISA math-score challenge moved from R² 0.76 to 0.78, and what hardware limits and CatBoost tuning taught us.
Notes from
the Wired
Working notes on the mathematics of learning, the systems that carry it, and the gap between proof and production.
“All models are wrong, but some are useful.”
George E. P. Box
- Articles
- 9
- Subjects
- 3
- Local time
Explanations and working notes on mathematics, machine learning, and applied systems.
All archives Read
The Causal Inference Series : Part I
Part I of a causal inference series: Simpson's paradox, Pearl's ladder of causation, confounding, selection bias, and why correlation is not enough.
Read
CatBoost : Hype or Not ?
A deep dive into CatBoost's ordered target statistics, oblivious trees, and feature combinations, and why they beat XGBoost on categorical data.
Read
Byte Pair Encoding: Explained
How Byte Pair Encoding turns text into tokens, how vocabulary size and merge rules affect compression, and the traps to avoid when implementing BPE.
Read
Neural Tangent Kernel: A theoretical view of Deep Learning Theory
A theoretical overview of the Neural Tangent Kernel and what it reveals about training dynamics and the behavior of wide neural networks.
Read
Demystifying Linear Regression: From Basic Equations to Loss Functions
Linear regression from first principles: the model, its assumptions, and how squared error, absolute error, Huber loss, and likelihood shape what it learns.
Read
A Deep Dive into Attention Mechanisms: The Secret Ingredient of Modern NLP
A practical tour of attention mechanisms, from Bahdanau soft attention and hard attention to self-attention and multi-head transformers.
Read
Choosing the Right Metrics: How to Actually Know if Your Model is Any Good
A guide to choosing evaluation metrics for classification, regression, and language models, and why accuracy alone can hide a failing model.
Read
Diving Deep: My First Public Project - Training a GPT-2 Level LLM from Scratch!
A from-scratch account of training a GPT-2-scale language model: the architecture, data pipeline, compute constraints, and lessons from my first public project.
Read
The ice, the sea, and the network share one horizon.
Tools, research projects, and applied systems built across data and ML.
All projects
LearnFlow (opens in a new tab)
A learning platform in development, with interactive resources and AI-supported guidance.
- Svelte
- SQL
- PostgreSQL
- +2
DementiaNet (opens in a new tab)
A research project that studies dementia detection in interviews through linguistic and audio features.
- NLP
- Audio Signal Analysis
- LLMs
- +5
HiParis 2025 - PISA Math Score Prediction (opens in a new tab)
A 48-hour hackathon project predicting PISA Math Scores from 1.17M student records. Features a two-stage architecture (classifier + regressor) using CatBoost, achieving R² = 0.78 and 3rd place. Explored kernel methods, ensemble strategies, and tackled compute limitations.
- CatBoost
- Python
- Scikit-learn
- +3