The backstory
In 2015, Andrej Karpathy published The Unreasonable Effectiveness of Recurrent Neural Networks. He also released min-char-rnn.py, a character-level vanilla RNN in ~112 lines of pure Python + NumPy. No framework, no magic, just the math, laid bare.
That code changed everything for me. In 2016 I wrote a line-by-line notebook explaining every single line of it, as a way to truly understand how a neural network learns. Thanks to this, I learned how deeplearning was working.
Fast forward to 2025: Karpathy does it again, this time with a Transformer. His microgpt is a complete GPT language model in ~226 lines of PyTorch. Same spirit as the RNN gist: all the core ideas (tokenization, embeddings, self-attention, layer norm, the training loop) are in one readable file.
I wanted to do the same thing I did in 2016: explain the code, line by line, until it fully makes sense. But this time AI is there, so there must be a twist!
What microgpt.dev does
microgpt.dev is an interactive companion site for Karpathy's
gpt.py. You can:- Read the full source with syntax highlighting (hand-rolled, no heavyweight library)
- Select any snippet and tap "Explain ✨" to get a scoped AI explanation
- Ask questions in a chat panel focused strictly on the code and LLM concepts
The AI answers are powered by Cloudflare Workers AI (Llama 3.2 3B), streamed in real-time. The system prompt enforces scope: it only talks about the microgpt code and how LLMs work. No general-purpose chatbot drift.
The landing page includes a visual walkthrough of how LLMs learn (the predict → loss → backprop → update loop) and an FAQ covering every technical term in the code — from tensors and tokens to dropout and Adam.
How is was created:
Fully done by an agent, including buying the domain name, configure the DNS, etc…
- Next.js 15 (App Router) on Cloudflare Pages
- React 18, TypeScript, Tailwind CSS (custom GitHub-dark palette)
- Cloudflare Workers AI (
@cf/meta/llama-3.2-3b-instruct, edge streaming)
- Vitest + Testing Library for tests
- Hand-rolled Python tokenizer for syntax highlighting
- Source auto-fetched from Karpathy's repo via
scripts/fetch-microgpt.ts