This is the course. Mica and AI states what the language offers machine learning; this page is where you learn it.
If you want to understand how machine learning works, you are in the right place. This course assumes no mathematics beyond school arithmetic and no machine learning at all. It assumes you can compile and run a Mica program, which Get started teaches in fifteen minutes, and it teaches everything else itself, in the order the ideas build on each other: first what learning is, then how a machine measures its own mistake, then how blame flows backward through a computation, then the shapes a language model is made of, then the model itself.
Mica is the tool. The ideas are the subject. The tool changes one thing about
the course, and the first chapter states it before anything else: every
number you print in this course is the same every time you run it, digit
for digit, at every optimization tier — the promise C makes for its
arithmetic, kept from the first chapter to the last. When a chapter shows an
output built from plain arithmetic — sums, products, quotients and square
roots — your output is that output, on every machine; where a chapter calls
Tanh, Exp or Ln, the digits rest on your machine’s C library, and they
repeat on it run after run.
The twelve chapters
| # | Chapter | You end up with |
|---|---|---|
| 1 | The guessing machine | what learning is, before any mathematics — a machine that fits a rule to observations with nothing but guess, measure, nudge, and a loss that falls to zero |
| 2 | Downhill in the fog | the gradient: direction and stride read off the guess itself, the measured and the computed slope agreeing to the last digit, and the descent that replaces search with arithmetic |
| 3 | The notebook that runs backward | the tape: ’tracked’ knobs, the recording window, and one ‘Backward’ walk answering every slope — chapter 2’s descent reproduced digit for digit with the algebra done by the machine |
| 4 | Many knobs at once | vectors and the dot product: the same loop when the guess is a vector and a bias, one backward walk answering a whole gradient vector, and the shape checked before the run |
| 5 | The bent wire | the wall no weighted sum can climb — XOR’s loss flattening at exactly one — and ‘Tanh’, the one bend that climbs it with the same descent |
| 6 | Layers | what a hidden layer learns: the four corners’ hidden coordinates printed before and after training, the want-1 corners landing together, and the one weighted sum that reads the learned space |
| 7 | Blame flows backward | backpropagation by hand, exactly once: three per-link rules, a ledger walked from the loss to the knobs, and the tape’s answers matching yours to the last digit |
| 8 | The art of the step | one unfair valley descended three ways: the wall pricing the plain step, momentum saving it, and Adam — with the GPT’s own β₁ and β₂ — repealing it |
| 9 | Text becomes numbers | tokens, one-hot rows, and the embedding as learned meaning — closed by the smallest language model there is, trained to exactly 1.000000 on its three transitions, with the unseen character learning exactly nothing |
| 10 | Looking at everything at once | one attention head at the smallest honest size: query, key, and value in five recorded lines, the causal mask’s exact zero, the soft split’s exact halves, and blame walked backward through the softmax — exactly |
| 11 | The GPT, assembled | Gpt.mica read as one page: the eighteen-sentence window with each line’s home chapter, the 786944-parameter arithmetic, nine tensors, and the few honest newcomers — scale, residuals, layer norm, cross-entropy |
| 12 | Train your own GPT | a corpus, one command, minutes of training, and text in your corpus’s style — reproducible to the digit, run after run, with the device twin’s diff showing exactly what moving training to a GPU means |
Three tensor chapters sit beside the course for the reader who wants the numbers’ machinery whole: deterministic reductions, WYSIWYG slicing and spans.
A GPT, part by part
By chapter 11 you read Gpt.mica, a complete GPT in one file, and recognize
every line. This is the map of that machine: each part, what it does in one
sentence, and the chapter that builds it.
| Part of a GPT | What it does | Where you build it |
|---|---|---|
| tokens and the embedding | text becomes numbers: each character an id, and each id a learned row of meaning | chapter 9 |
| attention | every position asks what it is looking for, every position announces what it holds, and the answer is a weighted mix, with a mask so nothing looks forward | chapter 10 |
| the feed-forward net | a widening, a bend, and a narrowing: the layer that gives depth its power | chapter 5 and chapter 6 |
| the residual stream | the block adds its input back to its output, so it refines a representation rather than replacing it, and blame can flow through many blocks | chapter 11 |
| layer normalization | each position re-centered and re-scaled, training’s stabilizer | chapter 11 |
| the loss | one number for how wrong the model was: the average surprise at the true next characters | chapter 1 and chapter 2 for the idea, chapter 11 for the verb |
| the tape | the forward computation recorded as it happens, so one backward walk answers every slope at once | chapter 3 |
| backpropagation | blame flowing backward through the recording, done by hand exactly once and checked against the tape | chapter 7 |
| the optimizer | the plain step, momentum, and Adam: the nudge that turns a slope into learning | chapter 8 |
| training | a corpus, one command, minutes of training, and text in the corpus’s style | chapter 12 |
After the course
- The chapters that carry the numbers further: batched training, image networks, quantized tensors and the GPU, without kernels.
Where the examples live
Every chapter’s program is a directory of the starter repository,
mica-lang/mica-container,
under examples/, and runs with one command, for instance
make -C examples/GuessingMachine run. Start has the four roads to
a machine with the compiler on it.