This is the course’s last page, and it is a doing page. You will train the machine you read in chapter 11 on a real corpus, watch its loss fall, save what it learned, load it back, and read the text it writes — on an ordinary processor, in minutes.

The working files are examples/Gpt/Gpt.mica and its device twin examples/Gpt/GptGpu.mica.

A corpus, and one command

Change into the example’s folder and put any book-sized plain text file named corpus.txt beside the program. The classic choice is the tiny Shakespeare corpus:

cd mica-container/examples/Gpt
curl -o corpus.txt https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt

Any plain text works — the model learns whatever rhythm you feed it. Then:

mica --optimize O2 --stack-size 33554432 Gpt.mica

Both flags are honest costs, stated in the file’s own header: the optimization tier buys the training loop its speed — a few minutes at O2 on an ordinary processor — and the value-typed model stages matrix-sized intermediates on the stack, so the program asks for a 32 MiB stack instead of the platform’s 8 MiB default.

What you will see

A first line naming the corpus’s characters, the model’s parameters and the steps to come; then a loss line every hundred steps — the number to watch falling, exactly as it fell in chapter 1. Then the program writes what it learned to gpt.ckpt, reads it back, and prints four hundred characters of generated text: structure, words, and rhythm recognizably shaped by your corpus — not literature. That is the honest result of a 786944-parameter model, and watching it emerge from random weights is the whole lesson.

Two thousand steps is a taste, chosen so the run fits in minutes. The constants at the file’s head are yours to turn: more steps, a different seed, a different corpus. Every experiment of the course applies — the loss column is chapter 1’s report card, the step size is chapter 8’s art, and if you change a tensor’s shape inconsistently, the build refuses before anything runs.

The same run, every time

The course opened with a promise, and this run is its largest test: the same corpus and the same seed print the same losses and the same sampled text every time the program runs on your machine — to the last digit. The deterministic reductions, the fixed fold orders, the seeded sampling and C’s own arithmetic carry the promise from chapter 1’s toy loop to a full training run. It is a claim you can test by diff: run it twice and compare bytes.

mica --optimize O2 --stack-size 33554432 Gpt.mica > first.txt
mica --optimize O2 --stack-size 33554432 Gpt.mica > second.txt
cmp first.txt second.txt && echo "the same, byte for byte"

Here is the column as a run printed it on an x86-64 machine, and digit for digit the same in an arm64 container on Apple silicon —

step 100  loss 2.754283
step 500  loss 2.736750
step 1000  loss 2.350751
step 2000  loss 2.209104

— the number to watch falling on yours. It wanders from one report to the next, because each step learns from one random window of the corpus, and it falls across the run; your column falls the same way. The softmax, the bend and the loss call the C library’s exp, tanh and log, and the layer verbs are the math unit’s own Mica, so a C library that gives another last digit, or a compiler release whose math unit computes a verb another way, parts your column from this one as training carries the difference forward — while every run under one compiler and one C library repeats its own.

The device twin

Beside Gpt.mica sits GptGpu.mica, the same program with training resident on the graphics device. Diff the two files — that is the recommended way to read it, with diff Gpt.mica GptGpu.mica in the terminal or Compare Selected on the two files in VS Code’s Explorer — and what you see is exactly what moving a training run to a GPU means in Mica: on gpu in the tensor declarations, a visible ToDevice or ToHost wherever data crosses between worlds, and the fused device AdamStep. Every sentence inside the tape window is identical. It runs with the same command on a machine with an NVIDIA card, NVIDIA’s driver, cuBLAS and NVRTC:

mica --optimize O2 --stack-size 33554432 GptGpu.mica

Without a card it builds all the same, and running it stops with a message naming what it looked for. On a card the training run finishes in seconds where the host run takes minutes, and it repeats its own loss column to the last digit every time it runs on that card. Set beside the host’s column, it tells the same descent and may part from it in a last digit at a few reports: the device sums its matrix products in cuBLAS’s own order and takes exp, tanh and log from NVIDIA’s functions, and either can round a last bit the other way.

The course, closed

Twelve chapters ago the machine was a loop: guess, measure, nudge. It still is. Everything between was the forward story growing richer — the gradient, the tape, layers, attention — until the guess had 786944 knobs and the story was a transformer, and the loop never changed.

You now hold something rare: a complete language model you have read every line of, trained yourself, and can reproduce to the digit. The constants at the top of Gpt.mica are the honest playground — and everything you learn there, you can trust, because nothing about this machine is hidden from you.