This is the course’s last page, and it is a doing page. You will train the machine you read in chapter 11 on a real corpus, watch its loss fall, save what it learned, load it back, and read the text it writes — on an ordinary processor, in minutes.

The working files are samples/gpt/gpt.mica and its device twin samples/gpt/gpt-gpu.mica.

A corpus, and one command

Put any book-sized plain text file named corpus.txt beside the program. The classic choice is the tiny Shakespeare corpus:

curl -o corpus.txt https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt

Any plain text works — the model learns whatever rhythm you feed it. Then:

mica --optimize O2 --stack-size 33554432 gpt.mica

Both flags are honest costs, stated in the file’s own header: the optimization tier buys the training loop its speed — about four minutes at O2 — and the value-typed model stages matrix-sized intermediates on the stack, so the program asks for a 32 MiB stack instead of the platform’s 8 MiB default.

What you will see

A loss line every hundred steps — the number to watch falling, exactly as it fell in chapter 1. Then the checkpoint being written, read back, and four hundred characters of generated text: structure, words, and rhythm recognizably shaped by your corpus — not literature. That is the honest result of a 786944-parameter model, and watching it emerge from random weights is the whole lesson.

Two thousand steps is a taste, chosen so the run fits in minutes. The constants at the file’s head are yours to turn: more steps, a different seed, a different corpus. Every experiment of the course applies — the loss column is chapter 1’s report card, the step size is chapter 8’s art, and if you change a tensor’s shape inconsistently, the build refuses before anything runs.

Reproducible, to the digit

The course opened with a promise, and this run is its largest test: the same corpus and the same seed print the same losses and the same sampled text on every supported machine, every time. Not close — identical. The deterministic reductions, the fixed fold orders, and the seeded sampling carry the promise from chapter 1’s toy loop to a full training run. It is a claim you can test by diff: run it twice, run it on another machine, compare bytes.

The suite behind this course holds that promise to a harder standard still: the same training, host against device, digit for digit —

  step 100   loss 1.757315        step 1000  loss 0.950289
  step 500   loss 1.501349        step 2000  loss 0.598289

— one loss column, printed by two different processors.

The device twin

Beside gpt.mica sits gpt-gpu.mica, the same program with training resident on the graphics device. Diff the two files — that is the recommended way to read it — and what you see is exactly what moving a training run to a GPU means in Mica: on gpu in the tensor declarations, a visible ToDevice or ToHost wherever data crosses between worlds, and the fused device AdamStep. Every sentence inside the tape window is identical. Measured on an RTX 4090 in an AMD64 machine, the training run finishes about nine times sooner than the same machine’s host run — and the loss column agrees with the host’s to the last digit, the twin contract kept as code.

The course, closed

Twelve chapters ago the machine was a loop: guess, measure, nudge. It still is. Everything between was the forward story growing richer — the gradient, the tape, layers, attention — until the guess had 786944 knobs and the story was a transformer, and the loop never changed.

You now hold something rare: a complete language model you have read every line of, trained yourself, and can reproduce to the digit. The constants at the top of gpt.mica are the honest playground — and everything you learn there, you can trust, because nothing about this machine is hidden from you.