A layer of a model is a handful of parameters that belong together. In Mica that is what it looks like: a record whose fields are tracked, and a model is an array of those records. The forward pass loops over the blocks and records through the places; the parameter walk pytorch hides behind model.parameters() is a loop you write, read, and step through.

The example is examples/TrackedRecords in the tutorial repository.

Getting the file

make -C examples/TrackedRecords run
TrackedRecords — records hold the parameters, the walk is your loop

loss = 6.000000
w0 after the step = 0.900000 -0.111111 -0.100000 0.888889
w1 after the step = 1.916667 -0.100000 -0.083333 1.900000
loss after the step = 4.262593

sgd over the elements = 0.875000 1.750000 2.875000 3.750000

Identical on every machine, at every optimization tier, on both architectures.

A block is a record, a model is an array

type
    Block = record
        w : tracked matrix[2, 2] of float64;
    end;

var
    blocks : array[0..1] of Block;

A tracked field is a place: blocks[k].w declares, records, answers Gradient, and re-adopts exactly as a tracked variable does. The member’s inner type is a float scalar or a static-shaped float tensor, and a variable whose type carries tracked members lives in the program’s own var section — the same home every tracked value has.

The depth loop records through the places

tape
    for k := 0 to 1 do
        h := blocks[k].w * h;
    l := Sum(h);
end;

Backward(l);

The loop is ordinary code inside the window. Each pass through the body reads a different place, and the tape records what actually executed — depth is unrolled by having run, not by a graph compiler.

The walk is your loop

Each block keeps a plain twin record beside it, carrying the weights the optimizer updates and its two moment runs — visible state, no optimizer object:

type
    Plain = record
        w : matrix[2, 2] of float64;
        m : matrix[2, 2] of float64;
        v : matrix[2, 2] of float64;
    end;
for k := 0 to 1 do
begin
    gw := Gradient(blocks[k].w);
    AdamStep(address twins[k].w, gw, address twins[k].m, address twins[k].v,
             1, 0.125, 0.5, 0.5, 0.5);
    blocks[k].w := twins[k].w;
end;

Three sentences per block: read the gradient at the place, step the plain twin, re-adopt. address twins[k].w lends the twin’s field to the update step through the conformant borrow — field steps with one index step, the same place grammar the tape itself reads — and the update writes the caller’s own storage. The re-adoption blocks[k].w := twins[k].w moves the stepped value back into the tracked place; the next window enrolls it fresh.

A second window over the updated places closes the cycle: the loss falls from 6 to 4.262593, and both numbers are the same on every machine.

Why the twin, and not the place itself

analyzer error 5119: data type 'tracked matrix[2, 2] of float64' cannot be
used in memory operation 'address of' (requires addressable)

A tracked place is a pair — the value and the id of the node that produced it — and a borrowed view writing the value behind the pair’s back is exactly the hidden mutation the pair refuses everywhere. The twin record keeps the optimizer’s writes in plain storage, and the adoption is the one visible moment the model changes.

The borrow serves bare elements too

for k := 0 to 1 do
    SgdStep(address ws[k], gsd, address vs[k], 0.25, 0.0);

An element of a plain array — ws[k], no record anywhere — lends through the same borrow, so parallel-array models update in place with the same three sentences.

Where this leads

The batched trainer uses exactly this shape at full size: eight parameters per block, two blocks, an embedding above them, and one visible walk stepping everything under a cosine schedule.