Mica 7.7.3: one engine for every constant

Mica 7.7.3 computes every constant one way. One exact engine gives a constant expression its value, or its refusal, when the program compiles, and gives it the same at every optimization level on both processors. Constants grow with it: they read each other in any order, join strings, call Succ, Ord or High, hold tables of names, truth values and an enumeration’s members, and hold integers past 64 bits.

Beside the language, 7.7.3 closes 235 defects, each compiler fix beside a test case that failed before it. A program generator writes small Mica programs every night and judges each one at every tier on both processors, and on the release’s final build 20,000 of its programs ran clean. Around them, characters index arrays and fill sets, a wide integer is written where it is needed, a program writes its diagnostics to standard error and ends with its pipeline, and the Mica libraries a C program loads share one runtime floor and leave the program as it made them.

The debugger finds every name at its own line, in the unit that declares it, the editor draws the export beside the source and shows a project file’s mistakes at the entries that cause them, and the devkit’s commands answer in one way.

The release at a glance

One engine for every constant

{
    Palette - a heading, a table of names and a limit, every one of them computed once, as the program compiles.

    The heading joins two constant strings, the colours' names stand in one typed array constant, and an enumeration's
    member and its successor are constants, so a case takes them as labels. The one engine that computes every constant
    gives each of them the same value at every optimization level.
}
program Palette;

imp
    WriteLn, Succ, High : std;

const
    { the heading joins two constant strings }
    Product = "palette";
    Heading = Product + " of three colours";

    { one name for each colour, read like any array }
    Names : NameTable = ("red", "green", "blue");

    { a member and its successor are constants, and so is the largest int64 }
    Favourite = Green;
    AfterFavourite = Succ(Favourite);
    Largest = High(int64);

    TextLine = "%ls";
    NoteLine = "%ls: %ls";
    LargestLine = "the largest int64 is %lld";
    FavouriteNote = "favourite";
    AfterNote = "after it";

type
    { The palette holds three colours, in the order of the table. }
    Colour = (Red, Green, Blue);

    { A name table holds one name for each colour. }
    NameTable = array[Red..Blue] of string;

var
    colour : Colour;

begin
    WriteLn(TextLine, Heading);

    for colour := Red to Blue do
        case colour of
            Favourite: WriteLn(NoteLine, FavouriteNote, Names[colour]);
            AfterFavourite: WriteLn(NoteLine, AfterNote, Names[colour])
        else
            WriteLn(TextLine, Names[colour]);
        end;

    WriteLn(LargestLine, Largest);
end.

It prints:

palette of three colours
red
favourite: green
after it: blue
the largest int64 is 9223372036854775807

Every value this program names in its const section is computed when it compiles: the heading joined from two strings, the table of names, the favourite colour and its successor, and the largest int64. One engine computes them, exactly, and a debug build holds the same values as an optimized one, so a constant has one value whatever the level. Because Favourite and AfterFavourite are constants, the case takes them as labels, as it takes a literal.

A typed constant may be a string, and a typed array constant may hold strings, truth values or an enumeration’s members, so a table of names is one declaration, read like any array, copied whole and handed to a routine by value. An element read at a constant index is itself a constant, so array[0..Codes[1]] of int64 or a case label Codes[0]: reads it. A character of a constant string is one too: with const Word = "abc";, Word[1] is the constant 'b', and Word[position] reads the character at a position the program computes, guarded against the string’s length like any index. A library’s contract carries its tables, so a program that imports one reads the same values.

The same engine refuses what has no value, once, at the operator, on every tier. Max + 1 over a typed constant that holds the largest value of its type is refused when the program compiles, at debug as at O2, and so is a constant that names an imported constant and leaves its type.

A constant reads the constants of its block in any order, as every other name in Mica does: Area = Width * Height; may stand above Width = 4; and Height = Width + 1;, in the same const section or a later one, and it keeps its type and its value, and draws the same refusal, as when it is written below them. A routine’s own constant hides the program’s constant of the same name wherever the routine writes it. Constants that read each other have no value, so they are refused once, by their names, at the one written first. Left = Right * 2; beside Right = Left; draws:

'Left' and 'Right' read each other, so none of them can be computed before the others: give one of them a value, a bound or a type that reads none of the others

A constant that reads itself is told so in the same way, and a type, a bound or a case label that reads a refused constant adds no refusal of its own.

Checked from every side

The program generator writes small Mica programs from seeds. Every program it writes is valid and free of traps by construction, and prints a trace of the values it computes. Each one is built at every tier on both processors, run, and judged against an interpreter of the generator’s own, under the checked build and under valgrind. A finding is shrunk to the smallest program that keeps it, and every night’s run on the release branch files each new finding as a bug, never the same one twice.

This is one of the shapes it found:

{
    Buckets - sorts the numbers 0 to 1000 into buckets sixteen wide, and sums each number's bucket and its place in it.

    A division and a remainder of an unsigned counter by a power of two are one shift and one mask at the optimizing
    levels, and they give the answers the program writes at every level.
}
program Buckets;

imp
    WriteLn : std;

const
    BucketWidth = 16;
    Last = 1000;
    SumLine = "buckets %llu, places %llu";

var
    number, buckets, places : uint64;

begin
    buckets := 0;
    places := 0;

    for number := 0 to Last do
    begin
        buckets := buckets + number / BucketWidth;
        places := places + number mod BucketWidth;
    end;

    WriteLn(SumLine, buckets, places);
end.

Built at every level on both processors, it prints:

buckets 30814, places 7476

7.7.2 answered the same at debug. At O1 and O2 it stopped this program with a floating-point exception on x86-64, and printed buckets 501, places 250000 on AArch64. A division or a remainder of an unsigned integer by a constant power of two is now one shift or one mask, and the answer is the one the program writes. The other wrong answers the generator found at the optimizing tiers are fixed the same way, each with a test case that failed before its fix.

The generator’s programs now also compute with unsigned integers, read typed constant tables and tables of truth values, and index arrays by characters, and each one is judged at O1 as well. On the release’s final build, 20,000 of its programs ran clean: every one printed, at every tier on both processors, the values its interpreter computes.

The generator is one of several checks this release passed. 7.7.3 closes 235 defects, and each compiler fix arrived with a test case that fails on the compiler before it and passes after it, so the defect stays fixed. Every program of the execution suite that valgrind can run ran under it without a leak or a memory error. Every tutorial on this site was walked, command by command, on a release candidate of 7.7.3, and the programs that compute on a GPU ran on real NVIDIA cards.

Characters as indexes and sets

{
    Vowels - counts each vowel of a sentence in a table whose elements are numbered by the letters themselves.

    The table's dimension is the range of small letters, so 'tally[character]' is the count of that letter, and a set
    of letters names the ones the program prints.
}
program Vowels;

imp
    WriteLn, Length : std;

const
    Sentence = "a table numbered by the letters it counts";
    CountLine = "%lc %d";

type
    { A letter is one of the small letters of the ASCII range. }
    Letter = 'a'..'z';

    { A tally holds one count for each letter, numbered by the letter itself. }
    Tally = array[Letter] of int32;

    { A set of letters names a choice among them. }
    Letters = set of Letter;

var
    text : string;
    tally : Tally;
    vowels : Letters;
    character : unicode;
    letter : Letter;
    position : int64;

begin
    text := Sentence;
    vowels := ['a', 'e', 'i', 'o', 'u'];

    for letter := 'a' to 'z' do
        tally[letter] := 0;

    for position := 0 to Length(text) - 1 do
    begin
        character := text[position];

        { a character outside the small letters has no element }
        if (character >= 'a') and (character <= 'z') then
            tally[character] := tally[character] + 1;
    end;

    for letter := 'a' to 'z' do
        if letter in vowels then
            WriteLn(CountLine, letter, tally[letter]);
end.

It prints:

a 2
e 6
i 1
o 1
u 2

An array’s dimension may be a range of characters. Tally numbers its elements from the code point of 'a', so tally[character] is the count of that letter, and the index is guarded against the dimension’s range like every other index. A set of characters is built, combined and tested like any set, with the operators +, -, *, = and #, and its characters may lie past ASCII: a set of Greek over 'α'..'ω' works under utf-32 and utf-8 alike.

Wide integers written in place

{
    Midpoint - the midpoint of two large int64 values and three quarters of one, computed in 128 bits written in place.

    The sum of two values near the top of the int64 range does not fit an int64, and neither does the product of one of
    them with three, so each operand is cast to 'int[128]' where it stands, the arithmetic runs in 128 bits, and each
    result, which fits again, comes back with 'as int64'.
}
program Midpoint;

imp
    WriteLn : std;

const
    Lower = 8000000000000000000;
    Upper = 9000000000000000000;
    Quarters = 3;
    Whole = 4;
    MidpointLine = "midpoint %lld";
    QuartersLine = "three quarters %lld";

var
    lower, upper : int64;

begin
    lower := Lower;
    upper := Upper;

    WriteLn(MidpointLine, (((lower as int[128]) + (upper as int[128])) / 2) as int64);
    WriteLn(QuartersLine, (((upper as int[128]) * Quarters) / Whole) as int64);
end.

It prints:

midpoint 8500000000000000000
three quarters 6750000000000000000

A cast may name a wide integer type where it stands, x as int[128] or x as uint[128], with no declaration. The arithmetic between the casts runs in 128 bits, so a sum or a product that passes the int64 range on its way to a result that fits is computed exactly, and the result comes back with as int64. A constant may hold such a value too, Big = 5 as int[128], and a typed constant of a wide type may hold one past 64 bits.

Standard error and pipelines

{
    Squares - writes the square of every number up to a million, after a note on standard error saying so.

    The note goes to standard error, so a pipeline that reads the squares never sees it, and the program ends as soon as
    the program reading its output has gone.
}
program Squares;

imp
    WriteLn, WriteErrLn : std;

const
    Count = 1000000;
    NoteLine = "writing the squares of 1 to %d";
    SquareLine = "%lld";

var
    number : int64;

begin
    WriteErrLn(NoteLine, Count);

    for number := 1 to Count do
        WriteLn(SquareLine, number * number);
end.

mica Squares.mica | head -3

writing the squares of 1 to 1000000
1
4
9

WriteErr and WriteErrLn write formatted text onto standard error with the format WriteLn takes, checked the same way when the program compiles, and a line one task writes is never split by another task’s. The note above goes to the terminal, and the squares go down the pipe.

When head has read its three lines and gone, the program ends at its next line, by SIGPIPE, which the shell reports as status 141, instead of writing squares nobody reads. A program whose standard output or standard error lost text, to a full disk or a closed descriptor for example, says so in one line on standard error when it ends, and ends with status 1. The standard streams reference states the rules, and the scripting tutorial uses them in a pipeline.

A good guest in a C program

A Mica shared library now leaves the process that loads it as that process made it. Loading one, at link time or with dlopen, no longer changes the host’s locale, its SIGPIPE disposition, its SIGSEGV and SIGBUS handlers or its main thread’s alternate signal stack. The library’s own text does not depend on the host’s locale either: each write, read and number conversion uses the runtime’s UTF-8 locale for that call, then gives the calling thread back its own.

The host keeps its exit status. When text the library wrote is lost, the library says so on standard error as it unloads, and a host may unload it with dlclose and exit as usual. A write to a pipe whose reader has gone follows the host’s SIGPIPE setting, as any C library’s does, and a stack overflow inside a library routine reaches the host as the fault it is. A Mica program starts exactly as before. The C calls Mica tutorial builds such a library and its host.

Every Mica shared object a process loads runs on one runtime floor. A library built with --output-modifier object names the shared floor, libmica-dragon-rt-<arch>.so.<release> or one of its variants, instead of carrying a copy of it, so a C host that loads two Mica libraries, at link time or with dlopen, runs their tasks on one scheduler and one carrier pool, and each library is about half its former size. The mica-dragon-rt package installs the shared floors in /usr/lib, where the dynamic loader finds them, so the machine that runs a Mica shared object installs that package.

A process holds one floor, so the Mica libraries it loads come from one compiler release, built at one tier, with one tasking flavor and one memory class. A library that names another floor is refused as it loads, in one line that names both floors, and the process ends with status 2. A host may unload a multicore library whose carriers have started and exit normally, and a host that unloads one Mica library while it keeps another ends cleanly: the unloaded library’s finalizers run, and what it bound to the process is released, as it unloads. The facts page lists the shared floors beside the archives.

Half precision on a GPU

{
    Blend - three quarters of one half-precision row plus another, computed on the GPU in one kernel.

    The expression '0.75 * a + b' over two float16 matrices on the card fuses into one kernel, which rounds each
    operation's answer into float16, so the row holds the digits the same two operations give one at a time.
}
program Blend;

imp
    WriteLn, ToDevice, ToHost : std;

const
    Cells = 4;
    Weight = 0.75 as float16;
    Offset = 0.5 as float16;
    CellLine = "%lf";

var
    a, b, blended : matrix[1, 4] of float16;
    onCardA, onCardB, onCardBlended : matrix[1, 4] of float16 on gpu;
    cell : int64;

begin
    for cell := 0 to Cells - 1 do
    begin
        a[0, cell] := (cell + 1) as float16;
        b[0, cell] := Offset;
    end;

    onCardA := ToDevice(a);
    onCardB := ToDevice(b);
    onCardBlended := Weight * onCardA + onCardB;
    blended := ToHost(onCardBlended);

    for cell := 0 to Cells - 1 do
        WriteLn(CellLine, blended[0, cell] as float64);
end.

On a machine with an NVIDIA card it prints, at debug and at O2:

1.250000
2.000000
2.750000
3.500000

A device expression of several operations over float16 or bfloat16 matrices, such as Weight * onCardA + onCardB, fuses into one kernel. The kernel rounds each operation’s answer into the half, so it gives the digits of the same two operations written one at a time, and the scalar reaches it as its value at every optimization level. A program built with fp_half_intermediates keeps a measured product’s split sums in the half on every card, as the runtime flags reference says.

The debugger finds every name at its place

gdb and perf find a routine’s caller from every instruction of the program: at a checked build’s trap, inside the runtime’s own helpers and on a routine’s last instructions, a backtrace names the routine that stopped, its callers and the program body, at every tier. gdb finds a type by the name the program writes, so ptype Point and ptype Shade answer beside ptype struct Point, ptype Alias answers for type Alias = Row;, and a routine’s own types are its own, also where they take the name of a unit’s type. print Defaults shows a typed array constant’s values, and a dynamic array shows its data, length and capacity, so print *ints.data@ints.length prints its elements.

Every name stands where the program declares it. Each named type sits at the line of its own declaration, a function’s result at the function’s name, and the program’s entry at its heading, program Name;. A record, an enumeration, an array or a constant imported from another unit of the project lies in that unit’s file, at its own line. A generic routine taken from a library keeps the library’s file and line: a warning about its body, a stop inside a copy the optimizer placed in your routine, and the records it declares in its own body all name the library’s source, and the debugger finds that source even when the library was built in another folder.

The editor: the export and the project file

In VS Code, an export’s control flow, dominator tree, placement lanes and call graph are drawn beside it by Graphviz, offline, and a click on a block, an instruction or a live range leads back to the source line it came from. Every line of every stage navigates, in an AArch64 export as in an x86-64 one: a hover names the source line it came from, the links above each routine open its control flow, its placement and the next stage, and Open Export Stage Beside stands a source and one of its stages side by side, each source line and the lines it became sharing a colour band. The later stages name each variable as the source declares it, so a hover says that v3.1 is sum in the source, and a placement lane reads sum (v3.1).

A mistake in mica.project is shown on the project file itself, at the entry that causes it: a source that does not exist at its line in the list, an unknown key at the key, a value of the wrong shape at the value. The file is read exactly as its schema states it, so a key written twice is refused at the second one:

{
  "sources": ["DuplicateKey.mica"],
  "encoding": "utf-8",
  "encoding": "utf-32"
}

duplicate key "encoding", first written at line 3: a project file states each key once

The command line gives the same refusal, naming the file and both lines. Text after the project’s object and a key written in another case, "Sources", are refused the same way. A change to the project file reaches every open source before any save, and Run File, the builds, Export File and the profiles save every file their build reads before they build, the project file first.

The devkit answers in one way

The devkit’s six commands, mica-devbox, mica-cloud, mica-provision, mica-measure, mica-bench-cloud and mica-bench-render, share one usage layout: the name and purpose, Usage:, the verbs or features, every option with its meaning and its default, and Examples: last. A command called with no argument prints its usage and changes nothing, --help prints the same text, and a word the command does not know is refused by name:

devkit/mica-provision --frobnicate

  error  unknown option '--frobnicate'
         'mica-provision --help' lists what is available

Every setting is a flag, on the command line or on make’s, and none is read from the environment: mica-provision --repos-dir DIR clones into the folder you name, make MICA=... picks the compiler of an example or a benchmark, and --no-color turns the colours off, which each stream otherwise shows only when it is a terminal. A failing run names its log on the error stream. The user image carries git, and the install page shows the one command that tells it who you are. The devkit post shows the whole kit.

Held to its own rules

Every fix below comes with a test case that failed before it and passes now, so it stays fixed.

• Wrong answers at the optimizing tiers. A value a call, a pointer or a nested routine wrote is read for what it holds, a variable assigned before an if keeps that value when the arm does not run, a function returns the last value assigned to its result, a negation in a loop keeps its sign, shifts of narrow values give Mica’s answer on both processors, and an operation on two constants inside a loop reads both of them at O2 on x86-64.
• Float conversions. A float converted to an integer type that does not hold it gives one answer on every tier and both processors: the type’s minimum below its range, its maximum above it, and 0 for NaN, as the traps reference says. The checked tier traps each such conversion at its as.
• Imports add exactly the names they state. Importing Sort brings Sort, not the helpers and templates it uses, so a program may declare a T, an E or a Length of its own, and imp * : arrays; brings every name the unit exports, its generic routines included.
• Ownership. A dispose of a variable’s own storage is refused when the program compiles, a leak that only drew a remark is refused, and a routine that waited inside a concurrent block when a child failed moves on to the block’s end.
• Deadlines and cancellations reach every wait. A server’s accept loop that skips a failed accept, accepted := Accept(listener) on fail reason do next;, leaves its wait when its block’s deadline fires, and its owner moves on to the block’s end; a task parked in such a loop ends when its owner cancels it, whatever the failure handler does. The echo server tutorial writes its accept loop this way, under a scope deadline too.
• One mistake, one refusal. A reserved word in a name’s place, a pointer to a pointer, two prefix operators in a row, a cast before + and a parameter written behind Pascal’s var are each refused once, in words that name the mistake and the form that compiles.
• Warnings where they belong. An import a unit never uses is warned about at its own entry, the name it brings marked, whether it comes from a unit of the build or through a library’s contract, and a type or constant a unit exports is never called unused in its own unit.

Upgrading

Things a 7.7.2 build meets differently in 7.7.3:

• Ord(letter) gives a character’s code point, and as converts no character to an integer. A subrange of characters converts as unicode does, so letter as int64 is refused for a letter of type Letter = 'a'..'z';:

  data type 'Letter' converts to an integer only with 'Ord', not with 'as'

• A typed constant, a cast and Ord keep their own types beside a variable of another integer type: n * K over an int32 n and K : int64 = 5 computes in int64, so a program that assigns it to an int32 or prints it with %d writes the cast that says so. n * 5 stays an int32.

• A mica.project is read exactly as its schema states it: text after its object, a key written in another case and a key written twice are refused by name, where a build went ahead without a word.

• A mica.project whose sources name a file that does not exist is refused when the project loads, a build that names its own --source included, so a project lists only the files that are there:

  project file mica.project: source "demo.mica" does not exist (looked at /home/mica/source/repos/notes/demo.mica)

• A constant computation that leaves its type is refused on every tier, a constant that names an imported constant included, and so is a hexadecimal or binary literal past 64 bits at a machine integer, which kept its low 64 bits.

• A dispose of a variable’s own storage, p := address n; dispose (p);, is refused:

  a variable's own storage is released as though 'new' made it: dispose releases the address of variable 'n', taken at line 9 — remove the dispose, since only a cell 'new' made may be disposed

• A program imports every name it writes. A program that writes SortList of int64 beside imp Sort : arrays; imports SortList too:

  'SortList' is a generic template of 'arrays', and this unit does not import it: write 'imp SortList : arrays;' in the unit's own import section

• A program links only the Mica libraries its own compiler built, so rebuild each library with 7.7.3. A library that exports a task links as an archive.

• A Mica shared object names the shared runtime floor, so the machine that runs it installs the mica-dragon-rt package of the same release, and the Mica libraries one process loads come from one release, tier, tasking flavor and memory class.

• A failure handler no longer runs on a wait that a scope’s deadline or a cancellation ended: the deadline moves its owner on to the block’s end, and the cancellation takes the task out, before the handler’s first statement.

• A name imported through a library’s contract and never used draws the unused-import warning at its entry, as a name imported from a unit of the build does.

• An optimization set holds one profile:

  the optimization set names two profiles, 'debug' and 'release'; a set holds one profile, debug or release

• A program ends by SIGPIPE when the program reading its output has gone, and ends with status 1 when its output lost text.

• The runtime floor’s contract is version 5, and a floor written to version 4 is refused by its version’s name.

The packages and the commands that install them are on the install page.