The Dragon SDK: our backend, as a deliverable

Writing a compiler front end is a pleasant few weeks. You design a grammar, write a scanner, write a recursive-descent parser, build a symbol table — and then you arrive at the part where the program has to become machine code, and the pleasant weeks end.

That is where most language projects stop. Not because the remaining work is uninteresting, but because it is disproportionate: register allocation alone is larger than everything you have written so far, and it is followed by instruction selection, then by the peephole passes, then by the discovery that your generated code is four times slower than gcc -O0 and you have no idea why.

With this release, that part is available separately. The Dragon SDK is the Mica compiler’s backend, published as a static archive and a C header, so a front end written in any language can hand it an intermediate-code unit and get optimized native code back.

What it actually is

Dragon is not a code-generation library. It is the whole back half of a production compiler, and the same one that compiles Mica — not a reduced or forked version of it.

A unit handed to Dragon goes through:

• validation against the intermediate language’s contract
• control-flow analysis — basic blocks, the flow graph, dominance
• an SSA middle-end — value numbering, constant propagation, dead-code elimination
• storage placement — what lives in a frame, what can live in a register
• register allocation
• instruction selection and emission for x86-64 and ARM64
• peephole optimization

The intermediate language you write is called Spectra. It is flat three-address code with named temporaries and labelled control flow — the family GCC’s GIMPLE and LLVM’s IR belong to — with no expressions, no nesting, and no types beyond machine types.

The whole consumer surface is one header, mica_dragon.h. A session, a type registry, a unit with its symbol and address tables, instruction builders, validate, build. From Mica it is reached by import instead:

imp
    ContextNew, RegistryNew, UnitNew : dragon;
    Validate, Build, Runtime : dragon;

Why we published it

Three reasons, in the order they actually mattered.

The backend was already separable, and we wanted to prove it. The Mica compiler is four modules, and the lower three — the foundation, the intermediate language, and the engine — import nothing from the Mica front end. That is a claim a build system can enforce, but the honest proof is a different front end compiling through the same engine. Publishing the SDK is how that claim stops being architectural intent and becomes a fact somebody outside the project can check.

Because a good backend is the expensive part, and it should be shared. A domain-specific language, a research language, a compiler for a course, a transpiler that currently emits C — all of them need what Dragon does, and none of them need to write it. The economics of that are not subtle.

Because it made our own compiler better. Every ambiguity in the interface that a stranger would trip over was a place where our own code had been relying on a shared assumption. Writing the C header forced us to name what a unit must satisfy, and then to make the engine check it rather than crash on it. The validator that resulted catches malformed input from the Mica front end too.

Validate, then build

The single design decision we would defend hardest is that validation is not optional and not advisory.

A back end that trusts its input fails deep inside register allocation, with a stack trace through code the front-end author has never seen and cannot read. That is survivable when the front end and the back end are written by the same people. It is not survivable across a published interface.

So Dragon refuses. Hand it a branch to a label that is never placed, an instruction whose operands have incompatible types, or arithmetic without the runtime-safety labels the contract demands, and you get a sentence naming the instruction and the rule it broke — before anything is emitted.

The worked example carries a deliberate probe for this. It has a -trap flag that sends one arithmetic instruction through the raw door without the safety labels, which is the mistake every new front end makes first, and the expected outcome is that validation names it. That probe runs on every change to the compiler.

A backend that validates is a backend a stranger can write a front end for. Everything else in the surface is downstream of that.

The worked example is a real compiler

We did not ship a hello-world. The SDK’s example is Niklaus Wirth’s PL/0 — the smallest complete compiler ever written down — implemented twice: once in C23, once in Mica, seven source files each, in the same order.

Each twin carries Wirth’s own stack machine as an embedded oracle: it interprets what it compiles, so whatever it prints is by definition what the program means. The Dragon road must print exactly the same bytes. The two twins must also agree with each other — oracle output, p-code listing, and the textual Spectra of every unit they build are held byte-identical between the C and Mica implementations, on both architectures and at every optimization level, on every change to the compiler.

That is a demanding test, and it is the point. The example is not a demonstration of the API; it is a continuously verified proof that the API works from two languages with very different ideas about what an interface is.

There is now a full course built on it: Build a compiler with Mica — eleven chapters from characters to native machine code, with Wirth’s 1975 original in an appendix. If you have ever wanted to read a whole compiler rather than a chapter about one, start there.

What we added to the intermediate language for this

Spectra was shaped by Mica’s needs, and Mica is not C. Publishing the surface meant closing the gaps a C-family front end would hit immediately:

• Indirect calls through a function address, so function pointers, vtables and callback tables lower directly. The indirect call reuses the direct call’s entire road, so no analysis had to learn a second shape.
• Byte-counted pointer displacement — pointerOffset, which is GCC’s POINTER_PLUS_EXPR verbatim. We chose it over the scaled form for one measured reason: a scaled offset cannot be formed over a pointer to an incomplete type, and C’s self-referential struct is exactly that.
• Initialized writable globals with relocations, so a file-scope initializer — including one whose value is the address of another global — becomes storage the linker fills in.

Each is on both architectures, each with validator rules that name a mistake rather than letting it through.

It is a preview, and we mean it

The Dragon and Spectra APIs are young and may change without a compatibility promise. What the SDK demonstrates today is the shape of the seam, not a frozen interface.

We would rather say that plainly than discover in a year that we cannot fix an awkward corner because somebody depends on it. If you build something on this now, expect to adjust it — and tell us where it hurt, because that is exactly the feedback that turns a preview into a stable interface.

Concretely, what is not settled yet: the exact shape of the type registry, how much of the optimizer’s regime selection belongs in the consumer’s hands, and whether the recording-and-splice pattern the examples use (buffering instructions because a front end’s natural order is not the target’s) should be something the engine does for you.

The licence, stated plainly

The Mica compiler and the Dragon SDK ship under different licences, and this is the one place in the project where that is true.

Anyone may try it. Install it, read the example, write a front end, run the course — that is inside the free grant for every reader of this post who is doing it for study or for their own interest. If it is work for a company, including internal work toward a commercial product, talk to us first.

The boundary is drawn at the package rather than inside one, because a .deb has no optional components: a single SDK file riding inside the compiler package would hand noncommercial-licensed material to every commercial user who ever installs Mica, and no correction reaches copies already taken. So the container image carries no SDK either — mica-install-sdk prints the terms and then installs, which is a decision someone takes rather than one that happens to them.

We chose PolyForm over writing our own licence for the same reason we chose GIMPLE’s pointer shape over inventing one: it is lawyer-drafted, and its definition of noncommercial is the industry’s.

Where this goes

The SDK exists because we needed the backend to be separable. Now that it is, three directions are open, and they are not equally near.

Nearest: making the preview a promise. The interface needs to survive a few more front ends before we freeze it. PL/0 twice was the first evidence; a third consumer written by somebody who is not us is the evidence that matters.

Next: what a C front end still needs. The three additions above were chosen against a small C compiler as the target. Getting the rest of the way — bit fields, the full initializer language, variadics beyond what the current shape carries — is a bounded list, and it is bounded precisely because there is a real front end pulling on it.

Further out: more targets. Dragon emits x86-64 and ARM64 today. The architecture is not the interesting constraint; the interesting one is that every new target must satisfy the same differential tests, which is what keeps a backend honest as it grows.

What will not change is the arrangement. The engine that compiles Mica is the engine you get — not a fork, not a subset, not a version that lags. When Mica’s optimizer improves, so does yours.

Try it:

docker run -it micalang/mica
mica-install-sdk

Or natively: installing the Dragon SDK is one .deb beside the compiler’s.

Then the course, or straight to the header at /usr/include/mica_dragon.h and the twins in sdk/pl0.