A Mica source is UTF-8 text, read into tokens: words, numbers, characters, strings, regular expressions and operators, with white space and comments between them.
Before a source is read, a byte-order mark at its start is set aside, a first line that begins with #! is set aside with it, and every carriage return is removed. A byte that is not UTF-8, a NUL, and a byte-order mark anywhere but at the start are refused. A tab stays: outside a literal it is white space, inside one it is a character of the literal, and the column a message reports counts it up to the next multiple of four.
What remains is read from left to right into tokens. At each place the longest token that can begin there is taken, and between tokens stand white space and comments. A spelling the scanner refuses by name is a token of its own here, so the page can say why it is refused.
source
source = [ byte-order-mark ] [ interpreter-line ] { white-space | comment | token } .A source: tokens with white space and comments between them.
byte-order-mark
byte-order-mark = ? U+FEFF ? .The invisible mark some editors write ahead of the first character.
It is set aside only as the first character; anywhere else it is refused.
Used in source.
interpreter-line
interpreter-line = "#!" { character - newline } .The line a script carries for the system to find the program that runs it.
It stands only on the first line; the line still counts, so later line numbers are the ones an editor shows.
Used in source.
white-space
white-space = ? Unicode white space ? .Spaces, tabs and line breaks, and every other Unicode white space character.
Used in source.
comment
comment = block-comment | line-comment .A comment is a block in braces or the rest of a line behind two slashes.
Used in source.
block-comment
block-comment = "{" { block-comment | character - ( "{" | "}" ) } "}" .A block comment runs from { to its matching } and may hold block comments of its own.
Used in comment, block-comment.
line-comment
line-comment = "//" { character - newline } [ newline ] .A line comment runs from // to the end of its line.
Used in comment.
character
character = ? any character ? .Any character of the prepared source.
Used in interpreter-line, block-comment, line-comment, string-literal, unicode-literal, raw-character, regex-atom, regex-class-character.
newline
newline = ? U+000A ? .The line break.
Used in interpreter-line, line-comment.
token
token = reserved-word | identifier | floating-point-literal | masked-integer-literal | integer-literal | string-literal
| unicode-literal | regex-literal | operator | refused-number | refused-regex | refused-operator .A token: a word, a literal or an operator; the last three alternatives are the spellings the scanner refuses by name.
Used in source.
reserved-word
reserved-word = "address" | "and" | "array" | "as" | "begin" | "bitrecord" | "bitset" | "borrowed" | "case" | "const"
| "defer" | "dispose" | "do" | "downto" | "else" | "end" | "exp" | "file" | "for" | "function" | "gen"
| "if" | "imp" | "in" | "is" | "leave" | "library" | "mod" | "new" | "nil" | "not" | "object" | "odd"
| "of" | "or" | "owning" | "packed" | "pointer" | "procedure" | "program" | "record" | "repeat"
| "reserve" | "set" | "shl" | "shr" | "then" | "to" | "type" | "until" | "value" | "var" | "while"
| "with" | "xor" .The words that belong to the language and cannot name anything.
Reserved words are written in lower case;
Beginis a name.withis reserved and no rule of the grammar uses it.
Used in token, identifier.
identifier
identifier = word - reserved-word .A name: a word that is not a reserved word.
Some words carry meaning in one position and stay ordinary names everywhere else, such as
next,when,taskandvector; the rules that use them say where. The namesmain,True,False,std,cstdandmathare names to this grammar, and a program cannot declare them.
Used in token, compilation-unit, generic-parameter, import-group, import-member, constant-entry, type-declaration, type-parameters, variable-group, intrinsic-declaration, function-heading, procedure-heading, task-heading, generator-heading, callback-heading, parameter-group, parameter-name, conformant-schema, result-type, field-group, dimension, subrange-type, enumeration-literal, sum-field, bitrecord-field-group, error-domain-type, device-scalar-type, value-type, qualified-type-reference, type-name, assignment-target, scheduled-stream, routine-name, for-statement, case-label, case-pattern, labeled-loop, leave-statement, next-statement, loop-target, until-owner, fail-statement, schedule-statement, select-arm, handler-section, value-consumption, statement-consumption, cast-expression, primary, aggregate-element, selector.
word
word = letter { [ "_" ] ( letter | decimal-digit | subscript-digit ) } .A word begins with a letter and continues with letters, digits and subscript digits, a single underscore allowed between two of them.
Used in identifier.
letter
letter = ? Unicode letter ? .A letter of any script: Latin, Greek and the rest.
Used in word.
decimal-digit
decimal-digit = ? Unicode decimal digit ? .A decimal digit of any script, as a word may continue with one.
Used in word, refused-number, foreign-digit.
subscript-digit
subscript-digit = "₀" .. "₉" .The subscript digits a formula writes, as in w₁.
Used in word.
integer-literal
integer-literal = hexadecimal-literal | binary-literal | decimal-literal .A whole number in decimal, hexadecimal or binary.
Used in token, subrange-type, type-argument, literal.
decimal-literal
decimal-literal = digit { [ "_" ] digit } .Decimal digits, a single underscore allowed between two of them.
Used in integer-literal, floating-point-literal, exponent, refused-number, number-head.
digit
digit = "0" .. "9" .A decimal digit of a number.
Used in decimal-literal, foreign-digit.
hexadecimal-literal
hexadecimal-literal = "0" ( "x" | "X" ) hex-digit { [ "_" ] hex-digit } .A hexadecimal number behind 0x.
Used in integer-literal.
hex-digit
hex-digit = "0" .. "9" | "a" .. "f" | "A" .. "F" .A hexadecimal digit.
Used in hexadecimal-literal, refused-number, hex-escape, braced-unicode-escape, long-unicode-escape.
binary-literal
binary-literal = "0" ( "b" | "B" ) binary-digit { [ "_" ] binary-digit } .A binary number behind 0b.
Used in integer-literal.
binary-digit
binary-digit = "0" | "1" .A binary digit.
Used in binary-literal, masked-integer-literal, refused-number.
masked-integer-literal
masked-integer-literal = "0" ( "b" | "B" ) { binary-digit [ "_" ] } "?" { [ "_" ] ( binary-digit | "?" ) } .A binary pattern in which ? marks a bit that may be either; a case label matches a value against it.
floating-point-literal
floating-point-literal = decimal-literal ( "." decimal-literal [ exponent ] | exponent ) .A number with a fraction, an exponent, or both.
Used in token, refused-number, literal.
exponent
exponent = ( "e" | "E" ) [ "+" | "-" ] decimal-literal .The power of ten a floating-point literal is scaled by.
Used in floating-point-literal.
refused-number
refused-number = "0" ( "x" | "X" ) (?! hex-digit ) | "0" ( "b" | "B" ) (?! binary-digit | "?" )
| decimal-literal [ "." decimal-literal ] ( "e" | "E" ) [ "+" | "-" ] ( (?! decimal-digit | "_" ) | "_" )
| floating-point-literal ( "." decimal-digit | ( "e" | "E" ) [ "+" | "-" ] ( decimal-digit | "_" ) )
| [ number-head ] foreign-digit .The number shapes the scanner refuses: a prefix with no digit behind it, an exponent letter with no digit behind it, a separator right behind an exponent’s letter or sign, a floating-point literal that runs on into a second fraction or exponent, and a number that holds a decimal digit other than 0 to 9, refused at that digit.
Mica writes numbers with the digits
0to9; a name may go on with a decimal digit of any script, asx٣does.
The compiler refuses each of these by name, with scanner errors 1007, 1008, 1016, 1017, 1023, 1025 and 1045.
Used in token.
number-head
number-head = decimal-literal [ "_" | "." [ decimal-literal [ "_" ] ]
| [ "." decimal-literal ] ( "e" | "E" ) [ "+" | "-" ] [ decimal-literal [ "_" ] ] ] .What a number holds before a decimal digit other than 0 to 9: its digits so far, its point, and its exponent’s letter and sign.
Used in refused-number.
foreign-digit
foreign-digit = decimal-digit - digit .A decimal digit other than 0 to 9, such as the Arabic-Indic three ٣.
Used in refused-number.
string-literal
string-literal = '"' { character - ( '"' | "\" ) | escape-sequence } '"' .Text between double quotes; it may span lines.
unicode-literal
unicode-literal = "'" ( character - ( "'" | "\" ) | escape-sequence ) "'" .Exactly one character between single quotes.
Used in token, subrange-type, literal.
escape-sequence
escape-sequence = "\" ( "n" | "a" | "t" | "r" | "b" | "f" | "v" | "\" | '"' | "'" | "?" | "0" ) | hex-escape
| braced-unicode-escape | long-unicode-escape .A backslash and what it stands for: a control character, a quote, a backslash, or a character by its number.
Used in string-literal, unicode-literal.
hex-escape
hex-escape = "\x" hex-digit hex-digit .A byte by two hexadecimal digits.
Used in escape-sequence.
braced-unicode-escape
braced-unicode-escape = "\u{" hex-digit [ hex-digit [ hex-digit [ hex-digit [ hex-digit [ hex-digit ] ] ] ] ] "}" .A character by one to six hexadecimal digits between braces.
The number names a Unicode scalar value: at most U+10FFFF, and none of the surrogates U+D800 to U+DFFF.
Used in escape-sequence.
long-unicode-escape
long-unicode-escape = "\U" hex-digit hex-digit hex-digit hex-digit hex-digit hex-digit hex-digit hex-digit .A character by exactly eight hexadecimal digits.
The number names a Unicode scalar value, as in a braced escape.
Used in escape-sequence.
regex-literal
regex-literal = "0" ( "r" | "R" ) '"' (?! '"' ) regex-pattern '"' .A regular expression the compiler checks while it compiles the program, written 0r and its pattern in double quotes.
The pattern is written as it is matched: a backslash in it is a backslash of the pattern, and
\"is a quote.
refused-regex
refused-regex = "0" ( "r" | "R" ) ( (?! '"' ) | '"' { raw-character } [ '"' ] ) .A regex literal the scanner refuses: 0r with no quote behind it, a pattern that never closes, or a pattern the rules below do not admit.
The compiler refuses each of these by name, with scanner errors 1028, 1029, 1030, 1031, 1032, 1033, 1034, 1042 and 1043.
Used in token.
raw-character
raw-character = "\" character | character - ( '"' | "\" ) .A character of a pattern as the scanner collects it: a backslash takes the character behind it along.
Used in refused-regex.
regex-pattern
regex-pattern = regex-branch { "|" regex-branch } .A pattern: branches separated by bars, any of which may match.
Used in regex-literal, regex-atom.
regex-branch
regex-branch = { regex-piece } .A branch: pieces matched one after another; a branch may be empty.
Used in regex-pattern.
regex-piece
regex-piece = regex-atom { "*" | "+" | "?" } .A piece: an atom, repeated by * (any number), + (at least once) or ? (at most once).
Used in regex-branch.
regex-atom
regex-atom = "(" (?! "?=" | "?!" | "?<=" | "?<!" ) regex-pattern ")" | regex-class | "." | "^" | "$" | regex-escape
| character - ( "(" | ")" | "[" | "|" | "*" | "+" | "?" | "\" | '"' ) .An atom: a group, a class, any character, the start or end of the text, an escape, or a character that stands for itself.
Lookaround is not admitted: the engine matches in time linear in the text, and lookaround is what would break that.
Used in regex-piece.
regex-class
regex-class = "[" [ "^" ] ( regex-class-bracket | regex-class-member ) { regex-class-member } "]" .A class: the characters between brackets, or all others when ^ leads; ] stands for itself as the first member.
Used in regex-atom.
regex-class-bracket
regex-class-bracket = "]" ( (?! "-" ) | (?= "-]" ) ) | "]" "-" regex-class-character .A ] as the first member of a class, alone or as the low end of a range.
Used in regex-class.
regex-class-member
regex-class-member = "\" ( "d" | "w" | "s" ) | regex-property | regex-class-range
| regex-class-character ( (?! "-" ) | (?= "-]" ) ) .A member of a class: a class escape, a property, a range, or one character.
Used in regex-class.
regex-class-range
regex-class-range = regex-class-character "-" regex-class-character .A range of characters, its low end first.
Used in regex-class-member.
regex-class-character
regex-class-character = "\" ( "n" | "t" | "\" | "." | "|" | "(" | ")" | "[" | "]" | "*" | "+" | "?" | "^" | "$" | "-"
| '"' )
| character - ( "]" | "\" | '"' ) .A character of a class: an escape that names one character, or any character but ], a backslash and a quote.
Used in regex-class-bracket, regex-class-member, regex-class-range.
regex-escape
regex-escape = "\" ( "d" | "w" | "s" | "n" | "t" | "\" | "." | "|" | "(" | ")" | "[" | "]" | "*" | "+" | "?" | "^" | "$"
| "-" | '"' )
| regex-property .An escape outside a class: a digit, word or space class, a control character, or an operator character made literal.
A backslash before a digit would name a backreference, which is not admitted.
Used in regex-atom.
regex-property
regex-property = "\" ( "p" | "P" ) "{" regex-property-name "}" .The characters of one Unicode general category, or with \P all others.
Used in regex-class-member, regex-escape.
regex-property-name
regex-property-name = "C" | "L" | "M" | "N" | "P" | "S" | "Z" | "Cc" | "Cf" | "Co" | "Cs" | "Ll" | "Lm" | "Lo" | "Lt"
| "Lu" | "Mc" | "Me" | "Mn" | "Nd" | "Nl" | "No" | "Pc" | "Pd" | "Pe" | "Pf" | "Pi" | "Po" | "Ps"
| "Sc" | "Sk" | "Sm" | "So" | "Zl" | "Zp" | "Zs" .A Unicode general category: a family of one letter or a category of two.
Used in regex-property.
operator
operator = ":=" | ".." | "<=" | "≤" | ">=" | "≥" | "+" | "-" | "*" | "×" | "·" | "⊙" | "/" | "=" | "#" | "≠" | "<" | ">"
| "(" | ")" | "[" | "]" | "," | ":" | ";" | "." .The operators and punctuation; a formula glyph is the same token as its twin: ≠ is #, ≤ is <=, ≥ is >=, × is *.
Used in token.
refused-operator
refused-operator = "<>" | "==" | "!=" | "−" .The spellings another language uses and Mica refuses by name: <>, == and != for which Mica writes #, = and #, and the typeset minus sign U+2212, for which Mica writes -.
The compiler refuses each of these by name, with scanner errors 1039, 1040, 1041 and 1044.
Used in token.