Python Frontend

The Python frontend is a direct subclass of plugin::ILanguageFrontend that implements its own line-based, indentation-sensitive parser. Python is the only whitespace-significant language in MPL_Compiler besides Ruby, but unlike Ruby, Python has been deeply elaborated with decorators, comprehensions, walrus operators, async/await, the match statement, and context managers.

File	Lines	Purpose
`frontends/python/PythonFrontend.h`	2,314	Frontend class with indent-stack scope tracker, full statement & expression parser
`frontends/python/PythonStdLib.h`	85	Inline `findStdLibMapping` table

Language registration

// PythonFrontend.h:19-20
std::string languageName() const override { return "python"; }
std::vector<std::string> fileExtensions() const override { return {".py"}; }

Only .py is recognised.

Parsing model

PythonFrontend::parse (PythonFrontend.h:22-33):

Preprocess: split source into LineInfo{indent, text} records (preprocess, line 165). Each leading tab counts as 4 spaces. Empty lines and comment lines (#…) are dropped.
Initialise state: scopeStack_.push_back({}) for module-level scope.
parseBlock(0) (line 1723): walks lines at indent 0 until line.indent < indent. Inner blocks use indent-strict descent.

State

// PythonFrontend.h:36-46
struct LineInfo { int indent = 0; std::string text; };
std::vector<LineInfo> lines_;
size_t index_ = 0;
std::vector<std::unordered_set<std::string>> scopeStack_;
std::vector<ir::EMPLNodePtr>* currentOut_ = nullptr;
int compCounter_ = 0;
int elseCounter_ = 0;

scopeStack_ tracks declared names per scope. currentOut_ points to the current statement list (used for nested for comprehensions). The two counters generate fresh identifiers for comprehensions and for/while-else.

Top-level statement kinds

parseBlock (PythonFrontend.h:1723-2310) recognises every Python top-level / block statement:

Statement	File:line	IR produced
`@decorator` (collected until next `def`/`class`)	1647-1704	`DecoratorEntry` list → applied via `emitDecoratorApplications`
`import module`	1755-1761	`ImportNode(module=…)`
`from module import a, b, *`	1763-1788	`ImportNode` with `bindings[]`
`pass`	1790-1793	(no-op)
`class Name(base): …`	1795-1842	`ClassDeclNode`, methods become `cls->methods`, `__init__` becomes `cls->constructor`
`with expr as var: …`	1844-1865	`TryCatchNode` with `tryBlock` containing `vdecl = expr`
`match subject: case pat: …` (PEP 634)	1867-1903	`SwitchNode` with `cases[]` and `defaultCase`
`try: … except E: … finally: …`	1905-1927	`TryCatchNode` with `tryBlock`, `catchBlocks[]`, `finallyBlock`
`if cond: … elif …: … else: …`	1929-1972	`IfNode` chain (with walrus support)
`def f(a, b=1, args, *kwargs): …`	1974-2040	`FunctionDeclNode` with parameters, decorator applications
`for var in iter: …` (with optional `else`)	2042-2152	`ForNode` (range special-cased)
`while cond: …` (with optional `else`)	2154-2197	`WhileNode` (with walrus support)
`return expr`	2199-2205	`ReturnNode`
`raise expr`	2207-2213	`ThrowNode`
`break`	2215-2218	`BreakNode`
`continue`	2220-2223	`ContinueNode`
`print(…)`, `len(…)`, etc. (bare calls)	2227-2249	`StandardLibraryCallNode`
`x = expr` / `x += expr` / `arr[i] = expr`	2251-2297	`VariableDeclNode` or `AssignmentNode`
Other	2299-2307	`UnsupportedNode` (or a `FunctionCallNode` if it parses as one)

Decorators

@logged
@logged_with("level")
def f(x):
    return x + 1

consumeDecoratorLines (PythonFrontend.h:1647) gathers every contiguous decorator line and emitDecoratorApplications emits the application sequence after the function declaration. The function gets metadata["decorators"] set with the comma-separated decorator expressions.

PEP 614 (@decorators on any expression) is supported.

Comprehensions

tryParseComprehension (PythonFrontend.h:487) and lowerComprehension (line 643) handle list/set/dict comprehensions and generator expressions. Examples:

squares = [x*x for x in range(10)]
evens   = [x for x in range(100) if x % 2 == 0]
pairs   = {(x, y) for x in a for y in b if x != y}
lookup  = {name: idx for idx, name in enumerate(names)}
total   = sum(x*x for x in range(10))   # generator expression

lowerComprehension builds a synthetic ForNode/ForEachNode IR and emits the value computation inside the innermost loop, then wraps the result in __mpl_make_list / __mpl_make_dict calls as appropriate.

Walrus operator `:=`

extractWalrus (PythonFrontend.h:235) and inline use sites in if (line 1929-1944) and while (line 2154-2184) extract name := expr from conditions:

if (n := len(items)) > 10:
    print(n)

In an if, the walrus declaration is emitted before the IfNode. In a while, the declaration is emitted before the WhileNode AND a re-assignment is appended to the body so the walrus is re-evaluated each iteration.

Type hints

Type annotations are type-erased at the IR level. The frontend simply ignores them; they don't affect codegen.

def add(a: int, b: int) -> int:
    return a + b

is parsed identically to:

def add(a, b):
    return a + b

TypedDict, Protocol, Generic[T], Optional[T], Union[A, B], etc. are all silently dropped.

Async / await

async def and await are parsed via parseExpr. The function becomes an FunctionDeclNode with metadata["async"] = "true". await expr inside the body becomes an AwaitNode wrapping the expression.

async def fetch_all(urls):
    results = await asyncio.gather(*(fetch(u) for u in urls))
    return results

Match statement (PEP 634)

match subject: case pat: … (line 1867-1903) lowers to SwitchNode. Patterns are currently restricted to literal patterns and the wildcard _. Class patterns and capture patterns are not yet supported (see limitations.md).

match cmd:
    case "quit":
        return
    case "go" | "run":
        start()
    case _:
        print("unknown")

| is parsed into multiple case entries (no fallthrough).

Context managers (`with`)

with expr as var: … (line 1844-1865) becomes a TryCatchNode whose tryBlock contains var = expr as the first statement. The __enter__/__exit__ calls are not yet auto-emitted (this is a known limitation; see limitations.md).

`try / except / finally`

try: … except E as e: … finally: … (line 1905-1927) lowers to TryCatchNode with:

tryBlock: body
catchBlocks: list of (exceptionClass, body) pairs
finallyBlock: optional body

Bare except: is permitted (catches BaseException).

`for` and `while … else`

for/while-else (PEP 3136) is supported via peekElseBlock (line 782) + instrumentBreaks (line 731) + emitForWhileElse (line 787). The body is wrapped so any break sets a flag, then the else body executes only when the loop completes without break.

for x in items:
    if x == target:
        found = True
        break
else:
    found = False

Standard library mapping (`PythonStdLib.h`)

The complete table at frontends/python/PythonStdLib.h:39-74:

Python	MPL module.function
`print(...)`	`Console.WriteLine(...)`
`input(...)`	`Console.ReadLine(...)`
`len(x)`	`Collection.Length(x)`
`abs(x)`	`Math.Abs(x)`
`round(x, n)`	`Math.Round(x, n)`
`min(...)`	`Math.Min(...)`
`max(...)`	`Math.Max(...)`
`int(x)`	`Convert.ToInt(x)`
`float(x)`	`Convert.ToFloat(x)`
`str(x)`	`Convert.ToString(x)`
`math.sqrt(x)`	`Math.Sqrt(x)`
`math.floor(x)`	`Math.Floor(x)`
`math.ceil(x)`	`Math.Ceil(x)`
`math.pow(x, y)`	`Math.Pow(x, y)`
`math.log(x)`	`Math.Log(x)`
`math.log2(x)`	`Math.Log2(x)`
`math.log10(x)`	`Math.Log10(x)`
`math.sin(x)`	`Math.Sin(x)`
`math.cos(x)`	`Math.Cos(x)`
`math.tan(x)`	`Math.Tan(x)`
`os.path.join(a, b)`	`IO.Path.Join(a, b)`
`os.path.dirname(p)`	`IO.Path.GetDirectoryName(p)`
`os.path.basename(p)`	`IO.Path.GetFileName(p)`
`os.path.exists(p)`	`IO.FileExists(p)`
`os.getcwd()`	`Environment.GetCurrentDirectory()`
`os.getenv(name)`	`Environment.GetVariable(name)`
`sys.exit(code)`	`Process.Exit(code)`
`json.loads(s)`	`Json.Parse(s)`
`json.dumps(obj)`	`Json.Stringify(obj)`

The frontend also emits StandardLibraryCallNodes for any bare function call whose name is in this table — parseBlock line 2227-2249.

Expression parsing (`parseExpr`)

parseExpr (PythonFrontend.h:1006) handles:

Operator / construct	Notes
`**` (power)	Right-associative
`+x -x ~x` (unary)
`* / // % @`	`@` is matrix multiplication
`+ -`
`<< >> &` `^` `\|`	bitwise
`in`, `not in`, `is`, `is not`, `<`, `<=`, `>`, `>=`, `==`, `!=`	comparisons
`not x`	boolean not
`and` / `or`	short-circuit
`lambda x: body`	`LambdaNode`
`x if cond else y`	ternary (`TernaryNode`)
`x := y` (inside expressions)	walrus
`f"Hello {name}"`	f-strings via `parseFString` (line 889)
`obj.attr` / `obj[i:j:k]`	member / slice
`obj(args)`	call
`obj.method(args)`	method call
`[...]` list, `{...}` dict/set, `(...)` tuple	literals
`x[i]`	index
`x[i:j]` / `x[i:j:k]`	slice → `buildSliceCall` (line 858)
`await x`	await
`yield x` / `yield from x`	generator
`args` / `*kwargs`	unpacking
Number, string, bytes, bool, None	literals

f-string parsing

parseFString (PythonFrontend.h:889-1005) supports:

f"Hello {name}" — simple substitution
f"{x + 1}" — expression substitution
f"{name=}" — debug expression (Python 3.8+)
f"{x:.2f}" — format spec
Triple-quoted f-strings
f"{{escaped}}" literal braces

Format specs are stored in metadata["format_spec"] for the C++ backend to honour.

IR examples

Example 1 — class with decorators and inheritance

@logged
@cached
class Counter(BaseCounter):
    def __init__(self, start=0):
        self.value = start

    def inc(self):
        self.value += 1
        return self.value

Generates:

ModuleNode {
  statements: [
    ClassDeclNode(name="Counter", metadata={ superclass="BaseCounter", decorators="logged,cached" })
      methods:
        FunctionDeclNode(name="__init__", isConstructor=true, parameters=[("auto","start")], body=…)
        FunctionDeclNode(name="inc", parameters=[], body=…)
  ]
}

Example 2 — comprehension + walrus

def find_first(items, target):
    if (idx := next((i for i, x in enumerate(items) if x == target), -1)) >= 0:
        return idx
    return -1

next(...) is not in the StdLib table, so it's emitted as a plain FunctionCallNode("next", …). The walrus declaration precedes the IfNode. The generator expression inside next is lowered to a synthetic for-loop.

Example 3 — async with match

async def handle(cmd):
    match cmd:
        case {"op": "echo", "msg": m}:
            print(m)
        case _:
            print("unknown")

async def sets metadata["async"]="true". match lowers to SwitchNode. Dict patterns are not yet recognised as patterns — {"op": "echo", "msg": m} parses as a literal-dict expression that won't match any SwitchNode case. The current implementation treats patterns literally, so this specific example would fall through to _. See limitations.md.

What is not supported (Python-specific gaps)

Dict / class / capture patterns in match (PEP 634)
Starred expressions inside patterns
Type alias statements (type X = ..., PEP 695)
Generic classes / methods (class Foo[T]: …)
Match guards (case X if cond:)
Exception groups (except*)
f-string conversion (!r, !s)
@override, @final decorators — accepted, ignored
Context manager protocol (__enter__ / __exit__ calls not emitted)
__slots__, descriptors, metaclasses