Elvelt
コミュニティ

Backends

Function & Variable Hoisting

Function & Variable Hoisting

This document explains how CppBackend decides which names need to be forward-declared before they are used, and the precise rules for module-level vs function-level vs lambda-level hoisting. Hoisting is what makes self-referencing functions, mutual recursion, and var + function declaration order work in C++ (where order of declaration matters) the same way they do in JS (where declarations are hoisted to the top of the scope).

The main entry points are:

Function Location Purpose
collectFunctionScopedVars CppBackend.h:2557 Hoisting pass 1 — module-level callable vars + their arities
collectNestedNonCallableAssignments CppBackend.h:2720 Hoisting pass 2 — nested non-callable re-assignments
collectDirectLexicalValueVars ~line 2935 Hoisting pass 3 — let / const / var with non-callable initializers
pruneShadowedCallableHoists private Strip functionVars whose name is shadowed by a closer valueVars
collectDirectHoistedCallables private Promote function-decl names to functionVars
closureCaptureModeFor CppBackend.h:3822 For each captured name, decide [=] / [&] / cell-wrap
buildLambdaCaptureList CppBackend.h:4894 Assemble the final [=, &x, __this] string

Why hoisting is necessary

JavaScript hoists function declarations and var bindings to the top of their enclosing scope:

console.log(square(5)); // 25 — works because of hoisting
function square(x) { return x * x; }

C++ does not hoist function declarations — they must be declared (at least as a forward declaration) before their first use. The C++ backend therefore:

  1. Scans the module's top-level statements before emitting anything, building a set of names that must be forward-declared.
  2. Emits static mpl_value <name> = __mpl_undefined__; lines before the statement stream so that every reference resolves.
  3. Rewrites the original function foo() {} or const foo = … statements so they assign to the already-declared static, instead of declaring a new local.

This is what the three hoisting-fix commits (referenced throughout the source as b43b4af, 32f5694, 5a3f87b) collectively enable.

Pass 1: collectFunctionScopedVars

File: CppBackend.h:2557-2718

static void collectFunctionScopedVars(
    const std::shared_ptr<ir::BlockNode>& block,
    std::unordered_set<std::string>& valueVars,
    std::unordered_map<std::string, size_t>& functionVars);

This is the core hoisting scan. It walks the module-level block and produces two parallel structures:

  • valueVars — names bound to values (let x = 5, const x = obj, etc.) that need forward declaration.
  • functionVars — names bound to callables (function f(), const f = () => …, f = function() {}) that need forward declaration with arity recorded.

What it inspects

IR node Effect on hoisting
Lambda Returns immediately (line 2568) — lambda contents are NOT scanned. Lambdas are hoisted to their enclosing function scope, not module scope.
FunctionDecl / FuncDecl Adds name → arity to functionVars; recurses into fn->body so nested function declarations are also hoisted (line 2576-2585).
VariableDecl with callable initializer Adds name → arity to functionVars (line 2608-2610).
VariableDecl with non-callable initializer Adds name to valueVars (line 2611-2612).
VariableDecl whose name starts with __mpl_destr_src_ Skipped — synthetic destructuring source (line 2592-2596).
Assignment with callable RHS Adds target to functionVars (line 2634-2635).
Assignment with non-callable RHS Adds target to valueVars, removes from functionVars (line 2637-2640).
ArrayLiteral / ObjectLiteral / Block / If / For / ForEach / While / DoWhile / Switch / TryCatch Recurse into children.

The "recurse into fn->body" rule (line 2580-2584) is the b43b4af fix — it ensures nested function declarations inside other function bodies are also hoisted to the enclosing block in the generated C++. Without this, a function declaration inside function outer() would only be hoisted to outer's scope, and any code at the module level that referenced the inner name would fail to link.

Arity tracking

functionVars is a std::unordered_map<std::string, size_t> — the value is the parameter count. Arity is later used by:

  • proxyTrapWrapperExpr (line 4050) to generate correctly-arityed proxy trap helpers
  • buildLambdaCaptureList (line 4894) to decide whether to capture by [=] or [&, x]
  • functionArity_ cache (line 2395) for downstream lookups

Pass 2: collectNestedNonCallableAssignments

File: CppBackend.h:2720-2886

static void collectNestedNonCallableAssignments(
    const ir::EMPLNodePtr& node,
    std::unordered_set<std::string>& valueVars,
    std::unordered_map<std::string, size_t>& functionVars);

Walks inside function bodies to find assignments to hoisted names. The crucial rule:

  • If a name was hoisted as a callable at module scope (in functionVars), but a nested body assigns a non-callable to that name, the name must be moved from functionVars to valueVars (line 2734-2736). The arity information is lost because the static storage now holds a value, not a function.

Example:

let f = () => 1;       // hoisted as function (functionVars["f"] = 0)
function inner() {
    f = 42;            // overwrites with non-callable — must downgrade to valueVars
}

The 32f5694 fix introduced this pass specifically to handle the re-assignment case. Without it, f would be forward-declared as static mpl_value f = …; with no value type tracking, and the assignment would produce an unexpected mpl_call(f, …) call site (which used the old callable typing).

Pass 3: collectDirectLexicalValueVars

Promotes any remaining lexical let / const / var declarations to valueVars. The hoisted name ends up as a static mpl_value slot.

Pass 4: forward-declaration emission

File: CppBackend.h:2105-2129

After all four passes, the backend has:

  • moduleValueVars — names to declare as static mpl_value
  • moduleFunctionVars — names to declare as static mpl_value (with arity tracked)

The emission order is:

for (const auto& name : hoistedFunctionNames) {
    ss << "    static mpl_value " << name << " = __mpl_undefined__;\n";
}
for (const auto& name : moduleValueVars) {
    if (hoistedFunctionNames.count(name) == 0) {
        ss << "    static mpl_value " << name << " = __mpl_undefined__;\n";
    }
}
Note Where Why
Callables are declared first line 2105-2119 So that later value declarations may depend on callables (e.g. const FOO = f; references f).
mpl_value (not std::function) is used for hoisted callable storage line 2118 comment Avoids std::function constructor crashes with large lambdas — the comment says explicitly: "Use mpl_value for hoisted callable forward declarations (avoids std::function construction crashes with large lambdas)". This is the 5a3f87b fix.
__mpl_undefined__ initial value line 2118 Declared but not yet initialized. The actual definition comes later as an assignment in the emit-order stream.
Dedup via hoistedFunctionNames.count(name) == 0 line 2121 Prevents emitting the same name twice if it's in both sets.

After emission, the backend stores the merged set into hoistedVarNames_ (line 2126-2129) so generateStatement can recognize these names and emit assignments (name = …) instead of declarations (auto name = …).

Emit-order traversal

File: CppBackend.h:2188-2231

The next pass produces the actual call graph. moduleHoistedCallableIndices maps each hoisted callable name to its position in module->statements:

std::unordered_map<std::string, size_t> moduleHoistedCallableIndices;
for (size_t i = 0; i < module->statements.size(); ++i) {
    if (!isModuleHoistedCallableStatement(module->statements[i])) continue;
    const std::string name = moduleHoistedCallableName(module->statements[i]);
    if (!name.empty()) moduleHoistedCallableIndices[name] = i;
}

isModuleHoistedCallableStatement (line 2136-2164) returns true for:

  • FunctionDecl / FuncDecl (line 2140)
  • Assignment to an identifier with a callable RHS (line 2143-2154)
  • VariableDecl whose name is hoisted and whose initializer is callable (line 2155-2164)

Then the recursive emit:

std::function<void(size_t)> emitModuleStatement = [&](size_t index) {
    if (moduleStatementEmitted[index]) return;
    for (const auto& refName : collectIdentifierRefs(stmt)) {
        auto it = moduleHoistedCallableIndices.find(refName);
        if (it != moduleHoistedCallableIndices.end() && it->second > index) {
            emitModuleStatement(it->second);   // emit callee first
        }
    }
    ss << generateStatement(stmt, indent);
    moduleStatementEmitted[index] = true;
};
for (size_t i = 0; i < N; ++i) emitModuleStatement(i);

This is the standard "emit dependencies before use" topological traversal. Self-references (function f() { f(); }) are detected by checking whether the body references the function's own name (line 2077-2087) and adding the name to legacyForwardDecls (line 2084).

Hoisting scope levels

Scope Where it's emitted What it hoists
Module-level static mpl_value name = __mpl_undefined__; (line 2118, 2122) All module-level callables and values, before any module statement runs
Function-level static mpl_value name = __mpl_undefined__; inside a function body Function-declared names visible across multiple lambdas in the same function (managed by localHoistedValueVarsStack_, localHoistedFunctionVarsStack_)
Lambda-level Captured via C++ lambda capture list [=, &x, __this] Names referenced by the lambda body but bound in an enclosing function (managed by closureCaptureModeFor)
Class-level static mpl_value __mpl_js_class_value_X = __mpl_undefined__; (line 5004) Hoisted JS class expression values

Local hoisting stacks (function-level)

For function-scoped hoisting, CppBackend maintains three parallel stacks (declared line 2412-2415):

std::vector<std::unordered_set<std::string>> localHoistedValueVarsStack_;
std::vector<std::unordered_map<std::string, size_t>> localHoistedFunctionVarsStack_;
std::vector<bool> localHoistedStaticStorageStack_;

When entering a function or lambda body, the backend pushes a fresh layer; when leaving, it pops. closureCaptureModeFor (line 3822-3884) consults these stacks to decide how a name should be captured.

A function-level hoist is emitted as static mpl_value name = __mpl_undefined__; inside the function body itself — this is the local equivalent of module-level hoisting.

Lambda capture list construction

File: CppBackend.h:4894-4972

buildLambdaCaptureList(node) returns a string like [=], [=, &x, __this], or [=, &x] depending on which names the lambda body references.

Algorithm:

  1. Walk the lambda body and collect referenced names (collectFreeIdentifierRefs).
  2. Collect locally bound names (parameters + let/const/var inside the body) — these are excluded from captures.
  3. For each hoisted name in hoistedVarNames_ (module-level), check if it's referenced and not locally bound → candidate for capture.
  4. For each name in localHoistedValueVarsStack_ and localHoistedFunctionVarsStack_ (function-level, walking outside-in), check if it's referenced and not locally bound → candidate for capture.
  5. For each candidate, ask closureCaptureModeFor(name):
    • ByRef → add &name to the capture list.
    • ByCell → don't add &name (the cell is captured by value via [=]); record the name in lastCapturedCellVars_ so the lambda body can emit auto& name = *__cell_name; aliases.
    • None → skip (the name has static storage and doesn't need explicit capture).
  6. If the lambda references this (bodyRefs.count("this") or "__v_this") and there is a usable this binding (hasAvailableThisBinding()), add __this to the captures.

The returned string is then placed inside […]. Examples:

Lambda body References Result
() => 42 none [=]
() => x (x is module-hoisted static) x [=] (static storage — closureCaptureModeFor(x) == None)
() => x (x is a local let) x [=, &x]
() => x (x is a cell-wrapped local) x [=] + body emits auto& x = *__cell_x;
() => this.prop inside a class method this [=, __this]

The result is always a string starting with [= — the backend never generates [&] alone (that would risk dangling references to stack-locals from earlier in the enclosing scope).

The three hoisting fixes

Throughout the source, three commits are referenced as the historical anchors of the current hoisting behavior. Their roles:

Commit What it fixed Where it lives now
b43b4af Recurse into fn->body inside collectFunctionScopedVars so nested function declarations are hoisted to the enclosing block in C++ (not just to the outer function's local scope). CppBackend.h:2576-2584
32f5694 Added collectNestedNonCallableAssignments to handle the "module-hoisted callable reassigned to a non-callable inside a nested scope" case (downgrade functionVars to valueVars). CppBackend.h:2720-2886
5a3f87b Switched hoisted callable forward-declaration from std::function to mpl_value to avoid std::function construction crashes with large lambdas. CppBackend.h:2118 comment

Together, they enable the following JS programs to lower correctly:

// 1. Nested function declarations (b43b4af)
function outer() { function inner() { return 1; } return inner(); }
inner();  // hoisted to module scope

// 2. Reassignment (32f5694)
let f = () => 1;
function reset() { f = 42; }
f();        // calls the lambda
f = 42;     // overwrites with value
f();        // TypeError at runtime — but the C++ storage is correctly value-typed

// 3. Large lambdas (5a3f87b)
const Big = () => { /* 200-line function body */ };
Big();

Without these fixes, the corresponding C++ would either fail to compile (inner undeclared), call the wrong overload (f() treated as mpl_call(f)), or crash in std::function's constructor for the large lambda.

Cross-references