MLIR for Lox: Part 11 — Two Files, One Program — Cross-Module Linking in MLIR

Every program in this tutorial has been a single file. print 1 + 2; — parse it, generate MLIR, JIT it, done. Real programs don’t work that way. You write math.lox with a square function, then main.lox calls square(4). Two files. Two modules. One program.

This part shows how that works in MLIR. The good news: MLIR was built for this. The symbol model, the module structure, the JIT’s symbol resolver — they all support multiple modules out of the box. The trick is understanding how the pieces connect.

The Problem

Here’s our goal. We have two Lox files:

Note on function signatures: Part 6 uses func.func @main() with no return type (the GC-aware model with numbers-only values). Part 7 uses func.func @main() -> !llvm.struct<(i8, i64)> (tagged-union model). This part uses func.func @main() -> i32 because a multi-file compiler needs a well-defined exit status — the i32 return value is the process exit code.

The IR examples in this part use the numbers-only model (f64 arithmetic) for clarity — the linking concepts are the same regardless of whether you’re passing raw floats or tagged (i8, i64) pairs. In the tagged-union model, every function call passes (i8, i64) pairs instead of bare f64 values, and @lox_print takes two arguments instead of one. The linking, mangling, and merging steps don’t change — only the value representation does.

// math.lox
fun square(n) {
  return n * n;
}

fun add(a, b) {
  return a + b;
}

// main.lox
print square(3);

When we compile main.lox, the code generator produces:

module {
  func.func private @lox_print(f64)

  func.func @main() -> i32 {
    // ... compute square(3) ...
    func.call @square(%three) : (f64) -> f64
    // ... print the result ...
    %zero = arith.constant 0 : i32
    func.return %zero : i32
  }
}

The JIT tries to resolve @square and fails — there’s no definition. We never compiled math.lox. The program crashes with a symbol-not-found error.

We need two things:

Compile each file to its own MLIR module. math.lox becomes a module with @square and @add, main.lox becomes a module with @main.

Link the modules together. When main.lox calls @square, the JIT needs to find it in math.lox’s module.

Let’s start with how MLIR thinks about symbols.

How MLIR Symbols Work

MLIR has a built-in symbol system. It works like you’d expect if you’ve used linkers before, but it’s worth spelling out because MLIR’s version has some specific rules.

Symbol Tables and Visibility

Every builtin.module has a symbol table. Operations that live in a module’s symbol table — func.func, llvm.mlir.global, llvm.func — are symbols. They have a name (like @square, @lox_print, or @main) and a visibility — public, private, or nested.

module {
  func.func @main() -> i32 { ... }       // public by default
  func.func private @helper() -> f64 { ... }  // private — not visible outside this module
}

The default visibility is public — any module can reference a public symbol. private means “this module only.” We also use private for runtime declarations like func.func private @lox_print(f64) — but that’s a different use: it’s a declaration (no body), not a private implementation. The JIT resolves @lox_print from registered host symbols, not from other Lox modules. We’ll explain the distinction shortly.

Why private for runtime declarations? A func.func private @lox_print(f64) with no body is a declaration — it says “this symbol exists, but I’m not defining it here.” The JIT resolves it from the registered runtime symbols. We use private to signal that these are implementation details, not part of the Lox program’s public API. After merging, all functions end up in the same module, so private doesn’t restrict access — any function in the merged module can call @lox_print. (MLIR always allows declarations without definitions, regardless of visibility — the error would come from the JIT at runtime, not the verifier.)

Nested Modules

MLIR allows modules inside modules:

module {
  module @math {
    func.func @square(%arg0: f64) -> f64 { ... }
    func.func @add(%arg0: f64, %arg1: f64) -> f64 { ... }
  }
  module @main_mod {
    func.func @main() -> i32 {
      // This CANNOT directly call @math::@square
      // Cross-module references use symbol references
    }
  }
}

Nested modules have their own symbol tables. A function in @main_mod can’t write func.call @square(...) directly — that name only exists in @math’s symbol table, not @main_mod’s.

You can reference symbols across modules using MLIR’s @module::@symbol syntax, but in practice, most compilers take a simpler approach: separate modules, link at JIT time.

That’s the approach we’ll use. Each Lox file compiles to a standalone builtin.module. The JIT links them together when it resolves symbols. This mirrors how real compilers work — each .c file compiles to a .o file, and the linker connects them.

Compiling Multiple Files

The compilation pipeline needs one change: instead of compiling one file, we compile a list of files, then feed all the resulting modules to the JIT.

The Multi-File Compiler

#![allow(unused)]
fn main() {
// src/compiler.rs

use anyhow::Result;
use melior::{
    ir::Module,
    Context,
};
use std::path::Path;

/// Compile a Lox source file into an MLIR module.
///
/// This is the same compilation pipeline from Parts 1–11. The only
/// difference is that we return a Module instead of immediately
/// running it through the JIT.
fn compile_file(context: &Context, source: &str, filename: &str) -> Result<Module<'_>> {
    let tokens = lexer::tokenize(source);
    let ast = parser::parse(tokens)?;
    let module = codegen::generate(context, &ast, filename)?;
    Ok(module)
}

/// Compile multiple Lox files and return all MLIR modules.
///
/// Each file gets its own module. All modules are later merged into one
/// before JIT execution (see `link_and_run`), so compilation order doesn't
/// affect symbol resolution — every symbol ends up in the same table
/// regardless of which module was compiled first. Order *would* matter if
/// we used per-module JIT linking (ORC multi-JITDylib) instead of merging.
pub fn compile_files(
    context: &Context,
    files: &[(String, String)],  // (filename, source)
) -> Result<Vec<Module<'_>>> {
    let mut modules = Vec::new();

    for (filename, source) in files {
        let module = compile_file(context, source, filename)?;
        modules.push(module);
    }

    Ok(modules)
}
}

We compile each file independently and collect the modules. The key question is what each module looks like.

What `math.lox` Compiles To

module {
  func.func @square(%arg0: f64) -> f64 {
    %result = arith.mulf %arg0, %arg0 : f64
    func.return %result : f64
  }

  func.func @add(%arg0: f64, %arg1: f64) -> f64 {
    %result = arith.addf %arg0, %arg1 : f64
    func.return %result : f64
  }
}

Both functions are public (default visibility). No @main function — this module is a library.

What `main.lox` Compiles To

module {
  func.func private @lox_print(f64)

  func.func @main() -> i32 {
    %three = arith.constant 3.0 : f64
    %result = func.call @square(%three) : (f64) -> f64
    // ... call lox_print with result ...
    %zero = arith.constant 0 : i32
    func.return %zero : i32
  }
}

@main calls @square — but @square isn’t defined in this module. In single-file mode, this would fail verification. In multi-file mode, the JIT resolves it from the other module after merging.

This is a simplification. A real compiler would verify that @square exists somewhere in the linked modules before running the JIT. We’re relying on the JIT’s symbol resolver to catch missing definitions at runtime. For a tutorial, this is fine — the error message is clear (“symbol not found: @square”). A production compiler would want a separate linking/verification step.

Linking at JIT Time

MLIR’s JIT uses LLVM’s ORC JIT underneath. ORC maintains a symbol table — when the JIT compiles @main and encounters func.call @square(...), it looks up @square in the symbol table. If it finds it (because we already compiled @math’s module), the call works. If not, symbol-not-found.

The linking strategy is straightforward: compile dependency modules first, then the main module.

#![allow(unused)]
fn main() {
// src/jit.rs

use anyhow::{Result, anyhow};
use melior::{
    execution_engine::ExecutionEngine,
    ir::Module,
    Context,
};
use melior::ir::RegionLike;

/// Link multiple MLIR modules and execute @main.
///
/// The approach: merge all modules into one, then JIT the merged module.
/// After merging, every `func.call` has its target in the same symbol
/// table — the JIT resolves symbols the same way it does for a
/// single-file program.
///
/// A more sophisticated approach would use ORC's multi-JITDylib API
/// to add each module to the JIT separately and let the symbol
/// resolver connect them. Merging is simpler and works for our use case.
///
/// This is the same pattern as object-file linking: compile .c files
/// to .o files, then link them into a single executable. Our merge step
/// is the "link" — we do it at the MLIR level, not the object-file level.
pub fn link_and_run(modules: Vec<Module<'_>>, context: &Context) -> Result<()> {
    // We need one execution engine that all modules share.
    // The first module creates the engine; subsequent modules are
    // added to the same JIT session.
    //
    // Note: Melior's ExecutionEngine API varies between versions.
    // In Melior 0.27, you typically:
    //   1. Create the engine from the first module
    //   2. Register runtime symbols (lox_print, lox_clock, etc.)
    //   3. Invoke @main

    // For multi-module support, we merge all modules into one
    // before creating the engine. This is simpler than managing
    // multiple JIT sessions and works for our use case.

    let merged = merge_modules(&modules, context);

    let engine = ExecutionEngine::new(&merged, 2, &[], false, false);
    //                               module  opt   libs  dump  pic ^
    //
    // ExecutionEngine::new takes five parameters in Melior 0.27:
    //   - module: the MLIR module to JIT
    //   - optimization_level: 0, 1, or 2
    //   - shared_library_paths: paths to shared libraries for the JIT
    //   - enable_object_dump: whether to allow dumping to object files
    //   - enable_pic: whether to enable position-independent code
    //
    // The API changes between Melior versions — verify against your version's docs.
    unsafe { register_runtime_symbols(&engine); }

    // Invoke @main
    //
    // The MLIR declares @main() -> i32 (exit status), but invoke_packed
    // here uses &mut [] (no return buffer). This means we don't capture
    // the exit code. A production compiler would pass &mut [0i64] to
    // capture the i32 return value. For this tutorial, success/failure
    // is enough.
    let result = unsafe {
        engine.invoke_packed("main", &mut [])
    };

    result.map_err(|e| anyhow!("JIT execution failed: {:?}", e))?;
    Ok(())
}
}

The register_runtime_symbols function wraps the individual register_symbol calls we showed in Part 9. It registers each runtime function that the JIT needs to resolve — lox_print, gc_push_frame, gc_pop_frame, and the allocator — so that func.call @lox_print(...) in our MLIR can find the actual Rust implementation at execution time:

#![allow(unused)]
fn main() {
/// Register runtime symbols with the execution engine.
///
/// Each `register_symbol` call maps a symbol name (like "lox_print")
/// to a Rust function pointer. When the JIT encounters
/// `func.call @lox_print`, it looks up the symbol here. Without
/// this registration, the JIT throws a symbol-not-found error at
/// runtime.
///
/// In Melior 0.27, the method is `register_symbol` and it takes
/// `*mut ()` — a raw mutable pointer. The `unsafe` block is
/// required because registering a symbol makes the pointer
/// accessible to JIT'd code. If the pointer is invalid or
/// misaligned, this is undefined behavior.
///
/// See Part 9 (Standard Library and Runtime) for the full details
/// on each runtime function and how the wrapper types work.
unsafe fn register_runtime_symbols(engine: &ExecutionEngine) {
    engine.register_symbol("lox_print", lox_print_wrapper as *mut ());
    engine.register_symbol("gc_push_frame", gc_push_frame_wrapper as *mut ());
    engine.register_symbol("gc_pop_frame", gc_pop_frame_wrapper as *mut ());
    engine.register_symbol("lox_runtime_alloc", lox_runtime_alloc_wrapper as *mut ());
}
}

The wrapper functions (lox_print_wrapper, etc.) are thin extern "C" functions that bridge between the LLVM calling convention and our Rust runtime. Part 9 shows how to define them. The key point for cross-module linking: every module’s runtime calls resolve through the same symbol table. Whether @lox_print appears in math.lox’s MLIR or main.lox’s MLIR, it finds the same lox_print_wrapper — because the merged module shares one engine, one symbol table.

Merging Modules

The simplest linking approach: merge all modules into one. Take every func.func from every module and put them in a single builtin.module:

#![allow(unused)]
fn main() {
use std::collections::HashSet;

/// Merge multiple MLIR modules into one.
///
/// This is the "poor man's linker" — we concatenate all the function
/// definitions into a single module. The MLIR verifier is happy because
/// every symbol reference now has a definition in the same symbol table.
///
/// A production compiler might use MLIR's symbol resolution or ORC's
/// proper multi-JITDylib linking. For a tutorial, merging is simpler
/// and achieves the same result.
fn merge_modules(modules: &[Module<'_>], context: &Context) -> Module<'_> {
    // Note: Location is from melior::ir::Location
    let merged = Module::new(Location::unknown(context));

    // Track which symbols we've already added to the merged module.
    // MLIR's verifier rejects duplicate symbols — even duplicate
    // *declarations* — in the same module. Every source module
    // has its own @lox_print declaration, so we'd get duplicates
    // if we blindly appended everything.
    let mut seen_symbols: HashSet<String> = HashSet::new();

    for module in modules {
        // Module::body() returns a Region, and iterating a Region
        // yields Blocks, not Operations. A builtin.module always has
        // exactly one block in its body region — we iterate the
        // operations *within* that block.
        for op in module.body().first_block().unwrap().iter() {
            // Check if this operation defines a symbol.
            // In Melior, OperationLike::name() returns the operation's
            // dialect.name (e.g., "func.func"). If the operation has a
            // symbol name attribute, we read it via
            //   op.as_operation().attribute("sym_name")
            // which returns an Attribute containing the @-prefixed name
            // like "@lox_print". We strip the @ to get the bare name
            // for deduplication.
            //
            // Not all operations in a module body are symbols — but
            // in practice, the only operations at module level in our
            // compiler are func.func definitions and declarations.
            // Both have sym_name attributes.
            if let Some(sym_attr) = op.as_operation().attribute("sym_name") {
                let sym_name = sym_attr.to_string();
                // sym_name comes back as "@lox_print" or "@main" etc.
                // Strip the @ for the HashSet key.
                let bare_name = sym_name.trim_start_matches('@');
                if seen_symbols.contains(bare_name) {
                    // Skip this operation — we already have a symbol
                    // with this name in the merged module.
                    //
                    // This is correct for declarations (they're interchangeable)
                    // and for definitions (name mangling should prevent duplicate
                    // definitions — see the "Duplicate Symbol Errors" section).
                    continue;
                }
                seen_symbols.insert(bare_name.to_string());
            }

            // Operation::clone() creates a deep copy (via mlirOperationClone)
            // so we can insert the copy into the merged module. The original
            // stays in the source module. This is safe but not free — each
            // clone allocates a new MLIR operation. For a tutorial compiler,
            // this is fine.
            merged.body().append_operation(op.clone());
        }
    }

    merged
}
}

After merging, our two modules become one:

module {
  // From math.lox:
  func.func @square(%arg0: f64) -> f64 {
    %result = arith.mulf %arg0, %arg0 : f64
    func.return %result : f64
  }
  func.func @add(%arg0: f64, %arg1: f64) -> f64 {
    %result = arith.addf %arg0, %arg1 : f64
    func.return %result : f64
  }

  // From main.lox:
  func.func private @lox_print(f64)
  func.func @main() -> i32 {
    %three = arith.constant 3.0 : f64
    %result = func.call @square(%three) : (f64) -> f64
    // ... print result ...
    %zero = arith.constant 0 : i32
    func.return %zero : i32
  }
}

Now @main’s call to @square resolves — the definition is in the same symbol table. The JIT compiles this merged module and everything works.

About private declarations after merging. The @lox_print declaration is marked private — a hint that these are implementation details, not part of the Lox program’s public API. After merging, all functions are in the same module, so private doesn’t restrict access.

Before merging, each source module has its own @lox_print declaration. MLIR allows the same declaration to appear in different modules — having func.func private @lox_print(f64) in both math.lox’s module and main.lox’s module is fine because they’re in separate symbol tables.

After merging, those separate symbol tables become one. MLIR’s verifier rejects duplicate symbols in a single module — even duplicate declarations. That’s why merge_modules uses a HashSet to track which symbols it has already added. The second @lox_print declaration is silently skipped — the first one is sufficient, and the JIT resolves it from the registered runtime symbols.

For runtime declarations that every module needs, public visibility would also work — we use private purely as a signal to the reader.

Handling Name Collisions

What happens when two files define a function with the same name?

// utils.lox
fun process(x) {
  return x + 1;
}

// main.lox
fun process(x) {
  return x * 2;
}

print process(3);

Both files define @process. After merging, we’d have two func.func @process definitions in the same module — MLIR’s verifier rejects this. Duplicate symbols.

Real compilers solve this with name mangling — each source file’s symbols get a unique prefix:

module {
  func.func @utils__process(%arg0: f64) -> f64 { ... }
  func.func @main__process(%arg0: f64) -> f64 { ... }
  func.func @main() -> i32 {
    // Calls main's process, not utils's
    func.call @main__process(%three) : (f64) -> f64
  }
}

The mangling scheme is up to you. Common patterns:

Scheme	Example	Pros	Cons
File prefix	`@math__square`	Simple, readable	Doesn’t handle nested modules
Encoded (C++ Itanium)	`@_Z6square`	Handles all cases	Unreadable, C++ ABI baggage
Path-based	`@src_math_square`	Unambiguous	Verbose

For Lox, where we don’t have namespaces or imports, file-prefix mangling is the right starting point:

#![allow(unused)]
fn main() {
use std::path::Path;

/// Mangle a function name with its source file.
///
/// "square" from "math.lox" becomes "math__square".
/// "main" from any file stays "main" (the JIT entry point).
/// MLIR's printer adds the `@` prefix when rendering the IR — the function
/// returns bare names because `FlatSymbolRefAttribute` expects them without `@`.
fn mangle_name(function_name: &str, source_file: &str) -> String {
    if function_name == "main" {
        return "main".to_string();  // Entry point is always @main in the rendered IR
    }
    // Strip extension and path, use only the base filename.
    // This means src/math.lox and test/math.lox would produce the same
    // mangled name — fine for a flat directory of Lox files, but a real
    // compiler would need to include the directory path to avoid collisions.
    let file_stem = Path::new(source_file)
        .file_stem()
        .and_then(|s| s.to_str())
        .unwrap_or("unknown");

    format!("{}__{}", file_stem, function_name)
}
}

⚠️ The main name is reserved. If a Lox program defines fun main() { ... }, it collides with the compiler-generated @main entry point. The mangling function silently lets this happen — it returns "main" for any function named main, regardless of the source file. The collect_symbols function catches this at compile time: it reports a duplicate function error ("duplicate function 'main': defined in both 'main.lox' and 'math.lox'") if two files define main. A production compiler would either reserve the name (and report a user error if the Lox program tries to define it), or use a different entry point convention (e.g., @__lox_main).

When main.lox calls square(3), the code generator needs to know which file defines square. This is the symbol resolution problem — and it’s where things get interesting.

Symbol Resolution: How `main.lox` Finds `square`

In Lox, there’s no import statement. Every function is visible to every other function in the same program. This is the same as C’s model: all functions share a single global namespace.

When the code generator for main.lox encounters square(3), it needs to emit func.call @math__square(...). But at compile time, main.lox doesn’t know that square lives in math.lox. We need a resolution step.

Two-Pass Approach

First pass — collect declarations. Scan all source files. For each function, record its name and source file. Build a symbol table:

#![allow(unused)]
fn main() {
use anyhow::{Result, anyhow};
use std::collections::HashMap;
use std::path::Path;

use crate::ast::Stmt;

/// A resolved symbol — a function name plus the file it's defined in.
struct ResolvedSymbol {
    mangled_name: String,   // "math__square" (no @ prefix — FlatSymbolRefAttribute adds it)
    source_file: String,    // "math.lox"
    original_name: String,  // "square"
}

/// Build a symbol table from all source files.
///
/// This is the compiler's "header" phase — we don't compile anything,
/// we learn what functions exist and where they live.
///
/// We parse each file twice — once here to collect symbols, once in
/// compile_file_with_symbols to generate code. A production compiler
/// would cache the AST from the first pass instead of reparsing.
fn collect_symbols(files: &[(String, String)]) -> Result<HashMap<String, ResolvedSymbol>> {
    let mut symbols = HashMap::new();

    for (filename, source) in files {
        let tokens = lexer::tokenize(source);
        let ast = parser::parse(tokens)?;

        // Walk the top-level statements for function declarations.
        // We only need function names — no bodies, no type info.
        for stmt in &ast {
            if let Stmt::Function(f) = stmt {
                // Duplicate check omitted for clarity — see the
                // "Duplicate Symbol Errors" section below for the
                // version that reports duplicate function names.
                let name = &f.name;
                let mangled = mangle_name(name, filename);
                symbols.insert(name.clone(), ResolvedSymbol {
                    mangled_name: mangled,
                    source_file: filename.clone(),
                    original_name: name.clone(),
                });
            }
        }
    }

    Ok(symbols)
}
}

Second pass — compile with resolved names. Each call to square(3) becomes func.call @math__square(%three) : (f64) -> f64 because the code generator knows square is in math.lox.

#![allow(unused)]
fn main() {
use anyhow::Result;

/// Compile all files with symbol resolution.
///
/// The flow:
///   1. Collect symbols from all files (what functions exist, where)
///   2. Compile each file, using the symbol table to resolve call targets
///   3. Merge all modules and run
pub fn compile_and_run(files: &[(String, String)]) -> Result<()> {
    let context = Context::new();

    // Step 1: Collect symbols
    let symbols = collect_symbols(files)?;

    // Step 2: Compile each file with symbol resolution
    let mut modules = Vec::new();
    for (filename, source) in files {
        let module = compile_file_with_symbols(&context, source, filename, &symbols)?;
        modules.push(module);
    }

    // Step 3: Merge and run
    link_and_run(modules, &context)
}
}

The code generator needs one change: when emitting a func.call, look up the function name in the symbol table instead of using the raw name.

#![allow(unused)]
fn main() {
// Before (single-file mode):
fn emit_call(&self, name: &str, args: &[Value<'c, '_>], block: &Block<'c>) -> Value<'c, '_> {
    let callee = FlatSymbolRefAttribute::new(self.context, name);
    // func.call @square(...)
}

// After (multi-file mode) — emit_call uses self.symbols from the CodeGenerator struct:
fn emit_call(&self, name: &str, args: &[Value<'c, '_>], block: &Block<'c>) -> Value<'c, '_> {
    // mangle_name returns names WITHOUT the @ prefix —
    // FlatSymbolRefAttribute::new expects bare names like "math__square",
    // and MLIR's printer adds the @ when rendering the IR.
    let resolved = self.symbols
        .and_then(|sym| sym.get(name))
        .map(|s| s.mangled_name.as_str())
        .unwrap_or(name);  // Safe in single-file mode (symbols is None,
                           // so and_then short-circuits). In multi-file mode,
                           // a missing entry means the function wasn't found
                           // in any compiled file — the JIT will throw a
                           // symbol-not-found error at runtime.
    let callee = FlatSymbolRefAttribute::new(self.context, resolved);
    // func.call @math__square(...)
}
}

That’s it. The code generator emits mangled names, the merged module has mangled definitions, and everything lines up.

The `compile_file_with_symbols` Function

The compile_and_run function above calls compile_file_with_symbols, which is the same as compile_file from earlier but passes the symbol table to the code generator:

#![allow(unused)]
fn main() {
use std::collections::HashMap;
use anyhow::Result;

/// Compile a single Lox source file with symbol resolution.
///
/// Same as compile_file, but the code generator uses the symbol
/// table to resolve function names to their mangled equivalents.
fn compile_file_with_symbols<'c>(
    context: &'c Context,
    source: &str,
    filename: &str,
    symbols: &HashMap<String, ResolvedSymbol>,
) -> Result<Module<'c>> {
    let tokens = lexer::tokenize(source);
    let ast = parser::parse(tokens)?;
    let module = codegen::generate_with_symbols(context, &ast, filename, symbols)?;
    Ok(module)
}
}

The only difference from compile_file: the code generator receives the symbol table and uses it in emit_call. Here’s how the CodeGenerator struct changes:

#![allow(unused)]
fn main() {
use std::collections::HashMap;

/// The code generator — unchanged from earlier parts, except for
/// the new `symbols` field for multi-file compilation.
struct CodeGenerator<'c, 'a> {
    context: &'c Context,
    symbols: Option<&'a HashMap<String, ResolvedSymbol>>,
    // ... other fields from earlier parts ...
}

impl<'c, 'a> CodeGenerator<'c, 'a> {
    /// Create a code generator for single-file mode (no symbol resolution).
    fn new(context: &'c Context) -> Self {
        Self { context, symbols: None }
    }

    /// Create a code generator with symbol resolution for multi-file mode.
    fn with_symbols(context: &'c Context, symbols: &'a HashMap<String, ResolvedSymbol>) -> Self {
        Self { context, symbols: Some(symbols) }
    }
}

/// The public entry point: compile an AST to MLIR with optional symbol resolution.
pub fn generate_with_symbols<'c>(
    context: &'c Context,
    ast: &[Statement],
    filename: &str,
    symbols: &HashMap<String, ResolvedSymbol>,
) -> Result<Module<'c>> {
    let mut generator = CodeGenerator::with_symbols(context, symbols);
    generator.compile(ast, filename)
}

/// Single-file mode: same as before, no symbol table.
pub fn generate<'c>(
    context: &'c Context,
    ast: &[Statement],
    filename: &str,
) -> Result<Module<'c>> {
    let mut generator = CodeGenerator::new(context);
    generator.compile(ast, filename)
}
}

The emit_call method checks self.symbols — when it’s Some, it resolves the function name through the symbol table. When it’s None (single-file mode), it falls back to the raw function name. What happens if self.symbols is Some but the function name isn’t in the table? The unwrap_or(name) fallback produces the raw, unmangled name — which won’t match any symbol in the merged module. The JIT will throw a symbol-not-found error at runtime. This is intentional: a missing symbol table entry in multi-file mode means the function wasn’t defined in any compiled file, and failing at JIT time with a clear error is better than silently producing incorrect code. If you’d rather catch this at compile time, replace the unwrap_or with expect or return a Result.

The Complete Flow

Let’s trace through our example end-to-end.

Input files:

// math.lox
fun square(n) { return n * n; }
fun add(a, b) { return a + b; }

// main.lox
print square(3);

Step 1: Collect symbols

"square" → ResolvedSymbol { mangled: "math__square", file: "math.lox" }
"add"    → ResolvedSymbol { mangled: "math__add",    file: "math.lox" }

Step 2: Compile each file

math.lox →:

module {
  func.func @math__square(%arg0: f64) -> f64 {
    %result = arith.mulf %arg0, %arg0 : f64
    func.return %result : f64
  }
  func.func @math__add(%arg0: f64, %arg1: f64) -> f64 {
    %result = arith.addf %arg0, %arg1 : f64
    func.return %result : f64
  }
}

main.lox →:

module {
  func.func private @lox_print(f64)
  func.func @main() -> i32 {
    %three = arith.constant 3.0 : f64
    %result = func.call @math__square(%three) : (f64) -> f64
    func.call @lox_print(%result) : (f64) -> ()
    %zero = arith.constant 0 : i32
    func.return %zero : i32
  }
}

Step 3: Merge and run

The merged module has @math__square, @math__add, @lox_print, and @main. The JIT resolves all symbols. Output: 9.

What About the Garbage Collector?

One thing to watch: the GC’s shadow stack. Each function pushes a frame on entry and pops it on exit. When main.lox calls math__square, the call crosses module boundaries — but the GC doesn’t care about modules. It cares about the call stack.

The shadow stack operations (lox.push_frame, lox.pop_frame) are already in each function’s MLIR. After lowering, they become func.call @gc_push_frame(...) and func.call @gc_pop_frame(...) — the connection between the Lox dialect operations and the lowered runtime calls is covered in Part 4’s lowering pipeline. These resolve to the same global RUNTIME we set up in Part 9. Cross-module calls work the same as intra-module calls — the GC frame management is per-function, not per-module.

func.func @math__square(%arg0: f64) -> f64 {
  %frame = func.call @gc_push_frame(%root_count) : (i32) -> !llvm.ptr
  // ... compute n * n ...
  func.call @gc_pop_frame() : () -> ()
  func.return %result : f64
}

func.func @main() -> i32 {
  %frame = func.call @gc_push_frame(%root_count) : (i32) -> !llvm.ptr
  %result = func.call @math__square(%three) : (f64) -> f64
  func.call @gc_pop_frame() : () -> ()
  // ...
}

gc_push_frame returns a roots array pointer (!llvm.ptr) — the same pointer the compiler uses with lox.set_root to store root values via GEP + store (see Part 4 for the lowering details). Even though math__square doesn’t need to root any values in this simplified example (its single f64 argument lives in a register), the function still pushes a frame — the GC needs to know about all live frames, even ones without roots, so it can walk the complete call stack during collection.

When @main calls @math__square, the runtime call stack looks like:

main → gc_push_frame
     → math__square → gc_push_frame
                     → gc_pop_frame  (square's frame removed)
     → gc_pop_frame  (main's frame removed)

Two separate frames on the shadow stack, two separate pops. The GC sees the full call chain. No special handling needed.

Duplicate Symbol Errors

What if two files define the same function? The symbol table catches it during the collection phase — here’s the collect_symbols function from earlier, with the duplicate check added:

#![allow(unused)]
fn main() {
use anyhow::{Result, anyhow};
use std::collections::HashMap;
use crate::ast::Stmt;

fn collect_symbols(files: &[(String, String)]) -> Result<HashMap<String, ResolvedSymbol>> {
    let mut symbols = HashMap::new();

    for (filename, source) in files {
        let tokens = lexer::tokenize(source);
        let ast = parser::parse(tokens)?;

        for stmt in &ast {
            if let Stmt::Function(f) = stmt {
                let name = &f.name;
                if let Some(existing) = symbols.get(name) {
                    return Err(anyhow!(
                        "duplicate function '{}': defined in both '{}' and '{}'",
                        name, existing.source_file, filename
                    ));
                }
                let mangled = mangle_name(name, filename);
                symbols.insert(name.clone(), ResolvedSymbol {
                    mangled_name: mangled,
                    source_file: filename.clone(),
                    original_name: name.clone(),
                });
            }
        }
    }

    Ok(symbols)
}
}

This is better than letting MLIR’s verifier catch it — we can report the error with file names and line numbers, not only “duplicate symbol @process.”

This is a simplification. Lox doesn’t have namespaces or a module system, so every function shares one global namespace. Real languages have import, pub, or visibility modifiers. The two-pass approach still works — you need richer symbol entries that carry visibility and namespace information alongside the name. The core idea is the same: collect names first, then compile with resolution.

What We Changed

The multi-file compiler adds three things on top of the single-file pipeline:

Symbol collection — scan all files for function names before compiling.

Name mangling — prefix each function with its source file to avoid collisions.

Module merging — combine all compiled modules into one before JIT execution.

Here’s the updated pipeline:

┌─────────────┐     ┌──────────────┐     ┌──────────────┐     ┌─────────┐
│ Source files │────▶│  Collect     │────▶│  Compile     │────▶│  Merge  │
│ *.lox        │     │  symbols     │     │  each file   │     │  modules│
└─────────────┘     └──────────────┘     └──────────────┘     └────┬────┘
                                                                  │
                                            ┌─────────────────────┘
                                            ▼
                                     ┌──────────────┐     ┌──────┐
                                     │  Lower + JIT │────▶│ Run  │
                                     └──────────────┘     └──────┘

The single-file pipeline was:

┌─────────────┐     ┌──────────────┐     ┌──────┐
│ Source file  │────▶│  Compile +   │────▶│ Run  │
│ *.lox        │     │  Lower + JIT │     │      │
└─────────────┘     └──────────────┘     └──────┘

Same destination, more steps. The extra steps are all about connecting symbols across files — the compilation within each file hasn’t changed.

Going Further

This is the last planned part of the tutorial. Here are directions you could take from here:

AOT compilation. We’ve used the JIT throughout, but MLIR supports ahead-of-time compilation too. The lowering pipeline already produces LLVM dialect MLIR — but you can’t pipe that directly to llc. First, you need to translate LLVM dialect MLIR to actual LLVM IR using mlir-translate --mlir-to-llvmir (or the Melior translation API). Then compile the resulting LLVM IR with llc and link it into a standalone executable. The runtime symbols (lox_print, gc_push_frame, etc.) become ordinary C functions you link against. This is how a real Lox compiler would ship.

The command-line pipeline looks like this:

Before you start: mlir-opt and mlir-translate are command-line tools from the MLIR project. If you installed MLIR from source, they’re in your build directory. If you’re using Melior only, you may not have them — Melior embeds the MLIR C library but doesn’t ship the command-line tools. In that case, use the Melior API to run the passes and translation in-process.

On Ubuntu 24.04+, you can install them from the LLVM project’s apt repository: apt install mlir-22-tools (adjust the version number to match your LLVM). On macOS, brew install llvm includes them as llvm/mlir-opt and llvm/mlir-translate. If neither option works, build from source — the MLIR Getting Started guide has step-by-step instructions.

# 1. Lower the Lox dialect MLIR to LLVM dialect MLIR
#    (same passes we run in the JIT: scf-to-cf, cf-to-llvm, arith-to-llvm, func-to-llvm)
mlir-opt --convert-scf-to-cf --convert-cf-to-llvm \
  --convert-arith-to-llvm --convert-func-to-llvm \
  output.mlir -o lowered.mlir

# 2. Translate LLVM dialect MLIR to LLVM IR
mlir-translate --mlir-to-llvmir lowered.mlir -o output.ll

# 3. Compile LLVM IR to an object file
llc -filetype=obj output.ll -o output.o

# 4. Link with the runtime
clang output.o liblox_runtime.a -o lox_program

Or, if you’re using Melior’s Rust API instead of command-line tools, the melior::dialect::llvm::translate_module function handles step 2 programmatically. Steps 3–4 stay the same — llc and the linker are external tools regardless of how you produce the LLVM IR.

Optimization passes. We run the standard conversion passes (scf-to-cf, cf-to-llvm, arith-to-llvm) but no optimization passes. MLIR has a rich set: common subexpression elimination, dead code elimination, loop invariant code motion, and more. Adding -O1 or -O2 level passes between the Lox dialect and the lowering pipeline would make the generated code significantly faster.

Type inference. Part 10 inserted runtime tag checks before every arithmetic operation — scf.if branches that call a type error function when the tag isn’t TAG_NUMBER. That’s a guard, not an analysis. The next step is a type inference pass that eliminates unnecessary checks: if a branch already tested the tag (e.g., if type(x) == "number"), the code inside that branch doesn’t need to check again. This is a flow-sensitive analysis — the type of each variable depends on the control-flow path that reached it. A simple version tracks tag sets per variable per block; a more sophisticated version uses abstract interpretation to infer types across loop boundaries.

Closures across modules. Part 5 showed closures within a single file. With cross-module linking, closures can capture variables defined in other modules. The capture analysis needs to work across the symbol table — when main.lox’s closure captures a variable from utils.lox, the code generator needs to resolve the variable’s location through the symbol table, not only the local scope. This is harder than function-level symbol resolution: captured variables need upvalue indices that are stable across modules, and the GC needs to know about cross-module references for rooting. A production implementation might use a module-level symbol table for functions but need a separate mechanism (e.g., cross-module upvalue indices) for captured variables.

Debugger integration. Part 10’s Approach 2 explains how debug info flows through the compilation pipeline — Location → DILocation → DWARF → addr2line. The next step is integrating with an actual debugger: producing DWARF that GDB or LLDB can consume, adding variable descriptions (DILocalVariable) so the debugger can show Lox variable names and values, and ensuring the debug info survives optimization passes (which can reorder or eliminate operations, invalidating stale location data). This turns the compiler from something that produces error messages into something you can step through interactively.

This is the final part of the MLIR for Lox tutorial series. The Review & Response section covers known limitations and potential improvements.

Keyboard shortcuts

MLIR for Lox: A Compiler Tutorial