DOCSEARCHbyAlgolia
Submit Docs

Roc Compiler

repository·main·Indexed 26 days ago

https://github.com/roc-lang/roc

The Roc compiler and its LLVM backend, which converts monomorphized IR into native machine code. The project includes a multi-stage pipeline consisting of canonicalization, type checking via Hindley-Milner inference, and code generation. It features a base utility module, core builtins for runtime functionality, and a CLI for orchestration, testing, and profiling.

Tokens
94.5K
Snippets
211
Records
537
Agent score
91%

What's inside roc

  1. Overview of the LLVM Compile module

    main
    The LLVM Compile module is responsible for compiling LLVM bitcode into native shared libraries. These libraries are intended for JIT (Just-In-Time) evaluation. The module manages the pipeline from LLVM IR to native compilation and handles platform-specific linking, allowing callers to load the resulting library via dlopen and execute code using the roc_eval function.

  2. Overview of the Collections module

    main

    The collections module provides efficient, specialized data structures designed for use within the Roc compiler. These collections are optimized for compiler-specific operations such as managing symbol tables, dependency graphs, and intermediate representations. Key characteristics include:

    • Specialized Data Structures: Tailored for compiler-specific use cases.
    • Memory Efficiency: Designed to minimize memory overhead during the compilation process.
    • Performance: Optimized for fast access patterns common in compiler operations.
    • Type Safety: Generic collections that integrate with Roc's type system.

  3. Overview of the Base module

    main

    The base module serves as the foundational layer for the Roc compiler. It provides essential utilities, data structures, and abstractions used across multiple compiler stages. Key capabilities include:

    • Common Data Structures: Shared collections and structures used throughout the compiler.
    • Memory Management: Utilities for safe memory allocation and management.
    • Parallel Processing: Tools for parallel execution and work distribution.
    • Error Handling: Standardized error types and handling patterns.
    • Debugging Support: Logging and debugging utilities.
    • Platform Abstractions: Cross-platform abstractions for common operations.

  4. Overview of Compiler Improvement Projects

    main

    The projects/ directory contains self-contained specifications for structural improvements to the Roc compiler. These projects are categorized by scope and complexity:

    • small/: Localized, mostly additive checks or deletions with low design risk. Estimated effort: hours to days.
    • big/: Cross-cutting projects requiring design decisions before implementation. Estimated effort: weeks.

    Projects are derived from root-cause analyses of bug clusters and audits aimed at eliminating 're-derivation' (where facts proven during checking are re-derived downstream from fragile identity like name strings instead of being carried as explicit data). The ultimate goal is to ensure every project's 'finishing move' is deleting the old re-derivation path.

  5. Overview of the Parse module

    main
    The parse module is the initial stage of the Roc compiler pipeline. Its primary purpose is to transform raw Roc source code into a structured Abstract Syntax Tree (AST) through a process of tokenization and parsing. This AST serves as the foundation for all subsequent compiler stages, including canonicalization and type checking.

  6. Overview of the Canonicalize stage

    main
    The canonicalize module is the second stage of the Roc compiler pipeline. It transforms the Abstract Syntax Tree (AST) into Canonical Intermediate Representation (CIR) by performing semantic analysis of the program. This stage is responsible for resolving names and normalizing the AST into a representation that is optimized for the subsequent type-checking phase.

  7. Overview of the Snapshot Tool

    main
    The Snapshot Tool is used for snapshot testing the Roc compiler. It verifies the behavior of various compiler stages by generating "golden snapshot" files—baseline outputs that are considered correct. During testing, the tool compares current compiler outputs against these golden files; any discrepancies cause a test failure, helping to detect regressions and unintended changes.

  8. Overview of the Roc type system implementation

    main
    The types module is the core implementation of the Roc language's type system. It provides the foundational structures used by the compiler's canonicalize, check, and eval stages to ensure type safety. It manages the representation and manipulation of all Roc types, including primitives, algebraic data types (ADTs), functions, and type variables used during Hindley-Milner type inference.

  9. Overview of Roc Builtins

    main

    The builtins module provides the core runtime functionality required by every Roc program. It serves as the bridge between Roc code and the host platform, providing:

    • Core Data Types: Fundamental types such as strings, numbers, and basic collections.
    • Runtime Operations: Essential operations including memory allocation, string manipulation, and arithmetic.
    • Host Platform Interface: The interface for the ABI and system calls.
    • Standard Library: Essential functions that are always available in Roc programs.

  10. Understand the Diagnostic Rendering Architecture

    main

    The Roc diagnostic system uses a single document model (DocumentElement and Annotation) to represent errors and warnings. This model is rendered into four distinct targets via the RenderTarget abstraction in src/reporting/renderer.zig.

    Supported renderers include:

    • Terminal: Uses ANSI codes and ColorPalette for colorization.
    • Markdown: Uses backticks for annotations.
    • HTML: Uses CSS classes based on Annotation.semanticName.
    • LSP: Formats diagnostics for Language Server Protocol consumers.

    While the traversal logic (element order, wrapping, and region math) is intended to be shared, target-specific differences exist in how annotations are opened/closed, how structure is escaped, and how source regions are drawn.

  11. Understand the Direct LIR Lowering process

    main

    The direct LIR (Low-level Intermediate Representation) lowering is the primary production path from Lambda Solved to LIR. It consumes Lambda Solved lifted syntax and an explicit inline plan to compute logical Lambda Mono decisions.

    Key characteristics:

    • There is no separate stored layout IR; the builder owns the layout, procedure, local, and pattern builders.
    • The builder maintains temporary work caches (e.g., TypeId -> layout.Idx, LiftedLocalId -> LirLocalId) which are not part of the final LIR.
    • Release builds are optimized to avoid allocating or traversing materialized Lambda Mono expression, pattern, or statement trees. They only allocate the specific decision data required for lowering (e.g., function specializations, capture records).

  12. Understand the Lambda Solved stage invariants

    main

    The Lambda Solved stage (src/postcheck/lambda_solved/) is responsible for deciding which lambdas flow to which call sites. Its correctness relies on four critical invariants that ensure monomorphic solving is sound:

    1. FnId Granularity (I1): Monotype specialization must key every LiftedFnId on the checked source function type digest (source_digest in src/postcheck/monotype/specialize.zig). This ensures that two call sites requiring different lambda sets always land in different FnIds. If they share an FnId, their lambda sets will be merged incorrectly.
    2. Positional Capture Matching (I2): unifyCaptures (lambda_solved/solve.zig:1473) matches capture lists positionally. This is only safe if every member's captures come from the same FnId's capture span in the same order.
    3. Lambda-set Member Order (I3): mergeLambdaSets (lambda_solved/solve.zig:1452) appends members in encounter order. Consumers (like Lambda Mono decisions or LIR lowering) must not rely on this order for data like discriminants; instead, members should be treated as canonically sorted by lambda Symbol.
    4. Erasure Trigger Completeness (I4): Closure erasure is triggered by markErasedCallablesReachedByType (lambda_solved/solve.zig:924) during box operations, layout requests, and runtime schema requests. All boundaries where a callable's specialization becomes unknowable must route through these triggers.