Nimony Documentation

repository·master·Indexed 19 days ago

https://github.com/nim-lang/nimony

Nimony is a new implementation of the Nim programming language (Nim 3) currently in development. It aims to provide a production-ready compiler featuring incremental recompilation, parallel builds, and a unified concurrency model. The project introduces the Nim Intermediate Format (NIF) for compiler communication, a continuation-based async system using the {.passive.} pragma, and tools such as nifmake for build management, nifler for Nim-to-NIF translation, and the Leng language compilers (lengc and arkham).

Tokens
58.7K
Snippets
168
Records
232
Agent score
62%

What's inside Nimony

  1. Overview of Nimony

    master
    Nimony is a minimal alternative frontend for the Nim language. It is specifically designed to provide essential editor integration features, such as "find usages" and "goto definition", which are useful for building or enhancing development environments and editors.
  2. Overview of the Nimony compilation pipeline

    master

    The Nimony compiler uses a series of specialized tools to transform Nim code into Leng code via NIF token streams. The pipeline follows these stages:

    1. nifler: Pure parsing (Nim $\rightarrow$ NIF).
    2. nimony: Semantic checking (symbol lookups, type checking, macro/template expansion).
    3. nimony: Effect inference (not yet implemented).
    4. nimony: Injection of dereferences and mutation checking.
    5. hexer: Iterator inlining, lambda lifting, and injecting duplicates.
    6. hexer: Lowering control flow expressions to statements.
    7. hexer: Injecting destructors.
    8. hexer: Mapping builtins (e.g., new, +) to compiler procs.
    9. hexer: Translating exception handling.
    10. hexer: Generating Leng code.
    11. Leng generator: Final expansion (copying imported symbols, translating types, and moving procs to toplevel).
  3. Overview of Leng and its compilers

    master

    Leng is a language that can be compiled using two different tools:

    1. Leng (lengc): A basic implementation that compiles Leng source code into C code.
    2. arkham: A tool that compiles Leng source code directly into native machine code.

    Use lengc if you want to target C for further compilation or portability, and use arkham for direct native execution.

  4. Overview of the nifgram tool

    master
    Nifgram is a code generation tool designed to work with NIF grammars (written in NIF syntax). It reads a specified NIF grammar and generates Nim code capable of traversing NIF files according to that grammar. The tool allows for significant customization, enabling developers to attach specific actions to matched constructs within the grammar.
  5. Key Design Properties of Passive Procs

    master

    Nimony's passive procedure model is designed with the following characteristics:

    • No colored functions: Passive procs can call regular procs, and regular procs can call passive procs. The compiler automatically inserts complete() to drive passive calls to completion. Use delay(call) if you want the continuation without immediately running it.
    • Zero-overhead: If no .passive procedures are used, no CPS (Continuation Passing Style) transformation occurs and no runtime types are introduced.
    • Composable: Passive procs use ordinary call syntax; the compiler handles the continuation chaining.
    • Destructor safety: Destructors are injected before CPS lowering, ensuring locals lifted into the coroutine environment are destroyed correctly at scope exit.
    • Stack safety: The system uses a trampoline loop. Each state function returns to the trampoline rather than calling the next state directly, preventing deep recursion issues.
  6. What is the Leng dialect?

    master

    Leng is a mid-level Intermediate Representation (IR) dialect of NIF (Nim Intermediate Format). It is designed to be easier to generate than C/C++ code while maintaining full NIF tooling support.

    Key features include:

    • Type Safety: Clearly distinguishes between array (value type), ptr (points to a single element), and aptr (points to an array).
    • Inheritance: Modeled directly in the type system.
    • Module System: Leverages NIF's module system to prevent C/C++ 'one definition rule' violations.
    • Name Mangling: Automatically handles symbol names to avoid conflicts with C/C++ keywords.
  7. What is NJVL and how does it work?

    master

    NJVL (No Jumps, Versioned Locations) is a structured intermediate representation (IR) for Nimony. It is designed to simplify control flow and data flow analysis by removing unstructured jumps and adding versioning to all locations.

    NJVL operates in two distinct phases:

    1. NJ (No Jumps) Pass: Translates unstructured control flow like return, break, raise, and short-circuiting logic (and/or) into control flow variables (cfvar) and guards. This restores a tree-like structure with clean join points.
    2. VL (Versioned Locations) Pass: Adds version information to all locations (variables, field accesses, array indices, etc.) using a variant of SSA (Static Single Assignment).

    Key Benefits:

    • Enables contract validation (requires/ensures).
    • Facilitates alias analysis, copy propagation, CSE, move analysis, and loop optimizations.
    • Simplifies code generation as cfvars can be easily mapped back to jumps.
  8. Overview of Nimony Plugins

    master

    Plugins are Nimony's metaprogramming mechanism, replacing the Nim 2 macro system. A plugin is a separate Nim program that transforms NIF (Nimony Intermediate Format) trees at compile time. Plugins run as external processes and communicate with the compiler via NIF files.

    There are five kinds of plugins:

    KindDeclarationScope
    Template plugintemplate foo(...) {.plugin: "path".}Invoked at each call site
    For-loop pluginiterator foo(...) {.plugin: "path".}Rewrites a for loop using the iterator
    Module plugin{.plugin: "path".} as statementTransforms the entire module
    Type plugintype T {.plugin: "path".} = ...Invoked for every module that uses T
    Import pluginimport (path/foo) {.plugin: "std/v2".}Imports the module path/foo from the plugin std/v2

    All plugins share the same API (plugins) and execution model.

  9. How local routines win over imported routines

    master

    Nimony uses a subtyping rule to prioritize local code. When a routine is imported, it is treated as having an implicit ImportScope parameter, which is a subtype of the local Scope.

    Because subtype matches have lower priority than exact matches, a local routine that matches exactly will beat an imported routine that matches via subtyping. However, an imported routine that matches exactly on real arguments can still beat a local routine that requires a user-defined conversion match.

    # module m:
    proc which*(x: int): string = "imported-int"
    proc which*(x: float): string = "imported-float"
    
    # caller:
    import m
    proc which(x: int): string = "local-int"
    
    doAssert which(1) == "local-int"        # local wins the tie
    doAssert which(1.0) == "imported-float" # conversion still beats scope
  10. How overload resolution works in Nimony

    master

    When a function call p(args) involves a symbol p that refers to multiple candidates, Nimony performs overload resolution to select the best match. If multiple candidates match equally well after all trials, a semantic analysis error (ambiguity) is reported.

    The Selection Process

    Resolution follows a hierarchy of trials:

    1. Category Matching

    Candidates are ranked based on the highest priority category of their argument matches. The categories, in descending order of priority, are:

    1. Exact match: Argument type a and formal parameter type f are identical.
    2. Literal match: Argument is an integer/floating-point literal within the range of the parameter type.
    3. Generic match: The parameter is a generic type (e.g., [T] or [T: int|char]) that matches the argument.
    4. Subrange or subtype match: Argument is a range[T] where T matches f exactly, or the argument is a subtype of f.
    5. Integral conversion match: Argument is a numeric type convertible to the parameter type.
    6. Conversion match: Argument is convertible via a user-defined converter.

    A candidate wins if it has more high-priority matches than others. For example, a candidate with one Exact match beats a candidate with multiple Generic matches.

    2. Module-of-origin (Scope)

    Nimony prefers routines declared in the calling module over those imported from another module. This is implemented by treating every routine as having an implicit Scope parameter:

    • Local routines: Match the Scope parameter exactly.
    • Imported routines: Match via ImportScope (a subtype of Scope), which counts as a subtype match (depth 1).

    3. Structural Type Comparisons

    For generic types, the more specific type is preferred. A type G[A[T]] is considered more specific than G[T] because it contains more information about the nesting of parameters.

    4. Inheritance Depth

    When multiple candidates match via subtyping, the one with the closest inheritance depth is selected (e.g., if C inherits from B which inherits from A, a procedure accepting B is a better match for C than one accepting A).

    # Example of category matching priority
    proc takesInt(x: int) = echo "int"
    proc takesInt[T](x: T) = echo "T"
    proc takesInt(x: int16) = echo "int16"
    
    takesInt(4)      # "int" (Exact match wins over Generic)
    
    var x: int32
    takesInt(x)      # "T" (Generic match)
    
    var y: int16
    takesInt(y)      # "int16" (Exact match)
    
    var z: range[0..4] = 0
    takesInt(z)      # "T" (Subrange match)
  11. How Return Values work in Passive Procs

    master

    Non-void passive procedures pass return values using a pointer parameter. Instead of returning a value directly through the continuation chain, the caller allocates space for the result within its own coroutine environment and passes the address (addr) of that space to the callee.

    This pattern ensures that return values are valid across suspension points without requiring additional dynamic allocation, as the result lives in the caller's existing environment.

    Generated Wrapper Signatures:

    • For non-void: foo_init(...args, result: ptr int, caller: Continuation)
    • For void: foo_init(...args, caller: Continuation)
    # Example of how a non-void passive proc is conceptually handled
    proc io2(): int {.passive.}
    
    # The caller (e.g., main2) manages the result storage:
    # let contVar = io2_init(
    #   addr this.x,  # result pointer into main2's coro struct
    #   Continuation(fn: main2_s1, env: cast[ptr CoroutineBase](this))
    # )
  12. Use Plugins for metaprogramming

    master

    Plugins are Nimony's metaprogramming mechanism, replacing the macro system found in Nim 2. A plugin is defined as a template that lacks a body and instead uses the {.plugin.} pragma to specify the Nim program that implements the transformation.

    Template Plugins

    When attached to a template, the plugin receives only the code related to that specific template invocation.

    Example of converting a template to a plugin:

    Original Template:

    template generateEcho(s: string) = echo s

    Plugin-based Template:

    import std / syncio
    
    template generateEcho(s: string) {.plugin: "deps/mplugin1".}
    
    generateEcho("Hello, world!")

    Plugin Implementation (deps/mplugin1.nim):

    import plugins
    
    proc tr(n: NifCursor): NifBuilder = 
      result = createTree()
      let info = n.info
      var n = n
      if n.stmtKind == StmtsS:
        inc n
      result.withTree CallS, info:
        result.addIdent "echo"
        result.takeTree n
    
    var inp = loadPluginInput()
    saveTree tr(inp)
    template generateEcho(s: string) {.plugin: "deps/mplugin1".}