PrecompileTools.jl

In Julia, code invalidation is the process of throwing away previously compiled code to ensure correctness when new methods are added to a function. This is necessary because Julia uses aggressive compilation to optimize code based on the current set of available methods.

Why invalidation occurs

Invalidation is triggered by the combination of:

1. Interactive development: Adding new code during a session.
2. Method overloading: Packages adding methods to functions they do not own.
3. Aggressive compilation: Optimizing code by assuming only certain methods are available (e.g., specializing a function because only one method of a called function exists).
4. Consistent compilation: Ensuring the same result regardless of the order in which code was loaded.

The trade-off: Aggressive vs. Runtime Dispatch

When a new method is added to a function that was previously used in optimized code, Julia has two choices:

• Option A (Runtime Dispatch): Always use runtime dispatch. This is slower but avoids invalidation.
• Option B (Recompilation): Throw away the optimized code and recompile it to account for the new method. This maintains high performance.

Julia typically follows Option B for up to 3 methods and then switches to Option A. Recent Julia versions offer experimental control over this via Base.Experimental.@max_methods.

Avoiding Spurious Invalidation

"Spurious invalidation" occurs when Julia defensively throws out code that is actually correct but cannot be proven correct by the type-inference machinery. This is most common with poorly-inferred code (e.g., using Any types).

To minimize unnecessary invalidations, use more specific types instead of Any. For example:

• Using Vector{Any} makes it impossible for Julia to predict which methods of a called function will apply, triggering invalidation when new methods are added.
• Using a specific Union type, such as Vector{Union{BinaryNode, TrinaryNode}}, allows Julia to track applicable methods more accurately and avoid invalidating code when a third type (like ArbitraryNode) is added.

Use PrecompileTools to reduce latency on first usage of Julia code by forcing the precompilation of specific workloads. This is effective for Julia 1.9+ (automatic disk saving) and 1.8 (limited benefit).

Macros

• @compile_workload:
  • Precompiles all code needed to execute the block, including all callees.
  • Works even if functions involve runtime dispatch.
  • Important: Must be used at the "top level", not inside a function or an anonymous function (e.g., avoid wrapping it inside withenv(...) do ... end).
• @setup_workload:
  • Marks a block of code as relevant for precompilation without forcing immediate compilation.
  • Useful for generating test data or setup objects that you want to be available during the precompilation process but don't want to include in the final precompile file (which keeps the file size smaller and loading faster).

Best Practices

• You can interleave multiple @compile_workload blocks with @setup_workload code.
• If the code was already inferred prior to the macro (e.g., from prior usage), you might omit external methods called via runtime dispatch. Use SnoopCompile to verify.

module MyPackage

using PrecompileTools: @setup_workload, @compile_workload

struct MyType
    x::Int
end
struct OtherType
    str::String
end

@setup_workload begin
    # Setup code (e.g., generating data) that doesn't need to be in the precompile file
    list = [OtherType("hello"), OtherType("world!")]

    @compile_workload begin
        # Code that MUST be precompiled
        d = Dict(MyType(1) => list)
        x = get(d, MyType(2), nothing)
        last(d[MyType(1)])
    end
end

end

Users can create custom "Startup" packages to precompile workloads that their existing packages haven't precompiled. This is useful for specific projects or analysis environments.

Workflow

1. Activate a project: pkg> activate .
2. Generate a Startup package: pkg> generate Startup
3. Develop the Startup package: pkg> dev ./Startup
4. Configure dependencies: Activate the Startup project and add PrecompileTools along with all packages required for your workload.
5. Define workloads: In Startup/src/Startup.jl, use @compile_workload to include "toy examples" of the code you want to be fast.
6. Use it: In your main project, simply call using Startup to load all packages and their precompiled code.

Tip: Use the Reexport.jl package within your Startup module to ensure all dependencies are also exported by Startup.

# Inside Startup/src/Startup.jl
module Startup

using LotsOfPackages...
using PrecompileTools

@compile_workload begin
    # put a "toy example" of everything you want to be fast
end

end

When Julia invalidates previously compiled code (often due to method extensions), PrecompileTools provides @recompile_invalidations to recompile that code and restore performance.

Usage Patterns

• In Startup packages: Wrap using statements to ensure all packages are loaded before recompiling invalidations.
• In Package development: Wrap method extensions that target Base or other packages to ensure users get a better out-of-the-box experience.

Trade-offs for Package Developers

• Pros: Delivers a better experience for users without requiring them to customize anything.
• Cons: Increases package precompilation time. Furthermore, what is invalidated once might be invalidated again by a later package, wasting the recompilation time.

Note: It is generally better to fix the underlying cause of invalidation (improving inferrability) using tools like SnoopCompile, JET, or Cthulhu rather than relying on @recompile_invalidations.

# Example 1: In a Startup package
module Startup
using PrecompileTools
@recompile_invalidations begin
    using LotsOfPackages...
end
end

# Example 2: In a package extending other packages
module MyContainers
using AnotherPackage
using PrecompileTools

struct Container
    list::Vector{Any}
end

@recompile_invalidations begin
    Base.push!(obj::Container, x) = ...
    function AnotherPackage.foo(obj::Container)
        # ...
    end
end
end