DOCSEARCHbyAlgolia
Submit Docs

Mitsuba 3 Documentation

repository·master·Indexed 25 days ago

https://github.com/mitsuba-renderer/mitsuba3

A research-oriented, retargetable forward and inverse renderer designed for light transport simulation. Mitsuba 3 supports differentiable rendering, spectral/polarized transport, and high-performance execution on CPUs and GPUs via the Dr.Jit JIT compiler. It provides various variants for scalar, LLVM (CPU), and CUDA (GPU) backends, supporting RGB, monochromatic, and spectral rendering.

Tokens
39.3K
Snippets
85
Records
180
Agent score
83%

What's inside Mitsuba 3

  1. Overview of Polarization Support in Mitsuba 3

    master

    Mitsuba 3 features a retargetable design that allows for optional tracking of the full polarization state of light. This simulates the electromagnetic wave oscillations perpendicular to the direction of travel. While not necessary for standard photorealistic rendering intended for human perception, polarization support is critical for:

    • Using measurement devices and specialized cameras.
    • Extracting information about material properties and object shapes.
    • Solving inverse problems.

    Polarization is implemented using bidirectional techniques based on the general path space formulation.

  2. Understand the Mitsuba 3 architecture and Dr.Jit foundation

    master
    Mitsuba 3 is built upon Dr.Jit, a Just-In-Time (JIT) compiler. Dr.Jit is responsible for the code transformations that enable systematic vectorization and automatic differentiation within the renderer. Developers looking to modify the core should thoroughly understand Dr.Jit before exploring the Mitsuba codebase.

  3. Explore the Mitsuba 3 code structure

    master

    The Mitsuba codebase is organized into several key functional areas:

    • src/core: Implements fundamental functionality including cross-platform file and bitmap I/O, data structures, scheduling, logging, and plugin management.
    • src/render: Contains abstractions for loading and representing scenes, including light sources, shapes, materials, and participating media.
    • src/python: Contains Python components that access Mitsuba via bindings, such as statistical tests (e.g., Chi^2) and differentiable rendering tooling.
    • src/[plugin_name]: Other folders within src implement specific Mitsuba 3 plugins, such as bsdf or shapes.

  4. Understand Image Reconstruction Filters

    master

    Image reconstruction filters in Mitsuba convert radiance samples generated by the sampler and integrator into the final output image. Choosing a filter involves balancing three competing factors:

    1. Sharpness: Capturing details at the requested image resolution.
    2. Aliasing: Preventing high-frequency detail from incorrectly leaking into low-frequency detail.
    3. Ringing: Avoiding oscillation artifacts near discontinuities (e.g., light-shadow transitions).

    There is no single superior filter; the choice depends on the specific trade-offs required for your scene and the computational efficiency desired.

  6. Understand Mueller-Stokes representation in Mitsuba 3

    master

    Mitsuba 3 implements polarized rendering by using a templated Spectrum type. To maintain a consistent API for both emitters and sensors (defined in include/mitsuba/render/endpoint.h), the renderer uses Mueller matrices even when representing Stokes vectors. In the case of a Stokes vector, only the first column of the Mueller matrix is non-zero.

    There are three Mueller matrix modes depending on the color representation:

    1. Monochrome mode (scalar_mono_polarized): $4 \times 4 \times 1$ matrices. Only intensity/luminance is simulated.
    2. RGB mode (scalar_rgb_polarized): $4 \times 4 \times 3$ matrices. Uses RGB colors, which is a less accurate approximation for polarized rendering.
    3. Spectral mode (scalar_spectral_polarized): $4 \times 4 \times 4$ matrices. The recommended approach for polarized rendering, where the last dimension corresponds to sampled wavelengths.

  7. Understand Mitsuba 3 Variants

    master

    Mitsuba 3 uses a system of "variants" to retarget the rendering engine. A variant name determines how the simulation handles color representation (monochromatic, RGB, spectral, or polarized), numerical precision, computational backends, and automatic differentiation.

    When installing via pip, only a subset of variants is provided to keep binary sizes manageable. For advanced use cases requiring specific combinations not included in the pip installation, you must compile Mitsuba from source.

  8. Understand polarization conventions in Mitsuba 3

    master

    Mitsuba 3 uses the Verdet convention for orienting electric (E) and magnetic (H) fields during specular reflection and transmission. In this convention, all electric fields are oriented parallel to each other. This differs from the Fresnel convention, where the reflected electric field is oriented such that the magnetic fields point in opposite directions.

    When working with polarization theory, note that switching between these conventions can be interpreted as a phase shift of $180^\circ$. Mitsuba 3 adopts all conventions recommended by Muller (1969) and Bennett to ensure consistency in its Mueller-Stokes calculus.

  10. Enable NVIDIA OptiX on WSL 2

    master

    Mitsuba uses the NVIDIA OptiX framework for hardware-accelerated ray tracing. While OptiX is not officially supported on Windows Subsystem for Linux 2 (WSL 2), you can enable it by extracting driver components and manually moving them to the WSL system library path.

    Warning: Using CUDA and OptiX through WSL degrades performance. Do not use WSL for performance benchmarking as results will not be representative. Use these instructions at your own risk.

  11. Implement a basic C++ plugin skeleton

    master

    To create a new Mitsuba plugin in C++, you should inherit from PluginInterface<Float, Spectrum> and use specific MI_* macros to handle type importing, variant instantiation, and RTTI.

    Key steps include:

    1. Use NAMESPACE_BEGIN(mitsuba) and NAMESPACE_END(mitsuba) to wrap your plugin.
    2. Use MI_IMPORT_BASE to import members from the parent class.
    3. Use MI_IMPORT_TYPES() to import core and rendering-related types.
    4. Use MI_DECLARE_CLASS(Name) inside the class for logging.
    5. Use MI_EXPORT_PLUGIN(Name) outside the class to instantiate the enabled variants.
    NAMESPACE_BEGIN(mitsuba)

template <typename Float, typename Spectrum>
class MyPlugin : public PluginInterface<Float, Spectrum> {
public:
    MI_IMPORT_BASE(PluginInterface, m_some_member, some_method)
    MI_IMPORT_TYPES()

    MyPlugin();

    Spectrum foo(..., Mask active) const override {
        MI_MASKED_FUNCTION(ProfilerPhase::MyEval, active)
        // ...
    }

    /// Indicate the name of this class (for logging)
    MI_DECLARE_CLASS(MyPlugin)
};

/// Implement RTTI data structures
MI_EXPORT_PLUGIN(MyPlugin)
NAMESPACE_END(mitsuba)

  12. Include external XML files and manage resource paths

    master

    Including Files

    Split large scenes using the <include> tag. The included file must be a valid scene file with a <scene> root tag.

    <include filename="nested-scene.xml"/>

    Managing Search Paths

    Use the <path> tag to add directories to Mitsuba's search path. This allows you to reference meshes or textures using relative paths from the added directory.

    <path value="../../my_resources"/>

    Note: Relative paths in <path> are first searched relative to the scene directory, then relative to existing search paths (which can be added via the -a command-line flag).

    <include filename="nested-scene-$version.xml"/>
<path value="../../my_resources"/>