DOCSEARCHbyAlgolia
Submit Docs

Marian NMT Documentation

repository·master·Indexed 23 days ago

https://github.com/marian-nmt/marian

A high-performance Neural Machine Translation (NMT) framework written in C++ supporting RNN and Transformer architectures on CPUs and GPUs. The documentation covers installation, API documentation using Doxygen, shortlist generation with fastalign and extract-lex, and integration with the Triton Inference Server via the Marian backend.

Tokens
30K
Snippets
75
Records
149
Agent score
80%

What's inside Marian

  1. Overview of CLI11

    master
    CLI11 is a powerful, header-only command line parser designed for C++11 and beyond. It provides a rich feature set with a simple, intuitive interface and has no external dependencies. It is suitable for both small projects and complex command-line applications, supporting standard shell idioms, subcommands, and configuration files (INI format).

  3. Overview of Marian NMT

    master

    Marian is an efficient Neural Machine Translation (NMT) framework implemented in pure C++ with minimal dependencies. It is designed for high-performance translation tasks, supporting both training and inference.

    Key Features:

    • High Performance: Efficient pure C++ implementation.
    • Hardware Acceleration: Fast multi-GPU training and support for both GPU and CPU translation.
    • Modern Architectures: Supports state-of-the-art NMT architectures, including deep RNNs and Transformers.
    • Permissive Licensing: Distributed under the MIT license.

  4. Overview of mio memory mapping library

    master

    mio is a header-only, cross-platform C++11 library for memory-mapped file I/O. It is designed to be a lightweight, dependency-free alternative to Boost.Iostreams.

    Key features include:

    • Support for establishing memory mappings with existing file handles/descriptors.
    • Automatic management of page boundaries (accepts any offset).
    • Two distinct usage models: move-only classes for zero-cost abstraction, and shared-semantics classes (using std::shared_ptr) for shared access.
    • Support for wide character types on Windows for path parameters.

  5. Overview of Pathie

    master
    Pathie is a C++ library designed for platform-independent pathname manipulation and filename handling, with a specific focus on Unicode support. It acts as a glue library that allows developers to write code using UTF-8 for all path operations, regardless of whether the underlying operating system uses UTF-8 (Linux/macOS) or UTF-16LE (Windows).

  6. Configure asynchronous logging in spdlog

    master

    To enable extremely fast asynchronous logging, call spd::set_async_mode(q_size) before creating your loggers. The q_size must be a power of 2. Once set, all loggers created afterwards will operate in asynchronous mode using lock-free queues.

    void async_example()
{
    size_t q_size = 4096; //queue size must be power of 2
    spd::set_async_mode(q_size);
    auto async_file = spd::daily_logger_st("async_file_logger", "logs/async_log.txt");
    for (int i = 0; i < 100; ++i)
        async_file->info("Async message #{}", i);
}

  7. How expression graphs work in Marian

    master

    Marian uses a deep learning framework based on reverse-mode automatic differentiation (backpropagation) with dynamic computation graphs.

    Key characteristics include:

    • Dynamic Declaration: A new graph is created for each training instance or batch. This allows for handling variably sized inputs and architectures with conditional logic or loops.
    • Memory Management: Marian uses careful memory management to minimize the overhead of dynamic graph construction and supports efficient execution on both CPU and GPU.
    • Graph Structure: A graph (implemented via the ExpressionGraph class) is a directed graph of Node objects. Nodes represent either data (tensors) or operations.

    Note: Because the graphs are dynamic, nodes are consumed during the forward or backward pass. If you need to visualize the graph using graphviz(), you must call it before performing any computation.

  8. Understand Tensor Operators and Backend Dispatch

    master

    Tensor operators in Marian provide a device-agnostic interface for interacting with tensor data. When an operator is called, it automatically dispatches the computation to the appropriate backend based on the device type configured in the graph:

    • CPU: Uses implementations in the cpu namespace. Supported libraries include CBLAS / OpenBLAS, FBGEMM, INTGEMM, and MKL. CPU libraries typically use row-major representation.
    • GPU: Uses implementations in the gpu namespace. Supported via CUDA (cuBLAS). Note that cuBLAS uses column-major representation.

    For performance, Marian uses OpenMPI and OpenMP for parallelization. Developers can also enable faster floating-point math using the MARIAN_FFAST_MATH macros, though standard caveats for fast_math apply.

    void TensorOp(marian::Tensor out, marian::Tensor in) {
#ifdef CUDA_FOUND
  if(out->getBackend()->getDeviceId().type == DeviceType::gpu)
    gpu::TensorOp(out, in);
  else
#endif
    cpu::TensorOp(out, in);
}

  9. Use powerful Validators with logical operators

    master

    Validators in CLI11 (version 1.6+) are highly flexible.

    • You can use any subclass of CLI::Validator.
    • Validators can define custom type names, such as PATH or INT in [1-4], which appear in help text.
    • Validators can be combined using logical operators: & (AND) and | (OR).
    • Built-in validators like ExistingPath (added in 1.4) are available.

  10. Handle complex logic in Expression Operators

    master

    Expression operators are not limited to simple node creation; they can include conditional logic or optimizations. For example, an operator like sum can check the shape of the input tensor to avoid unnecessary computation if the dimension being reduced is already 1.

    Additionally, complex operators can be composed of multiple existing expression operators (e.g., weighted_average composed of scalar_product, sum, and operator/), though lower-level implementations are generally more efficient.

    Expr sum(Expr a, int ax) {
  if(a->shape()[ax] == 1) {
    return a;
  }
  return Expression<ReduceNodeOp>(a, ax, ReduceNodeOpCode::sum);
}

Expr weighted_average(Expr in, Expr weights, int ax) {
  auto p = scalar_product(in, weights, ax);
  auto s = sum(weights, ax);
  return p / s;
}

  11. Model compatibility limitations for Triton-AML

    master
    Currently, Triton-AML is specifically optimized for the nlxseq2seq model. The ModelState::SetMarianConfigPath function contains hard-coded logic for this model. If you intend to run other models with Marian using this backend, you must modify this function to support different model configurations.

  12. How factors work in Marian

    master

    Factors allow you to incorporate additional features (like capitalization or subword divisions) into the translation model alongside the main word/token (the lemma).

    Core Concepts

    • Lemmas: Factor group zero, representing the actual words or tokens in the text.
    • Factor Groups: Collections of features. Each group represents a different feature type (e.g., capitalization).
    • Factors: Individual features within a group. Factors within a group must share a common prefix (e.g., c0, c1, c2 for a capitalization group _c).
    • Data Format: In training data, factors are appended to lemmas using a pipe |. Multiple factor groups are also separated by pipes.

    Example Data Transformation

    Original: Trump tested positive for COVID-19. Preprocessed: trump test@@ ed positive for c@@ o@@ v@@ i@@ d - 19 . Factored: trump|c1|s0 test|c0|s1 ed|c0|s0 positive|c0|s0 for|c0|s0 c|c2|s1 o|c2|s1 v|c2|s1 i|c2|s1 d|c2|s0 -|c0|s0 19|c0|s0 .|c0|s0