DOCSEARCHbyAlgolia
Submit Docs

MongoDB Server Source Code

repository·master·Indexed 12 days ago

https://github.com/mongodb/mongo

Source code for the MongoDB high-performance, general-purpose database system, including the core database engine (mongod) and sharding router (mongos). Documentation covers Bazel build rules for resmoke test suites, jstestfuzz generation, multiversion testing, WASI SDK configuration, and Test Composer scripts for Antithesis.

Tokens
390.5K
Snippets
726
Records
1.6K
Agent score
98%

What's inside MongoDB

  1. Overview of JSON for Modern C++

    master

    JSON for Modern C++ is a single-header, C++11 compliant library designed for intuitive JSON manipulation. It aims to make JSON feel like a first-class data type in C++ using operator overloading and modern syntax.

    Key Characteristics

    • Trivial Integration: The library consists of a single header file json.hpp. It has no external dependencies and requires no complex build system or special compiler flags.
    • Intuitive Syntax: Uses C++ operator magic to provide a syntax similar to dynamic languages like Python.
    • High Reliability: Heavily unit-tested with 100% coverage, checked with Valgrind and Clang Sanitizers, and continuously fuzzed by Google OSS-Fuzz.
    • Memory Model: By default, it uses std::string for strings, int64_t/uint64_t/double for numbers, std::map for objects, std::vector for arrays, and bool for Booleans. The basic_json class can be templated to use different underlying types if needed.

  2. Overview of the Xsum library

    master

    The Xsum library is a vendored, reduced version of the original Xsum library (found at https://gitlab.com/radfordneal/xsum). It is used specifically to implement the Math.sumPrecise proposal.

    Because it is a manual vendor of the original source, it includes the following modifications to ensure compatibility with the C++ environment:

    • Files have been converted from .c to .cpp.
    • The C-specific restrict type qualifier has been removed to comply with C++ syntax standards.

  3. Overview of OpenTelemetry Collector Proto packages

    master

    The OpenTelemetry Collector Proto package defines the protocols used by the OpenTelemetry collector. It is organized into several specialized packages based on the type of telemetry data being handled:

    • common: Contains shared messages used across different services.
    • trace: Defines the Trace Service protocols.
    • metrics: Defines the Metrics Service protocols.
    • logs: Defines the Logs Service protocols.

  4. Overview of MongoDB IDL

    master

    The Interface Definition Language (IDL) is a custom YAML-based Domain Specific Language (DSL) used to generate C++ code for handling BSON documents, MongoDB commands, server parameters, and configuration options. It automates the creation of BSON parsers and serializers, reducing the need for manual code and testing.

    Key Capabilities:

    • Generates C++ classes representing BSON documents (structs).
    • Generates C++ classes representing MongoDB BSON commands.
    • Handles Enums (string or integer based).
    • Declares and manages server parameters and configuration options.
    • Uses a C++ exception-based design for error handling.

  5. Overview of the MongoDB Query System

    master

    The MongoDB query system interprets user requests, determines optimal execution paths, and computes results. It is the core engine used by several primary commands and operations:

    Primary Commands

    • find
    • aggregate

    Associated Read Commands

    • count
    • distinct
    • mapReduce

    Associated Write Commands

    • update
    • delete
    • findAndModify

  6. Overview of the MongoDB Balancer

    master

    The balancer is a background daemon running on the config server primary that monitors data distribution across sharded collections and issues commands to improve it. It is enabled by default but can be enabled/disabled for the entire cluster or for specific collections. You can also configure a 'balancing window' to restrict balancer activity to specific hours of the day.

    The balancer operates using two separate threads and three primary action policies:

    1. ChunkSelectionPolicy: Handles normal collection balancing (splits and migrations).
    2. DefragmentationPolicy: Handles collection defragmentation.
    3. AutoMergerPolicy: Handles merging of contiguous chunks.

  7. Overview of MongoDB Extension Utilities

    master

    The resmokelib/extensions module provides utilities for managing MongoDB extensions within resmoke test suites. Extensions are implemented as dynamically loaded shared objects (.so files). The module automates three primary tasks:

    1. Discovery: Locating extension .so files within build directories.
    2. Generation: Creating .conf configuration files required to load extensions.
    3. Cleanup: Removing configuration files after test execution.

  8. What is Zstandard (zstd)?

    master

    Zstandard, or zstd, is a fast lossless compression algorithm designed for real-time compression scenarios. It aims to provide zlib-level compression ratios with significantly better performance. It is backed by a fast entropy stage using the Huff0 and FSE library.

    Key characteristics:

    • Stable Format: Documented in RFC8878.
    • Reference Implementation: A dual BSD/GPLv2 licensed C library.
    • CLI Utility: Supports producing and decoding .zst, .gz, .xz, and .lz4 files.
    • Configurable Trade-offs: Users can trade compression ratio for speed using compression levels or the --fast=# flag (negative levels).

  9. Overview of double-conversion library

    master

    The double-conversion library provides efficient binary-decimal and decimal-binary routines for IEEE doubles. These routines were extracted from the V8 JavaScript engine and refactored for general use.

    For detailed API documentation, refer to:

    • double-conversion/string-to-double.h
    • double-conversion/double-to-string.h

    For implementation examples, see test/cctest/test-conversions.cc.

  10. Overview of MongoDB Query Optimization Architecture

    master

    The MongoDB Query Optimization (QO) system is responsible for transforming client requests (like find, aggregate, or update) into efficient execution plans. The architecture follows a flow from parsing client commands into a CanonicalQuery, applying logical rewrites, enumerating potential plans, and finally selecting a winning QuerySolution based on cost models and cardinality estimation.

    High-Level Query Flow

    1. Command Entry: Client operations such as find, count, distinct, delete, update, findAndModify, aggregate, and mapReduce enter the system.
    2. Canonicalization: Commands are converted into a CanonicalQuery or a Pipeline.
    3. Logical Optimization:
      • MatchExpression rewrites are applied to filters.
      • Pipeline::optimize() is used for aggregation pipelines.
    4. Planning & Enumeration: The Plan Enumerator generates Candidate QuerySolutions by considering projections and sorts.
    5. Ranking: Multiple planners (e.g., Classic Multiplanner, Classic Multiplanner for SBE) and a Cost Based Ranker (utilizing Cardinality Estimation and a Cost Model) evaluate candidates to select the Winning QuerySolution.
    6. Execution: The winning plan is passed to the PlanExecutor, which interacts with DocumentSourceExecution and the Storage API to retrieve data.
    ---
config:
  themeVariables:
    fontSize: 32px
---
flowchart TD
 subgraph Client[" "]
        mr["mapReduce"]
        agg["aggregate"]
        find["find"]
        count["count"]
        dist["distinct"]
        del["delete"]
        update["update"]
        fam["findAndModify"]
  end
 subgraph s1[" "]
        D1["DocumentSourceExecution"]
        D2["PlanExecutor"]
        D3["Storage API"]
  end
    find --> cq["CanonicalQuery"]
    count --> cq
    dist --> cq
    cq -- filter --> me["MatchExpression"]
    del -- filter --> cq
    update -- filter --> cq
    fam -- filter --> cq
    me --> C1["optimizeMatchExpression()"]
    C1 --> C3["Plan Enumerator"]
    C3 --> n2["Candidate QuerySolutions"]
    n2 --> C4["Classic Multiplanner"] & C5["Classic Multiplanner for SBE"] & C6["Cost Based Ranker"]
    C7["Cardinality Estimation"] --o C6
    C8["Cost Model"] --o C6
    C4 --> C12["Winning QuerySolution"]
    C5 --> C12
    C6 --> C12
    C11{"Can pushdown to find?"} -- Yes --> cq
    C12 --> D2
    C11 -- No --> D1
    D1 --> D2
    D2 --> D3
    n1["Pipeline::optimize()"] --> C11
    agg --> B1["Pipeline"]
    mr -- "Deprecated in v5.0 in favor of agg" --> B1
    cq -- projection<br>sort --> C3
    B1 --> n1
    D1@{ shape: subproc}
    D2@{ shape: lin-rect}
    D3@{ shape: cyl}
    cq@{ shape: dbl-circ}
    me@{ shape: dbl-circ}
    C1@{ shape: subproc}
    C3@{ shape: subproc}
    n2@{ shape: docs}
    C4@{ shape: subproc}
    C5@{ shape: subproc}
    C6@{ shape: subproc}
    C7@{ shape: lean-r}
    C8@{ shape: lean-r}
    C12@{ shape: doc}
    n1@{ shape: subproc}
    B1@{ shape: dbl-circ}

  12. Check OpenTelemetry C++ compatibility and requirements

    master

    Before integrating OpenTelemetry C++, ensure your environment meets the following requirements:

    Supported C++ Standards

    The library generally supports:

    • ISO/IEC 14882:2014 (C++14)
    • ISO/IEC 14882:2017 (C++17)
    • ISO/IEC 14882:2020 (C++20)

    Note: Supporting the C programming language is not a goal of this project.

    Supported Platforms

    The project is built and tested on x86-64 platforms including:

    • Ubuntu (22.04, 20.04): Using GCC and Clang with CMake or Bazel.
    • macOS 12.7: Using Xcode 14.2 with Bazel.
    • Windows Server (2019, 2022): Using Visual Studio 2019/2022 with CMake or Bazel.

    Project Status

    The implementation is considered Stable across all three primary signals: Logs, Metrics, and Traces.