Marten Documentation

repository·master·Indexed 25 days ago

https://github.com/jasperfx/marten

A .NET library that transforms PostgreSQL into a fully-featured document database and an ACID-compliant event store using native JSON capabilities. Includes documentation on development environment setup, integration testing with Marten.Testing, and load testing via Marten.ScaleTesting for async daemon projection rebuilds and multi-node HotCold scenarios.

Tokens
257.1K
Snippets
573
Records
1.1K
Agent score
85%

What's inside Marten

  1. Overview of Marten

    master

    Marten is a .NET Transactional Document DB and Event Store built on top of PostgreSQL. It leverages PostgreSQL's JSON support and ACID compliance to provide a robust document database and an event-sourcing engine for building event-sourced systems.

    Key capabilities include:

    • Document Database: A full-fledged document store for .NET objects.
    • Event Store: Tools to store events and streams, including support for projections to create read-side views.
    • PostgreSQL Integration: Uses PostgreSQL as the underlying data store to benefit from its proven engine and JSON capabilities.
  2. Overview of Marten features

    master

    Marten is a .NET library that enables using PostgreSQL as both a document-oriented database and a full-fledged event store. It provides strong data consistency through PostgreSQL transactions and supports various advanced features for building event-driven architectures.

    Key capabilities include:

    • Document Storage: Store entities as JSON for flexible data modeling.
    • Event Store: Capture business facts using Event Sourcing.
    • Strong Consistency: Leverages PostgreSQL transactions for both document and event storage.
    • Advanced Querying: Supports LINQ queries, full-text search, and custom SQL.
    • Events Projections: Store both events and read models in the same storage, with support for inline (same transaction) or asynchronous (async daemon) projections.
    • Automatic Schema Management: Simplifies relational schema management using JSON formats.
    • Flexible Indexing: Define various indexing strategies to optimize performance.
    • Multi-tenancy: Supports data isolation via multiple databases, different schemas, or sharded-tables for both documents and events.
    • Tooling: Includes ASP.NET integration and Command Line tooling.
  3. Understand StoreOptions and Configuration Precedence

    master

    The StoreOptions object is the root configuration for a DocumentStore. Configuration for a specific document type is applied in a specific order, with later steps overriding earlier ones.

    Order of Precedence (Lowest to Highest):

    1. ConfigureMarten(DocumentMapping) method defined on the document type.
    2. IDocumentPolicy registered on StoreOptions (applies to all document types).
    3. MartenAttribute decorations on the document type or its members.
    4. Explicit configuration via MartenRegistry (using StoreOptions.Schema).

    To avoid confusion, Marten recommends choosing one consistent configuration style.

  4. Understand Marten connection handling

    master

    By default, Marten opens a database connection only immediately before an operation requiring one and closes it immediately after (returning it to the Npgsql pool). This allows IQuerySession or IDocumentSession objects to be used safely across multiple threads.

    Exceptions where a connection becomes "sticky" (stays open for the duration of the session) include:

    • Creating a session with an existing connection or transaction.
    • Using serializable transaction isolation levels (e.g., via IDocumentStore.SerializableSessionAsync()).
    • Enrolling in ambient transactions using SessionOptions.ForCurrentTransaction().
    • Using explicit transaction boundaries.
    • Accessing the session's underlying connection directly for user-defined querying.
  5. Use Marten as a Document Database

    master

    Marten allows you to use PostgreSQL as a document database for .NET applications. Instead of mapping .NET types to flat relational tables using an ORM like EF Core, Marten uses JSON serialization to persist and load .NET objects (referred to as "documents") directly into PostgreSQL's JSONB data type.

    This approach is ideal for systems with relatively self-contained entities and provides several benefits:

    • Eliminates explicit ORM mapping: No need to define complex relational mappings.
    • Schema flexibility: Entities can evolve without requiring heavy database migrations.
    • Simplified lifecycle: Supports built-in database initialization and runtime migrations.
  6. Alternative for SQL Server: Polecat

    master
    If your infrastructure requires SQL Server instead of PostgreSQL, use Polecat. Polecat is a sibling project in the 'Critter Stack' ecosystem that brings a similar document database and event sourcing model to SQL Server 2025 using native JSON types and modern T-SQL.
  7. Understand the Marten LINQ execution workflow

    master

    The Marten LINQ provider operates through several internal models that transform a .NET LINQ expression into a PostgreSQL query:

    • MartenLinqQueryable<T>: The entry point implementation of IQueryable that users interact with. It holds the Expression tree.
    • CollectionUsage: An intermediate model that organizes the raw Expression into logical steps like Select, Where, OrderBy, Take, Skip, SelectMany, and Include.
    • Statement: A model (structured as a double-linked list) that knows how to generate the necessary SQL and read raw data into results. Complex queries involving Include(), SelectMany(), or Distinct() may require multiple statements.
    • IQueryHandler: The execution model that uses the Statement to run the query and return results via an IMartenSession.
  8. Configure Async Daemon Deployment Modes

    master

    The Marten Async Daemon, responsible for background projections, supports two primary deployment modes:

    1. Polling-Based Mode: A classic approach where the daemon polls for new events. It can use PostgreSQL advisory locks for leader election to manage active/standby nodes.
    2. Messaging-Based Mode: Uses external messaging infrastructure (e.g., RabbitMQ, Azure Service Bus, Kafka) to gather events and connect them to running projection builders. This mode typically requires a centralized Distributor process to assign projection segments to different nodes.
  9. Understand Event Append Modes

    master

    Marten's closed-shape event storage uses three distinct implementations based on the EventAppendMode configured in your EventGraph. The specific storage class instantiated depends on the mode:

    Append modeStorage classDescriptor
    RichRich/RichEventStorage<TId>Rich/RichEventStorageDescriptor
    QuickQuick/QuickEventStorage<TId>Quick/QuickEventStorageDescriptor
    QuickWithServerTimestampsQuickWithServerTimestamps/QuickWithServerTimestampsEventStorage<TId>QuickWithServerTimestamps/QuickWithServerTimestampsEventStorageDescriptor
    • Rich (Full mode): Uses individual INSERT statements per row. It is highly extensible via IEventMetadataBinder but has a higher operation count per stream.
    • Quick (Batch modes): Uses batched function calls (e.g., select mt_quick_append_events(...)) with array parameters. This is optimized for performance by reducing the operation count to one per batch.
    • QuickWithServerTimestamps: A specialized version of Quick that includes server-side now() timestamps.
  10. Understand Aggregate Projections

    master

    Aggregate Projections in Marten combine groups of events to create a single aggregated document representing the state of those events. There are two primary types:

    1. Single Stream Projections: Create a rolled-up view of all or a segment of events within a single event stream. These can be implemented using the SingleStreamProjection<TDoc, TId> base type or via a "self-aggregating" Snapshot approach using Create, Apply, and ShouldDelete methods.
    2. Multi Stream Projections: Create a rolled-up view of a user-defined grouping of events across multiple streams (e.g., an accounting query model rolling up unpaid invoices by client). These are implemented by subclassing MultiStreamProjection<TDoc, TId>.

    Projections can be implemented using conventional methods (the original Marten pattern) or explicit code (introduced in Marten 8.0).

  11. Planned LINQ Improvements in Marten v4

    master

    The upcoming Marten v4 release includes a significant overhaul of the LINQ provider to improve performance and feature support. Key planned improvements include:

    • Revamped IField Model: A new model within the LINQ provider designed to improve performance, clean up internal conditional code, and better support JSON serialization customizations (like [JsonProperty]) and F# types (such as discriminated unions).
    • Rewritten Include() Functionality: A move away from complex JOIN operations in favor of PostgreSQL Common Table Expressions (CTE) and UNION queries. This is intended to enable Include() on child collections and simplify ISelector implementations by removing InnerJoin logic.
    • Optimized IdentityMap Interactions: Finer-grained code generation to allow query sessions to skip IdentityMap interactions entirely, reducing overhead for purely read-only operations.
    • Modular ILinqDialect: A proposed architectural change to make LINQ support more modular, allowing the expression parser to delegate logical detection to a dialect. This would facilitate support for different JSON/SQL dialects (e.g., JSONPath for PostgreSQL v12+ or future SQL Server versions).
    • Improved SelectMany(): Potential replacement of the current SelectMany() implementation with CTE-based SQL statements to support deeper nested models.
    • JSON Streaming: Implementation of JSON streaming to improve performance, particularly for the read-side of CQRS architectures using Marten's event store.
  12. Understand Default Tenancy behavior

    master

    When multi-tenancy is enabled, Marten associates every record with a tenant. If no tenant is explicitly specified (via session scoping, mapping policies, or method overloads), Marten defaults to the identifier defined in StorageConstants.DefaultTenantId, which has a constant value of *DEFAULT*.

    • Non-tenanted schemas: If a schema is not configured as multi-tenanted (e.g., opts.Schema.For<User>()), documents are treated as non-tenanted and are accessible regardless of the session's tenant.
    • Tenanted schemas: If a schema is configured as multi-tenanted (e.g., opts.Schema.For<Target>().MultiTenanted()), documents without an explicit tenant will be assigned to the *DEFAULT* tenant.