DotMim.Sync Documentation

repository·master·Indexed 21 days ago

https://github.com/mimetis/dotmim.sync

A lightweight, high-performance framework for synchronizing relational databases across different platforms and database engines. Built on .NET 8 and compatible with .NET Standard 2.0, it supports IoT, Xamarin, UWP, and standard .NET applications. Features include multi-database support, a dedicated CLI for project management, and specialized providers like SqlSyncChangeTrackingProvider for native SQL Server Change Tracking.

Tokens
52.7K
Snippets
141
Records
167
Agent score
75%

What's inside DotMim.Sync

  1. Dotmim.Sync Overview

    master

    DotMim.Sync (DMS) is a framework designed for synchronizing relational databases. It is built on top of .NET 8 and is compatible with .NET Standard 2.0, making it suitable for various platforms including IoT, Xamarin, UWP, and standard .NET applications.

    Key features include:

    • Multi-Database Support: Works across different database engines.
    • Cross-Platform: Compatible with multiple operating systems and runtimes.
    • High Performance: Designed for fast synchronization of changes.
  2. What is a scope in Dotmim.Sync?

    master

    A scope is an object that defines a specific synchronization setup and its database schema. It is identified by a unique name and contains a list of tables to be synchronized.

    Scopes are persisted in a metadata table (defaulting to scope_info) on both the Server and all Client data sources. This allows the synchronization engine to track what has been synchronized and what the schema looks like for specific groups of tables.

    You can define multiple scopes to handle different synchronization scenarios, such as:

    • A "products" scope containing only product-related tables.
    • A "customers" scope containing only customer and order tables.
    • A default scope (named "DefaultScope") containing all tables.
    // Example of defining a setup that will be part of a scope
    var setup = new SyncSetup("ProductCategory", "ProductModel", "Product",
        "Address", "Customer", "CustomerAddress", "SalesOrderHeader", "SalesOrderDetail" );
    
    var agent = new SyncAgent(clientProvider, serverProvider);
    var s1 = await agent.SynchronizeAsync(setup);
  3. What is a scope client and how does it work?

    master

    In Dotmim.Sync, a scope represents a set of tables (the "FROM" part of a query), which is stored in the scope_info table.

    A scope client is the association of a specific scope with a set of filter parameters (the "WHERE" part of a query). It is stored in the scope_info_client table.

    For example, if you want to synchronize Product and ProductCategory tables but only for the category "Books", you define a scope client where:

    • Scope: Product, ProductCategory tables.
    • Filter parameters values: ProductCategoryID = "Books".

    Each scope client is independent and can be synchronized separately because each has its own unique combination of scope name and scope parameters, which are used to track its specific synchronization state (timestamps, etc.).

    // Example concept: 
    // Scope = Tables (Product, ProductCategory)
    // Scope Client = Scope + Filter (ProductCategoryID = 'Books')
  4. How serializers and converters work in HTTP mode

    master

    When using HTTP architecture, DMS utilizes two components to handle data transmission:

    1. Serializer: Transforms a database row into a serialized stream. The default is JSON.
    2. Converter: Converts data types (e.g., converting a byte[] to a base64 string). DMS does not use default converters and relies on the serializer's default converter.

    Note: These components are only relevant when using an HTTP architecture.

  5. How synchronization works over HTTP

    master

    When clients are not on the same local network as the server (e.g., mobile devices), synchronization must occur over HTTP rather than TCP. In this scenario, the standard RemoteOrchestrator cannot be used because it expects a direct TCP connection. Instead, DMS uses a proxy pattern involving two new components:

    • WebRemoteOrchestrator: Runs on the client side. It acts as an orchestrator for the SyncAgent but generates HTTP requests containing the synchronization payload instead of direct database commands.
    • WebServerAgent: Runs on the server side, typically hosted within an ASP.NET Core Web API. It receives the incoming HTTP requests from the WebRemoteOrchestrator and translates them into calls to the server-side provider.

    This architecture allows the server-side database to remain protected behind a web API.

  6. Use SqlSyncChangeTrackingProvider for high-performance SQL Server synchronization

    master

    The SqlSyncChangeTrackingProvider is a specialized provider for Dotmim.Sync that leverages the native SQL Server Change Tracking feature. This is an alternative to the standard SqlSyncProvider and offers several advantages:

    • Improved Performance: Changes are tracked directly by the SQL Engine rather than using triggers.
    • Reduced Database Overhead: No need for manual tracking tables or triggers on your user tables.
    • Automated Metadata Management: Retention and cleanup of change metadata are managed by SQL Server itself.

    This provider is compatible with all other sync providers. You can use SqlSyncChangeTrackingProvider on your server while using different providers (like SqlSyncProvider) on your client databases.

    var serverProvider = new SqlSyncChangeTrackingProvider("Data Source=...");
    var clientProvider = new SqlSyncChangeTrackingProvider("Data Source=...");
  7. Monitor synchronization progress using IProgress<ProgressArgs>

    master
    For standard progress tracking within awaitable methods, the recommended best practice is to use the .NET IProgress<T> pattern. You can pass an IProgress<ProgressArgs> instance to the synchronization methods to receive updates as the process moves through different SyncStage values. This is the preferred method for simple progress reporting (e.g., updating a UI progress bar).
  8. Understand the difference between SyncSetup and SyncOptions

    master

    Dotmim.Sync uses two primary configuration objects to define the synchronization model:

    • SyncSetup: Contains parameters related to your database schema. These are shared between the server and all clients. In Http mode, the server defines the SyncSetup parameters and sends them to the clients.
    • SyncOptions: Contains parameters that are not shared between the server and the clients (local-only settings).

    Use SyncSetup to define which tables, columns, and rows are part of the synchronization scope.

  9. Configure error resolution policies in Dotmim.Sync

    master

    You can control how the synchronization engine handles errors during the ApplyChanges phase using the ErrorResolution enum. This can be configured globally via the SyncOptions.ErrorResolutionPolicy property or dynamically per error using the OnApplyChangesErrorOccured interceptor on the LocalOrchestrator (or RemoteOrchestrator).

    When an error occurs, the interceptor provides an args object containing the Exception, the ErrorRow, and a Resolution property that you must set to determine the next step.

    agent.LocalOrchestrator.OnApplyChangesErrorOccured(args =>
    {
        // Handle error logic here
        args.Resolution = ErrorResolution.ContinueOnError;
    });
  10. Use SetupFilter to implement horizontal filtering

    master

    Horizontal filtering allows you to remove specific rows from the source during synchronization. You achieve this by creating a SetupFilter instance for a specific table and adding it to your SyncSetup.Filters collection.

    To implement a filter, you must:

    1. Define the target table in the SetupFilter constructor.
    2. Register parameters using .AddParameter().
    3. Define how those parameters map to columns using .AddWhere().
    4. (Optional) Use .AddJoin() to link the target table to other tables that contain the filtering criteria.

    This is useful when you only want to synchronize a subset of data (e.g., only customers from a specific city).

    var setup = new SyncSetup(new string[] { "Table1", "Table2" });
    var filter = new SetupFilter("Table1");
    // ... configure parameters and where clauses ...
    setup.Filters.Add(filter);
  11. How SyncType works

    master

    The SyncType enumeration determines how the client database is treated during synchronization. This is useful for recovering from bugs or out-of-sync states by re-downloading the entire schema and rows from the server.

    • SyncType.Normal: The default mode. It performs a standard synchronization, uploading local changes and downloading new changes from the server.
    • SyncType.Reinitialize: Marks the client to be fully resynchronized. Warning: This will delete all rows on the client and download them again from the server. Any local changes on the client that haven't been uploaded will be lost.
    • SyncType.ReinitializeWithUpload: Similar to Reinitialize, but it attempts to upload all local client changes to the server before performing the full reinitialization/download process.
    public enum SyncType
    {
        /// <summary>
        /// Normal synchronization
        /// </summary>
        Normal,
    
        /// <summary>
        /// Reinitialize the whole sync database, applying all rows from the server to the client
        /// </summary>
        Reinitialize,
        
        /// <summary>
        /// Reinitialize the whole sync database, applying all rows from the server to the client, 
        /// after tried a client upload
        /// </summary>
        ReinitializeWithUpload
    }
  12. Configure synchronization using SyncSetup

    master

    The SyncSetup object allows you to define the scope of your synchronization. You can either pass a simple array of table names to SynchronizeAsync() (which creates a default SyncSetup automatically) or provide a fully customized SyncSetup instance.

    Customizing SyncSetup allows you to control schemas, column filtering, row filtering, and naming conventions for database objects.

    // Option 1: Automatic setup using only table names
    var tables = new string[] {"Product", "Customer"};
    var agent = new SyncAgent(clientProvider, serverProvider);
    var r = await agent.SynchronizeAsync(tables);
    
    // Option 2: Manual setup for full customization
    var setup = new SyncSetup("Product", "Customer");
    var agent = new SyncAgent(clientProvider, serverProvider);
    var r = await agent.SynchronizeAsync(setup);