PhenX.EntityFrameworkCore.BulkInsert

PhenX BulkInsert supports the following database providers:

• SQL Server
• PostgreSQL
• SQLite
• MySQL
• Oracle

When using PhenX.EntityFrameworkCore.BulkInsert, be aware of the following unsupported features:

• Navigation properties: You can only insert simple entities. The library does not support inserting entities that contain navigation properties (relationships).
• Change tracking: The library does not track changes to the entities being inserted. Consequently, DbContext.ChangeTracker will not reflect changes to these entities after the bulk insert operation is complete.
• Inheritance: The library does not support entity types using inheritance hierarchies (such as TPT, TPH, or TPC). You must insert entities of a single, concrete type.

Install the specific NuGet package corresponding to your database provider to enable bulk insert capabilities. You should also install the common library PhenX.EntityFrameworkCore.BulkInsert if required by your project structure, though the provider packages typically pull in necessary dependencies.

# For SQL Server
Install-Package PhenX.EntityFrameworkCore.BulkInsert.SqlServer

# For PostgreSQL
Install-Package PhenX.EntityFrameworkCore.BulkInsert.PostgreSql

# For SQLite
Install-Package PhenX.EntityFrameworkCore.BulkInsert.Sqlite

# For MySql
Install-Package PhenX.EntityFrameworkCore.BulkInsert.MySql

# For Oracle
Install-Package PhenX.EntityFrameworkCore.BulkInsert.Oracle

# For OpenTelemetry instrumentation
Install-Package PhenX.EntityFrameworkCore.BulkInsert.OpenTelemetry

To use the library, you must register the bulk insert provider within your DbContext configuration using the extension methods provided by the specific provider package. This is done during service registration (e.g., in Startup.cs or Program.cs).

services.AddDbContext<MyDbContext>(options =>
{
    options
        // Choose the method corresponding to your provider:
        .UseBulkInsertPostgreSql()
        // OR
        .UseBulkInsertSqlServer()
        // OR
        .UseBulkInsertSqlite()
        // OR
        .UseBulkInsertMySql()
        // OR
        .UseBulkInsertOracle()
        ;
});

Register the instrumentation within your OpenTelemetry TracerProvider configuration by calling AddPhenXEntityFrameworkCoreBulkInsertInstrumentation(). This allows the library to automatically track bulk insert operations as Activities.

using OpenTelemetry.Trace;

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddPhenXEntityFrameworkCoreBulkInsertInstrumentation() // <-- add this
        .AddOtlpExporter()
    );

To enable OpenTelemetry tracing for bulk insert operations, install the PhenX.EntityFrameworkCore.BulkInsert.OpenTelemetry NuGet package.

Install-Package PhenX.EntityFrameworkCore.BulkInsert.OpenTelemetry

To use the bulk insert functionality, you must register the appropriate provider in your DbContext configuration using the UseBulkInsert... extension methods. This should be called alongside your standard database provider (e.g., UseSqlServer, UseNpgsql, etc.) during service registration.

services.AddDbContext<MyDbContext>(options =>
{
    options
        // .UseSqlServer(connectionString) // or UseNpgsql or UseSqlite, as appropriate

        .UseBulkInsertPostgreSql()
        // OR
        .UseBulkInsertSqlServer()
        // OR
        .UseBulkInsertSqlite()
        // OR
        .UseBulkInsertMySql()
        // OR
        .UseBulkInsertOracle()
        ;
});

Bulk insert operations emit EF Core-style logs. To see them, ensure a loggerFactory is configured on your DbContext options.

Key log events:

• 1004 (Information): Bulk insert completed, including elapsed time and destination table.
• 1005 (Debug): Auxiliary SQL commands (like temporary table DDL) executed by the library.

services.AddDbContext<MyDbContext>(options =>
{
    options
        .UseSqlite(connectionString)
        .UseLoggerFactory(loggerFactory)
        .UseBulkInsertSqlite();
});

Bulk insert operations emit EF Core-style logs. To see these, ensure a loggerFactory is configured in your DbContext setup.

Key log events:

• 1004 (Information): Bulk insert completed, including elapsed time and destination table.
• 1005 (Debug): Auxiliary SQL commands executed by the library (e.g., temporary table creation).

services.AddDbContext<MyDbContext>(options =>
{
    options
        .UseSqlite(connectionString)
        .UseLoggerFactory(loggerFactory)
        .UseBulkInsertSqlite();
});

If you require lower-level access to traces, you can subscribe directly to the activity source using the name PhenX.EntityFrameworkCore.BulkInsert via an ActivityListener.

using System.Diagnostics;

ActivitySource.AddActivityListener(new ActivityListener
{
    ShouldListenTo = source => source.Name == "PhenX.EntityFrameworkCore.BulkInsert",
    Sample = (ref ActivityCreationOptions<ActivityContext> _) => ActivitySamplingResult.AllDataAndRecorded,
    ActivityStarted = activity => Console.WriteLine($"Started: {activity.DisplayName}"),
    ActivityStopped = activity => Console.WriteLine($"Stopped: {activity.DisplayName} ({activity.Duration})"),
});

You can customize the behavior of bulk inserts by passing an Action<BulkInsertOptions<T>> to the execution methods. Options can be common across providers or specific to a provider (e.g., SqlServerBulkInsertOptions).

// Common options
await dbContext.ExecuteBulkInsertAsync(entities, options =>
{
    options.BatchSize = 1000;
});

// Provider specific options (SQL Server)
await dbContext.ExecuteBulkInsertAsync(entities, (SqlServerBulkInsertOptions o) => 
{
    o.EnableStreaming = true;
});

// Supporting multiple providers in one call
await dbContext.ExecuteBulkInsertAsync(entities, o =>
{
    o.MoveRows = true;

    if (o is SqlServerBulkInsertOptions sqlServerOptions)
    {
        sqlServerOptions.EnableStreaming = true;
    }
    else if (o is MySqlBulkInsertOptions mysqlOptions)
    {
        mysqlOptions.BatchSize = 1000;
    }
});