Downloader .NET Library

Downloader is a high-performance, cross-platform multipart downloader for .NET with the following capabilities:

• Multipart Downloads: Downloads files in multiple parallel parts for increased speed.
• Resilience: Supports automatic and manual resume, and is resilient to client/server errors.
• Efficiency: Pre-allocates file size before downloading and saves chunks directly into the final file without temporary files (though it uses a .download extension during the process).
• Control: Supports dynamic speed limits, configurable ChunkCount, and downloading specific byte ranges.
• Flexibility: Supports downloading to disk or directly to memory streams, and allows injecting custom HttpClient or HttpMessageHandler.
• Metadata: Provides real-time speed, progress updates, and remote file resolution (name/size) without downloading the file via RemoteFileResolver.

The recommended way to handle interruptions (crashes, network drops) is to enable EnableAutoResumeDownload = true in your DownloadConfiguration.

How it works:

1. Temporary Files: The downloader always uses a temporary file with the extension specified by DownloadFileExtension (default is .download).
2. Metadata Embedding: When EnableAutoResumeDownload is true, the downloader appends a JSON-serialized DownloadPackage (containing chunk positions and file size) to the end of the .download file.
3. Automatic Recovery: On a subsequent call to DownloadFileTaskAsync for the same file, the downloader detects the .download file, reads the metadata from the end, verifies the server still supports ranges, and resumes from the last known chunk positions.
4. Cleanup: Upon successful completion, the downloader truncates the file to remove the metadata and renames it to the original filename.

var downloadOpt = new DownloadConfiguration()
{
    EnableAutoResumeDownload = true,
    DownloadFileExtension = ".download" 
};

var downloader = new DownloadService(downloadOpt);

To integrate with IHttpClientFactory, add custom authentication, or use specific SSL settings, you can provide a custom HttpClient or HttpMessageHandler via DownloadConfiguration or the DownloadBuilder.

Important Precedence Rule

If both CustomHttpClientFactory and CustomHttpMessageHandlerFactory are set, CustomHttpClientFactory takes precedence and CustomHttpMessageHandlerFactory is ignored.

Option 1: Custom HttpClient

Use CustomHttpClientFactory for full control. The Downloader will skip all internal handler and header configuration.

Option 2: Custom HttpMessageHandler

Use CustomHttpMessageHandlerFactory to customize only the handler (e.g., for caching or custom SSL), while letting the Downloader still configure default request headers and timeouts.

// Using DownloadConfiguration with CustomHttpClientFactory
var downloadOpt = new DownloadConfiguration()
{
    ChunkCount = 8,
    ParallelDownload = true,
    CustomHttpClientFactory = () => {
        return httpClientFactory.CreateClient("MyDownloader");
    }
};

// Using DownloadConfiguration with CustomHttpMessageHandlerFactory
var downloadOpt = new DownloadConfiguration()
{
    ChunkCount = 8,
    ParallelDownload = true,
    CustomHttpMessageHandlerFactory = () => {
        return new SocketsHttpHandler {
            MaxConnectionsPerServer = 500,
            PooledConnectionLifetime = TimeSpan.FromMinutes(10)
        };
    }
};

// Using the fluent builder API
await DownloadBuilder.New()
    .WithUrl(url)
    .WithDirectory(@"C:\temp")
    .WithHttpClient(() => httpClientFactory.CreateClient("MyDownloader"))
    .Build()
    .StartAsync();

await DownloadBuilder.New()
    .WithUrl(url)
    .WithDirectory(@"C:\temp")
    .WithHttpMessageHandler(() => new SocketsHttpHandler {
        MaxConnectionsPerServer = 500,
        PooledConnectionLifetime = TimeSpan.FromMinutes(10)
    })
    .Build()
    .StartAsync();

Quick Pause/Resume

To temporarily suspend a download while keeping the network streams alive, use the Pause() and Resume() methods on the DownloadService.

downloader.Pause();
downloader.Resume();

Manual Stop and Resume (Persistent)

If you need to stop a download and resume it later (even after an application restart), you must manually manage the DownloadPackage object.

1. Capture the state: Access downloader.Package to get a snapshot of the current download state.
2. Stop: Use downloader.CancelAsync() (fire-and-forget) or await downloader.CancelTaskAsync() (waits for stop).
3. Resume: Pass the captured DownloadPackage back into DownloadFileTaskAsync(pack).

Note: Resuming requires the server to support HTTP range requests.

// Quick pause/resume
downloader.Pause();
downloader.Resume();

// Manual persistent resume
DownloadPackage pack = downloader.Package;

// Stop
await downloader.CancelTaskAsync();

// Resume later (even after restart)
await downloader.DownloadFileTaskAsync(pack);

The DownloadConfiguration object allows you to customize the behavior of the DownloadService. You can control chunking, parallelism, speed limits, retries, and HTTP request headers.

Common Configuration Scenarios

Simple Configuration

Use this for basic parallel downloads.

Recommended Setup for Speed and Reliability

A good general-purpose starting point that uses parallel chunks, retries, and automatic resume capabilities.

Complex Configuration

Use this for fine-grained control over buffers, timeouts, range downloads, and custom RequestConfiguration (headers, proxy, authentication).

// Recommended setup
var downloadOpt = new DownloadConfiguration
{
    ChunkCount = 8,         
    ParallelDownload = true,
    ParallelCount = 4,              
    MaxTryAgainOnFailure = 5,       
    EnableAutoResumeDownload = true,
    MaximumMemoryBufferBytes = 50 * 1024 * 1024, 
    CheckDiskSizeBeforeDownload = true, 
    MaximumBytesPerSecond = 0,   
};

// Complex configuration with RequestConfiguration
var downloadOpt = new DownloadConfiguration()
{
    // ... other properties ...
    RequestConfiguration = 
    {
        Accept = "*/*",
        CookieContainer = cookies,
        Headers = ["Accept-Encoding: gzip, deflate, br"],
        KeepAlive = true, 
        ProtocolVersion = HttpVersion.Version11,
        UserAgent = "Mozilla/5.0",
        Proxy = new WebProxy() {
           Address = new Uri("http://YourProxyServer/proxy.pac"),
           UseDefaultCredentials = false,
           Credentials = System.Net.CredentialCache.DefaultNetworkCredentials,
           BypassProxyOnLocal = true
        },
        Authorization = new AuthenticationHeaderValue("Bearer", "XX_YOUR_TOKEN_XX")
    }
};