FileDownloader

repository·master·Indexed 11 days ago

https://github.com/lingochamp/filedownloader

An Android multi-task file download engine supporting chunked, serial, and parallel tasks. It features customizable components for connections, output streams, and databases, including specialized implementations like RemitDatabase for high-frequency small tasks and NoDatabaseImpl for database-less downloading. Supports foreground services for Android 8.0 and 9.0, multi-connection downloads, and serial download queues.

Tokens
11.1K
Snippets
20
Records
61
Agent score
94%

What's inside FileDownloader

  1. Note on FileDownloader2 (OkDownload)

    master
    The maintainers recommend using OkDownload (FileDownloader2-OkDownload) for new projects. All new features and enhancements are being implemented in OkDownload, while FileDownloader is primarily focused on bug fixes due to low unit-test coverage.
  2. Key features of FileDownloader

    master

    FileDownloader is an Android file download engine with the following capabilities:

    • Ease of use: Simple API for common tasks.
    • Advanced downloading: Supports single-task multi-threading, multi-connection, and chunked downloading. You can customize this behavior using a ConnectionCountAdapter.
    • High concurrency: Efficiently handles multiple simultaneous downloads.
    • Flexibility: Supports both independent and non-independent process modes.
    • Resumable downloads: Automatic support for breakpoint resumption (resuming interrupted downloads).
  3. Configure UI thread callback frequency to avoid frame drops

    master

    To prevent high-frequency callbacks from causing UI frame drops (especially during high-concurrency file detection), FileDownloader uses an 'avoid drop frame' mechanism. By default, this is enabled. It limits the frequency of messages sent to the UI thread and batches the number of callbacks processed per message.

    Key configuration methods:

    • FileDownloader.getImpl().disableAvoidDropFrame(): Disables the mechanism. All callbacks will be sent to the UI thread immediately, which may cause UI stuttering under high load.
    • FileDownloader.setGlobalPost2UIInterval(intervalMillisecond: int): Sets the maximum interval (in milliseconds) between messages sent to the UI thread. Setting this to a value < 0 disables the mechanism.
    • FileDownloader.setGlobalHandleSubPackageSize(packageSize: int): Sets how many callbacks are processed per message on the UI thread. Default is 5.
    • FileDownloader.isEnabledAvoidDropFrame(): Checks if the mechanism is currently enabled.
    // Enable/Disable and configure the avoid-drop-frame mechanism
    FileDownloader.getImpl().enableAvoidDropFrame();
    FileDownloader.getImpl().setGlobalPost2UIInterval(10); // 10ms interval
    FileDownloader.getImpl().setGlobalHandleSubPackageSize(5);
  4. How FileDownloader handles low memory situations

    master

    FileDownloader manages memory by separating the UI process (non-download process) from the download process (service process). This architecture ensures download continuity even when the system is under memory pressure.

    Non-download process (UI Process)

    This process holds queue data. Its behavior depends on its state when memory is reclaimed:

    • Foreground Process: If reclaimed, the application is likely already in a crashed state (extremely low probability).
    • Background Process: If the UI process is in the background and reclaimed, behavior depends on the task type:
      • Serial Queue Tasks: The download process completes the task currently pending in the download process. However, tasks not yet pending in the download process (driven by the UI process) will be interrupted. Upon app restart, the queue will resume from the last successful download.
      • Parallel Queue Tasks: The download process continues all tasks that were already pending. Since pending tasks are added quickly, most tasks will continue. However, because the UI process was reclaimed, the app will miss the listeners/callbacks for those tasks. Upon app restart, the queue behaves normally and listeners will be received.

    Download Process (Service Process)

    This process uses buffers and is an active service process, making it a low-priority target for memory reclamation compared to background apps.

    Resilience and Recovery: If the download process is reclaimed, it will attempt to restart automatically via START_STICKY. When the system provides sufficient memory, the download process restarts, reconnects to the UI process, and resumes all pending tasks (both parallel and serial) using breakpoint resumption (resuming from where they left off). This ensures a seamless experience.

    Note on Service Lifecycle: To prevent the service from restarting automatically, you can manually call:

    • FileDownloader#unBindService
    • FileDownloader#unBindServiceIfIdle
    // To prevent automatic service restart if desired:
    FileDownloader.unBindService(context);
    // OR
    FileDownloader.unBindServiceIfIdle(context);
  5. Handle large files (> 1.99GB) in FileDownloader

    master

    When downloading files larger than 1.99GB ($2^{31}-1 = 2,147,483,647$ bytes), the standard FileDownloadListener may encounter issues. To correctly handle large files, you must use the following specialized components:

    • Listener: Use FileDownloadLargeFileListener instead of FileDownloadListener.
    • Progress Tracking: Use getLargeFileSofarBytes() and getLargeFileTotalBytes() to retrieve progress data.
  6. Optimize performance and UI responsiveness

    master

    Avoid UI Thread DDoS

    By default, the engine includes processing to avoid frame drops. This prevents the FileDownloadListener from sending callbacks too frequently, which could otherwise overwhelm the UI thread. If you need to disable this and revert to the behavior of version 0.1.9 (where all callbacks are immediately dispatched to the UI thread via a Handler), you can adjust the engine configuration.

    Reduce IPC I/O

    To reduce the I/O overhead caused by Inter-Process Communication (IPC) during callbacks, set process.non-separate to true in your filedownloader.properties configuration file, unless you have a specific requirement for independent processes.

  7. Configure FileDownloader process execution

    master

    By default, the FileDownloadService runs in a separate process.

    • To run in the main process: Configure this setting in your filedownloader.properties file.
    • To check the current process: Use FileDownloadUtils.isDownloaderProcess(Context) to determine if the service is currently running in the downloader process or the main process.
  8. Customize connection count per task

    master

    Starting from version 1.5.0, FileDownloader supports multi-connection (multi-threaded) downloads for a single task. You can customize the number of connections used per task by implementing a ConnectionCountAdapter and providing it during initialization.

    By default, the connection strategy is based on file size:

    • 1 connection: [0, 1MB)
    • 2 connections: [1MB, 5MB)
    • 3 connections: [5MB, 50MB)
    • 4 connections: [50MB, 100MB)
    • 5 connections: [100MB, ∞)
  9. Customize FileDownloader components

    master

    FileDownloader allows you to replace its default components with your own implementations. This is useful if you want to use okhttp for connections or if you want to disable the database entirely.

    Supported customizable components:

    NameInterfaceDefault Impl
    ConnectionFileDownloadConnectionFileDownloadUrlConnection
    OutputStreamFileDownloadOutputStreamFileDownloadRandomAccessFile
    DatabaseFileDownloadDatabaseRemitDatabase
    ConnectionCountAdapterConnectionCountAdapterDefaultConnectionCountAdapter
    IdGeneratorIdGeneratorDefaultIdGenerator
    ForegroundServiceConfigForegroundServiceConfigForegroundServiceConfig

    Tips:

    • To use okhttp as the connection component, you can use a dedicated adapter like filedownloader-okhttp3-connection.
    • To disable the database (which is used for persisting task breakpoint info), you can use NoDatabaseImpl.java.
  10. Understand RemitDatabase for high-frequency small tasks

    master

    Introduced in version 1.6.9, RemitDatabase is a specialized database implementation designed to handle a large number of very small tasks (e.g., tasks completing in < 2 seconds) without overwhelming the file system with frequent I/O operations.

    How it works:

    • Short-lived tasks (< 2s): If a task's entire lifecycle (start to finish) occurs within a configurable threshold (default 2s), data is kept entirely in memory and no database operations are performed.
    • Long-lived tasks (> 2s): For tasks exceeding the threshold, the engine stores data in memory for the first 2 seconds, then begins simultaneous memory and database updates.
    • Error/Pause states: If a task ends in a pause or error state, the final status update is always persisted to both memory and the database to ensure reliability.
  11. Android 8.0 (Oreo) Compatibility

    master

    Due to background service restrictions in Android 8.0+, FileDownloader (starting from version 1.7.6) automatically runs as a foreground service when started from the background.

    • A notification titled "FileDownloader" will be displayed.
    • You can customize the content of this notification by following the guidance in the project Wiki.