mediafire_rs

repository·main·Indexed 42 days ago

https://github.com/nickoehler/mediafire_rs

An asynchronous Rust rewrite of the mediafire_bulk_downloader for efficiently downloading files and folders from MediaFire. Version 0.2.2 provides a CLI tool (mdrs) and a library featuring concurrent downloads, proxy support, and retry logic.

Tokens
2.9K
Snippets
12
Records
13
Agent score
53%

What's inside mediafire_rs

  1. Install mediafire_rs from source

    main

    To install mediafire_rs, you must have Rust installed on your system. Clone the repository, navigate to the project directory, and compile the release binary using cargo.

    git clone https://github.com/NicKoehler/mediafire_rs.git
    cd mediafire_rs
    cargo build --release
  2. Reference the mediafire_rs CLI options

    main

    The mdrs CLI supports the following options:

    FlagLong FormDescription
    -i--input <FILE>File containing URLs
    -o--output <OUTPUT>Output directory [default: .]
    -m--max <MAX>Maximum number of concurrent downloads [default: 10]
    -t--tries <MAX>Maximum number of tries to repeat for every download [default: 1]
    -r--reverseDownload files in reverse order (largest first)
    -p--proxy <FILE>Specify a file to read proxies from
    N/A--proxy-downloadDownloads files through proxies (default is to use proxies for the API only)
    -h--helpPrint help
    -V--versionPrint version
  3. Use mediafire_rs as a library to download files

    main

    You can use the MediafireDownloader from the md_downloader crate to programmatically download files from Mediafire. The workflow involves initializing a downloader with a retry count, configuring proxies and download order, and then generating download jobs with progress tracking.

    Key steps:

    1. Initialize MediafireDownloader::new(tries).
    2. Configure behavior using .reverse_downloads(bool) and .set_proxies(proxies, use_proxies_for_downloads).
    3. Call .get_download_jobs_with_progress(urls, output_path, progress_callback) to prepare the jobs.
    4. The progress callback receives the current filename being processed, allowing you to update UI elements like progress bars.
    use md_downloader::MediafireDownloader;
    
    // Initialize downloader with 3 retries
    let downloader = MediafireDownloader::new(3)?
        .reverse_downloads(false)
        .set_proxies(proxies, true);
    
    // Get jobs with a progress callback
    let jobs = downloader
        .get_download_jobs_with_progress(&urls, "./downloads", |filename| {
            println!("Currently downloading: {}", filename);
        })
        .await?;
  4. Use the mediafire_rs CLI to download files

    main

    After building, locate the mdrs binary in target/release. You can download a file by providing its Mediafire URL as a positional argument. By default, files are saved to the current working directory.

    cd target/release
    ./mdrs "https://www.mediafire.com/file/examplefile12345/file.zip/file"
  5. Load proxy configurations from a file

    main

    Use load_proxies to read a list of proxy strings from a file. The function accepts an Option<PathBuf>. If None is provided, it returns None. If a path is provided, it returns Some(Vec<String>) containing the lines from the file. If the file cannot be opened, it returns an empty vector wrapped in Some.

    use std::path::PathBuf;
    use mediafire_rs::io::load_proxies;
    
    let proxies = load_proxies(Some(PathBuf::from("proxies.txt")));
  6. Download a file with retry logic using `download_file`

    main

    The download_file function manages the lifecycle of a single file download, including progress bar updates and automatic retries upon failure. It uses indicatif for visual feedback and integrates with md_downloader types.

    Parameters:

    • download_job: A reference to a DownloadJob containing the file metadata and download logic.
    • tries: A u64 representing the number of retry attempts allowed.
    • multi: An Arc<MultiProgress> used to manage multiple concurrent progress bars.
    • styles: An Arc<ProgressStyles> defining the visual appearance of the progress bars for different states (normal, download, error).

    Behavior:

    • It creates a new ProgressBar within the provided MultiProgress container.
    • If a download attempt fails, it waits for 2 seconds (RETRY_SECS) before retrying, provided tries > 0.
    • It updates the progress bar prefix and style based on the current state (e.g., "Getting link...", "Checking hash...", "Retrying...").
    • If all retries are exhausted, it returns the last encountered DownloadError and marks the progress bar with a ❌ icon.
    use std::sync::Arc;
    use indicatif::MultiProgress;
    // Assuming these types are available from the crate
    // use mediafire_rs::{DownloadJob, ProgressStyles, DownloadError};
    
    // Example invocation pattern:
    // download_file(&job, 3, multi_progress, styles).await?;
  7. Configure the mediafire_rs Config struct

    main

    The Config struct defines the operational parameters for the application. You can instantiate it using Config::new(). This struct controls how URLs are processed, where files are saved, concurrency limits, retry logic, and proxy settings.

    use std::path::PathBuf;
    
    let config = Config::new(
        vec!["https://www.mediafire.com/file/example".to_string()], // urls
        None,                                                      // input_file
        PathBuf::from("./downloads"),                            // output_path
        5,                                                         // max (concurrency/limit)
        3,                                                         // tries (retries)
        false,                                                     // reverse_order
        None,                                                      // proxy_file
        false,                                                     // proxy_downloads
    );
  8. Load and sanitize URLs from a file

    main

    Use load_urls to read a list of URLs from a file. The function performs the following sanitization steps on each line:

    • Trims whitespace.
    • Removes empty lines.
    • Ignores lines starting with # (comments).
    • Removes duplicate URLs (only the first occurrence is kept).

    It returns an io::Result<Vec<String>>.

    use std::path::PathBuf;
    use mediafire_rs::io::load_urls;
    
    let urls = load_urls(&PathBuf::from("urls.txt"))?;
  9. Reference the `mdrs` CLI flags and options

    main

    The following flags and arguments are available for the mdrs command:

    Positional Arguments:
      URLS                List of folders or files to download
    
    Options:
      -i, --input <FILE>  File containing URLs
      -o, --output <OUTPUT> Output directory (default: ".")
      -m, --max <MAX>     Maximum concurrent downloads (default: "10", range: 1-100)
      -t, --tries <TRIES> Maximum retries per download (default: "1", range: 1-10)
      -r, --reverse       Download largest files first
      -p, --proxy <FILE>  File containing proxy list
      --proxy-download    Use proxies for file downloads
  10. Reference the Config struct fields

    main

    The following fields are available in the Config struct to control application behavior:

    pub struct Config {
        pub urls: Vec<String>,           // A list of MediaFire URLs to process
        pub input_file: Option<PathBuf>, // Optional path to a file containing URLs
        pub output_path: PathBuf,        // Directory where downloads will be saved
        pub max: u64,                    // Maximum number of concurrent operations or limit
        pub tries: u64,                  // Number of retry attempts for failed operations
        pub reverse_order: bool,         // Whether to process URLs in reverse order
        pub proxy_file: Option<PathBuf>, // Optional path to a file containing proxy configurations
        pub proxy_downloads: bool,      // Whether to use proxies for downloads
    }