down

By default, Down::ChunkedIO caches downloaded content into a Tempfile so that you can #rewind the stream and re-read data.

If you want to reduce disk usage and IO calls and do not need to rewind, you can disable caching by passing rewindable: false to Down.open.

# Default behavior (rewindable)
remote_file = Down.open("http://example.com/image.jpg")
remote_file.read(1*1024*1024)
remote_file.rewind
remote_file.read(1*1024*1024) # reads from cache

# Disable caching
Down.open("http://example.com/image.jpg", rewindable: false)

Down supports multiple backends for downloading files. You can use a backend directly by calling its class methods, or set a backend globally for the Down module.

Available backends:

• Down::NetHttp (default)
• Down::Http (requires http gem)
• Down::Httpx (requires httpx gem)

To set a backend globally, use Down.backend :backend_name.

require "down"

# Set the backend globally
Down.backend :http

# Use the global Down methods
Down.download("...")
Down.open("...")

# OR use a backend directly
require "down/http"
Down::Http.download("...")

Add the down gem to your project. It is recommended to use the ~> 5.0 version constraint.

gem "down", "~> 5.0"

Down supports basic authentication in two ways:

1. In the URL: Include credentials directly in the remote URL (e.g., https://user:pass@example.com/file.zip).
2. Via Proxy: If using a :proxy option, provide a URL containing credentials (e.g., proxy: "http://user:pass@proxy.example.com:8080").

Note: When following redirects, Down::NetHttp will not leak credentials to the new location unless auth_on_redirect: true is explicitly passed in the options.

The Down::Http backend uses the http.rb gem. It is optimized for low memory usage (up to 10x less than Net::HTTP) and supports persistent connections and global timeouts.

Installation:

gem "down", "~> 5.0"
gem "http", "~> 5.0"

Usage: Options are passed to HTTP::Client#request. It is recommended to use the chainable builder API when using Down::Http.open to configure the client.

require "down/http"

# Download to a Tempfile
tempfile = Down::Http.download("http://nature.com/forest.jpg")

# Open as an IO object
io = Down::Http.open("http://nature.com/forest.jpg")

# Using the builder API for configuration
Down::Http.open("http://example.org/image.jpg") do |client|
  client.timeout(connect: 3, read: 3)
end

You can control where files are saved or how the temporary files are named using the following options in Down.download:

• :destination: Specifies a specific path on disk. Note that when using this, Down.download returns nil instead of a Tempfile.
• :extension: Overrides the file extension of the returned Tempfile.
• :tempfile_name: Overrides the default prefix used for the Tempfile name.

# Download to a specific path
Down.download("http://example.com/image.jpg", destination: "/path/to/destination")

# Override extension
tempfile = Down.download("http://example.com/some/file", extension: "txt")

# Override tempfile prefix
tempfile = Down.download("http://example.com/image.jpg", tempfile_name: "custom-prefix")

The Down.download method downloads a remote file into a Tempfile. The returned Tempfile includes additional metadata extracted from the HTTP response, such as content_type, original_filename, and charset.

require "down"

tempfile = Down.download("http://example.com/nature.jpg")
tempfile.content_type      #=> "text/plain"
tempfile.original_filename #=> "document.txt"
tempfile.charset           #=> "utf-8"

To process data in chunks without any disk caching (regardless of the :rewindable setting), use the #each_chunk method on a Down::ChunkedIO object.

remote_file = Down.open("http://example.com/image.jpg")
remote_file.each_chunk { |chunk| ... }
remote_file.close

Down.download and Down.open automatically detect and apply HTTP basic authentication if the credentials are provided within the URL.

Down.download("http://user:password@example.org")
Down.open("http://user:password@example.org")

To protect your server from large files, use the :max_size option. Down will terminate the download as soon as it receives a Content-Length header exceeding the limit, or as soon as the downloaded content surpasses the limit if the header is missing. If the limit is exceeded, a Down::TooLarge error is raised.

Down.download("http://example.com/image.jpg", max_size: 5 * 1024 * 1024) # 5 MB
# Raises Down::TooLarge if file exceeds 5MB

Use :content_length_proc and :progress_proc to track download status:

• :content_length_proc: A proc called with the Content-Length value once received.
• :progress_proc: A proc called with the current filesize whenever a new chunk is downloaded.

Down.download "http://example.com/movie.mp4",
  content_length_proc: -> (content_length) { ... },
  progress_proc:       -> (progress)       { ... }