DOCSEARCHbyAlgolia
Submit Docs

Gemstash Documentation

repository·main·Indexed 21 days ago

https://github.com/rubygems/gemstash

Gemstash is a caching proxy and private gem server that allows teams to cache gems from public sources like RubyGems.org to save bandwidth and speed up builds, or to host private gems. It includes a CLI for server management and authorization, support for multiple database adapters (SQLite, Postgres, MySQL), and configurable caching via memory or Memcached.

Tokens
18.6K
Snippets
92
Records
112
Agent score
70%

What's inside Gemstash

  1. Understand Gemstash caching and storage

    main

    Gemstash manages data in the ~/.gemstash directory, which contains cached gems, the SQLite database, server logs, and configuration.

    Caching Behavior

    • Gem files (*.gem): Cached indefinitely (permanently).
    • Gem dependencies metadata: Cached for 30 minutes. If you bundle again within this window, Gemstash can resolve dependencies without an internet connection.

    Storage Components

    • Storage Directory: ~/.gemstash (can be customized).
    • Database: Uses SQLite to store details about private gems. The database file appears in ~/.gemstash once private gems are used.
    • Memory Cache: Gem dependencies are temporarily cached in memory for 30 minutes. This can be replaced with memcached for customization.

  2. Use ERB for dynamic configuration

    main

    If you create a file named ~/.gemstash/config.yml.erb, Gemstash will use it instead of the standard .yml file. This allows you to use Ruby code and environment variables within your configuration.

    # ~/.gemstash/config.yml.erb
---
:db_adapter: postgres
:db_url: <%= ENV["DATABASE_URL"] %>

  3. Use ERB in Gemstash configuration

    main

    If you create a file named ~/.gemstash/config.yml.erb, Gemstash will use it instead of the standard .yml file. This allows you to use Ruby code and environment variables within your configuration.

    # ~/.gemstash/config.yml.erb
---
:db_adapter: postgres
:db_url: <%= ENV["DATABASE_URL"] %>

  4. Gemstash storage and caching behavior

    main

    Gemstash manages files and metadata using the following logic:

    • Storage Location: By default, all cached gems, private gems, logs, and the SQLite database are stored in ~/.gemstash.
    • Gem Files (*.gem): Cached indefinitely.
    • Gem Dependencies Metadata: Cached for 30 minutes in memory. If you attempt to bundle again within this window, Gemstash can resolve dependencies without an internet connection.
    • Database: Uses SQLite to store details about private gems. The database file appears in ~/.gemstash once private gems are used.

  5. Configure Gemstash with a custom config file

    main

    To save your configuration to a specific location instead of the default ~/.gemstash/config.yml, use the --config-file option.

    If you intend to use ERB (Embedded Ruby) within your configuration file, it is recommended to use a filename ending in .yml.erb (e.g., ~/.gemstash/config.yml.erb).

    Note: If you use a custom config file, you must include the --config-file <file> flag in every Gemstash command you run thereafter.

    gemstash setup --config-file /path/to/your/config.yml

  6. Install and start Gemstash

    main

    To use Gemstash as a local cache, install the gem and start the server. By default, the server runs on port 9292.

    1. Install the gem: gem install gemstash

    2. Start the server: gemstash start

    gem install gemstash
gemstash start

  7. Configure Gemstash via interactive setup

    main

    You can use the interactive gemstash setup command to configure your environment. The command will prompt you for settings such as file storage location, cache type, Memcached servers, and database adapter.

    If a configuration check fails (e.g., the database is unavailable), the settings will not be saved. To force a reconfiguration after fixing issues, use the --redo flag.

    Successful configuration is stored in ~/.gemstash/config.yml by default.

    # Run the interactive setup
$ gemstash setup

# Force a reconfiguration if the previous attempt failed
$ gemstash setup --redo

  8. Authenticate with multiple gem sources

    main

    You can provide authentication credentials for upstream sources in two ways:

    1. Directly in the Gemfile

    Include the credentials (basic auth or API keys) within the escaped URL string.

    2. Using Environment Variables

    To keep credentials out of your Gemfile, use the GEMSTASH_<HOST> environment variable pattern. The variable name is constructed as follows:

    1. Prefix with GEMSTASH_.
    2. Convert the hostname to uppercase.
    3. Replace all . (dots) with __ (double underscores).
    4. Replace all - (hyphens) with ___ (triple underscores).

    Example Mapping: my.gem-source.local becomes GEMSTASH_MY__GEM___SOURCE__LOCAL.

    # Method 1: In Gemfile
require "cgi"
source "http://localhost:9292/upstream/#{CGI.escape("user:password@my.gem-source.local")}" do
  gem "my-gem"
end

# Method 2: Via ENV variable (Gemfile remains clean)
source "http://localhost:9292/upstream/my.gem-source.local" do
  gem "my-gem"
end
    # Running Gemstash with credentials
GEMSTASH_MY__GEM___SOURCE__LOCAL=user:password gemstash start --config-file config.yml.erb

  9. Bundle with multiple gem sources using /upstream

    main

    To bundle against multiple gem sources simultaneously, use the /upstream/ prefix in your Gemfile sources. This tells Gemstash to fetch from a source other than the configured default.

    When providing a full URL (including the scheme like https://) as an upstream source, you must use CGI.escape to properly URL-encode the source URL so Gemstash can parse it correctly.

    If the source URL does not contain special characters and you rely on the default https:// scheme, escaping may not be strictly necessary, but using CGI.escape is the recommended pattern for reliability.

    require "cgi"

# Uses the Gemstash default source
source "http://localhost:9292"
source "gem_from_default"

# Uses an upstream source (requires escaping the URL)
source "http://localhost:9292/upstream/#{CGI.escape("https://my.gem-source.local")}" do
  gem "my-gem"
end

# Another upstream source
source "http://localhost:9292/upstream/my-other.gem-source.local" do
  gem "my-other-gem"
end

  10. Redo or update Gemstash configuration

    main

    If you have already run gemstash setup and wish to change your settings, you must use the --redo flag. Without this flag, Gemstash will detect the existing configuration and simply inform you that the setup is already complete without prompting you for new answers.

    gemstash setup --redo

  11. Bundle with multiple gem sources using the /upstream prefix

    main

    To use gem sources other than the default configured in ~/.gemstash/config.yml, use the /upstream/ prefix in your Gemfile. This tells Gemstash to fetch from the specified external source and stash the gems locally.

    When using a full URL as the upstream source, you must use CGI.escape to ensure the URL is properly escaped for the Gemstash path. If the source does not contain special characters (and you omit the scheme, as Gemstash defaults to https://), escaping may not be strictly necessary.

    require "cgi"
source "http://localhost:9292"

# Using an escaped full URL
source "http://localhost:9292/upstream/#{CGI.escape("https://my.gem-source.local")}" do
  gem "my-gem"
end

# Using a host without a scheme (defaults to https)
source "http://localhost:9292/upstream/my-other.gem-source.local" do
  gem "my-other-gem"
end

  12. Deploy and upgrade Gemstash using Bundler

    main

    To manage Gemstash versions and ensure a consistent environment, use Bundler. Create a Gemfile that includes gemstash, run bundle to generate a Gemfile.lock, and use bundle update to upgrade. When running Gemstash via Bundler, prefix the command with bundle exec.

    Gemstash handles database migrations automatically upon upgrading. For a safe upgrade process, it is recommended to stop the server, perform the update, and then start the server again.

    # ./Gemfile
source "https://rubygems.org"
gem "gemstash"

    Recommended upgrade workflow

    $ bundle exec gemstash stop $ bundle update $ bundle exec gemstash start