ruby-pg

repository·master·Indexed 21 days ago

https://github.com/ged/ruby-pg

The official Ruby interface to the PostgreSQL RDBMS. It provides low-level access to PostgreSQL features, including connection management, advanced type casting via PG::BasicTypeMapForResults and PG::BasicTypeMapForQueries, and support for Ruby 3.0's Fiber.scheduler and Ractors.

Tokens
14.1K
Snippets
57
Records
71
Agent score
74%

What's inside pg

  1. Dependency management for pg

    master

    The pg gem follows Semantic Versioning. It is recommended to use pessimistic version constraints when adding pg as a dependency in your gemspec to ensure compatibility while allowing minor updates.

    Example:

    spec.add_dependency 'pg', '~> 1.0'
  2. Use pg with Ractors

    master

    Since version 1.5.0, pg is compatible with Ruby 3.0 Ractors.

    Sharing Data:

    • Type en/decoders and type maps can be shared between Ractors if they are made shareable using Ractor.make_shareable.
    • Frozen PG::Result and PG::Tuple objects can be shared.
    • All frozen objects (except PG::Connection) can be used to communicate with the PostgreSQL server or to read retrieved data.

    Connection Management: PG::Connection is not shareable. You must create a new PG::Connection instance within each Ractor to establish a dedicated connection for that Ractor.

    # Example of sharing a result across Ractors
    # (Assumes 'result' was created and then frozen)
    Ractor.new(result) do |shared_result|
      # Use the frozen PG::Result here
      puts shared_result.getvalue(0, 0)
    end
  3. How TypeMaps work

    master

    A PG::TypeMap defines which encoder or decoder should be used for specific values. You can assign a type map to a specific result set, a connection, or a single query. Type maps can also be used for PG::Connection#copy_data streaming.

    Available TypeMap strategies:

    • PG::TypeMapAllStrings: The default. Encodes/decodes everything as strings.
    • PG::TypeMapByClass: Selects an encoder based on the Ruby class of the value being sent.
    • PG::TypeMapByColumn: Selects encoders/decoders based on column position.
    • PG::TypeMapByOid: Selects decoders based on the PostgreSQL OID.
    • PG::TypeMapInRuby: Allows defining custom type mappings in Ruby.

    Pre-configured TypeMaps (based on PG::BasicTypeRegistry):

    • PG::BasicTypeMapForResults: A PG::TypeMapByOid pre-loaded with decoders for common PostgreSQL column types.
    • PG::BasicTypeMapBasedOnResult: A PG::TypeMapByOid pre-loaded with encoders for common PostgreSQL column types.
    • PG::BasicTypeMapForQueries: A PG::TypeMapByClass pre-loaded with encoders for common Ruby value classes.

    You can chain multiple type maps together by setting PG::TypeMap::DefaultTypeMappable#default_type_map.

  4. Thread safety in pg

    master

    The pg gem is thread-safe in that different threads or fibers can use different PG::Connection objects concurrently.

    Warning: It is not safe to access a single PG object (like a connection or result) simultaneously from more than one thread or fiber unless that object is frozen. To avoid concurrency issues, open a new database connection for every new thread or use a connection pooler/wrapper like ActiveRecord.

  5. Use Fiber IO scheduler with pg

    master

    Since version 1.3.0, pg is fully compatible with Ruby 3.0's Fiber.scheduler. All potentially blocking IO operations are automatically routed through the registered Fiber.scheduler for the running thread. This is achieved by pg internally using the asynchronous libpq interface even for synchronous method calls and using Ruby's DNS resolution instead of libpq's built-in functions.

    Note on Non-blocking Mode: Internally, pg always uses libpq's non-blocking connection mode. If you call PG::Connection#setnonblocking(true), the non-blocking state remains enabled, but the automatic handling of blocking states is disabled, requiring your program to handle them manually.

    Exceptions to Fiber compatibility: Some operations will work but will not allow the IO scheduler to switch to another Fiber during waiting states (they will block the thread):

    • Large object methods (e.g., PG::Connection#lo_create)
    • Authentication using external libraries (GSSAPI, LDAP)
    • LDAP lookup of connection parameters
    • Connection strings/hashes using the service parameter without explicit host and port.

    Workarounds for service-based connections: To maintain Fiber.scheduler compatibility when using service files, use one of these instead:

    • Set the service via the PGSERVICE environment variable.
    • Provide host and port explicitly in the connection string or hash.
    • Set the hostaddr in the service file.
    # Example of a connection that might block the Fiber scheduler if 'service' is used without host/port
    conn = PG.connect(service: 'my_service') 
    
    # Recommended for Fiber compatibility:
    conn = PG.connect(service: 'my_service', host: 'localhost', port: 5432)
  6. Fiber IO Scheduler support

    master

    Since version 1.3.0, pg supports Ruby 3.0's Fiber.scheduler. If the thread running the scheduler is registered, all blocking IO operations performed by pg will be handled via the scheduler.

    Internally, pg always uses libpq's non-blocking connection mode but behaves as if it were in blocking mode, ensuring that IO waits are controlled by the Ruby Fiber.scheduler whenever possible.

    Note:

    • On Windows, Fiber.scheduler support requires Ruby 3.1 or later.
    • If you call PG::Connection.setnonblocking(true), the non-blocking state remains active, and you must manually manage the blocking state.
    • Certain methods like PG::Connection#lo_create or authentication methods (e.g., GSSAPI) are incompatible with Fiber.scheduler and will block the thread without allowing the scheduler to switch to other fibers.
  7. Thread safety and concurrency

    master

    The pg gem is thread-safe in the sense that individual threads or fibers can each use their own PG::Connection object concurrently.

    CRITICAL: It is NOT safe for one or more threads/fibers to access the same PG object simultaneously. You must either open a new database connection for every thread/fiber or use a thread-safe connection manager like ActiveRecord.

    If you see the following error messages in stderr, it is a sign that multiple threads are attempting to use a single connection:

    message type 0x31 arrived from server while idle
    message type 0x32 arrived from server while idle
    message type 0x54 arrived from server while idle
    message type 0x43 arrived from server while idle
    message type 0x5a arrived from server while idle
  8. How Encoders and Decoders work

    master

    Encoders and Decoders are low-level classes used to transform data between Ruby objects and the DBMS. They are categorized by the format used for transfer:

    • PG::TextEncoder / PG::TextDecoder: For text-based formats.
    • PG::BinaryEncoder / PG::BinaryDecoder: For binary-based formats.

    These objects can be assigned OIDs, format codes, or custom names. They can also be composed to build complex types. PG::Coder objects can be used to set up PG::TypeMap or to perform simple single-value conversions between Ruby and string representations.

  9. Ractor compatibility

    master

    Since version 1.5.0, pg is compatible with Ruby 3.0 Ractors.

    • Sharing: You can share type encoders, decoders, and type maps between Ractors if they are made shareable via Ractor.make_shareable (i.e., they are frozen). Frozen PG::Result and PG::Tuple objects can also be shared. Most frozen objects can be used to interact with the server or read data.
    • Connections: PG::Connection objects cannot be shared. You must establish a new connection within each Ractor.
  10. Install the pg gem from source

    master

    If you need specific features not supported by binary gems (such as LDAP connection options) or if the binary gem fails, you can force installation from source. You may need to provide the path to the pg_config executable provided by your PostgreSQL installation.

    Steps to force source installation:

    1. Uninstall existing versions: gem uninstall pg --all
    2. Install using the ruby platform: gem install pg --platform ruby

    Providing pg_config path:

    • Via CLI: gem install pg -- --with-pg-config=<path to pg_config>
    • Via Bundler: bundle config build.pg --with-pg-config=<path to pg_config>
    gem uninstall pg --all
    gem install pg --platform ruby
    
    # Or with a specific pg_config path
    gem install pg -- --with-pg-config=/usr/local/bin/pg_config
  11. Run tests for the pg gem

    master

    You can run the test suite using rake test. This will execute tests against the PostgreSQL version pointed to by pg_config --bindir.

    To run a specific test file at a specific line number using a specific PostgreSQL version, prepend the desired PostgreSQL bin directory to your PATH and use rspec with the -I (lib path) and -fd (file and line) flags.

    # Run all tests against the current pg_config version
    $ rake test
    
    # Run a specific test at a specific line with a specific PostgreSQL version
    $ PATH=/usr/lib/postgresql/14/bin:$PATH rspec -Ilib -fd spec/pg/connection_spec.rb:455
  12. Use the 'pg' gem instead of 'postgres'

    master

    The postgres gem is an old, deprecated version of the Ruby PostgreSQL driver that has not been maintained since early 2008. For all modern development, you should install and require the pg gem instead.

    If you are working on modern projects, use:

    gem install pg