doobie Documentation

repository·main·Indexed 24 days ago

https://github.com/typelevel/doobie

A pure functional JDBC layer for Scala that provides a type-safe way to interact with relational databases. It includes abstractions for SQL queries via Query and Query0, SQL composition using Fragment, and tools for managing ResultSets and Statements, including fs2 streaming support.

Tokens
34K
Snippets
81
Records
148
Agent score
81%

What's inside doobie

  1. Explore Doobie ecosystem and related projects

    main

    Doobie can be supplemented by several community and ecosystem projects to extend its functionality:

    • DoobieRoll: A collection of utilities designed to simplify working with Doobie and SQL.
      • TableColumns: Used to ensure that fields in your SQL queries are consistently named and ordered.
      • Assembler: Used to assemble SQL query results into hierarchical domain models.
    • doobie-typesafe: Provides type-safe table definitions for use with Doobie.
    • otel4s-doobie: Provides integration with Otel4s for observability.
  2. Use the low-level JDBC API with `doobie.free`

    main

    Doobie provides a low-level API that encodes JDBC operations as a free monad. This allows you to perform fine-grained operations on JDBC objects. Each standard JDBC type has a corresponding free monad counterpart:

    • doobie.free.ConnectionIO / doobie.FC: Operations over java.sql.Connection.
    • doobie.free.PreparedStatement / doobie.FPS: Operations over java.sql.PreparedStatement.
    • doobie.free.ResultSetIO / doobie.FRS: Operations over java.sql.ResultSet.

    Warning: When using the low-level API, you are responsible for resource management. You must ensure that Connections, PreparedStatements, and ResultSets are properly closed (e.g., using .bracket or similar resource safety patterns) to avoid leaks.

  3. How PostgreSQL type mappings work in doobie

    main

    doobie provides extended support for many PostgreSQL types not natively supported by JDBC. Most mappings are provided in the pgtypes module. To enable them, use:

    import org.typelevel.doobie.postgres._
    import org.typelevel.doobie.postgres.implicits._

    Java 8 Time Types (JSR310)

    Use the following mappings for correct timezone handling:

    • TIMESTAMP $\rightarrow$ java.time.LocalDateTime
    • TIMESTAMPTZ $\rightarrow$ java.time.Instant or java.time.OffsetDateTime
    • DATE $\rightarrow$ java.time.LocalDate
    • TIME $\rightarrow$ java.time.LocalTime

    Array Types

    doobie supports single-dimensional arrays of these types:

    • bit[] $\rightarrow$ Array[Boolean]
    • int4[] $\rightarrow$ Array[Int]
    • int8[] $\rightarrow$ Array[Long]
    • float4[] $\rightarrow$ Array[Float]
    • float8[] $\rightarrow$ Array[Double]
    • varchar[], char[], text[], bpchar[] $\rightarrow$ Array[String]
    • uuid[] $\rightarrow$ Array[UUID]

    Note: You can also map to List and Vector. Arrays of int2 are not supported and are incorrectly mapped as Array[Int] by the driver.

    Other Nonstandard Types

    • uuid $\rightarrow$ java.util.UUID
    • inet $\rightarrow$ java.net.InetAddress
    • hstore $\rightarrow$ java.util.Map[String, String] or Map[String, String]
  4. Handle NULLs in SQL arrays

    main

    When working with arrays, you must distinguish between the column itself being NULL and the individual cells within the array being NULL. There are four primary mapping strategies. Note that if you attempt to read a NULL cell into a non-optional type, a NullableCellRead exception will be thrown.

    ScenarioMapping Type
    Non-nullable column, non-nullable cellsList[String]
    Nullable column, non-nullable cellsOption[List[String]]
    Non-nullable column, nullable cellsList[Option[String]]
    Nullable column, nullable cellsOption[List[Option[String]]]

    Example of the four mapping variations:

    sql"select array['foo','bar','baz']".query[List[String]].quick.unsafeRunSync()
    sql"select array['foo','bar','baz']".query[Option[List[String]]].quick.unsafeRunSync()
    sql"select array['foo',NULL,'baz']".query[List[Option[String]]].quick.unsafeRunSync()
    sql"select array['foo',NULL,'baz']".query[Option[List[Option[String]]]].quick.unsafeRunSync()
  5. Understand the `Write` typeclass and parameter setting

    main

    When using high-level APIs like the sql interpolator, Doobie uses the Write[A] typeclass to map Scala values to SQL parameters.

    • Write instances are automatically derived for types that have a Put instance.
    • Write instances are also derived for products (like case classes or tuples) of other writable types.
    • The HPS.set constructor takes a value with a Write instance and sets the unrolled sequence of values starting at the specified index (defaulting to 1).

    At the low level (doobie.free), you must use specific methods for each type (e.g., FPS.setString, FPS.setBoolean), but the Write and Put typeclasses abstract this away.

  6. Handle whitespace in Fragment composition

    main

    By default, fragments created with fr or Fragment.const have a single trailing space appended. To control whitespace:

    1. Remove trailing space: Use the fr0 interpolator or the Fragment.const0 constructor. These yield fragments without a trailing space.
    2. Ensure spacing: Use the +~+ operator to concatenate two fragments. This operator ensures that at least one space exists between them, which is useful when combining fragments that might not have trailing/leading whitespace.

    Note: sql is an alias for fr0.

    // Using fr0 to avoid trailing spaces
    fr"IN (" ++ List(1, 2, 3).map(n => fr0"$n").intercalate(fr",") ++ fr")"
    
    // Using +~+ to ensure spacing between fragments
    import Fragment.const0
    def codeCondFrag: Fragment = fr0"code = 'USA'"
    def populationCondFrag: Fragment = fr0"population > 1000000"
    
    const0("SELECT code, name, population FROM country\n") +~+
      fr0"WHERE" +~+ codeCondFrag +~+ fr0"AND" +~+ populationCondFrag +~+
      fr0"ORDER BY population DESC"
  7. Use the high-level API with `doobie.hi`

    main

    The doobie.hi (High-level Interface) modules build upon the low-level doobie.free API but automatically handle resource management (closing Connections, PreparedStatements, and ResultSets) and logging. This is the recommended way to perform custom JDBC operations without the risk of resource leaks.

    import cats.effect.IO
    import cats.effect.unsafe.implicits.global // To allow .unsafeRunSync
    import org.typelevel.doobie.Transactor
    
    // Create the transactor
    val xa: Transactor[IO] = Transactor.fromDriverManager[IO](
      driver = "org.postgresql.Driver", 
      url = "jdbc:postgresql:world",   
      user = "postgres",
      password = "password",
      logHandler = None                  
    )
    
    import org.typelevel.doobie.HC  // High-level API over java.sql.Connection
    import org.typelevel.doobie.HRS  // High-level API over java.sql.ResultSet
    import org.typelevel.doobie.ConnectionIO
    import org.typelevel.doobie.util.log.{LoggingInfo, Parameters}
    import cats.effect.unsafe.implicits.global
    import org.typelevel.doobie.util.unlabeled
    
    val sql = "SELECT * FROM (VALUES (1, '1'), (2, '2'))"
    val program: ConnectionIO[List[(Int, String)]] = HC.executeWithResultSet(
      create = FC.prepareStatement(sql),
      prep = FPS.unit,
      exec = FPS.executeQuery,
      process = HRS.list[(Int, String)],
      loggingInfo = LoggingInfo(sql, Parameters.NonBatch(List.empty), label = unlabeled)
    )
    
    program.transact(xa).unsafeRunSync()
  8. How Column Vector Mappings work with Read and Write

    main

    While Get and Put handle mappings between Scala types and single columns, Read and Write handle mappings between Scala types and heterogeneous vectors (multiple columns).

    • Read[A]: Maps a vector of schema types to a Scala type A. This is used to read rows as composite values.
    • Write[A]: Maps a Scala type A to a vector of schema types. This is used to set multiple statement parameters (e.g., in a VALUES clause).

    Automatic Derivation Rules

    Doobie can automatically derive Read and Write instances for most composite types based on these rules:

    1. Base Cases:
      • Zero-width types: Unit and HNil.
      • Single-column types: Any type with a Get or Put instance, including Options of those types.
    2. Inductive Cases:
      • Shapeless HLists (if elements are readable/writable).
      • Shapeless records (if values are readable/writable).
      • Product types (case classes or tuples) via their Generic representation.

    Handling Nullability

    Doobie provides Read[Option[A]] and Write[Option[A]] for the cases above, mapping all columns to nullable schema types. This is useful for mapping results from an OUTER JOIN to optional data types, such as (Parent, Option[Child]).

  9. Use checkOutput to work around limited JDBC metadata

    main

    Some JDBC drivers (notably MySQL and MS-SQL) may have incomplete metadata implementation. If a driver fails to provide metadata for prepared statement parameters, the .check method might throw an exception or return inaccurate results.

    In these cases, you can use .checkOutput instead of .check. This instructs doobie to ignore the input parameter metadata and only validate the output column metadata. This is a useful workaround when parameter metadata is unavailable but column metadata is present.

  10. Stream results directly as an IO Stream

    main

    While most queries are discharged into a collection (like List[A]), you can treat the Stream[ConnectionIO, A] as your top-level type. This is useful for integration with libraries like http4s, where you can stream a resultset directly to a network socket by calling .transact(xa) directly on the stream.

    Example of creating a Stream[IO, A]:

    val p: Stream[IO, Country2] = {
      sql"select name, population, gnp from country"
        .query[Country2]
        .stream
        .transact(xa)
    }
    val p: Stream[IO, Country2] = {
      sql"select name, population, gnp from country"
        .query[Country2] // Query0[Country2]
        .stream          // Stream[ConnectionIO, Country2]
        .transact(xa)    // Stream[IO, Country2]
    }
    
    p.take(5).compile.toVector.unsafeRunSync().foreach(println)
  11. How nested programs and lifting work in Doobie

    main
    Doobie is a monadic API where different data types describe computations in different contexts. For example, ConnectionIO describes computations that require a java.sql.Connection. These programs compose naturally via lifting, allowing you to mirror the lifecycles of the underlying JDBC carrier types. Many common patterns, such as reading query results as a stream or validating schemas, are provided via the high-level API to allow most programs to be written entirely in terms of ConnectionIO.