sqlcommenter

repository·master·Indexed 20 days ago

https://github.com/google/sqlcommenter

A set of tools to inject application-level metadata into SQL queries as comments, enabling database administrators and developers to trace queries back to the specific application code, service, or request. It provides support for Go (including gorilla/mux and net/http) and Java (including Spring and Hibernate) to improve database observability.

Tokens
16K
Snippets
61
Records
76
Agent score
70%

What's inside sqlcommenter

  1. Overview of sqlcommenter implementations

    master

    sqlcommenter is a project donated to OpenTelemetry that provides various language-specific implementations to inject metadata into SQL comments. This allows for better observability by linking database queries back to the application code that generated them.

    Available implementations include support for:

    • Python: Django, psycopg2, and SQLAlchemy
    • Java: Hibernate and Spring+Hibernate
    • Ruby: Rails
    • Node.js: Knex.js and Sequelize.js
    • PHP: Laravel
  2. How sqlcommenter_rails works with Marginalia and OpenCensus

    master

    This gem works by registering an opencensus component and appending it to the list of default Marginalia components.

    In the default configuration, the OpenCensus trace is automatically appended to the end of the Marginalia comments. The trace consists of OpenCensus Span names from the current span up to the root, joined by the ~ character.

    For detailed configuration options regarding the trace format, refer to the Marginalia::OpenCensus documentation.

  3. How to use Spring and Hibernate together with SQLCommenter

    master

    To achieve full correlation when using Spring with Hibernate, you must apply both integrations:

    1. Add the Spring interceptor (SpringSQLCommenterInterceptor) to your Spring MVC configuration to capture web context.
    2. Add the Hibernate StatementInspector (com.google.cloud.sqlcommenter.schibernate.SCHibernate) to your Hibernate configuration to inject that context into SQL comments.
  4. Run the Express + OpenTelemetry example app

    master

    This example demonstrates an Express application connecting to a Cloud SQL database (Postgres or MySQL) using either Sequelize or Knex. It uses OpenTelemetry to collect trace data and send it to Cloud Trace. SQLCommenter automatically attaches the OpenTelemetry trace and span ID to the SQL queries as comments for correlation.

    Prerequisites

    Set the following environment variables to configure your database connection:

    • DBHOST: The hostname, IP, or path to the unix socket directory for the Cloud SQL database (e.g., /cloudsql/my-project:us-central1:my-db when using Cloud SQL proxy).
    • DBUSERNAME: Database username.
    • DBPASSWORD: Database password.
    • DBDIALECT: (Optional) Set to postgres (default) or mysql.

    Setup and Execution

    1. Install dependencies:

      npm install
    2. Initialize data: Before the first run, create sample data using:

      npm run createTodos
    3. Start the server: You can choose to run either the Sequelize-based server or the Knex-based server:

      # For Sequelize
      npm run sequelizeServer
      
      # For Knex
      npm run knexServer
    4. Test the endpoint: In a separate terminal, trigger the server with curl:

      curl http://localhost:8000
    # Example setup sequence
    export DBHOST="<hostname or unix socket path>"
    export DBUSERNAME="<database user>"
    export DBPASSWORD="<database password>"
    export DBDIALECT="postgres"
    
    npm install
    npm run createTodos
    npm run sequelizeServer
  5. Set up the sqlcommenter_rails demo application

    master

    This demo application demonstrates the integration of sqlcommenter_rails.

    Note: If you are using Rails version 7.0 or higher, use the [sqlcommenter-ruby README] instead.

    Prerequisites

    • Ruby v2.6.3 or higher.
    • A specific fork of marginalia that includes required formatting functionality.

    Installation Steps

    1. Clone the required marginalia fork (must be installed one directory above the demo):
      git clone https://github.com/modulitos/marginalia.git ../marginalia
      git -C ../marginalia checkout formatting
    2. Install dependencies and prepare the database:
      bin/setup
    3. Start the Rails server:
      bin/rails s
    4. Monitor SQL queries (run in a separate terminal):
      tail -f log/development.log | grep 'Post '
    # Install marginalia fork
    git clone https://github.com/modulitos/marginalia.git ../marginalia
    git -C ../marginalia checkout formatting
    
    # Setup demo
    bin/setup
    
    # Start server
    bin/rails s
    
    # Monitor logs
    tail -f log/development.log | grep 'Post '
  6. Use SQLCommenter middleware with gorilla/mux

    master

    The sqlcommentermux package provides a middleware that extracts SQLCommenter HTTP request tags from a request handled by gorilla/mux and attaches them to the request's context.

    To ensure these tags and trace information (like traceparent) are passed into SQL comments, you must use the context returned by the middleware when executing queries via sqlcommenter/go/database/sql.

    import (
        "net/http"
        sqlcommentermux "github.com/google/sqlcommenter/go/gorrila/mux"
        "github.com/gorilla/mux"
    )
    
    func runApp() {
        r := mux.NewRouter()
        r.Use(sqlcommentermux.SQLCommenterMiddleware)
    
        r.HandleFunc("/", ActionHome).Methods("GET")
    
        http.ListenAndServe(":8081", r)
    }
  7. Use the go-sql-driver with context

    master

    To use SQLcommenter, use the gosql.Open method provided by the github.com/google/sqlcommenter/go/database/sql package.

    Because of how Go handles information passing, it is highly recommended to use context-aware methods from the DB interface (such as QueryContext, ExecContext, and PrepareContext) to ensure application-related information is correctly passed from your framework to the database driver.

    import (
        gosql "github.com/google/sqlcommenter/go/database/sql"
        sqlcommentercore "github.com/google/sqlcommenter/go/core"
        _ "github.com/lib/pq" // or any other database driver
    )
    
    db, err := gosql.Open("<driver>", "<connectionString>", sqlcommentercore.CommenterOptions{
        Config: sqlcommentercore.CommenterConfig{<flag>:bool}
        Tags  : sqlcommentercore.StaticTags{<tag>: string} // optional
    })
  8. Install sqlcommenter_rails from source

    master

    Note: If you are using Rails 7.0 or higher, use the sqlcommenter-ruby gem instead of this one.

    sqlcommenter_rails is not currently available on RubyGems and must be installed from source. It requires a specific fork of marginalia that includes functionality from an open PR.

    Follow these steps to install:

    1. Clone the required marginalia fork into the directory above your project:
      git clone https://github.com/modulitos/marginalia.git ../marginalia
    2. Add the gems to your Gemfile using local paths:
      gem 'sqlcommenter_rails', path: '../sqlcommenter_rails'
      gem 'marginalia', path: '../marginalia'
      gem 'marginalia-opencensus', path: '../marginalia-opencensus'
    3. Run the setup command to install dependencies:
      bin/setup
    gem 'sqlcommenter_rails', path: '../sqlcommenter_rails'
    gem 'marginalia', path: '../marginalia'
    gem 'marginalia-opencensus', path: '../marginalia-opencensus'
  9. Configure SpringSQLCommenterInterceptor via XML

    master

    You can add the SpringSQLCommenterInterceptor as a bean in your Spring XML configuration. You can apply it globally to all interceptors or restrict it to specific URL mappings.

    <!-- Global configuration -->
    <mvc:interceptors>
        <bean class="com.google.cloud.sqlcommenter.interceptors.SpringSQLCommenterInterceptor"></bean>
    </mvc:interceptors>
    
    <!-- Specific path configuration -->
    <mvc:interceptors>
        <mvc:interceptor>
            <mvc:mapping path="/flights"></mvc:mapping>
            <bean class="com.google.cloud.sqlcommenter.interceptors.SpringSQLCommenterInterceptor"></bean>
        </mvc:interceptor>
    </mvc:interceptors>
  10. Install marginalia-opencensus

    master

    To use marginalia-opencensus in your Ruby application, add it to your Gemfile and run bundle install, or install it directly via the gem command.

    This gem adds OpenCensus support to Marginalia, allowing OpenCensus trace information to be appended to SQL comments.

    # Add to your Gemfile
    gem 'marginalia-opencensus'

    Then execute in your terminal

    $ bundle

    Or install manually

    $ gem install marginalia-opencensus