Rails Semantic Logger

repository·main·Indexed 16 days ago

https://github.com/reidmorrison/rails_semantic_logger

A gem that replaces the default Rails logger with structured logging (JSON), enabling Rails, application code, and gems to emit machine-readable logs. It provides configurable appenders for different environments (server, console, and general), supports container platforms like Docker and Kubernetes, and includes specialized logging for Sidekiq jobs and Rack requests.

Tokens
3.7K
Snippets
18
Records
21
Agent score
65%

What's inside rails_semantic_logger

  1. How Rails Semantic Logger works out of the box

    main

    If no configuration is provided, the gem automatically:

    • Writes to log/<environment>.log (colorized if Rails colorized logging is enabled).
    • Logs to standard out ($stdout) when running rails server.
    • Logs to standard error ($stderr) when running rails console (to prevent log lines from mixing with command return values).
    • Replaces multi-line Rails request logs with a single structured "Completed" line containing searchable fields like controller, action, status, and duration.
  2. Run the test suite with Appraisal

    main

    The project uses appraisal to manage and run tests across different versions of Rails and ActiveRecord. You can run the full suite, target a specific Rails version, or run individual test files and cases.

    # Run all supported Rails/ActiveRecord versions
    rake
    
    # Run tests for a specific rails version
    appraisal rails_4.2 rake
    
    # Run a specific test file for a specific rails version
    appraisal rails_5.0 ruby test/controllers/articles_controller_test.rb 
    
    # Run a single test case within a file
    appraisal rails_5.0 ruby test/controllers/articles_controller_test.rb -n "/shows new article/"
  3. Install Rails Semantic Logger

    main

    Add rails_semantic_logger to your Gemfile to replace the default Rails logger with structured logging. You may also optionally add amazing_print to colorize structured payloads in development.

    Important: Remove lograge, rails_stdout_logging, or rails_12factor if they are present, as they conflict with this gem.

    gem "rails_semantic_logger"
    gem "amazing_print" # optional
  4. Configure log destinations using the appenders block

    main

    An appender is a destination for log output (e.g., a file, stdout, or a centralized service). You declare appenders within the config.rails_semantic_logger.appenders block.

    CRITICAL: As soon as you declare any appender in this block, Rails Semantic Logger stops using its automatic default appenders. The block becomes the single source of truth for all log destinations.

    Appender Methods

    The method used determines when the appender is created:

    MethodCreated when...Default destination
    addAlways, during Rails initialization(must specify one)
    add_serverOnly when serving requests (rails server, rack server, Sidekiq in server mode)$stdout
    add_consoleOnly inside a rails console session$stderr

    All methods accept the same arguments as SemanticLogger.add_appender.

    # Example: Development setup with a color log file and color screen output
    config.rails_semantic_logger.appenders do |appenders|
      appenders.add(file_name: "log/#{Rails.env}.log", formatter: :color)
      appenders.add_server(formatter: :color) # → $stdout, only when serving
    end
  5. How appender contexts (add, add_server, add_console) work

    main

    The Appenders class uses different methods to define the context in which an appender is created. All methods accept the same arguments as SemanticLogger.add_appender (such as io, file_name, formatter, etc.).

    • add: Appenders created this way are always created during Rails initialization.
    • add_server: Appenders created this way are only created when the application is serving requests (e.g., rails server, Sidekiq in server mode). If no destination is provided, it defaults to $stdout with a :color formatter.
    • add_console: Appenders created this way are only created inside a rails console session. If no destination is provided, it defaults to $stderr with a :color formatter to prevent log output from interfering with command results.

    Because each call appends to its specific context, you can mix and match. For example, you can have a file appender that always runs (add) and a stdout appender that only runs during a server session (add_server).

    config.rails_semantic_logger.appenders do |appenders|
      # Always active
      appenders.add(file_name: "log/production.log") 
    
      # Only active in rails server / Sidekiq
      appenders.add_server(io: $stdout)
    
      # Only active in rails console
      appenders.add_console(io: $stderr)
    end
  6. Understand Sidekiq job latency and duration metrics

    main

    The JobLogger automatically emits metrics for Sidekiq jobs:

    1. Latency: The time between when the job was enqueued and when it started executing. It is logged with the metric name sidekiq.queue.latency.
    2. Duration: The time taken to run the job. It is logged using measure_info with the metric name sidekiq.job.perform.

    Latency calculation behavior:

    • Sidekiq <= 7: Uses seconds since epoch. The logger converts this to milliseconds.
    • Sidekiq 8+: Uses milliseconds since epoch.
  7. Configure server appenders for non-hooked app servers

    main

    While rails_semantic_logger automatically detects and sets up appenders for rails server and Sidekiq (server mode), it cannot reliably detect other app servers like bare Puma, Rackup, Passenger, or Unicorn. For these servers, you must manually call RailsSemanticLogger.add_server_appenders within the server's definitive boot hook.

    Example for config/puma.rb:

    on_booted { RailsSemanticLogger.add_server_appenders }
  8. Configure JSON logging for container platforms

    main

    For platforms like Docker, Kubernetes, or Heroku, you should log JSON to standard out. This allows the platform to collect logs and enables centralized logging systems to parse fields (including payload and metric data) into a searchable hierarchy.

    Note that by declaring this appender, you replace the default file appender, making JSON to stdout your only destination.

    # Example: Container platform recipe
    config.rails_semantic_logger.appenders do |appenders|
      appenders.add(io: $stdout, formatter: :json)
    end
  9. Configure appenders via config.rails_semantic_logger.appenders

    main

    You can define custom log appenders within a config.rails_semantic_logger.appenders block in your Rails configuration.

    Important Behavior: When you declare at least one appender using this block, Rails Semantic Logger stops building its own default file appender. Standard configuration options like format, ap_options, filter, and add_file_appender will no longer apply; instead, only the appenders explicitly declared in this block will be created.

    config.rails_semantic_logger.appenders do |appenders|
      appenders.add(file_name: "log/#{Rails.env}.log", formatter: :json)
      appenders.add_server(io: $stdout, formatter: :color)
      appenders.add_console(io: $stderr, formatter: :color)
    end