Panko Serializer

repository·master·Indexed 20 days ago

https://github.com/yosiat/panko_serializer

A high-performance Ruby library for serializing ActiveRecord and Ruby objects to JSON. Optimized for speed via the Oj engine, pre-computed serialization descriptors, and C-level type casting, Panko provides a declarative syntax inspired by ActiveModelSerializers 0.9. It supports field and method attributes, has_one and has_many associations, and advanced filtering using :only and :except options.

Tokens
8.1K
Snippets
31
Records
37
Agent score
67%

What's inside panko_serializer

  1. Overview of Panko Serializer

    master

    Panko is a high-performance library designed for serializing ActiveRecord and Ruby objects into JSON strings. It is inspired by ActiveModelSerializers 0.9 but optimized for speed.

    To achieve high performance, Panko utilizes several key strategies:

    • Oj Integration: Uses the Oj gem for fast, incremental serialization via Oj::StringWriter.
    • Serialization Descriptors: Computes most metadata ahead of time to minimize overhead during the actual serialization process.
    • Self-contained Type Casting: Performs its own type casting rather than relying on the slower ActiveRecord implementation.
  2. Introduction to Panko Serializer

    master

    Panko is a high-performance library designed for serializing ActiveRecord and Ruby objects to JSON strings. It is inspired by ActiveModelSerializers 0.9 but optimized for speed.

    To achieve its performance goals, Panko utilizes:

    • Oj: It relies on the Oj library for fast, incremental serialization using Oj::StringWriter.
    • Serialization Descriptors: Most metadata is computed ahead of time to minimize overhead during the actual serialization process.
    • Self-contained Type Casting: Panko performs its own type casting rather than relying on ActiveRecord, reducing latency.
  3. Compare Panko performance against ActiveModelSerializers

    master

    Panko is designed for high-performance serialization. Benchmarks show significant improvements in iterations per second (ip/s) compared to ActiveModelSerializers (AMS) for both simple and complex (HasOne) associations. In real-world scenarios involving large datasets (e.g., 7,884 entries with 48 attributes), Panko significantly reduces average response times and increases total request throughput.

    ### Microbenchmark Comparison (ip/s)
    | Benchmark         | AMS ip/s | Panko ip/s |
    | ----------------- | -------- | ---------- |
    | Simple_Posts_2300 | 11.72    | 523.05     |
    | Simple_Posts_50   | 557.29   | 23,011.9   |
    | HasOne_Posts_2300 | 5.91     | 233.44     |
    | HasOne_Posts_50   | 285.8    | 10,362.79  |
    
    ### Real-world Benchmark (Large Dataset)
    | Metric             | AMS   | Panko |
    | ------------------ | ----- | ----- |
    | Avg Response Time  | 4.89s | 1.48s |
    | Max Response Time  | 5.42s | 1.83s |
    | 99th Response Time | 5.42s | 1.74s |
    | Total Requests     | 61    | 202   |
  4. How Panko achieves high performance

    master

    Panko achieves high performance through three primary design choices:

    1. Oj::StringWriter: Instead of building intermediate Ruby hashes (which is memory and CPU intensive), Panko uses Oj::StringWriter to incrementally serialize values directly into a JSON string in C.
    2. Ahead-of-time Metadata (Serialization Descriptor): Panko builds a Serialization Descriptor during initialization. This descriptor pre-calculates which fields are properties, which are methods, and handles only/except filtering. This minimizes the number of questions asked during the actual serialization loop.
    3. C-level Type Casting: Panko performs type casting in C to avoid the overhead of ActiveRecord's Ruby-based type casting. It avoids expensive operations like duplicating strings or converting time strings into full Ruby Time objects if they can be formatted directly for JSON.
  5. How Panko infers serializers for associations

    master

    Panko can automatically detect the correct serializer by looking at the relationship name, allowing you to omit the serializer: or each_serializer: option.

    Inference Logic:

    1. Take the relationship name (e.g., :author or :comments).
    2. Singularize and camelize the name.
    3. Look for a constant defined with that name followed by the Serializer suffix (using Object.const_get).

    Example:

    class PostSerializer < Panko::Serializer
      attributes :title, :body
    
      has_one :author
      has_many :comments
    end

    Error Handling: If Panko cannot find the inferred serializer, it will throw an error at startup time, such as: Can't find serializer for PostSerializer.author has_one relationship.

  6. How Panko handles Time type casting

    master

    Panko optimizes time serialization by targeting the final JSON requirement: a UTC ISO8601 formatted string. Instead of converting database strings into Ruby Time objects, it uses optimized logic:

    • If a string ends with Z and matches the UTC ISO8601 regex, it is returned as-is.
    • If a string is in a standard database timestamp format, Panko uses regex and string concatenation to convert it to UTC ISO8601 directly in C.
    • If neither condition is met, it falls back to ActiveRecord's standard type casting.
  7. Define attributes in a Panko Serializer

    master

    Attributes determine which data from a record is included in the serialized output. Panko supports two types of attributes:

    1. Field Attributes: Simple columns defined directly on the ActiveRecord object. Panko performs its own type casting on these fields for improved performance.
    2. Method Attributes: Derived values calculated via methods defined within the serializer. These methods can access the record being serialized using the object keyword.

    You can also pass a context hash to the serializer instance to provide external data (like feature flags) that the serializer methods can access.

    class UserSerializer < Panko::Serializer
      # Field attribute
      attributes :id, :email
    
      # Method attribute using 'object'
      attributes :full_name
      def full_name
        "#{object.first_name} #{object.last_name}"
      end
    
      # Method attribute using 'context'
      attributes :feature_flags
      def feature_flags
        context[:feature_flags]
      end
    end
    
    # Usage with context
    serializer = UserSerializer.new(context: { feature_flags: ['flag_a'] })
    serializer.serialize(User.first)
  8. Use nested filters to restrict association attributes

    master

    Panko allows you to use the only: option to filter attributes not just for the main object, but also for its associations. This allows you to reuse existing serializers while controlling the depth and breadth of the data returned.

    The only option structure:

    • instance: A list of attributes (and associations) to serialize for the current level of the serializer.
    • [association_name]: A list of attributes to serialize for a specific association.

    Note: Nested filters are recursive. You can filter an association's associations by nesting further keys inside the association's configuration.

    Example: Filtering a list of posts with specific author and comment fields:

    posts = Post.all
    
    Panko::ArraySerializer.new(posts, each_serializer: PostSerializer, only: {
      instance: [:title, :body, :author, :comments],
      author: [:id],
      comments: [:id],
    })

    Example: Deeply nested filtering (filtering the author of a comment):

    posts = Post.all
    
    Panko::ArraySerializer.new(posts, each_serializer: PostSerializer, only: {
      instance: [:title, :body, :author, :comments],
      author: [:id],
      comments: {
        instance: [:id, :author],
        author: [:name]
      }
    })

    In the second example, the instance key inside comments refers to the attributes for the CommentSerializer.

  9. Create a Panko serializer

    master

    Define a serializer by inheriting from Panko::Serializer. Use the attributes method to specify which fields to include. You can also define associations using has_many or has_one, specifying the target serializer if necessary.

    class PostSerializer < Panko::Serializer
      attributes :title
    end
    
    class UserSerializer < Panko::Serializer
      attributes :id, :name, :age
    
      has_many :posts, serializer: PostSerializer
    end
  10. How to use Panko to serialize ActiveRecord objects

    master

    Panko is a high-performance serializer specifically for ActiveRecord objects. It uses a declarative syntax inspired by ActiveModelSerializers to define which attributes and methods should be included in the JSON output.

    To use Panko:

    1. Define a serializer class inheriting from Panko::Serializer.
    2. Use the attributes method to specify fields.
    3. Define custom methods within the serializer to handle complex logic or combined fields.
    4. Instantiate the serializer and call serialize_to_json(object).

    Note: Panko cannot serialize objects other than ActiveRecord objects.

    class UserSerializer < Panko::Serializer
      attributes :name, :age, :email
    
      def name
        "#{object.first_name} #{object.last_name}"
      end
    end
    
    # Usage
    user = User.first
    serializer = UserSerializer.new
    json_string = serializer.serialize_to_json(user)
  11. Use Panko::Response to avoid double-encoded JSON strings

    master

    When constructing a JSON payload that includes Panko serialization results, using standard Ruby hashes can result in a JSON string being nested inside another JSON string (double encoding). Panko::Response solves this by correctly handling Panko serializer objects within a response hash, ensuring they are serialized as part of the main JSON structure rather than as pre-encoded strings.

    To serialize a collection, pass the Panko::ArraySerializer instance directly into the Panko::Response.new constructor.

    class PostsController < ApplicationController
      def index
       posts = Post.all
       render json: Panko::Response.new(
         success: true,
         total_count: posts.count,
         posts: Panko::ArraySerializer.new(posts, each_serializer: PostSerializer)
       )
      end
    end