Alaveteli Documentation

repository·develop·Indexed 19 days ago

https://github.com/mysociety/alaveteli

An open-source, internationalised platform for facilitating Freedom of Information (FOI) requests globally to promote transparency between citizens and governments. The project includes the Alaveteli core application, the alaveteli_features gem for feature management via Flipper, and the ExcelAnalyzer gem for inspecting XLSX files for hidden data.

Tokens
9.9K
Snippets
49
Records
57
Agent score
65%

What's inside Alaveteli

  1. Overview of Alaveteli

    develop
    Alaveteli is an open-source, internationalised platform designed for making Freedom of Information (FOI) requests. It provides a standardized way for citizens to interact with governments to promote transparency. The software originated from the UK-based 'WhatDoTheyKnow' website.
  2. How indexing works in Alaveteli

    develop

    Alaveteli uses a search_documents table to store searchable data, which decouples the search index from the primary model tables. This allows for efficient searching using PostgreSQL's ts_vector capabilities.

    There are two main ways to handle indexing:

    1. object.reindex: Upserts content for a specific instance into the search_documents table. This should be called whenever a model's searchable content is modified.
    2. Model.reindex_all: Overwrites all existing entries for a specific model in batches. This is useful for initial backfilling or full refreshes and can be run without service interruption.

    Note: Reindexing large datasets (e.g., FoiAttachment) can take significant time (minutes to days).

    # Reindex a single instance
    PublicBody.find(123).reindex
    
    # Reindex all instances of a model
    PublicBody.reindex_all
  3. How the Search module architecture works

    develop

    The Search module acts as a decoupled facade that allows the application to interact with search engines without being tied to a specific implementation. It uses a pluggable backend architecture:

    1. Public Facade (Search): The entry point for controllers, models, and mailers (e.g., Search.search).
    2. Abstract Backend (Search::Backend): Defines the interface that all concrete implementations must follow.
    3. Adapters (Search::Adapter): Concrete implementations (like Adapters::Xapian or Adapters::PostgreSQL) that map generic search requests to backend-specific logic.
    4. Search Operations: Specific search types like FullText, Typeahead, and SimilarRequests are handled by adapters and return a unified Search::Results object.
    Controllers / Models / Mailers
            |
            v
      Search module          (app/search/search.rb)    -- public facade
            |
            v
      Search::Backend        (app/search/backend.rb)   -- abstract interface
            |
            v
      Adapters::Xapian       (app/search/adapters/)    -- concrete implementation
            |
            v
      Search::Adapter        (app/search/adapter.rb)   -- base for search types
            |
        +---+---+------------------+
        |       |                  |
    FullText  Typeahead   SimilarRequests
        |       |                  |
        v       v                  v
      Search::Results        (app/search/results.rb)   -- unified result object
  4. Define searchable fields on a model

    develop

    To make a model searchable, add a searchable definition to it. This definition specifies which fields (or Ruby methods) should be included in the search index and assigns them weights (A being the highest) for ranking results.

    • Use a leading dot (e.g., .name) to call a Ruby method.
    • Use the column name directly (e.g., home_page) to access the PostgreSQL column, which is faster but less flexible.

    You can define both a public index and an admin_index for content only visible to administrators.

    searchable index: {
                   ".name": "A",
                   "home_page": "B",
                 },
                 admin_index: {
                   # same logic as above
                 }
  5. Rename view templates from .rhtml to .html.erb

    develop

    Standard view templates must be renamed from the .rhtml extension to .html.erb. You can automate this in your theme's root directory using the following command:

    Note: Mailer templates are an exception; they should be renamed to .text.erb because Alaveteli uses text-only emails.

    for r in $(find lib/views -name '*.rhtml'); do echo git mv $r ${r%.rhtml}.html.erb; done
  6. Install the alaveteli_features gem

    develop

    To add alaveteli_features to your Alaveteli application, add the gem to your Gemfile and run bundle, or install it directly via the gem command. After installation, run the generator to set up necessary migrations and an example initializer file.

    # Add to Gemfile
    gem 'alaveteli_features'
    
    # Then run in terminal
    $ bundle
    $ rails g alaveteli_features:install
  7. Ruby version compatibility and setup

    develop

    Alaveteli is tested against ruby-3.4.

    If you are using a Ruby version manager like RVM or .rbenv, you can automatically switch to the recommended development version (currently 3.4.7) by creating a .ruby-version symlink pointing to .ruby-version.example within the project directory.

    ln -s .ruby-version.example .ruby-version
  8. How to build a new search backend

    develop

    To implement a new search backend, follow these steps:

    1. Subclass Search::Backend

    Create an adapter class (e.g., app/search/adapters/my_backend.rb) that implements the required interface:

    • search(query, models:, sort_by: nil, sort_ascending: true, collapse_by: nil): Must return an object responding to .results(page:, per_page:) which returns a Search::Results object.
    • typeahead(query, model:, exclude_tags: []): Must follow the same contract as search.
    • similar(record): Must return an object responding to .results or .first.

    2. Implement search operation classes

    Each method on your adapter should return an object that implements the results logic. You can use Search::Adapter as a base class to help create Search::Results objects using create_search_results.

    3. Wire it up

    Assign your new adapter to Search.backend in an initializer, or set the SEARCH_BACKEND value in your configuration (e.g., config/general.yml).

    # In an initializer
    Search.backend = Search::Adapters::MyBackend::Adapter.new

    4. Indexing (Optional)

    If your backend requires specific indexing setup (like PostgreSQL triggers), place the configuration in your adapter's namespace (e.g., Search::Adapters::MyBackend::Indexing) and call it from an initializer.

    module Search
      module Adapters
        module PostgreSQL
          class Adapter < Search::Backend
            def search(query, models:, sort_by: nil, sort_ascending: true, 
                       collapse_by: nil)
              # Return an object that responds to .results(page:, per_page:)
              # and returns a Search::Results
            end
    
            def typeahead(query, model:, exclude_tags: [])
              # Same contract as search
            end
    
            def similar(record)
              # Return an object responding to .results / .first for records
              # similar to the given one
            end
          end
        end
      end
    end
  9. Update mailer template patching logic

    develop

    If your theme patches mailer paths in lib/patch_mailer_paths.rb, update the method used to add view paths:

    • Replace ActionMailer::Base.view_paths.unshift ... with ActionMailer::Base.prepend_view_path ...
    • Replace ActionMailer::Base.view_paths << ... with ActionMailer::Base.append_view_path ...
    # Old
    ActionMailer::Base.view_paths.unshift File.join(File.dirname(__FILE__), "views")
    
    # New
    ActionMailer::Base.prepend_view_path File.join(File.dirname(__FILE__), "views")
  10. Previewing Action Mailer emails

    develop
    Alaveteli uses Rails' Action Mailer previews to allow developers to inspect email templates without sending actual emails. You can view individual email previews by visiting specific URLs generated by the framework, or browse a complete list of all available mailer previews by navigating to the /rails/mailers endpoint in your local development environment.