DOCSEARCHbyAlgolia
Submit Docs

EvilSeed Documentation

repository·master·Indexed 20 days ago

https://github.com/evilmartians/evil-seed

A tool for creating partial, anonymized database dumps based on ActiveRecord models. EvilSeed allows developers to bring production-like data into staging or local environments by defining root models, applying constraints, and using a declarative DSL to anonymize sensitive information or customize record data.

Tokens
3.8K
Snippets
20
Records
21
Agent score
68%

What's inside EvilSeed

  1. Restore a SQL dump

    master

    The output is a plain SQL file. You can restore it using standard CLI tools like psql, mysql, or sqlite3. To restore via Ruby/ActiveRecord, use:

    ActiveRecord::Base.connection.execute(File.read('path/to/new_dump.sql'))

    Important Tips:

    • Reset Sequences: After restoration, reset primary key sequences to ensure new seeds work correctly:
      ActiveRecord::Base.connection.tables.each do |table|
  ActiveRecord::Base.connection.reset_pk_sequence!(table)
end
    • PostgreSQL Circular Dependencies: If you encounter circular dependencies in PostgreSQL, you may need to make foreign keys NOT DEFERRABLE before restoration and use SET CONSTRAINTS ALL DEFERRED within a transaction during the load.
    # Restore via Ruby
ActiveRecord::Base.connection.execute(File.read('path/to/new_dump.sql'))

  3. Configure global EvilSeed settings

    master

    You can set several global options within the EvilSeed.configure block:

    • config.ignore_columns(model, column): Removes specific columns from the dump for a model, even if it is reached via an association. Useful for encrypted columns.
    • config.dont_nullify = true: Prevents EvilSeed from nullifying foreign keys for records not included in the dump (default is false).
    • config.unscoped = true: Tells EvilSeed to ignore ActiveRecord default scopes (useful for including soft-deleted records).
    • config.verbose = true: Enables progress printing to the console.
    • config.verbose_sql = true: Enables SQL logging during the dump process.

  4. Customize and anonymize record data

    master

    EvilSeed provides two ways to modify data during the dump process:

    1. customize(model_name): Allows direct mutation of the attribute hash. Note that you are working with a hash of attributes, not the ActiveRecord object itself. Use encrypt(value) to help with password hashing.
    2. anonymize(model_name): A declarative DSL for transforming attributes. This is ideal for using libraries like Faker to generate realistic but fake data.

    Both methods allow you to hide sensitive information or standardize data (like resetting passwords) for development environments.

    EvilSeed.configure do |config|
  # Custom mutation
  config.customize("User") do |u|
    u["encrypted_password"] = encrypt("qwerty")
    u["metadata"].merge!("foo" => "bar")
  end

  # Declarative anonymization
  config.anonymize("User") do
    name  { Faker::Name.name }
    email { Faker::Internet.email }
    login { |login| "#{login}-test" }
  end
end

  5. Configure root models and constraints

    master

    Use config.root to define the starting points for your database dump. You can pass constraints to limit which records are selected, similar to an ActiveRecord .where clause.

    Key configuration methods for a root block:

    • limit(n): Limits the number of records for this root model.
    • order(hash): Specifies the sort order for selecting records.
    • exclude(pattern, *paths): Excludes specific associations using regex patterns or dot-delimited association paths (e.g., 'forum.users.questions').
    • include(hash_or_regex): Explicitly includes specific association chains.
    • limit_associations_size(n, path_or_hash): Limits the number of associated records (has_many/has_one) per parent.
    • limit_deep(n): Limits the maximum depth of association traversal.
    EvilSeed.configure do |config|
  config.root('Forum', featured: true) do |root|
    root.limit(100)
    root.order(created_at: :desc)
    root.exclude(/\btracking_pixels\b/, 'forum.popular_questions')
    root.include(parent: {questions: %i[answers votes]})
    root.limit_associations_size(5, 'forum.questions')
    root.limit_deep(10)
  end
end

  6. Use EvilSeed as a standalone script

    master

    You can run EvilSeed as a standalone Ruby script by including bundler/inline to manage dependencies, defining your models, and establishing a database connection manually.

    #!/usr/bin/env ruby

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'activerecord'
  gem 'evil-seed'
  gem 'mysql2'
end

class Category < ActiveRecord::Base
  has_many :translations, class_name: "Category::Translation"
end

class Category::Translation < ActiveRecord::Base
  belongs_to :category, inverse_of: :translations
end

EvilSeed.configure do |config|
  config.root("Category", "id < ?", 1000)
end

ActiveRecord::Base.establish_connection(ENV.fetch("DATABASE_URL"))
EvilSeed.dump(File.join(__dir__, "dump.sql").to_s)

  8. Configure root models and constraints in EvilSeed

    master

    When configuring EvilSeed, you define root models to be dumped. You can pass constraints to the model initialization, which are passed directly to ActiveRecord's where method to limit the number of records selected for dumping.

    Example constraints:

    • Forum.where(featured: true)
    • User.where(active: true)
    # Conceptual usage of the Root configuration
# model: Name of the model class
# dont_nullify: Boolean flag
# constraints: Arguments for .where()
EvilSeed::Configuration::Root.new("Forum", true, featured: true)

  11. Configure record ordering and nullification

    master

    The Root configuration allows you to control the order of record selection and how nullification is handled:

    • order(order): Specifies the order for records to be selected for the dump (e.g., passing an ActiveRecord order string or hash).
    • do_not_nullify(nullify_flag): Sets a flag to control nullification behavior during the seeding process.
    # Set order and nullification behavior
root_config.order("created_at DESC")
root_config.do_not_nullify(true)

  12. Exclude associations from the dump

    master

    You can prevent certain associations from being included in the dump using the exclude method. This method accepts String or Regexp patterns.

    Association paths are dot-delimited strings starting from the model itself (e.g., forum.users.questions).

    # Exclude specific association paths using strings or regex
root_config.exclude("users.questions", /posts/)