MiniSearch

MiniSearch is designed for providing rich full-text search functionalities in local setups (e.g., client-side in the browser).

Core Goals:

• Small memory footprint for the index.
• Fast document indexing.
• Versatile and performant search features.
• Simple API surface.
• Support for adding/removing documents at any time.

Ideal Use Cases:

• Client-side search in web applications.
• Small to medium-sized datasets where memory is constrained.

Not Recommended For:

• Large index data structures that exceed local memory.
• Distributed setups requiring synchronized nodes.
• Scenarios requiring built-in, opinionated locale support (stemmers, stopwords, etc.). MiniSearch provides the building blocks for you to implement these yourself.

To run the client-side demonstration application locally, follow these steps:

1. Open a terminal and cd to the docs/demo directory.
2. Start a local HTTP server using one of the following commands:
   • If you have Python installed: python3 -m http.server
   • If you have NodeJS installed: npx http-server -p 8000
3. Open your browser and navigate to the server URL (e.g., http://localhost:8000).

Create a new MiniSearch instance by providing an Options object. At minimum, you must specify the fields to be indexed. You can also define an idField (defaults to 'id'), storeFields to include data in search results, and custom functions for field extraction, tokenization, and term processing.

// Basic initialization
const miniSearch = new MiniSearch({
  fields: ['title', 'text']
});

// Initialization with custom ID field and stored fields
const miniSearch = new MiniSearch({
  idField: 'key',
  fields: ['title', 'text'],
  storeFields: ['title', 'category']
});

Use the processTerm option to normalize, filter, or apply stemming to terms. The function receives (term, fieldName) and should return the processed string or a falsy value to discard the term. Terms are downcased by default.

let stopWords = new Set(['and', 'or', 'to', 'in', 'a', 'the'])

let miniSearch = new MiniSearch({
  fields: ['title', 'text'],
  processTerm: (term, _fieldName) =>
    stopWords.has(term) ? null : term.toLowerCase()
})

// Use different processing for search queries
let miniSearch = new MiniSearch({
  fields: ['title', 'text'],
  processTerm: (term, _fieldName) =>
    stopWords.has(term) ? null : term.toLowerCase(), // index term processing
  searchOptions: {
    processTerm: (term) => term.toLowerCase() // search query processing
  }
})

// To get the default term processor:
// const defaultProcessor = MiniSearch.getDefault('processTerm')

Control how strings are split into tokens using the tokenize option. You can set a tokenizer for indexing and a different one for searching via searchOptions.tokenize.

// Custom tokenizer for both indexing and searching
let miniSearch = new MiniSearch({
  fields: ['title', 'text'],
  tokenize: (string, _fieldName) => string.split('-')
})

// Different tokenizers for indexing vs searching
let miniSearch = new MiniSearch({
  fields: ['title', 'text'],
  tokenize: (string) => string.split('-'), // indexing tokenizer
  searchOptions: {
    tokenize: (string) => string.split(/[\s-]+/) // search query tokenizer
  }
})

// To get the default tokenizer:
// const defaultTokenizer = MiniSearch.getDefault('tokenize')

By default, MiniSearch treats documents as plain key-value objects. Use the extractField option to implement custom logic for nested fields or non-string values. The function receives (document, fieldName) and should return the value to be indexed.

let miniSearch = new MiniSearch({
  fields: ['title', 'author.name', 'pubYear'],
  extractField: (document, fieldName) => {
    // Extract year from a Date object
    if (fieldName === 'pubYear') {
      const pubDate = document['pubDate']
      return pubDate && pubDate.getFullYear().toString()
    }

    // Access nested fields (e.g., 'author.name')
    return fieldName.split('.').reduce((doc, key) => doc && doc[key], document)
  }
})

// To get the default extractor:
// const defaultExtractor = MiniSearch.getDefault('extractField')

Vacuuming cleans up obsolete references left by discard(). You can control this via the autoVacuum option in the constructor.

AutoVacuumOptions includes:

• batchSize: Number of terms to traverse per batch (default: 1000).
• batchWait: Milliseconds to wait between batches (default: 10).
• minDirtCount: Minimum number of discarded documents required to trigger vacuum (default: 20).
• minDirtFactor: Minimum proportion of discarded documents required to trigger vacuum (default: 0.1).