Informers

Add informers to your application's Gemfile to use fast transformer inference in Ruby via ONNX.

gem "informers"

The ImageFeatureExtractor#preprocess method executes a sequence of transformations to prepare an image for a model. The typical lifecycle is:

1. Color Conversion: Converts to RGB or Grayscale based on config.
2. Resizing: Applies resize or thumbnail logic.
3. Cropping: Applies center_crop if configured.
4. Pixel Manipulation:
   • Rescales pixel values (e.g., to [0, 1]).
   • Normalizes using image_mean and image_std.
   • Applies padding if do_pad is enabled.
5. Format Conversion: Converts the data from HWC (Height, Width, Channels) to CHW (Channels, Height, Width) format.

The call method allows processing an array of images, returning a hash containing pixel_values (stacked tensors), original_sizes, and reshaped_input_sizes.

When downloading files from Hugging Face (huggingface.co or hf.co), Informers automatically checks for the HF_TOKEN environment variable. If present, it is added to the request headers as a Bearer token:

Authorization: Bearer <YOUR_TOKEN>

This allows the library to access private or gated models on the Hugging Face Hub without manual header configuration.

Models in this library return specialized output objects that inherit from ModelOutput. These objects allow for easy access to model predictions via key-based lookup or specific reader methods. Common output types include:

• Seq2SeqLMOutput: Contains logits, past_key_values, encoder_outputs, decoder_attentions, and cross_attentions.
• SequenceClassifierOutput: Contains logits.
• TokenClassifierOutput: Contains logits.
• MaskedLMOutput: Contains logits.
• QuestionAnsweringModelOutput: Contains start_logits and end_logits.
• DetrObjectDetectionOutput: Contains logits and pred_boxes.
• DetrSegmentationOutput: Contains logits, pred_boxes, and pred_masks.

Certain pipelines require external system dependencies to handle specific file formats:

• Vision Pipelines (e.g., image-classification, object-detection): Requires the ruby-vips gem to load images.
• Audio Pipelines (e.g., audio-classification): Requires ffmpeg installed on the system to load audio files.

Reranking pipelines take a query and a list of documents, returning a score or ranked result that indicates how relevant the documents are to the query. This is often used as a second stage after an initial retrieval step.

Example using mixedbread-ai/mxbai-rerank-base-v1:

query = "How many people live in London?"
docs = ["Around 9 Million people live in London", "London is known for its financial district"]

model = Informers.pipeline("reranking", "mixedbread-ai/mxbai-rerank-base-v1")
result = model.(query, docs)

Embedding pipelines convert text into numerical vectors. Many models require specific prefixes for queries or documents to improve performance (e.g., query: or passage: ).

Example using sentence-transformers/multi-qa-MiniLM-L6-cos-v1 for semantic similarity:

query = "How many people live in London?"
docs = ["Around 9 Million people live in London", "London is known for its financial district"]

model = Informers.pipeline("embedding", "sentence-transformers/multi-qa-MiniLM-L6-cos-v1")
query_embedding = model.(query)
doc_embeddings = model.(docs)

# Manual cosine similarity calculation
scores = doc_embeddings.map { |e| e.zip(query_embedding).sum { |d, q| d * q } }
doc_score_pairs = docs.zip(scores).sort_by { |d, s| -s }

When calling .generate, you can pass a generation_config hash to control the output. Supported keys include:

• max_new_tokens: Maximum number of new tokens to generate.
• max_length: Maximum total length of the sequence.
• num_beams: Number of beams for beam search.
• repetition_penalty: Penalty for repeating tokens.
• no_repeat_ngram_size: Prevents n-gram repetition.
• bad_words_ids: List of token IDs to avoid.
• min_length: Minimum length of the generated sequence.
• min_new_tokens: Minimum number of new tokens to generate.
• forced_bos_token_id: Forces a specific Beginning Of Sentence token.
• forced_eos_token_id: Forces a specific End Of Sentence token.
• num_return_sequences: Number of sequences to return (currently limited in implementation).

To prevent the library from making network requests and force it to only use files already present in the local cache, you can use one of two methods:

1. Per-call: Pass local_files_only: true to get_model_file or get_model_json.
2. Globally: Set Informers.allow_remote_models = false.

If a file is not found in the local cache while these settings are active, the library will raise an error (unless fatal: false is passed to the specific method call).