Jina AI Examples

This repository contains a collection of examples demonstrating Jina's capabilities in neural search. Examples are categorized by complexity:

Simple Examples

• Semantic Wikipedia Search: Text-search using Transformers and DistilBERT.
• Lyrics Search: Searching a lyrics database with chunking support and a front-end.
• Audio Search: Finding similar audio clips.

Advanced Examples

• Simultaneous Querying and Indexing: Wikipedia search example that supports both operations at once.
• Cross-Modal Search: Searching images using text captions and vice-versa.

For a more comprehensive list and detailed documentation, visit docs.jina.ai.

Querying while indexing allows you to perform searches on your data even while new data is being inserted, updated, or deleted.

Jina achieves this using a dump-reload mechanism. This requires splitting your application into two separate Flows:

1. Index Flow (Storage Flow): Responsible for data ingestion and updates. It uses a Storage Indexer.
2. Query Flow: Responsible for serving search requests. It uses a Compound Searcher.

To ensure zero downtime during updates, the Query Flow uses replicas. While one replica is being reloaded with new data (via a ROLLING_UPDATE), the other replica continues to serve incoming search requests.

This example demonstrates how chunking (breaking large text into smaller pieces like sentences) enables granular semantic search.

Indexing Flow:

1. Splits songs into sentences (chunks).
2. Encodes chunks and indexes them at the sentence level.
3. Simultaneously indexes the full song to allow for later retrieval.

Query Flow:

1. Breaks the query into chunks (sentences).
2. Encodes query chunks and performs a nearest neighbor search against the indexed chunks.
3. Uses the MinRanker class (found in flows/executors.py) to aggregate match scores and calculate a relevance score for the whole song.

Relevance Score: The average of the numeric match values (0 to 1) for all matches within a song. A song with many high-scoring matches will have a high relevance score.

This application follows a linear flow of Executors to perform neural search:

1. Gateway: Receives input Documents from the user.
2. Transformer: Uses a language model (specifically distilbert-base-nli-stsb-mean-tokens) to compute embeddings based on the text of the document.
3. Indexer:
   • At Index Time: Stores the documents and their embeddings on disk (in the workspace folder).
   • At Query Time: Compares the embedding of the query document against all stored embeddings and returns the closest matches.

To support querying while indexing, you must configure specific indexers for each flow:

• For the Index Flow: Use LMDBStorage (from jinahub/indexers/storage). This uses LMDB as a disk-based key-value storage engine.
• For the Query Flow: Use FaissLMDBSearcher (from jinahub/indexers/searcher/compound). This combines the faiss algorithm for fast vector search with LMDB for retrieving document metadata.

In the Query Flow, the indexer is split into two components: one for vectors and one for document metadata, whereas the Storage Flow writes them into a single Storage Indexer.

Most Jina examples follow a standard directory structure for managing Flows, Pods, and workspace data:

• flows/: Contains Flow configuration files (e.g., index.yml for indexing and query.yml for querying).
• pods/: Contains Pod configuration files (e.g., encoder.yml to configure an encoder Pod).
• workspace/: A directory automatically created after the first indexing to store indexed files, such as embeddings and documents.

After indexing, you can query the data using two methods:

1. REST API

Start the query Flow:

python app.py -t query

Then, send a POST request using cURL. The request body should include parameters (like top_k) and the data array containing your search string:

curl --request POST -d '{"parameters": {"top_k": 10}, "data": ["hello world"]}' -H 'Content-Type: application/json' 'http://0.0.0.0:45678/search'

2. Web Interface

In a separate terminal, start a local server for the frontend:

cd static
python -m http.server

Then, open http://0.0.0.0:8000/ in your browser.

# Start Query Flow
python app.py -t query

# Query via cURL
curl --request POST -d '{"parameters": {"top_k": 10}, "data": ["hello world"]}' -H 'Content-Type: application/json' 'http://0.0.0.0:45678/search'

# Start Web UI
cd static
python -m http.server

Indexing is the process of preparing your data for search. In most examples, this is triggered by running the application with an index task type via the -t flag.

Command:

python app.py -t index

Successful indexing is typically indicated by a message stating that the flow is closed and resources are released (e.g., Flow@...[S]:flow is closed and all resources are released).

To index the data, run the app.py script with the index task. This triggers a Flow that breaks songs into sentences, encodes them using a language model (e.g., distilbert-based-uncased), and stores them in an Index.

python app.py -t index

Indexing converts images and their captions into vectors and stores them in an index. You can use a small toy dataset for testing or the full Flickr 8k dataset for production-like results.

Index the toy dataset:

python app.py -t index

Index the full Flickr 8k dataset: First, download the dataset using sh get_data.sh (requires a Kaggle account and API token). Then run:

python app.py -t index -d f8k -n 8000

Arguments:

• -t index: Sets the task to indexing.
• -d: Dataset choice (f8k for Flickr 8k, f30k for Flickr 30k).
• -n: Number of documents to index (e.g., 8000 for f8k).
• -s: Request size (optional).

Before running the audio-to-audio search example, you must install system-level dependencies (libsndfile1 and ffmpeg) and the Python requirements. Navigate to the directory containing requirements.txt to perform the installation.

sudo apt-get -y update && sudo apt-get install libsndfile1 ffmpeg
pip install -r requirements.txt