vision-parse

After installation, you need to pull the required vision model and ensure the server is running. Vision Parse requires the llama3.2-vision:11b model.

1. Pull the model: ollama pull llama3.2-vision:11b
2. Start the server: ollama serve
3. Verify the server is responding by checking the version endpoint via curl.

# Pull the required vision model
ollama pull llama3.2-vision:11b

# Start the Ollama server
ollama serve

# Verify server status
curl http://localhost:11434/api/version

You can install the core package via pip. To include all additional dependencies required for using OpenAI or Gemini models, use the [all] extra.

Core installation:

pip install vision-parse

Full installation (with OpenAI/Gemini dependencies):

pip install 'vision-parse[all]'

Install from source:

pip install 'git+https://github.com/iamarunbrahma/vision-parse.git#egg=vision-parse[all]'

You can evaluate the performance of Vision Parse on your own machine using the provided benchmarking script.

1. Install the required benchmark dependencies:

pip install --no-cache-dir -r benchmarks/requirements.txt

2. Run the scoring script (ensure you update the pdf_path and benchmark_results_path within the script or via arguments if supported):

python benchmarks/scoring.py

Follow these steps to deploy the application:

1. Enable GPU (Optional): If you have an Nvidia GPU, uncomment the deploy section in your docker-compose.yml to reserve the GPU device.
2. Build and Start: Use docker compose build to create the image and docker compose up -d to start the container in detached mode.
3. Verify: Run docker ps to ensure the container is active.
4. Execute Application: Run the Gradio application inside the container using docker compose exec.

# 1. (Optional) Edit docker-compose.yml to include:
# deploy:
#   resources:
#     reservations:
#       devices:
#         - driver: nvidia
#           count: 1
#           capabilities: [gpu]

# 2. Build and start
docker compose build
docker compose up -d

# 3. Verify
docker ps

# 4. Run the application
docker compose exec vision-parse python docs/examples/gradio_app.py

To use Vision Parse with a local LLM provider, you must first install Ollama on your operating system. Use the following commands based on your platform:

• Linux: Use the curl install script.
• MacOS: Use Homebrew.
• Windows: Download the installer directly from the official website.

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# MacOS
brew install ollama

To customize OpenAI connections, pass a configuration dictionary to the openai_config parameter during VisionParser initialization.

Model-specific parameters (like temperature or max_tokens) should be passed as additional keyword arguments (kwargs) to the VisionParser class, not inside the openai_config dictionary.

# Example usage pattern
parser = VisionParser(
    openai_config={
        'OPENAI_BASE_URL': 'https://api.openai.com/v1',
        'OPENAI_MAX_RETRIES': 5,
        'OPENAI_TIMEOUT': 120.0
    },
    max_tokens=500  # Model-specific parameter passed as kwarg
)

To use Azure OpenAI, pass the required credentials and endpoint information via the openai_config parameter when initializing VisionParser.

parser = VisionParser(
    openai_config={
        'AZURE_OPENAI_API_KEY': 'your-api-key',
        'AZURE_ENDPOINT_URL': 'https://your-endpoint.azure.com/',
        'AZURE_DEPLOYMENT_NAME': 'your-deployment-name',
        'AZURE_OPENAI_API_VERSION': '2024-08-01-preview'
    }
)

Before running the container, you must export the necessary environment variables. MODEL_NAME is required and should be selected from the list of supported models. API keys for OpenAI or Gemini are optional but required if using those specific models.

# Required: Choose one of the following models
export MODEL_NAME=llama3.2-vision:11b

# Optional: API keys (required only for specific models)
export OPENAI_API_KEY=your_openai_api_key
export GEMINI_API_KEY=your_gemini_api_key

When initializing the VisionParser class, you can customize the Ollama connection by passing a configuration dictionary to the ollama_config parameter.

For model-specific parameters (such as temperature or top_p), pass them directly as additional keyword arguments (kwargs) to the VisionParser constructor rather than inside the ollama_config dictionary.

# Example usage pattern
parser = VisionParser(
    ollama_config={
        'OLLAMA_HOST': 'http://your-host:11434',
        'OLLAMA_NUM_PARALLEL': 4,
        'OLLAMA_REQUEST_TIMEOUT': 300.0
    },
    temperature=0.2  # Model-specific parameter passed as kwarg
)

To use Vision Parse with OpenAI models, install the package with the openai extra using pip:

!pip install 'vision-parse[openai]' -Uqq

To use Google Gemini models with Vision Parse, install the package with the [gemini] extra to ensure all necessary dependencies are included.

!pip install 'vision-parse[gemini]' -Uqq