DOCSEARCHbyAlgolia
Submit Docs

prometheus-fastapi-instrumentator

repository·master·Indexed 23 days ago

https://github.com/trallnag/prometheus-fastapi-instrumentator

A configurable and modular Prometheus Instrumentator for FastAPI applications (version 8.1.0). It provides default metrics such as http_requests_total and http_request_duration_seconds, and allows for easy customization through closures, custom instrumentation functions, and the Info object. Supports exposing the /metrics endpoint on separate ports and provides guidance on handling Prometheus multi-process mode with Gunicorn.

Tokens
6.5K
Snippets
17
Records
37
Agent score
80%

What's inside prometheus-fastapi-instrumentator

  1. Create custom metrics with the Info object

    master

    To create a custom metric, define a function that returns a closure. The outer function handles persistent elements (like the prometheus_client metric instance), and the inner closure performs the actual instrumentation using the provided Info object.

    The Info object contains:

    • request: The raw FastAPI request object.
    • response: The FastAPI response object (can be None if an error occurred).
    • handler: The matched route template.
    • status: The (potentially grouped) status code.
    • duration: The request duration.

    Warning: Errors thrown in the handler are not caught by the instrumentator. Always check if the response is None before accessing it.

    from typing import Callable
from prometheus_fastapi_instrumentator.metrics import Info
from prometheus_client import Counter

def http_requested_languages_total() -> Callable[[Info], None]:
    # Persistent metric instance
    METRIC = Counter(
        "http_requested_languages_total",
        "Number of times a certain language has been requested.",
        labelnames=("langs",)
    )

    # The instrumentation closure
    def instrumentation(info: Info) -> None:
        langs = set()
        lang_str = info.request.headers.get("Accept-Language", "")
        for element in lang_str.split(","):
            element = element.split(";")[0].strip().lower()
            if element:
                langs.add(element)
        for language in langs:
            METRIC.labels(language).inc()

    return instrumentation

# Usage:
instrumentator.add(http_requested_languages_total())

  2. Understand metric limitations in Prometheus multi-process mode

    master

    When using the Prometheus Python client library in multi-process mode (e.g., with Gunicorn), certain metrics are not supported or will be missing:

    • created_by metrics: These are not supported by the Prometheus client library in multi-process mode.
    • System/Process metrics: Metrics provided by components like ProcessCollector (CPU/Memory) and PlatformCollector are not supported in multi-process mode.

    In the provided example, main_total reflects the number of Gunicorn workers (e.g., if --workers 2 is used, main_total will be 2).

  3. Quickstart: Instrument FastAPI with default metrics

    master

    To quickly instrument your FastAPI application with a set of default Prometheus metrics and expose them via an endpoint, use the Instrumentator().instrument(app).expose(app) pattern.

    By default, this provides:

    • http_requests_total: Counter with handler, status, and method labels.
    • http_request_size_bytes: Summary with handler label.
    • http_response_size_bytes: Summary with handler label.
    • http_request_duration_seconds: Histogram with handler and method labels (low cardinality).
    • http_request_duration_highr_seconds: Histogram with many buckets (high cardinality).

    Note: Status codes are grouped (e.g., 2xx) and unmatched routes are grouped under the none handler.

    from prometheus_fastapi_instrumentator import Instrumentator

# Fast track
Instrumentator().instrument(app).expose(app)

  4. Quickstart: Instrument FastAPI using startup event

    master

    If your application logic requires a more controlled lifecycle, you can instantiate the Instrumentator, call .instrument(app), and then use the FastAPI @app.on_event("startup") handler to call .expose(app).

    from prometheus_fastapi_instrumentator import Instrumentator

instrumentator = Instrumentator().instrument(app)

@app.on_event("startup")
async def _startup():
    instrumentator.expose(app)

  5. Run commands in the Poetry environment

    master

    You can execute commands within the project's managed environment using two methods:

    1. Interactive Shell: Use poetry shell to spawn a new shell session with the virtual environment activated.
    2. Direct Execution: Prepend any command with poetry run to execute it within the environment without manually activating a shell.

  8. Install and set up pre-commit hooks

    master

    The project uses pre-commit to maintain Git hooks. To set up the environment after cloning the repository, you must first ensure pre-commit is installed on your system (e.g., via pipx install pre-commit).

    Once installed, run the following commands to install the hooks and ensure they are active for both standard commits and commit messages:

    pre-commit install --install-hooks
pre-commit install --install-hooks --hook-type commit-msg

  9. Update Poetry dependencies and Poetry itself

    master

    Update dependencies

    To automatically update dependencies and bump versions in pyproject.toml, you can use the poetry-plugin-up plugin.

    1. Install the plugin: poetry self add poetry-plugin-up
    2. Run the update: poetry up

    Update Poetry

    To update the Poetry tool itself to the latest version, use: poetry self update

    # Install the update plugin
poetry self add poetry-plugin-up

# Use the plugin to update dependencies
poetry up

# Update Poetry itself
poetry self update

  10. Run pre-commit hooks manually

    master

    You can manually trigger pre-commit checks using the following commands:

    • Run all hooks against all files: Use pre-commit run -a.
    • Run a specific hook against all files: Use pre-commit run -a <hook> (replace <hook> with the specific hook ID).
    # Run all hooks against all files
pre-commit run -a

# Run specific hook against all files
pre-commit run -a <hook>

  11. Run the `prom-multi-proc-gunicorn` example

    master

    This example demonstrates how to integrate FastAPI with Gunicorn in Prometheus multi-process mode without using prometheus-fastapi-instrumentator. It is intended to highlight which metrics are lost or unsupported when using the standard Prometheus client library in multi-process mode.

    Prerequisites

    Run poetry install and poetry shell in the root of the repository before executing the commands below.

    Setup and Execution

    1. Set the multi-process directory: Define the PROMETHEUS_MULTIPROC_DIR environment variable to a unique, unused location.
    2. Initialize the directory: Clear and recreate the directory to ensure a clean state.
    3. Start Gunicorn: Launch the application using Gunicorn with Uvicorn workers.
    4. Interact: Use curl to hit the application endpoints and the /metrics endpoint.
    # 1. Set environment variable
export PROMETHEUS_MULTIPROC_DIR=/tmp/python-testing-pfi/560223ba-887f-429a-9c48-933df56a68ba

# 2. & 3. Prepare directory and start Gunicorn
rm -rf "$PROMETHEUS_MULTIPROC_DIR"
mkdir -p "$PROMETHEUS_MULTIPROC_DIR"
gunicorn main:app \
  --config gunicorn.conf.py \
  --workers 2 \
  --worker-class uvicorn.workers.UvicornWorker \
  --bind 0.0.0.0:8080

# 4. Interact with app
for i in {1..5}; do curl localhost:8080/ping; done
curl localhost:8080/metrics