DOCSEARCHbyAlgolia
Submit Docs

aioprometheus

repository·master·Indexed 41 days ago

https://github.com/claws/aioprometheus

A Prometheus Python client library designed for asyncio-based applications. It provides tools for collecting metrics (Counter, Gauge, Summary, and Histogram) and serving them via a standalone HTTP service or ASGI middleware for frameworks like FastAPI, Starlette, and Quart. It also supports pushing metrics to a Prometheus Pushgateway.

Tokens
12K
Snippets
40
Records
50
Agent score
52%

What's inside aioprometheus

  1. Overview of aioprometheus

    master

    aioprometheus

    aioprometheus is a Prometheus Python client library specifically designed for asyncio-based applications. It enables metrics collection and serving for Prometheus and compatible monitoring systems.

    Key Capabilities:

    • Metrics Collection: Supports standard Prometheus metrics including histograms.
    • Exporting: Can export metrics in text format.
    • Pushing: Supports pushing metrics to a Prometheus Pushgateway.
    • Integration: Compatible with asyncio networking apps and web frameworks such as:
      • FastAPI / Starlette
      • Quart
      • aiohttp

  2. Manage metric labels and constant labels

    master

    Labels allow you to group related time series. You can provide labels dynamically during metric updates (e.g., metric.inc({'type': 'virtual'})).

    To avoid repeating the same labels in every update, you can use const_labels when instantiating a metric. These labels will be automatically included in every observation made by that metric instance.

    import socket
from aioprometheus import Gauge

# Using constant labels to avoid repeating 'host' in every call
ram_metric = Gauge(
    "memory_usage_bytes", 
    "Memory usage in bytes.",
    const_labels={'host': socket.gethostname()}
)

# The 'host' label is automatically included
ram_metric.set({'type': "virtual"}, 100)
ram_metric.set({'type': "swap"}, 100.1)

  3. Push metrics to a Prometheus Push Gateway

    master

    If your components are short-lived or behind a firewall and cannot be scraped directly, use the Pusher object to push metrics to a Prometheus Push Gateway.

    Usage:

    • Initialize Pusher with a job name and the PUSH_GATEWAY_ADDR.
    • Use grouping_key to add additional labels to the Push Gateway URL.
    • Call await pusher.replace(registry) to push the current state of the registry.
    • You can pass auth (e.g., aiohttp.BasicAuth) to the replace method to handle authentication.

    Requires aioprometheus[aiohttp] extras.

    from aioprometheus import REGISTRY, Counter
from aioprometheus.pusher import Pusher
import asyncio
import aiohttp

PUSH_GATEWAY_ADDR = "http://127.0.0.1:61423"
AUTH = aiohttp.BasicAuth("Joe", password="4321")

async def main():
    # Initialize Pusher with job name and grouping keys
    pusher = Pusher("my-job", PUSH_GATEWAY_ADDR, grouping_key={"instance": "127.0.0.1:1234"})
    
    # Create and increment a metric
    c = Counter("total_requests", "Total requests.", {})
    c.inc({'url': '/p/user'})
    
    # Push the registry to the gateway (with optional auth)
    resp = await pusher.replace(REGISTRY, auth=AUTH)

if __name__ == "__main__":
    asyncio.run(main())

  4. Use decorators for automatic instrumentation

    master

    The library provides several decorators to simplify instrumentation. These decorators work with both regular functions and coroutines (returning an awaitable if the wrapped function is a coroutine).

    Available decorators include:

    • @timer: Measures how long a function takes to run.
    • @inprogress: Tracks the number of requests currently in progress.
    • @count_exceptions: Tracks the number of exceptions occurring within a function block.

  5. Run unit tests and check coverage

    master

    Use make test to run the standard library unittest suite. For more detail, use make test-verbose. You can also run individual tests using the unittest module directly. To check test coverage, use make coverage.

    # Run all unit tests
(aioprom) $ make test

# Run tests with verbose output
(aioprom) $ make test-verbose

# Run a specific test case
(aioprom) $ cd aioprometheus/tests
(aioprom) $ python -m unittest test_negotiate.TestNegotiate.test_text_default

# Check code coverage
(aioprom) $ make coverage

  6. Instrument software with Counter, Gauge, Summary, and Histogram

    master

    To instrument your software, import the desired metric type from aioprometheus, instantiate it with a name and help string, and then update it during runtime. By default, all metrics are registered to aioprometheus.REGISTRY. You can provide a custom registry via a keyword argument during instantiation.

    from aioprometheus import Counter

# Create and update a Counter
events_counter = Counter(
    "events_counter",
    "Total number of events.",
)
events_counter.inc({"kind": "event A"})

  7. Use ASGI Middleware in FastAPI or Starlette

    master

    For web applications built on Starlette or FastAPI, use the MetricsMiddleware to automatically collect standard metrics (total requests, responses, exceptions, and status codes).

    To include custom application metrics in the exposed /metrics endpoint, it is recommended to attach them to app.state. This allows you to access them easily within route handlers, even if they were defined in different modules.

    Note: This requires the starlette extra.

    from fastapi import FastAPI, Request, Response
from aioprometheus import Counter, MetricsMiddleware
from aioprometheus.asgi.starlette import metrics

app = FastAPI()

# Attach custom metrics to app.state for easy access in handlers
app.state.users_events_counter = Counter("events", "Number of events.")

app.add_middleware(MetricsMiddleware)
app.add_route("/metrics", metrics)

@app.get("/users/{user_id}")
async def get_user(request: Request, user_id: str):
    # Access the metric via request.app.state
    request.app.state.users_events_counter.inc({"path": request.scope["path"]})
    return Response(f"{user_id}")

  8. Set up a development environment for aioprometheus

    master

    To develop aioprometheus, you need Python 3.8+. It is recommended to use a virtual environment to isolate dependencies. After creating the environment, install the development dependencies and install the package in editable mode with optional integration dependencies (aiohttp, starlette, quart) to ensure changes take effect immediately.

    # 1. Clone the repository
$ git clone git@github.com:claws/aioprometheus.git
$ cd aioprometheus

# 2. Create and activate a virtual environment
$ python3 -m venv venv --prompt aioprom
$ source venv/bin/activate

# 3. Upgrade core tools
(aioprom) $ pip install pip --upgrade
(aioprom) $ pip install wheel

# 4. Install development dependencies and the package in editable mode
(aioprom) $ pip install -r requirements.dev.txt
(aioprom) $ pip install -e .[aiohttp,starlette,quart]

  9. Run code style, linting, and type checks

    master

    The project uses Black for formatting, isort for import sorting, Pylint for static analysis, and mypy for type checking. You can run these checks using the provided Makefile rules.

    # Apply code style (Black and isort)
(aioprom) $ make style

# Run linting (Pylint)
(aioprom) $ make check-lint

# Run type annotation checks (mypy)
(aioprom) $ make check-types

  10. Expose metrics in Web Frameworks (Starlette, FastAPI, Quart)

    master

    For web applications, use the aioprometheus ASGI middleware. This middleware automatically tracks a default set of metrics: total requests received, total responses sent, exceptions raised, and response status codes for route handlers.

    Key Details:

    • Exclusions: Common paths like /favicon.ico and /metrics are excluded from metric updates (defined in aioprometheus.agsi.middleware.EXCLUDE_PATHS).
    • Quart Note: Exceptions are not propagated to the ASGI layer by Quart, so default exception tracking may not work.
    • Custom Metrics: Any custom application metrics you create are automatically included in the exposed metrics.
    • Manual Control: If you need finer control or want to avoid default metrics, use the aioprometheus render function to implement a custom metrics handler.

    To use the Starlette-specific rendering function, you must install the starlette extra.

    $ pip install aioprometheus[starlette]

  11. Install aioprometheus

    master

    Install the core library using pip:

    pip install aioprometheus

    Depending on your use case, you may need to install optional extra dependencies:

    • Starlette / FastAPI: pip install aioprometheus[starlette]
    • Quart: pip install aioprometheus[quart]
    • HTTP Service (aiohttp): pip install aioprometheus[aiohttp]

    You can combine multiple extras in a single command:

    pip install aioprometheus[aiohttp,starlette,quart]
    pip install aioprometheus

  12. Release process and distribution

    master

    When releasing a new version, follow these steps:

    1. Verify style, linting, type checks, and docs build pass.
    2. Update the version in __init__.py using the YY.MM.MICRO CalVer scheme.
    3. Update CHANGELOG.md.
    4. Build the distribution wheel using make dist.
    5. Test the distribution using make test-dist.
    6. Upload to PyPI using make upload-dist.
    7. Tag the release in Git and push tags to origin.
    # Build distribution
(aioprom) $ make dist

# Test the distribution
(aioprom) $ make test-dist

# Upload to PyPI
(aioprom) $ make upload-dist

# Tag and push to GitHub
$ git tag YY.MM.MICRO -m "A meaningful release tag comment"
$ git push --tags origin master