pingap

repository·main·Indexed 23 days ago

https://github.com/vicanso/pingap

A high-performance, extensible reverse proxy powered by Cloudflare Pingora. It features zero-downtime hot-reloading, a Web UI, and a plugin system for authentication, security, and traffic control. The project is organized into specialized crates covering load balancing, health checks, ACME automation, OpenTelemetry tracing, and various caching and compression backends.

Tokens
100.5K
Snippets
228
Records
526
Agent score
79%

What's inside pingap

  1. Overview of Pingap Modules

    main

    Pingap is composed of several specialized modules that handle different aspects of proxying, security, and observability. Understanding these modules helps in identifying which component is responsible for specific behaviors like SSL management, routing, or monitoring.

    Core Modules

    • proxy: The main proxy server component.
    • core: Essential functionality and shared components used across the system.
    • config: Manages configuration parsing and distribution.
    • util: Shared utility functions and helper methods.

    Security and Certificates

    • acme: Handles the ACME protocol for automated SSL/TLS certificate issuance and renewal.
    • certificate: Manages SSL/TLS certificate storage and validation.

    Traffic and Routing

    • location: Handles URL routing and location-based request processing.
    • upstream: Manages backend server connections and load balancing.
    • discovery: Implements service discovery for dynamic backend detection.
    • health: Manages health checks and monitoring of backend services.
    • webhook: Supports various webhook protocols for external notifications.

    Performance and Optimization

    • cache: Manages caching mechanisms to reduce backend load.
    • imageoptim: Optimizes images in png, jpeg, webp, and avif formats.
    • performance: Provides performance metrics.

    Observability and Monitoring

    • logger: Provides logging and log management.
    • otel: Implements OpenTelemetry integration for distributed tracing and metrics.
    • sentry: Provides error tracking via Sentry integration.
    • pyroscope: Integrates with Pyroscope for continuous profiling.

    Extensibility

    • plugin: Manages the plugin system for extending Pingap's functionality.
  2. Overview of Pingap

    main

    Pingap is a high-performance reverse proxy driven by the Cloudflare Pingora framework. It is built with Rust to ensure memory safety and top-tier performance. It supports HTTP/1.1, HTTP/2, and gRPC-web proxying.

    Key capabilities include:

    • Dynamic Configuration: Zero-downtime hot updates via TOML files or etcd.
    • Extensibility: A powerful plugin system for authentication (JWT, Key Auth), security (CSRF, IP/Referer/UA limits), traffic control (rate limiting, caching), and content modification.
    • Observability: Native Prometheus metrics (pull/push), OpenTelemetry integration for distributed tracing, and highly customizable access logs.
    • Service Discovery: Built-in support for static lists, DNS, or Docker tags.
    • Automated HTTPS: Let's Encrypt integration supporting HTTP-01 and DNS-01 challenges.
  3. Understand the Pingap Location Module features

    main

    The Pingap Location Module provides an intelligent request routing system for reverse proxies and API gateways. It supports routing based on hostnames and URL paths with the following capabilities:

    • Dynamic Host Matching: Supports exact matches, wildcard/suffix matches (e.g., *.example.com), and regex matches with named captures.
    • Flexible Path Matching: Supports exact matches (=), regex matches (~), and prefix matches (default).
    • URL Rewriting: Modify request paths using variables from named captures.
    • Request Control: Implement request throttling (concurrency limits) and body size limiting.
    • Header Modification: Add or set custom HTTP headers before forwarding to upstream services.
    • gRPC-Web Support: Translates gRPC-Web requests to standard gRPC.
    • Extensible Plugins: Attach custom logic via a plugin system.
  4. Understand the Pingap Core architecture

    main
    Pingap Core is a foundational Rust library built on top of the pingora framework. It provides a modular toolkit for building high-performance proxy and networking applications. The architecture centers around a request lifecycle managed by a central context, an extensible plugin system, and specialized services for background tasks and rate limiting.
  5. Usage notes for basic_auth

    main

    When using the basic_auth plugin, keep the following security and performance considerations in mind:

    • Use TLS: Basic authentication sends the password encoded in Base64, not encrypted. Only use this plugin over a TLS connection.
    • Delay Performance: The delay option blocks the request task for its duration. On busy listeners, keep this value well under one second, or combine a short delay with the limit plugin instead.
    • Credential Privacy: Setting hide_credentials = true is recommended if the upstream server does not need the credentials. This prevents the Authorization header from appearing in upstream logs.
  6. Understand certificate storage and persistence

    main

    Issued certificates are written back through the configuration storage used by Pingap. The behavior depends on your storage backend:

    • etcd: Every instance sharing the backend picks up the new certificate automatically. Only one instance needs to perform the ordering process.
    • file: The certificate is saved directly in the configuration directory.
    • Quick Start: Certificates are persisted to ~/.pingap/acme/<domains>.toml (owner-readable only) and restored upon restart.

    Warning on Rate Limits: Let's Encrypt allows only 5 duplicate certificates per week for the same set of domains. Avoid deleting persisted certificates during restarts or deployments, and be cautious with ephemeral containers that do not persist their configuration directory, as crash loops can quickly exhaust your weekly quota.

  7. Core concepts of Pingap Location: Location and Indexing

    main

    Location

    The Location is the central struct that encapsulates a complete set of routing rules. It is initialized from a LocationConf and contains the logic to determine if a request matches and how to handle it.

    LocationHostIndex and ServerLocationRoute

    To optimize performance, Pingap builds a LocationHostIndex when server locations are loaded. For any given request host, the index returns a weight-ordered list of candidates including:

    • Exact host matches
    • Matching suffixes
    • All regex-host locations
    • "Any host" locations

    This allows the system to skip unrelated hosts and only run full path/condition matching on relevant candidates, ensuring high performance even with diverse hostnames.

  8. Understand Referer Restriction behavior

    main

    The behavior of the plugin depends on whether type is set to allow or deny:

    Referertype = "allow"type = "deny"
    Host in the listallowed403
    Host not in the list403allowed
    Header absent403allowed
    Header unparseable as a URL403allowed

    Important Usage Notes:

    • Allow Mode & Direct Access: In allow mode, requests without a Referer header are blocked. This will break direct navigation, bookmarks, and clients using strict Referrer-Policy settings. For hot-link protection where you want to permit direct access, consider using deny mode or scoping the plugin to specific paths (e.g., only /images).
    • Security Warning: The Referer header is client-controlled and can be easily forged. Use this plugin as a convenience measure (like hot-link protection) rather than a primary security control. For actual security, use key_auth or signed URLs.
  9. Understand the Pingap Proxy request lifecycle

    main

    The Pingap Proxy manages the request lifecycle by mapping Pingora callbacks to specific PluginStep values. Plugins are executed at exactly one step; if a plugin does not implement a specific step, it is treated as a silent no-op.

    Plugin Lifecycle Steps:

    • PluginStep::EarlyRequest: Triggered by early_request_filter.
    • PluginStep::Request: Triggered by request_filter after a location has been matched.
    • PluginStep::ProxyUpstream: Triggered by proxy_upstream_filter.
    • PluginStep::UpstreamResponse: Triggered by upstream_response_filter.
    • PluginStep::Response: Triggered by response_filter or response_body_filter.

    Operational Hooks (Non-plugin steps):

    • upstream_peer: Selects the backend and applies retry budgets (max_retries, max_retry_window).
    • connected_to_upstream: Records TCP connect and TLS handshake timings.
    • request_body_filter: Enforces client_max_body_size for the location.
    • fail_to_proxy: Renders error pages using the configured HTML template.
                      ┌──────────────────────────────────────────┐
       client ───────▶│ early_request_filter                     │  PluginStep::EarlyRequest
                      ├──────────────────────────────────────────┤
                      │ request_filter        (location matched) │  PluginStep::Request
                      ├──────────────────────────────────────────┤
                      │ proxy_upstream_filter                    │  PluginStep::ProxyUpstream
                      ├──────────────────────────────────────────┤
                      │ upstream_peer         (backend selected) │
                      │ upstream_request_filter                  |
                      ├──────────────────────────────────────────┤
                      │ upstream_response_filter                 │  PluginStep::UpstreamResponse
                      ├──────────────────────────────────────────┤
                      │ response_filter / response_body_filter   │  PluginStep::Response
                      ├──────────────────────────────────────────┤
       client ◀───────│ logging                                  |
                      └──────────────────────────────────────────┘
  10. Use the directory plugin to serve static files

    main

    The directory plugin allows Pingap to serve static files from a local directory. It includes features like MIME detection, ETag support, Cache-Control headers, HTTP range requests (for partial content), chunked streaming for large files, and an optional HTML directory index.

    To use it, register a plugin with category = "directory" and specify a path (the root directory). You can then assign this plugin to specific locations in your configuration.

    [plugins.web]
    category = "directory"
    path = "/var/www/app"
    index = "index.html"
    chunk_size = "64kb"
    max_age = "1h"
    charset = "utf-8"
    headers = ["X-Content-Type-Options: nosniff"]
    
    [locations.web]
    path = "/"
    plugins = ["web"]
  11. How the HTTP-01 challenge validation works

    main

    Pingap supports the HTTP-01 challenge for ACME certificate issuance. The process works as follows:

    1. Listening: The service listens for incoming HTTP requests on port 80 specifically targeting the ACME challenge path.
    2. Token Retrieval: When a challenge request is received, the service attempts to retrieve the required validation token from a local file (which was previously saved during the authorization step).
    3. Validation:
      • If the token is found in the file, the challenge is successful.
      • If the token is not found, the validation fails.
  12. Configure health check types via URL schemas

    main

    Health checks are configured using a URL-like string where the schema determines the protocol used:

    • tcp://<host>: Performs a TCP health check.
    • http://<host>/<path>: Performs an HTTP health check.
    • https://<host>/<path>: Performs an HTTPS health check.
    • grpc://<host>: Performs a gRPC health check.