NginxPulse Documentation

The NginxPulse Wiki provides detailed guides for the following topics:

• Quick Start
• Deployment Methods
• SQLite to PostgreSQL Migration
• Configuration Details
• Log Source Configuration
• Supported Log Formats
• Agent Collection
• Log Parsing Mechanism
• IP Geo Resolution
• Database Schema
• FAQ

NginxPulse resolves IP locations using a tiered approach from fastest to slowest:

1. DB cache (ip_geo_cache)
2. Local ip2region (v4/v6)
3. Remote ip-api.com (batch lookup)

If local lookups fail or return "unknown", the system falls back to the remote lookup.

Processing Flow

1. Parsing writes IPs into the ip_geo_pending queue.
2. A background task processes these in batches (default size: 500).
3. Results are saved to ip_geo_cache and backfilled into log tables.
4. The cache is automatically trimmed when it exceeds the system.ipGeoCacheLimit threshold.

The project is organized into the following key directories:

• cmd/nginxpulse/: The main entry point for the application.
• internal/: Core logic including:
  • app/: Initialization and task scheduling.
  • analytics/: Statistical logic and aggregation.
  • enrich/: IP geolocation and PV filtering.
  • ingest/: Log scanning, parsing, and database ingestion.
  • server/: HTTP service and middleware.
  • store/: PostgreSQL repository and schema.
  • web/: API routing.
  • webui/: Embedded frontend static assets.
• webapp/: Frontend source code.
• webapp_mobile/: Mobile frontend source code (accessible via /m).
• configs/: Configuration files (nginxpulse_config.json, nginx_frontend.conf).
• scripts/: Build and deployment scripts (build_single.sh, dev_local.sh, publish_docker.sh).
• var/: Runtime data directory.

NginxPulse follows a multi-stage process to ingest and enrich log data:

1. Initial Scan: Parses logs from the "recent window" upon startup.
2. Incremental Scan: A scheduled task continuously scans for new content based on system.taskInterval.
3. Historical Backfill: Gradually processes historical logs in the background without blocking real-time parsing.
4. IP Geo Backfill: Asynchronously parses and enriches IP geolocation data after logs are stored in the database.

NginxPulse follows a four-stage process for handling logs:

1. Initial scan: Parses the recent window of logs immediately after startup.
2. Incremental scan: Scans newly appended logs based on the system.taskInterval setting.
3. Historical backfill: Processes older logs in the background to ensure historical data is available without blocking real-time parsing.
4. IP geo backfill: Asynchronously resolves IP locations after the log entries have been inserted into the database.

NginxPulse resolves IP locations using a tiered approach from fastest to slowest to optimize performance:

1. Database Cache: Checks the ip_geo_cache first.
2. Local Libraries: Uses ip2region (supporting both v4 and v6) for local lookups.
3. Remote API: If the local library returns "未知" (Unknown) or fails, it calls the remote interface (e.g., ip-api.com) in batches.

Optimization Tip: To minimize remote API calls and latency, ensure your local ip2region libraries are updated to cover as many IP ranges as possible.

The frontend implements a full-screen or masked initialization wizard that is only displayed when the backend reports setup_required=true.

Wizard Features:

• Multi-step workflow with a progress bar.
• "Advanced" collapsible sections for each step to hide complexity.
• Real-time validation for required fields and backend-driven validation error messages.
• JSON preview mode for the configuration before saving.
• Post-save notification: Users must be prompted to restart the service after saving configuration changes (specifically for changes involving ports, databases, or access keys) to ensure they take effect.

To use host-side logs in a Docker container, mount the host directory to a path inside the container, then point NginxPulse's logPath to that internal path.

# Docker Compose volume mount
volumes:
  - /var/log/nginx:/share/logs/nginx:ro

# NginxPulse configuration
{
  "name": "主站",
  "logPath": "/share/logs/nginx/access.log",
  "domains": ["example.com"],
  "logType": "nginx"
}

The mobile interface (/m/) allows temporary overrides of the navigation bar position using URL parameters. These settings are written to local storage and persist across subsequent page navigations.

Parameter Priority: tabbarBottom takes precedence over tabbar.

Values:

• Bottom Navigation (Truth): 1, true, yes, on, bottom.
• Top Navigation (False): 0, false, no, off, top.

Usage Examples:

• Force bottom navigation: https://example.com/m/?tabbarBottom=true
• Force top navigation: https://example.com/m/?tabbarBottom=false