OpenWISP Network Topology

repository·master·Indexed 42 days ago

https://github.com/openwisp/openwisp-network-topology

A specialized tool for collecting, storing, and visualizing network topology data. It provides a web application for users and an API for integration with networking software such as OpenVPN or mesh routing protocols. The system utilizes InfluxDB 1.8-alpine and Redis for data persistence and management.

Tokens
10K
Snippets
29
Records
44
Agent score
49%

What's inside OpenWISP Network Topology

  1. Overview of OpenWISP Network Topology features

    master

    The OpenWISP Network Topology module provides tools for managing, visualizing, and monitoring network topologies. Key capabilities include:

    • Topology Collection: Supports multiple formats including NetJSON NetworkGraph, OLSR, batman-adv, BMX6, CNML 1.0, OpenVPN, Wireguard, and ZeroTier. You can extend this by writing custom netdiff parsers.
    • Visualization: Uses netjsongraph.js for rendering topologies.
    • Data Access: Provides a REST API that exposes data in the NetJSON NetworkGraph format.
    • Management: An admin interface for managing, auditing, visualizing, and debugging nodes, links, and topology data.
    • Data Ingestion: Supports receiving network topology data from multiple nodes.
    • Topology History: Enables saving and viewing daily snapshots of topologies in the frontend.
    • Monitoring Integration: Integrates with OpenWISP Controller and OpenWISP Monitoring for rapid detection of critical network events.
  2. Overview of OpenWISP Network Topology

    master

    OpenWISP Network Topology is a web application and API designed to collect, store, visualize, and edit network topology data. It can ingest data from various networking software, such as dynamic mesh routing protocols and OpenVPN.

    Key capabilities include:

    • Data Collection: Ingests topology information from different networking sources.
    • Visualization & Editing: Provides a UI to view and modify network details.
    • Event Hooks: Uses Django signals to trigger code execution whenever a link status changes.
    • Monitoring Integration: When used with the OpenWISP Controller and OpenWISP Monitoring, it accelerates the detection of network changes.
  3. Overview of the WebSocket API

    master

    The WebSocket API provides real-time topology updates for OpenWISP Network Topology. It is designed for observing changes to an existing topology in real time rather than performing CRUD operations.

    Key Characteristics:

    • Push-only: The server pushes real-time updates to the client after the connection is established.
    • Unidirectional: The server does not accept client messages; any data sent from the client is ignored.
    • Format: All messages on the wire are JSON-encoded.
    • Complementary to REST: Use the REST API for creating, retrieving, updating, or deleting topology data. Use the WebSocket API to observe those changes live.
  4. Use the RECEIVE strategy for topology collection

    master

    The RECEIVE strategy allows topology data to be pushed directly from one or more network nodes to the collector via HTTP POST requests.

    How it works:

    1. The collector waits for a POST request containing a payload.
    2. It checks the key parameter in the request against the Topology instance's key.
    3. If authorized, the collector updates the topology.

    Configuration based on node count:

    • Single Node: If only one node sends data, set the expiration_time of the Topology instance to 0 (seconds). This mimics the FETCH strategy behavior where missing links are flagged as "down" immediately.
    • Multiple Nodes: If data is sent from multiple nodes, set the expiration_time to a value slightly higher than the interval used by the nodes to send data. This prevents links from being flagged as "down" prematurely due to network splits or timing jitter, though it introduces a delay in detecting offline links.
  5. Extend Management Commands

    master

    To add or extend management commands in your custom application, create a management/commands directory within your app. You can use the existing commands as templates by referencing the implementation in the test suite.

    Example structure: your_app_name/management/commands/ containing your command files (e.g., save_snapshot.py, update_topology.py).

    your_app_name/
    └── management/
        └── commands/
            ├── save_snapshot.py
            └── update_topology.py
  6. Configure ZeroTier topology with RECEIVE strategy

    master

    To set up ZeroTier topology visualization:

    1. Create Topology: Visit admin/topology/topology/add. Set Label to ZeroTier, select Format as ZeroTier, and set Strategy to RECEIVE. Ensure Published is checked. Copy the generated receive URL.
    2. Create Script: Create a script (e.g., /opt/send-zt-topology.sh) that uses zerotier-cli peers -j to fetch data in JSON format and sends it via curl to the OpenWISP receive URL.
    3. Permissions: Because zerotier-cli requires root privileges for kernel interaction, you must add the script to the root crontab using sudo crontab -e.

    Note: If using the OpenWISP controller integration, ensure the ZeroTier topology organization matches the device organization.

    #!/bin/bash
    # command to fetch zerotier controller peers data in json format
    COMMAND="zerotier-cli peers -j"
    UUID="<TOPOLOGY-UUID-HERE>"
    KEY="<TOPOLOGY-KEY-HERE>"
    OPENWISP_URL="https://<OPENWISP_DOMAIN_HERE>"
    $COMMAND |
        # Upload the topology data to OpenWISP
        curl -X POST \
            --data-binary @- \
            --header "Content-Type: text/plain" \
            $OPENWISP_URL/api/v1/network-topology/topology/$UUID/receive/?key=$KEY
  7. Access the Network Topology REST API documentation

    master

    The Network Topology project provides two ways to explore its API surface:

    1. Live OpenAPI Documentation: A general live API documentation following the OpenAPI specification is available at /api/v1/docs/.
    2. Browsable Web Interface: Every endpoint can be opened directly in a browser to use the Django-REST-Framework browsable API interface, which provides a user-friendly way to inspect endpoint details and interact with resources.
  8. Install openwisp-network-topology from GitHub

    master

    You can install the latest development version directly from GitHub using either the tarball via HTTPS or the git protocol.

    # Install via tarball
    pip install https://github.com/openwisp/openwisp-network-topology/tarball/master
    
    # Install via git protocol
    pip install -e git+git://github.com/openwisp/openwisp-network-topology#egg=openwisp_network_topology
  9. Import and run automated tests for custom applications

    master

    When developing a custom application based on openwisp-network-topology, you should import and run the base tests to ensure your changes do not break existing features.

    To set up testing:

    1. Inherit Tests: Refer to the tests.py file in the sample_network_topology app. You can overwrite tests defined in base classes to test your specific behavior if you introduce breaking changes.
    2. Extend Fixtures: Copy openwisp_network_topology/fixtures/test_users.json into your application's fixtures/ directory.
    3. Run Tests: Use the Django test runner, specifying your app name.

    Note: The --parallel flag is optional.

    # Substitute sample_network_topology with your app name
    ./manage.py test --parallel sample_network_topology
  10. Configure Celery Beat for WiFi Mesh topology generation

    master

    If you have enabled the WiFi Mesh integration, you must schedule the create_mesh_topology task in your CELERY_BEAT_SCHEDULE.

    This task generates the mesh topology from monitoring data. It requires two arguments:

    1. A list of organization UUIDs: The topology will only be created for devices belonging to these organizations.
    2. A lookback period (in seconds): The task will ignore monitoring data reported before this time.
    from datetime import timedelta
    
    CELERY_BEAT_SCHEDULE.update(
        {
            "create_mesh_topology": {
                "task": "openwisp_network_topology.integrations.device.tasks.create_mesh_topology",
                "schedule": timedelta(minutes=5),
                "args": (
                    [
                        "4e002f97-eb01-4371-a4a8-857faa22fe5c",
                        "be88d4c4-599a-4ca2-a1c0-3839b4fdc315",
                    ],
                    6 * 60,  # 6 minutes
                ),
            },
        }
    )