DOCSEARCHbyAlgolia
Submit Docs

ns.py Network Simulator

repository·main·Indexed 41 days ago

https://github.com/tl-system/ns.py

A Pythonic discrete-event network simulator built on SimPy. It allows developers to model complex network topologies using components such as packet generators, switches, schedulers (WFQ, DRR, SP), and links. Version 0.4.5 includes an emulation mode to proxy real-world TCP/UDP traffic and a specialized TCP implementation with specific timing and state contracts for accurate latency accounting.

Tokens
3.2K
Snippets
6
Records
13
Agent score
51%

What's inside ns.py

  1. Overview of ns.py network components

    main

    The ns.py simulator provides a variety of pre-implemented networking components that can be connected to build complex topologies. Key categories include:

    Packet Generation

    • Packet: Represents a network packet with creation time, size, ID, flow ID, source, and destination.
    • DistPacketGenerator: Generates packets based on inter-arrival time and size distributions.
    • TracePacketGenerator: Generates packets from a trace file.
    • TCPPacketGenerator: Generates packets using the TCP transport protocol.
    • ProxyPacketGenerator: Redirects real-world packets into the simulation.

    Sinks and Monitoring

    • PacketSink: Receives packets and records delay statistics.
    • TCPSink: Receives packets, records statistics, and produces TCP acknowledgements.
    • ProxySink: Redirects received packets to a real-world TCP server.
    • PortMonitor: Records the number of packets in a Port at intervals defined by a distribution.
    • ServerMonitor: Records performance statistics for scheduling servers (e.g., WFQServer, SPServer).

    Network Elements and Links

    • Wire: A network cable with propagation delay following a distribution (bandwidth is modeled by upstream ports/schedulers).
    • Port: An output port on a switch with a specific rate and buffer size (bytes or packet count) using tail-drop.
    • REDPort: An output port using the Early Random Detection (RED) mechanism.
    • Splitter / NWaySplitter: Elements that duplicate and distribute packets to one or multiple downstream ports.
    • TrTCM: A two-rate three-color marker (RFC 2698).
    • RandomDemux / FlowDemux / FIBDemux: Elements that route packets based on random choice, flow ID, or a Flow Information Base (FIB).

    Traffic Shaping and Scheduling

    • TokenBucketShaper / TwoRateTokenBucketShaper: Traffic shaping elements.
    • SPServer: Static Priority (SP) scheduler.
    • WFQServer: Weighted Fair Queueing (WFQ) scheduler.
    • DRRServer: Deficit Round Robin (DRR) scheduler.
    • VirtualClockServer: Virtual Clock scheduler.

    Switches

    • SimplePacketSwitch: A switch with FIFO bounded buffers on outgoing ports.
    • FairPacketSwitch: A switch supporting WFQ, DRR, Static Priority, or Virtual Clock scheduling on outgoing ports, with support for class-based scheduling via the flow_classes parameter.

  2. Understand the TCP Timing Contract in ns.py

    main

    The TCP implementation in ns.py follows a specific timing and state contract that defines how sequence numbers, ACKs, and latency are handled. Developers implementing or extending TCP components must adhere to these semantics to ensure correct end-to-end latency accounting and retransmission behavior.

    Key Semantics:

    • ACK Tracking: TCPSink tracks the cumulative ACK frontier (RCV.NXT) based only on the contiguous prefix of received data.
    • Segment State: TCPPacketGenerator manages logical segment state, using the segment's start sequence number as the key.
    • Packet Emission: Every send or retransmit operation generates a new Packet object derived from the existing logical segment state.
    • Latency Accounting: For TCP data packets, Packet.time represents the original first-transmit timestamp. Sinks use this specific timestamp to calculate end-to-end latency.
    • RTT and RTO: Round-Trip Time (RTT) and Retransmission Timeout (RTO) updates are managed by the sender and are designed to be conservative. The sender does not estimate RTT from retransmitted or ambiguous ACKs.
    • Supported Features: The current transport rewrite assumes a simplified TCP model: no TCP timestamps, no Selective Acknowledgments (SACK), and no delayed-ACK modeling.

  3. Use emulation mode to proxy real-world traffic

    main

    ns.py supports an emulation mode that acts as a proxy between real-world clients (like web browsers) and real-world servers (like Node.js servers).

    This is achieved using two main components:

    • ProxyPacketGenerator: Handles incoming traffic from a real-world client.
    • ProxySink: Forwards the simulated traffic to a real-world server.

    This allows you to test real network applications within a simulated network topology.

  4. Configure conservative RTT/RTO measurement in TCP

    main

    To avoid errors in RTT (Round Trip Time) and RTO (Retransmission Timeout) estimation, especially when TCP timestamps or SACK are not used, follow a conservative sampling policy:

    Eligibility Rule for RTT Updates: Only update SRTT, RTTVAR, and RTO when an ACK satisfies all the following conditions:

    1. The ACK advances the cumulative frontier by exactly one logical segment.
    2. That specific segment has a retransmit_count == 0 (it was never retransmitted).

    When to skip RTT updates:

    • During cumulative jumps (advancing multiple segments at once).
    • When an ACK newly covers data that was previously retransmitted.
    • When dealing with ambiguous recovery ACKs.

    Handling Ambiguous ACKs: If an ACK is ambiguous (cannot be safely attributed to a specific transmission), you should still process it for congestion control, but pass the current smoothed_rtt or the last valid RTT sample into the congestion control callback instead of attempting a new measurement.

  5. Implement logical segment state for TCP senders

    main

    To support correct TCP transport semantics (including sub-MSS chunks and tail segments), the sender should be refactored to use a SegmentState abstraction. This separates the durable logical state of a data segment from the transient Packet objects emitted during transmission.

    SegmentState should track:

    • seq: Starting sequence number
    • size: Total size of the segment
    • first_tx_time: The original timestamp for latency accounting
    • last_tx_time: The timestamp of the most recent transmission attempt
    • retransmit_count: Number of times this segment has been sent
    • timer state / backoff state: Current retransmission timer status

    Key Behaviors:

    • Fresh Packets: Every attempt (initial send or retransmit) must emit a new Packet object. These packets inherit packet_id and size from the SegmentState but use Packet.time = first_tx_time.
    • MSS Handling: All send paths must compute sendable_bytes = min(mss, available) to correctly handle buffered sub-MSS chunks and tail segments.

  6. Understand the TCP implementation design invariants

    main

    When working with or extending the TCP implementation in ns.py, you must adhere to several core design invariants to ensure correct network simulation behavior:

    1. Packet Timing: Packet.time on TCP data packets must always represent the original first-transmit timestamp. This is critical for sinks to perform accurate end-to-end wait/latency accounting. Do not mutate this timestamp during retransmissions.
    2. State Ownership: Sender timing and retransmission state must reside in sender-owned logical segment state, not within mutable Packet objects.
    3. Cumulative ACKs: The Receiver ACK must represent the cumulative frontier (RCV.NXT), which is the next expected byte based strictly on the contiguous prefix of received data.
    4. Segment Cleanup: Use the logic seq + size <= ack for cleaning up segments, rather than simple sequence number comparisons.
    5. Packet Immutability: Every send or retransmit operation must emit a fresh Packet object derived from the segment state, rather than reusing and mutating an existing object.
    6. RTT Estimation: RTT estimation must rely exclusively on sender-side metadata.
    7. Conservative RTT: In the absence of TCP timestamps, SACK, or delayed-ACK modeling, RTT updates must be conservative (e.g., following Karn/Partridge principles).

  7. Test emulation mode with TCP/UDP echo servers

    main

    You can verify the emulation mode using the provided echo server/client examples.

    TCP Echo Example

    1. Start the TCP echo server:
    python examples/real_traffic/tcp_echo_server.py 10000
    1. Start the ns.py proxy (listening on 5000, redirecting to localhost:10000):
    python examples/real_traffic/proxy.py 5000 localhost 10000 tcp
    1. Run the TCP echo client:
    python examples/real_traffic/tcp_echo_client.py localhost 5000

    UDP Echo Example

    1. Start the UDP echo server:
    python examples/real_traffic/udp_echo_server.py 10000
    1. Start the ns.py proxy (listening on 5000, redirecting to localhost:10000):
    python examples/real_traffic/proxy.py 5000 localhost 10000 udp
    1. Run the UDP echo client:
    python examples/real_traffic/udp_echo_client.py localhost 5000 Hello World
    # TCP Proxy Command
python examples/real_traffic/proxy.py 5000 localhost 10000 tcp

# UDP Proxy Command
python examples/real_traffic/proxy.py 5000 localhost 10000 udp

  8. Install ns.py

    main

    You can install ns.py from PyPI using pip, or set up a local development environment using uv for better dependency management and editable installs.

    # From PyPI
pip install ns.py

# Local development with uv
uv sync
uv run examples/basic.py

  9. Validate TCP and BBR implementations

    main

    To validate the correctness of the TCP and BBR transport implementations, use the following commands to run the test suites and examples:

    # Run the full pytest suite
uv run pytest -q

# Run the classic TCP example
uv run python examples/tcp.py

    Validation focus areas:

    • Classic TCP: Cumulative ACK frontier handling, fresh packet-per-attempt behavior, and conservative RTT/RTO measurement.
    • BBR: Rate-sample stability after retransmission, correct timing/metadata alignment, and avoidance of packet-object reuse.
    uv run pytest -q
uv run python examples/tcp.py

  10. Test emulation mode with an HTTPS server

    main

    To test HTTPS emulation, follow these steps:

    1. Generate a self-signed certificate:
    openssl req -new -x509 -keyout server_cert.pem -out server_cert.pem -days 365 -nodes
    1. Start the HTTPS server:
    python examples/real_traffic/https_server.py 4443 server_cert.pem
    1. Start the ns.py proxy:
    python examples/real_traffic/proxy.py 5000 localhost 4443
    1. Connect using curl:
    curl -v https://localhost:5000 --insecure
    # Generate certificate
openssl req -new -x509 -keyout server_cert.pem -out server_cert.pem -days 365 -nodes

# Start HTTPS server
python examples/real_traffic/https_server.py 4443 server_cert.pem

# Start proxy
python examples/real_traffic/proxy.py 5000 localhost 4443

# Test with curl
curl -v https://localhost:5000 --insecure

  11. Implement new network components using SimPy

    main

    ns.py is built on SimPy. To implement new components, you should understand how to interact with the SimPy environment (env).

    Core SimPy primitives used in ns.py

    • env.process(generator): Adds a component's run() method to the simulation environment. Use this for traffic shapers, packet generators, ports, and schedulers.
    • env.run(until=...): Starts the simulation. Example: env.run(until=100) runs for 100 simulation seconds.
    • yield env.timeout(value): Schedules an event in the future. This is used to model transmission delays or waiting for resources.

    Component Lifecycle and Patterns

    • Processes: Components like schedulers and shapers should be implemented as SimPy processes (generator functions) so they can run concurrently.
    • Non-Processes: Components like flow demultiplexers, packet sinks, or traffic splitters do not need to be processes; they can simply be logic applied to packets.
    • Resource Sharing: Use yield store.get() to retrieve items from a simpy.Store (FIFO buffer).
    • Sending Packets: Use the .put(packet) method on a downstream component to pass data through the network.
    • Flow IDs: Packets are assigned flow IDs (often integers or strings) which are used to index per-flow parameters (like weights or deficit counters) in dictionaries or lists.
    # Example: Scheduling a transmission delay in a scheduler
yield self.env.timeout(packet.size * 8.0 / self.rate)

# Example: Retrieving a packet from a buffer
packet = yield store.get()

# Example: Sending a packet downstream
self.out.put(packet)

  12. Use the Config singleton for global settings

    main
    The Config utility is a global singleton instance that reads parameter settings from a configuration file. To access the configuration instance anywhere in your code, call Config().