nnInteractive Documentation

The server supports multiple concurrent client sessions up to the limit defined by --max-sessions.

Session Lifecycle

• Claiming: A session is automatically claimed when nnInteractiveRemoteInferenceSession is instantiated. The client manages a private lease token automatically.
• Releasing: Use a context manager or call session.close() to release the session slot back to the server immediately.
• Concurrency: While multiple sessions can exist, the GPU is a shared resource. Prediction calls are serialized via a global GPU lock. Non-prediction calls (like set_image) can run concurrently across sessions.

Handling Capacity

If the server is at its --max-sessions limit, constructing a session will raise ServerAtCapacityError. You should implement a retry mechanism.

import time
from nnInteractive.inference.remote import (
    nnInteractiveRemoteInferenceSession,
    ServerAtCapacityError,
)

for attempt in range(6):
    try:
        session = nnInteractiveRemoteInferenceSession(server_url, api_key=KEY)
        break
    except ServerAtCapacityError:
        time.sleep(10)
else:
    raise SystemExit("server has been at capacity for too long")

The nnInteractiveRemoteInferenceSession allows you to run heavy 3D segmentation models on a remote GPU server while driving the interaction from a lightweight GUI client over HTTP.

Key architectural concepts:

• Shared Model: The server loads the model weights into GPU memory only once at startup. All client sessions share these weights.
• Per-Client Sessions: Each client is assigned a unique session via a lease token. Each session maintains its own independent state, including the current image, target_buffer, and interaction history.
• Concurrency: While multiple clients can preprocess images simultaneously, predictions are GPU-serialized (only one prediction runs at a time) to manage GPU resources.
• Session Management: Sessions are managed via --max-sessions. If a client becomes inactive or loses connection, the server reaps the session based on --idle-timeout-seconds or --liveness-timeout-seconds to free up slots.

[GUI client A]  ─┐
                 │ HTTP
[GUI client B]  ─┼────►  nninteractive-server  ──►  one shared model on GPU
                 │       (per-client sessions:           (loaded once at startup)
[GUI client C]  ─┘        image, target_buffer, 
                          interactions per session)

The server reaps sessions based on two independent timeout mechanisms. If a session is reaped, the next request will raise SessionExpiredError. Because all state (image, buffer, interactions) is stored on the server, a reaped session cannot be restored; the user must start the workflow over.

Timeout Types

1. Liveness Timeout (--liveness-timeout-seconds, default 60s): Triggered if the client process stops responding (e.g., crash). The client library automatically sends heartbeats in a background thread to prevent this.
2. Idle/Inactivity Timeout (--idle-timeout-seconds, default 600s): Triggered if the user is inactive. Heartbeats do not reset this timer. Only real interactions (like set_image or add_*_interaction) reset the idle timer.

Monitoring

• session.lease_status(): Returns the remaining seconds until the idle timeout. This is a read-only probe and does not affect the timer.

from nnInteractive.inference.remote import SessionExpiredError

try:
    session.add_point_interaction([60, 70, 30], include_interaction=True)
except SessionExpiredError:
    # The server-side session is gone. There is nothing to restore.
    # The user has to start the segmentation workflow over.
    session = nnInteractiveRemoteInferenceSession(server_url, api_key=KEY)
    session.set_image(image)
    session.set_target_buffer(buf)

When designing your deployment, be aware of the following constraints:

• GPU Serialization: Within a single server process, predictions are serialized. Multiple clients can preprocess concurrently, but only one prediction runs at a time on the GPU. For higher throughput, run multiple server processes on different GPUs.
• Authentication Model: Authentication is a single shared bearer token. There is no per-user identity, quota, or login flow.
• Fixed Checkpoint: The model checkpoint is loaded at startup and remains fixed for the lifetime of the server process.
• No TLS by Default: The server does not terminate TLS; use a reverse proxy for secure deployments.
• No Client-side Retries: The client does not implement automatic reconnection. If a network blip or SessionExpiredError occurs, the error is raised to the caller; the application (e.g., a GUI) must handle re-initialization.