llm-sandbox

When executing code that generates visualizations, you can control whether plots from previous runs persist or are cleared using the clear_plots parameter in session.run().

To enable plotting capabilities, you must initialize the ArtifactSandboxSession with enable_plotting=True.

• To clear plots: Set clear_plots=True (or clear_plots=not accumulate_plots in iterative logic) to ensure each execution starts with a clean slate.
• To accumulate plots: Set clear_plots=False to keep plots from previous runs in the session context.

Results from session.run() include a plots attribute, which is a list of plot objects containing content_base64 (the encoded image data) and format (the file format).

from llm_sandbox import ArtifactSandboxSession, SandboxBackend

# Initialize session with plotting enabled
with ArtifactSandboxSession(
    lang="python",
    backend=SandboxBackend.DOCKER,
    enable_plotting=True
) as session:
    # Clear plots before this run
    result = session.run(code, clear_plots=True)
    
    # Access plots
    for plot in result.plots:
        print(f"Format: {plot.format.value}")
        # plot.content_base64 contains the image data

When the pool reaches max_pool_size and all containers are busy, the exhaustion_strategy determines how new requests are handled:

• ExhaustionStrategy.WAIT (Default): Blocks the caller until a container becomes available. Use this for production applications with predictable loads where slight delays are acceptable. Respects acquisition_timeout.
• ExhaustionStrategy.FAIL_FAST: Immediately raises a PoolExhaustedError. Use this for real-time systems that cannot tolerate delays or when you want to implement custom retry logic.
• ExhaustionStrategy.TEMPORARY: Creates a new container outside the pool limits. This container is destroyed after use and not returned to the pool. Use this for handling occasional traffic spikes, but monitor system resources as it bypasses pool limits.

# Example of FAIL_FAST with error handling
from llm_sandbox import SandboxSession
from llm_sandbox.pool import PoolExhaustedError

try:
    with SandboxSession(lang="python", pool=pool) as session:
        result = session.run("print('Hello')")
except PoolExhaustedError as e:
    print(f"Pool exhausted: {e}")

For Docker, Podman, and Micromamba backends, you can pass SANDBOX_* environment variables which are translated into runtime_configs for every sandbox session.

Note: These settings do not apply to the Kubernetes backend.

Security Recommendation: For hardened environments, use SANDBOX_NETWORK_MODE=none, SANDBOX_READ_ONLY=true, SANDBOX_CAP_DROP=ALL, and restrictive SANDBOX_SECURITY_OPT values.

{
  "mcpServers": {
    "llm-sandbox": {
      "command": "python3",
      "args": ["-m", "llm_sandbox.mcp_server.server"],
      "env": {
        "BACKEND": "podman",
        "DOCKER_HOST": "unix:///run/podman/podman.sock",
        "SANDBOX_NETWORK_MODE": "none",
        "SANDBOX_READ_ONLY": "true",
        "SANDBOX_CAP_DROP": "ALL",
        "SANDBOX_MEMORY": "4g"
      }
    }
  }
}

A common pattern for LLM-driven development is to iteratively improve code based on execution feedback. You can use SandboxSession to run generated code and capture exit_code or stderr. If the code fails, feed the error back into the LLM prompt to request a fix, repeating this up to a maximum number of iterations.

from llm_sandbox import SandboxSession
import openai

class SelfCorrectingCodeGenerator:
    """Generate and iteratively improve code using LLM feedback""

    def __init__(self, api_key: str):
        self.client = openai.OpenAI(api_key=api_key)
        self.max_iterations = 3

    def generate_and_test_code(self, task: str, test_cases: list) -> dict:
        """Generate code and iteratively improve it based on test results""

        iteration = 0
        current_code = None
        last_error = ""

        while iteration < self.max_iterations:
            iteration += 1

            # Generate or improve code
            if current_code is None:
                prompt = f"Write Python code to: {task}\n\nInclude proper error handling and documentation."
            else:
                prompt = f"""
                The previous code failed. Here's what happened:

                Code: {current_code}
                Error: {last_error}

                Fix the issues and improve the code to: {task}
                """

            response = self.client.chat.completions.create(
                model="gpt-4",
                messages=[
                    {"role": "system", "content": "You are an expert Python developer. Write clean, efficient, well-tested code."},
                    {"role": "user", "content": prompt}
                ]
            )

            current_code = response.choices[0].message.content

            # Test the generated code
            with SandboxSession(lang="python") as session:
                test_result = session.run(current_code)

                if test_result.exit_code == 0:
                    all_passed = True
                    test_outputs = []

                    for test_case in test_cases:
                        test_code = f"""
# Test case: {test_case['description']}
try:
    result = {test_case['code']}
    expected = {test_case['expected']}
    passed = result == expected
    print(f"Test '{test_case['description']}': {'PASS' if passed else 'FAIL'}")
    if not passed:
        print(f"  Expected: {expected}, Got: {result}")
except Exception as e:
    print(f"Test '{test_case['description']}': ERROR - {e}")
    passed = False
"""
                        test_output = session.run(test_code)
                        test_outputs.append(test_output.stdout)

                        if "FAIL" in test_output.stdout or "ERROR" in test_output.stdout:
                            all_passed = False

                    if all_passed:
                        return {
                            "success": True,
                            "code": current_code,
                            "iterations": iteration,
                            "test_results": test_outputs
                        }
                    else:
                        last_error = "Some test cases failed: " + "\n".join(test_outputs)
                else:
                    last_error = test_result.stderr

        return {
            "success": False,
            "code": current_code,
            "iterations": iteration,
            "final_error": last_error
        }

The security scanner follows a multi-step pipeline to analyze code before execution:

1. Filter Comments: Removes comments using language-specific handlers to prevent false positives (e.g., code mentioned in a comment won't trigger an alert).
2. Generate Module Patterns: Converts restricted modules (defined in RestrictedModule) into regex patterns via language handlers.
3. Apply Patterns: Matches all defined SecurityPattern regexes against the filtered code.
4. Severity Check: Compares detected violations against the SecurityPolicy.severity_threshold to determine if the code should be blocked.
5. Return Results: Reports the safety status (is_safe) and a list of violations.

# Example of the analysis logic flow
code = """
import os  # This imports the OS module
# os.system('commented out') - this won't be detected
os.system('whoami')  # This will be detected
"""

policy = SecurityPolicy(
    severity_threshold=SecurityIssueSeverity.MEDIUM,
    restricted_modules=[
        RestrictedModule("os", "OS interface", SecurityIssueSeverity.HIGH)
    ],
    patterns=[
        SecurityPattern(r"os\.system\s*\(", "System commands", SecurityIssueSeverity.HIGH)
    ]
)

# The process results in: is_safe=False, violations=[2]

For high-frequency code execution (e.g., web APIs, production services, or batch processing), use container pooling to reuse pre-warmed containers. This can provide up to 10x performance improvement by avoiding the overhead of creating new containers for every execution.

To use pooling:

1. Create a pool manager using create_pool_manager with a PoolConfig.
2. Pass the pool instance to the SandboxSession constructor.
3. Ensure you call pool.close() when finished to clean up resources.

from llm_sandbox import SandboxSession
from llm_sandbox.pool import create_pool_manager, PoolConfig

# Configure the pool
pool = create_pool_manager(
    backend="docker",
    config=PoolConfig(
        max_pool_size=10,
        min_pool_size=3,
        enable_prewarming=True,
    ),
    lang="python",
    libraries=["numpy", "pandas"],
)

try:
    # Use pooled session - containers are reused automatically
    with SandboxSession(lang="python", pool=pool) as session:
        result = session.run("import pandas as pd; print(pd.__version__)")
        print(result.stdout)
finally:
    pool.close()