Runme Documentation

Runme Harbor is an optional Python adapter for runme eval. Its primary purpose is to bridge Harbor's environment interface with a local runme harbor stdio process.

While task scaffolding is managed by the Go CLI (via runme eval task new), the Harbor package is specifically focused on runtime integration. It builds upon the upstream Harbor Python package.

When using runme eval, the command delegates to runme-harbor.

• Runme Environment (--env runme): This is the default. Runme environments default to one concurrent trial because they share the host workspace.
• Other Environments (e.g., --env docker): These use Harbor's concurrency defaults. When using a non-Runme environment, the selected agent is delegated to Harbor without the Runme-specific agent wrappers.
• Concurrency: You can override concurrency defaults by using passthrough arguments. For example, to set concurrency to 4, use -- --n-concurrent 4.
• Metadata: Each runme eval execution creates Harbor job and trial metadata under the .runme/evals/jobs directory.

The Runme Agent server provides several core services required for notebook and cell execution:

• WebSocket Transport (/ws): A bidirectional WebSocket used to execute cells and stream terminal I/O. This is available when assistantServer.runnerService is enabled.
• Runner Service: Manages execution sessions and related runner operations. Enabled via assistantServer.runnerService.
• Parser Service: Handles parsing and serialization of notebook content. Enabled via assistantServer.parserService.
• Jupyter Proxy: Manages Jupyter servers and forwards kernel channel WebSockets.
• Authentication & Telemetry: Provides optional OIDC authentication, authorization, and telemetry via the HTTP server.
• Static Asset Hosting: Serves web applications from a configured directory.

Runme allows you to control how environment variables are handled during execution using the promptEnv option in a code block's metadata.

• promptEnv: "auto": Runme will automatically detect and prompt the user for any environment variables used in the block that are not currently set in the environment.
• promptEnv: "no": Runme will not prompt for missing environment variables. If a variable is missing, the command will execute with an empty value or fail as per standard shell behavior.

$ export VAR_NAME1='Placeholder 1'
$ echo "1. ${VAR_NAME1}"

You can attach tags to snippets using the tag key in the code block metadata. Tags can be single values or comma-separated lists. This allows you to group and filter snippets during execution or listing.

echo "Tag A,B"

Runme turns Markdown files into interactive runbooks by executing code inside fenced code blocks.

Key Features:

• Runtime Support: Supports Shell/Bash, Python, Ruby, JavaScript/TypeScript, Lua, PHP, Perl, and more via shebangs.
• State Persistence: Environment variables are retained across the execution of different cells, similar to a terminal session.
• Piping: You can pipe the output of one code cell into a subsequent cell.
• Compatibility: Works alongside existing task definitions like Makefile, NPM scripts, or Pipfile.

You can configure OpenTelemetry (OTLP) telemetry for the Runme Agent by specifying an otlpHTTPEndpoint in your config.yaml.

To test tracing locally, you can run Jaeger using Docker:

telemetry:
  otlpHTTPEndpoint: localhost:4318

docker run --rm --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 5778:5778 \
  -p 9411:9411 \
  jaegertracing/jaeger:2.6.0

Runme Harbor is a Python adapter used by runme eval to connect Harbor's environment interface to a local runme harbor stdio process. It is installed as an isolated Python CLI tool using uv.

Note: The runme CLI must be installed separately and must be available on your PATH for the adapter to function correctly.

uv tool install runme-harbor

You can configure a Markdown file to run non-interactively by setting skipPrompts: true in the runme frontmatter block. This is useful for automated environments or CI/CD pipelines where manual input for variable prompts is not possible.

Example configuration:

---
runme:
  id: <your-id>
  version: v3
  skipPrompts: true
---

---
runme:
  id: 01HF7BT3HBDTRGQAQMHDAQCQEP
  version: v3
skipPrompts: true
---

Tool definitions are defined via RPCs in Protobuf. To generate the corresponding Model Context Protocol (MCP) and JSONSchemas, you must use the protoc-gen-go-mcp plugin. Currently, generation is performed locally using buf rather than through Bazel.

To regenerate the tool definitions:

1. Download the protoc-gen-go-mcp plugin.
2. Execute the build script: ./build_tool_mcps.sh.

The tool protos are located in api/proto/agent/tools/v1.

You can run runme directly in a web browser by using its WebAssembly (WASM) build. This allows for parsing Markdown snippets within a web environment.

Setup Steps

1. Build the WASM binary: From the root of the project, execute the make command to build runme and populate the necessary files for the web example.

   make wasm

2. Start a web server: Navigate to the examples/web directory and start a local web server (e.g., using Python).

   python -m http.server 9000

3. Verify: Open http://localhost:9000 in your browser. Open the Developer Tools > Console to see the parsed snippets from the Markdown files.

Before interacting with Runme gRPC services, you must set the following environment variables to define the server address, session strategy, and TLS directory:

• RUNME_SERVER_ADDR: The address of the Runme server (e.g., 127.0.0.1:9999).
• RUNME_SESSION_STRATEGY: The strategy for managing sessions (e.g., recent).
• RUNME_TLS_DIR: The directory containing the TLS certificates (e.g., /tmp/runme/tls).

export RUNME_SERVER_ADDR="127.0.0.1:9999"
export RUNME_SESSION_STRATEGY="recent"
export RUNME_TLS_DIR="/tmp/runme/tls"