OpenCost Documentation

OpenCost uses standard Kubernetes and cloud terminology to define cost-incurring entities. Key concepts include:

• Cluster Assets: Observable entities in a Kubernetes cluster that directly incur costs (e.g., nodes, persistent volumes, attached disks, load balancers).
• Pod: A Kubernetes-specific concept consisting of a group of containers, treated as a single block of resources for scheduling and scaling.
• Namespace: A virtual cluster within Kubernetes used to deploy and observe pods/containers discreetly.
• Pod Labels: Key/Value pairs used to identify objects and group multiple namespaces associated with a specific workload.
• Container: An instance of a container image; the smallest identifiable unit of resource usage.

OpenCost calculates inference costs using two different bases:

• Allocation (cost_basis=allocation): Includes max(request, usage) × price plus idle and shared infrastructure shares. Use this for chargeback, showback, and bill reconciliation.
• Usage (cost_basis=usage): Includes only actual resource consumption. Use this for pure workload efficiency analysis.

Note that usage does not reconcile to the infrastructure bill as it excludes idle and shared infrastructure costs.

The OpenCost Specification provides a vendor-neutral methodology for measuring and allocating Kubernetes infrastructure and container costs. It categorizes costs into three main pillars:

1. Total Cluster Costs = Cluster Asset Costs + Cluster Overhead Costs.
2. Cluster Asset Costs = Resource Allocation Costs (provisioned/reserved) + Resource Usage Costs (consumed per unit).
3. Asset Costs Distribution = Workload Costs (attributed to tenants) + Cluster Idle Costs (unallocated resources).

This hierarchy allows organizations to distinguish between what is being paid for (Assets), what is being used by workloads, and what is being wasted (Idle).

OpenCost calculates inference costs using two primary methods:

1. Compute-Time Based Allocation (Default)

OpenCost splits infrastructure costs between input (prefill) and output (decode) based on processing time:

• InputCost = TotalCost × (PrefillTime / TotalTime)
• OutputCost = TotalCost × (DecodeTime / TotalTime)

If vLLM timing metrics are unavailable, OpenCost falls back to a multiplier method where output tokens are assumed to cost 2.5× input tokens (allocationMethod=multiplier).

2. KV Cache Savings

OpenCost reports cacheSavingsFraction (range 0–1) using the ratio of cachedTokens / promptTokens. This is derived from the vllm:prefix_cache_hits_total metric. This fraction makes the cost benefit of KV caching explicit in the API.

OpenCost is transitioning to a modular architecture to decouple core cost calculation logic from specific data sources. The architecture relies on three primary components:

1. Metric Collection and Queries: Traditionally uses Prometheus to scrape emitters and expose metrics required for cost calculation. The goal is to replace the direct Prometheus dependency with a DataSource contract.
2. Cloud Provider: An abstraction that provides specific cost data for a given resource, used to calculate the cost of that resource.
3. Kubernetes API: Acts as the glue between raw metric data (from the metric source) and cost data (from the Provider).

By implementing a DataSource contract, OpenCost aims to allow substitution of Prometheus with other sources like InfluxDB or in-memory sources for testing, without refactoring the core implementation.

The Model Context Protocol (MCP) server is disabled by default. It provides AI agents with access to cost allocation, asset, and cloud cost data via HTTP on port 8081. You can enable it and configure it using Helm --set flags.

Common Configuration Options:

• opencost.mcp.enabled=true: Enables the MCP server.
• opencost.mcp.port: Sets a custom port (default is 8081).
• opencost.mcp.extraEnv.MCP_LOG_LEVEL: Sets the logging level (e.g., debug).

To activate the inference cost tracking feature, set the following environment variable on your OpenCost deployment:

INFERENCE_COST_ENABLED=true

When enabled, OpenCost initializes the collector, calculator, exporter, and runner to process vLLM metrics and compute token-based costs.

The proposed DataSource contract is intended to provide an abstract interface for metric collection and querying. This allows developers to substitute Prometheus with other metric sources (e.g., InfluxDB or in-memory sources for testing) while maintaining the existing OpenCost data model and metric emission process.

In the modular structure, each data source can be maintained within its own Go module inside the opencost repository. This allows consumers to select only the specific modules they require rather than inheriting the entire dependency tree.