EcoLogits

EcoLogits estimates GPU energy consumption per output token using a parametric model derived from the ML.ENERGY Leaderboard (specifically targeting NVIDIA H100 80GB HBM3 GPUs).

The model distinguishes between total parameters (used to calculate how many GPUs are needed to load the model) and active parameters ($P_{\text{active}}$) (used to estimate energy consumption per GPU).

For Sparse Mixture-of-Experts (SMoE) models, $P_{\text{active}}$ is the number of active experts, whereas for dense models, $P_{\text{active}} = P_{\text{total}}$.

The energy consumption per output token is modeled as a function of active parameters and batch size ($B$), following a linear-exponential form:

$$f_E(P_{\text{active}}, B) = \alpha e^{\beta B} P_{\text{active}} + \gamma$$

In current implementations, the batch size is assumed to be fixed at $B = 64$. The total GPU energy for a request is calculated as:

$$E_{\text{GPU}}(#T_{\text{out}}, P_{\text{active}}) = #T_{\text{out}} \times f_E(P_{\text{active}}, 64)$$

If the model requires multiple GPUs, this value is multiplied by the number of required GPUs.

The video_impacts function returns an ImpactsOutput object. You can access various metrics like energy, global warming potential (GWP), and water consumption.

Because video impact values are often returned as RangeValue intervals, you can access the mean, min, and max values directly to get central estimates or bounds.

# Accessing specific metrics
print(impacts.energy.value)       # Direct electricity consumption, in kWh
print(impacts.gwp.value)          # Global warming potential, in kgCO2eq
print(impacts.adpe.value)         # Abiotic resource depletion, in kgSbeq
print(impacts.pe.value)           # Primary energy, in MJ
print(impacts.wcf.value)          # Water consumption footprint, in L

# Accessing phase-specific impacts
print(impacts.usage.gwp.value)    # Usage phase only
print(impacts.embodied.gwp.value) # Embodied phase only

# Accessing RangeValue components
print(impacts.gwp.value.mean) # Central estimate
print(impacts.gwp.value.min)  # Lower bound
print(impacts.gwp.value.max)  # Upper bound

EcoLogits makes several assumptions to enable environmental impact estimation for LLMs. Users should be aware of these constraints:

Model Assumptions

• Infrastructure: Assumes models are deployed with a vLLM backend and quantized to 16 bits.
• Capabilities: Only accounts for text-to-text generation (no multi-modality).
• Proprietary Models: Since technical details are often undisclosed, EcoLogits relies on parameter count estimations.

Hardware Assumptions

• GPU Type: Assumes models run on NVIDIA H100 GPUs with 80GB of memory.
• Server Type: Assumes base servers are similar to AWS p5.48xlarge instances.
• Exclusions: Does not account for TPUs, other accelerators, networking, storage primitives, or infrastructure overheads.

Data Center & Impact Factor Assumptions

• Metrics: Uses PUE (Power Usage Effectiveness) and WUE (Water Usage Effectiveness).
• Electricity Mix: Uses averaged impact factors per country (GWP from Our World in Data, ADPe/PE from ADEME).
• Limitations: Does not account for local electricity generation specific to a data center or the specific cooling methods used.

Usage impacts are estimated by calculating the generation latency and the power draw of the hardware configuration. The process involves:

1. Latency Estimation: Uses model-specific regressions based on spatiotemporal volume ($T = \frac{W \times H \times F}{1000}$), where $W$ is width, $H$ is height, and $F$ is frame count (calculated at 24 fps).
2. Energy Consumption: Calculated as $E_{\text{server}} = \frac{\Delta T}{3600} \times P_{\text{server}}$, where $P_{\text{server}}$ is the power of the full machine in kW.
3. Data Center Overhead: The server energy is multiplied by the Power Usage Effectiveness (PUE) to get $E_{\text{request}} = \text{PUE} \times E_{\text{server}}$.
4. Environmental Impact: The electricity consumption is multiplied by the country-specific electricity-mix factor ($F_{\text{em}}$).

Water Consumption Footprint (WCF) for usage phase is calculated using: $$\text{WCF}^{\text{u}}{\text{request}} = E{\text{server}} \times [\text{WUE}{\text{on-site}} + \text{PUE} \times \text{WUE}{\text{off-site}}]$$