repository·main·Indexed 23 days ago
https://github.com/benchflow-ai/skillsbenchA gym-style benchmark for evaluating how AI agents leverage modular skills—including instructions, scripts, and resources—to perform specialized workflows. It measures skill effectiveness and agent behavior across various coding-agent harnesses such as claude-code, opencode, openhands, codex, and pi. The package includes tools for running evaluation experiments, managing agent timeouts, and generating research figures for the NeurIPS 2026 paper.
The Lean 4 skills suite is composed of three primary plugins:
SkillsBench 1.1 is a benchmark designed to evaluate how AI agents utilize Agent Skills: structured packages containing instructions, scripts, and reference material that are mounted at inference time.
The benchmark features:
no-Skills and curated-Skills conditions to measure "Skill Lift" (the improvement in resolution rate provided by the skills).senior-data-scientist skill is designed for production-grade AI/ML/Data systems. It provides expertise in statistical modeling, experimentation, causal inference, and advanced analytics using a tech stack that includes Python (NumPy, Pandas, Scikit-learn), R, SQL, and various ML frameworks like PyTorch and TensorFlow. Use this skill when you need to design experiments, build predictive models, perform causal analysis, or drive data-driven decisions.web-interface-guidelines skill is used to review UI code for compliance with Vercel's web interface standards. It focuses on accessibility, performance, animations, and visual stability. Use this skill when building or auditing UI components to ensure they meet best practices for modern web development.geometric-layout-repair skill is used when the required deliverable is a repaired 2D plan-view layout and a modified CAD/DXF file, rather than a rendered drawing. It focuses on producing a minimally changed repaired layout in JSON format based on extracted architectural geometry and rule-based clearance constraints.The conditioning skill provides data conditioning techniques for gravitational wave detector data, specifically designed for preprocessing raw detector strain data before matched filtering. It works with PyCBC TimeSeries data and includes high-pass filtering, resampling, removing filter wraparound artifacts, and estimating power spectral density (PSD).
Dependencies:
Requires pycbc:
pip install pycbcThe gpt-multimodal skill uses OpenAI's vision-capable models (like gpt-4o or gpt-5) to perform image analysis, scene understanding, OCR (text extraction), and temporal analysis of video frames. It supports single images, multiple images in one request, and sequential frame processing.
Supported Formats:
Constraints:
from openai import OpenAI
import base64
import json
import os
from pathlib import Pathexoplanet-workflows skill provides general guidance and best practices for exoplanet detection and characterization using light curve data. It is designed to help users plan analysis pipelines, select appropriate period search algorithms, and troubleshoot detection issues such as low signal-to-noise ratios or period aliasing.The xlsx skill provides comprehensive capabilities for creating, editing, and analyzing spreadsheet files (.xlsx, .xlsm, .csv, .tsv). It is designed for tasks such as creating new spreadsheets with formulas and formatting, reading/analyzing data, modifying existing files while preserving formulas, and performing data visualization or recalculation.
Key Requirements:
openpyxl only write formula strings, you must use the recalc.py script (which uses LibreOffice) to calculate the actual values and verify there are no errors.#REF!, #DIV/0!, #VALUE!, #N/A, or #NAME?).The following features are planned for the lean4-memories skill to improve management and utility:
economic-dispatch skill is used for generator economic dispatch and cost optimization in power systems. It is suitable for minimizing generation costs, computing optimal generator setpoints, calculating operating margins, or working with generator cost functions.The motor-model-dynamics skill is used for simulating quadrotor physical dynamics. It provides a two-layer physics model:
[F, Mx, My, Mz] to individual motor RPMs. It accounts for propeller allocation (X-frame), applies first-order motor lag, and returns actual thrust, moments, and the motor RPM derivative.This skill is ideal when you need to simulate the mapping of control inputs to physical movement while accounting for motor latency and rotational/translational physics.