DOCSEARCHbyAlgolia
Submit Docs

CVEfixes Dataset

repository·main·Indexed 43 days ago

https://github.com/secureit-project/cvefixes

An automated dataset of vulnerabilities and their fixes extracted from NVD CVE records. It provides granular source code data at the CVE, repository, commit, file, and method levels to support security research. As of v1.0.8, it includes 12,107 vulnerability fixing commits across 4,249 open-source projects, covering 11,873 CVEs and 272 CWE types.

Tokens
9.3K
Snippets
18
Records
22
Agent score
57%

What's inside CVEfixes

  1. Overview of CVEfixes dataset

    main

    CVEfixes is an automated collection of vulnerabilities and their corresponding fixes from open-source software. It is curated from Common Vulnerabilities and Exposures (CVE) records in the U.S. National Vulnerability Database (NVD).

    The dataset provides interlinked information at multiple levels of abstraction to support data-driven security research:

    • CVE level
    • Repository level
    • Commit level
    • File level
    • Method/Function level

    As of v1.0.8 (covering CVEs up to 23 July 2024), the dataset includes 12,107 vulnerability fixing commits across 4,249 open-source projects, covering 11,873 CVEs and 272 CWE types. It contains source code snapshots (before and after changes) for 51,342 files and 138,974 functions.

  2. Understand the CVEfixes relational database structure

    main

    The CVEfixes dataset is organized as a relational database where multiple tables represent artifacts at different abstraction levels. These tables are interconnected to link vulnerability information (CVEs) with their corresponding fixes (commits), the repositories they belong to, and specific code-level changes.

    Key abstraction levels include:

    • CVE level: Vulnerability metadata (severity, description, CVSS scores).
    • Fix level: Links CVEs to specific commit hashes and repository URLs.
    • Commit level: Detailed git commit metadata and code metrics.
    • Repository level: Metadata about the source projects (stars, forks, language, etc.).
    • Code change level: Granular details regarding file and method changes.

  4. Convert the CVEfixes dataset from the compressed SQL dump

    main

    If you have downloaded the compressed SQL dump from Zenodo (DOI: 10.5281/zenodo.4476563), you can convert it into an SQLite3 database using the provided script or manually via the command line.

    Using the provided script:

    $ sh Code/create_CVEfixes_from_dump.sh

    Manual conversion (Linux/Mac):

    $ gzcat Data/CVEfixes.sql.gz | sqlite3 Data/CVEfixes.db

    Manual conversion (Windows): Use sqlite3.exe instead of sqlite3 in the command above.

    $ sh Code/create_CVEfixes_from_dump.sh

  5. Gather the CVEfixes dataset from scratch

    main

    To recreate the entire CVEfixes database (covering records from 2002 to the latest available), run the following script.

    Note: This process can take a significant amount of time (8-10 hours on a fast connection, up to 3 days on a home connection). Running this script will overwrite any existing CVEfixes.db file in your configured database_path.

    Prerequisites:

    1. A configured .CVEfixes.ini file.
    2. A GitHub token configured in .CVEfixes.ini to prevent rate-limiting.
    $ sh Code/create_CVEfixes_from_scratch.sh

  6. How to use or gather CVEfixes

    main

    Depending on whether you want to use the existing dataset or replicate the collection process from scratch, refer to the following documentation:

    • To use the existing dataset: See the first section of INSTALL.md.
    • To gather the dataset from scratch:
      • Review the requirements in REQUIREMENTS.md.
      • Follow the gathering instructions in the second section of INSTALL.md.

  7. Set up a GitHub token for data collection

    main

    Since over 98% of CVEfixes repositories are on GitHub, using the GitHub API is necessary to retrieve repository metadata. Without a token, the API is rate-limited to approximately 25 repositories.

    To avoid rate limits, add a [GitHub] section to your .CVEfixes.ini file with your credentials:

    • user: Your GitHub username.
    • token: Your GitHub personal access token.

    Warning: Do not disclose your token in public repositories.

    [GitHub]
user = YOUR_USERNAME
token = YOUR_GITHUB_TOKEN

  8. Explore the CVEfixes vulnerability data

    main

    The CVEfixes database is an SQLite3 database. You can explore its structure using an ER diagram (available at Doc/ER_diagram.png) and query it with any SQLite tool, such as DB Browser for SQLite.

    For guided exploration, a Jupyter Notebook is provided with example queries for extracting data at different abstraction levels, generating statistics, and replicating plots from the original paper.

    Example: Extracting C programming language method-level vulnerability data

    SELECT m.method_change_id, m.name, m.code, m.before_change, f.programming_language
from file_change f, method_change m
WHERE m.file_change_id=f.file_change_id
AND f.programming_language='C';

    Example: Extracting Java code patches with single-statement changes

    SELECT cv.cve_id, f.filename, f.num_lines_added, f.num_lines_deleted, f.code_before, f.code_after, cc.cwe_id
FROM file_change f, commits c, fixes fx, cve cv, cwe_classification cc
WHERE f.hash = c.hash
AND c.hash = fx.hash
AND fx.cve_id = cv.cve_id
AND cv.cve_id = cc.cve_id
AND f.num_lines_added<=1
AND f.num_lines_deleted<=1
AND f.programming_language='Java';

  9. Gather a sample of CVEfixes for demonstration

    main

    If you want to test the collection process without waiting days, use the sample_limit variable in .CVEfixes.ini.

    When sample_limit is set to a value greater than 0, the system collects only that many commits from CVEs for the current year. To minimize collection time, the tool automatically filters out five major projects when a sample limit is active.

    For example, setting sample_limit = 25 takes approximately 12 minutes.

    [CVEfixes]
sample_limit = 25

  10. Configure the .CVEfixes.ini file

    main

    To collect the dataset from scratch, you must create a .CVEfixes.ini configuration file in the project root. You can use example.CVEfixes.ini as a template.

    The [CVEfixes] section supports the following keys:

    • database_path: The directory where the CVEfixes database file will be stored and where temporary files will be used during extraction.
    • sample_limit: The number of samples to extract.
      • Set to 0 for unlimited samples (full dataset).
      • Set to a positive integer to collect only that many commits for CVEs in the current year (useful for demonstrations).

    Example configuration structure:

    [CVEfixes]
database_path = /path/to/data
sample_limit = 25
    [CVEfixes]
database_path = /path/to/data
sample_limit = 0

  11. Connect to the CVEfixes SQLite database

    main

    To interact with the CVEfixes dataset, you must establish a connection to the CVEfixes.db SQLite database. The following pattern uses sqlite3 to create a connection. You can also use sqlalchemy for more advanced database operations.

    import sqlite3 as lite
from sqlite3 import Error
from pathlib import Path

def create_connection(db_file):
    """
    create a connection to sqlite3 database
    """
    conn = None
    try:
        conn = lite.connect(db_file, timeout=10)
    except Error as e:
        print(e)
    return conn

# Assuming the database is in a 'Data' directory relative to the parent of the current working directory
DATA_PATH = Path.cwd().parents[0] / 'Data'
conn = create_connection(DATA_PATH / "CVEfixes.db")

  12. Extract added and deleted statements from diffs

    main

    When analyzing patches at the method level, you can use difflib to generate a diff between the vulnerable code (before_change) and the fixed code (code). You can then parse this diff to separate added statements from deleted statements. This is useful for creating datasets for patch prediction tasks.

    import difflib

def to_diff_file(old_str, new_str):
    """
    produces a diff file comparing two files/methods.
    """
    diff = difflib.ndiff(old_str.splitlines(1), new_str.splitlines(1))
    result = '\n'.join(diff)
    return result

# Example usage to extract added/deleted lines from a diff string
def to_diff_dict(str1):
    lines = str1.splitlines()
    diff_dict = {}
    added_lines = []
    deleted_lines = []
    for line in lines:
        if(line[0]=='+'):
            added_lines.append(line[1:].strip())
        elif(line[0]=='-'):
            deleted_lines.append(line[1:].strip())
    diff_dict = {'added':added_lines, 'deleted':deleted_lines}
    return diff_dict