DOCSEARCHbyAlgolia
Submit Docs

Copulas

repository·main·Indexed 20 days ago

https://github.com/sdv-dev/copulas

A Python library for modeling multivariate distributions and generating synthetic numerical data using Gaussian, Archimedian, and Vine Copulas. It provides tools to learn statistical properties of numerical datasets, sample new synthetic rows via a fit-and-sample workflow, and visually compare real versus synthetic data using 1D histograms, 2D scatterplots, and 3D scatterplots.

Tokens
2K
Snippets
10
Records
12
Agent score
70%

What's inside Copulas

  1. Overview of Copulas features

    main

    Copulas is a Python library designed for modeling multivariate distributions and sampling from them using copula functions. It is primarily used to learn the statistical properties of a table of numerical data and generate new synthetic data that follows those same properties.

    Key capabilities:

    • Multivariate Modeling: Supports various univariate distributions and copulas, including Archimedian, Gaussian, and Vine Copulas.
    • Visual Comparison: Provides tools for 1D histograms, 2D scatterplots, and 3D scatterplots to compare real and synthetic datasets.
    • Parameter Manipulation: Allows users to access and tune the learned internal parameters of the models.

  2. Install Copulas for development

    main

    If you are contributing to or modifying the source code, install the package in development mode using make install-develp. It is recommended to create a new branch from main before proceeding.

    git clone git@github.com:sdv-dev/Copulas
cd Copulas
git checkout main
git checkout -b <your-branch-name>
make install-develp

  5. Install Copulas from source

    main

    To install from source, clone the repository and run make install while on the stable branch. Note that this method is only supported on Unix-based systems (e.g., GNU/Linux and macOS).

    git clone https://github.com/sdv-dev/Copulas
cd Copulas
git checkout stable
make install

  7. Generate synthetic data using Gaussian Copulas

    main

    To generate synthetic data, you follow a three-step workflow:

    1. Load or prepare your real numerical data.
    2. Initialize a multivariate copula model (e.g., GaussianMultivariate) and call .fit(real_data) to learn the distribution.
    3. Call .sample(n_samples) to generate new synthetic rows.

    Copulas also supports Archimedian Copulas and Vine Copulas for different modeling needs.

    from copulas.datasets import sample_trivariate_xyz
from copulas.multivariate import GaussianMultivariate

# 1. Prepare data
real_data = sample_trivariate_xyz()

# 2. Fit the model
copula = GaussianMultivariate()
copula.fit(real_data)

# 3. Sample synthetic data
synthetic_data = copula.sample(len(real_data))

  9. Quickstart: Model and generate synthetic multivariate data

    main

    This quickstart demonstrates the end-to-end workflow of modeling a multivariate dataset using a Gaussian copula and generating synthetic data that resembles the original.

    1. Load Data: Use copulas.datasets to obtain sample data.
    2. Fit Copula: Initialize a GaussianMultivariate model and call .fit(real_data) to learn the dependencies between columns.
    3. Sample Data: Use .sample(n_samples) to generate new synthetic rows.
    4. Visualize: Use copulas.visualization.compare_3d to compare the distributions of the real and synthetic datasets.
    import warnings
warnings.filterwarnings('ignore')

from copulas.datasets import sample_trivariate_xyz
from copulas.multivariate import GaussianMultivariate
from copulas.visualization import compare_3d

# Load a dataset with 3 columns that are not independent
real_data = sample_trivariate_xyz()

# Fit a gaussian copula to the data
copula = GaussianMultivariate()
copula.fit(real_data)

# Sample synthetic data
synthetic_data = copula.sample(len(real_data))

# Plot the real and the synthetic data to compare
compare_3d(real_data, synthetic_data)

  10. Generate synthetic training data using GaussianMultivariate

    main

    You can use the GaussianMultivariate class from copulas.multivariate to generate a synthetic copy of a dataset. To preserve the relationship between features ($X$) and targets ($y$), concatenate them into a single dataset before fitting the model. After fitting, use the .sample() method to generate new synthetic observations, which can then be split back into $X$ and $y$ components.

    This approach is useful for scenarios where real data cannot be shared due to privacy or proprietary concerns, but a synthetic version that preserves statistical properties is required for machine learning research.

    import numpy as np
from copulas.multivariate import GaussianMultivariate

# 1. Prepare the dataset by concatenating features and target
dataset = np.concatenate([X_train, np.expand_dims(y_train, 1)], axis=1)

# 2. Fit the Gaussian copula model
model = GaussianMultivariate()
model.fit(dataset)

# 3. Generate synthetic samples
synthetic = model.sample(len(dataset))

# 4. Split synthetic data back into features and target
X_synthetic = synthetic.values[:, :-1]
y_synthetic = synthetic.values[:, -1]

  11. Visualize real vs synthetic data with compare_3d

    main

    After generating synthetic data, you can visually compare it to the original dataset using the copulas.visualization module. The compare_3d function creates a 3D scatterplot to show how well the synthetic data captures the multivariate relationships of the real data.

    from copulas.visualization import compare_3d

compare_3d(real_data, synthetic_data)

  12. Use GaussianMultivariate for synthetic data generation

    main

    The GaussianMultivariate class is used to model multivariate distributions using a Gaussian copula. It follows a standard fit and sample workflow.

    • fit(dataset): Learns the statistical properties and dependencies of the provided dataset.
    • sample(n_samples): Generates n_samples of synthetic data based on the fitted model.
    from copulas.multivariate import GaussianMultivariate

model = GaussianMultivariate()
model.fit(dataset)
synthetic = model.sample(len(dataset))