fg-data-synthetic

The ydata-synthetic package is an open-source Python library designed for experimenting with generative models to create synthetic data. It serves as an alternative or supplement to real-world data, particularly when real data is scarce or for boosting Machine Learning performance.

Supported data types and models include:

• Tabular Data: Generatable via Variational Auto Encoders (VAE), Gaussian Mixture Models (GMM), and Conditional Generative Adversarial Networks (CTGAN).
• Time-Series Data: Generatable via TimeGAN.

The package also features a user-friendly UI interface to guide users through the synthetic data generation process.

The ydata-synthetic library uses telemetry to collect anonymous, non-personal usage data to improve performance and prioritize features. The following data points are collected:

• Python version
• ydata-synthetic version
• Frequency of use of ydata-synthetic features
• Errors or exceptions thrown within the library

The package supports two primary types of data:

1. Tabular Data: Structured data organized in rows (observations) and columns (features). It handles both numeric (quantitative) and categorical (qualitative, including ordinal, binary/boolean, and nominal) features. Tabular data is characterized by a lack of temporal dependence.
2. Time-Series Data: Sequential data exhibiting temporal dependency between records. This includes patterns such as seasonality (repeating calendar patterns) and periodicity (repeating patterns over time).

TimeGAN is not designed to replicate the full long-term behavior or temporal patterns of a dataset. Instead, it operates on the concept of "windows".

It learns to map the data distribution of short-term frames within the specific time windows provided. Because it treats these windows as independent, it cannot return the long-term temporal patterns typically expected in time-series analysis.

Synthetic data quality is evaluated across three essential pillars:

• Privacy: The ability of the synthetic data to withhold personal, private, or sensitive information and prevent data leakage or connections to the original data.
• Fidelity: How closely the synthetic data preserves the properties and distributions of the original data ("how faithful/precise" it is).
• Utility: The ability of the synthetic data to be used in downstream applications to derive the same insights as the original data.

Use the validator object to define assertions (Expectations) about your data. After defining your expectations, call validator.save_expectation_suite() to persist them.

# Create expectations
validator.expect_column_values_to_not_be_null("age")
validator.expect_column_values_to_be_between("workclass", auto=True)
validator.save_expectation_suite()

Since version 1.0.0, ydata-synthetic includes a GUI experience powered by Streamlit. This UI supports the full data synthesization workflow, from reading data to profiling synthetic data generation.

To install the GUI, use the [streamlit] extra during installation.

Note: The Streamlit app is not supported in Jupyter or Colab Notebooks; it must be run in a standard Python environment.

pip install "ydata-synthetic[streamlit]"

To use the UI-guided experience for generating structured tabular data, you must install the streamlit extra for the ydata-synthetic package.

pip install ydata-synthetic[streamlit]

Install the package via PyPI using pip. This package is used to generate synthetic tabular and time-series data using generative models like GANs and Gaussian Mixtures.

Note: The package requires Python 3.9 or 3.10.

pip install fg-data-synthetic

If you have installed the package with the streamlit extra, you can launch an interactive UI to synthesize data by calling streamlit_app.run() from the data_synthetic module. The console will provide the URL to access the application.

from data_synthetic import streamlit_app

streamlit_app.run()

The package includes a Streamlit-based UI for a low-code experience to train synthesizer models and generate/profile synthetic data samples for structured tabular data.

Supported models in the UI:

• CGAN
• WGAN
• WGANGP
• DRAGAN
• CRAMER
• CTGAN

Installation: You must install the streamlit extra to use the UI:

pip install fg-data-synthetic[streamlit]

Running the UI: You can run the UI from within a Python script (note: Jupyter Notebooks are not supported) or via the command line.

from data_synthetic import streamlit_app

streamlit_app.run()